can-i-finetune-this

Recipes now preserve native system/multi-turn conversations and use one versioned Transformers runtime with explicit loss masks, truncation and requested/effective configuration. Loader errors cannot silently change precision or attention. Real updates, staged model/adapter saves and reload checks replace template-only validation.

Start with pip install canifinetune==0.4.0, an offline metadata estimate, or canifinetune demo for the local interactive preflight. Evidence export is opt-in, redacted and manual; community uploads remain unreviewed. GPU discovery records memory sources and uses the lower CUDA-driver/nvidia-smi free reading for an unambiguous single-device budget.

Compatibility: core Python 3.10–3.14; qualified training Python 3.12/torch 2.6, minimum and recommended constraints. CPU full/LoRA were exercised on Linux and Windows. Local RTX 4080 CUDA LoRA/QLoRA candidate checks execute updates, save and reload outside the checkout. WSL GPU, other GPU hardware, Flash Attention and Liger are not qualified; Liger remains experimental.

The prospective pinned 0.5B cohort has four three-update successes. Old/new estimates are unchanged and conservatively high (46.2% MAPE); this release does not claim an accuracy improvement or calibrated OOM probability. Historical observations retain their original memory scope and are separate from fits and final candidate qualification.

Migration: regenerate recipes, install their exact package requirement, remove unknown legacy YAML keys and opt into right truncation deliberately. Default all-token loss is retained. Old calibration caches need regeneration; legacy measurement fits cannot apply to the new estimator. Inference saves do not implement full training resume.

Select an unquantized base and request quantization explicitly. Pre-quantized checkpoints are rejected before weights: Transformers can retain their own configuration instead of the requested one. Main README links are versioned absolute URLs so they also work on PyPI.

Quickstart · Training contract · Validation and limits

Release assets include the exact qualified wheel/sdist, SHA256SUMS, candidate.json and qualification.json. The existing Trusted Publishing workflow verifies these files, uploads them without rebuilding and checks public PyPI hashes and installation. See the attached receipts for commit, software, actual update counts and allocator peaks. Maintainer: Daoyuan Li.

Post-release verification: the official PyPI wheel and sdist match the qualified hashes exactly. Fresh public-index installations outside the checkout passed torch-free core, CPU full/LoRA and CUDA LoRA/QLoRA updates, saves and reloads. The pinned 0.5B QLoRA quickstart also completed two updates and adapter reload/generation. All three release jobs passed, and PyPI exposes normal publishing attestations. Verification report · Redacted public-install receipt.

This release improves estimate accuracy for mixture-of-experts models and tightens the benchmark, calibration, and recipe workflows.

Highlights:

• use exact safetensors parameter counts when available
• account for stored and active experts in MoE models
• separate quantization metadata from packed weight memory
• calibrate dynamic memory without rescaling static weights
• fail clearly when QLoRA or 8-bit optimizer dependencies are unavailable
• add safer recipe generation and optional Liger kernels
• expand CI coverage through Python 3.14

See CHANGELOG.md for the complete list of changes.

Estimator accuracy overhaul

The static estimate was 0.5–3.9× off versus real training peaks — worst at long seq_len and for untied-embedding models, and in the dangerous direction (configs called feasible could OOM). 0.2.0 rewrites the memory model against measured torch.cuda traces on an RTX 4080:

• New logits / loss component — the (batch × seq × vocab) logits + fp32 cross-entropy chain (~14 B per token per vocab entry). For a 152k-vocab model at seq 2048 this is ~4.1 GiB, the single largest training buffer, and gradient checkpointing does not remove it. Previously unmodeled.
• QLoRA weights are no longer "all params × 0.5 B" — only Linear layers are 4-bit; embeddings, lm_head (twice for untied models), and norms stay full precision and are upcast to fp32 by prepare_model_for_kbit_training. Measured: Qwen2.5-7B loads at 7.2 GiB, not 3.9. tie_word_embeddings is now part of model metadata.
• Activation coefficients re-fitted for the modern stack (SDPA, SwiGLU-width MLPs, fp32 k-bit intermediates); checkpointing charges saved block inputs plus a single-layer recompute peak; NF4 double-quant absmax overhead corrected to the measured 0.017 B/param.

Across ten measured configurations (0.5B–7B, seq 512–4096, batch 1–2, checkpointing on/off, LoRA + QLoRA) the estimated real components land within 0.93–1.05× of max_memory_allocated, with the headroom terms keeping the total slightly conservative. Those runs ship as regression fixtures in tests/test_estimator_accuracy.py, and docs/rtx4080_baselines.md shows estimated-vs-measured for every row.

Expect estimates to be higher than 0.1.x — they were wrong before, not the workload. A 12 GB card asking about 7B QLoRA now correctly hears "no" (11.2 GB measured).

Fixes

• Generated train.py recipes no longer train on padding: labels are masked to -100 via attention_mask (a real EOS survives even when pad_token == eos_token).
• Infeasible full fine-tunes now get "switch to QLoRA" as the first degradation suggestion instead of no-op LoRA-rank reductions.
• Bench result filenames encode checkpointing / quantization / scope / forward-only, so A/B runs stop overwriting each other.
• Reports show - instead of a misleading quantization scheme for non-QLoRA rows; calibration skips smoke-scale samples and scales the logits term consistently.

Data

The committed benchmark grid grew to 12 configurations (seq 512–4096, batch 1–2, ckpt on/off, LoRA + QLoRA, 0.5B–7B) with fresh estimates embedded, and scripts/run_4080_baselines.sh reproduces the whole table.

Docs-only patch release — first release published via PyPI trusted publishing.

• README masthead: SVG hero banner (also rendered on the PyPI project page), centered badges
• Architecture diagram now uses an absolute URL so it renders on PyPI
• No code changes since 0.1.0

Install: pip install canifinetune