Fine-tuning & Adaptation
Customize a foundation model on your own data. Manage datasets, run hyperparameter sweeps as variants, and pick the winning checkpoint — all from one graph.
The pattern
Foundation model adaptation usually means:
- Load a base model from a registry.
- Prepare a labeled adaptation dataset.
- Train one or more variants (LR sweeps, LoRA rank sweeps, full-fine-tune vs. adapter).
- Evaluate each on a held-out set.
- Promote the best checkpoint.
Connectify lets you do all five in a single project, with each training run as its own variant.
Building the base graph
-
Add the base model
Click + Node → Model and select your foundation model from the registry (e.g.,
llama-3-8b-base). The Model node exposes the model as a tensor producer. -
Wire in adaptation data
Add a Dataset node pointed at your fine-tuning corpus. Add a Logic
tokenizenode configured for the model's tokenizer. Wire Dataset → tokenize → Model (training input). -
Configure training
In the Model node's Config tab set:
mode:lorarank:8lr:2e-4epochs:3
-
Add eval
Add a held-out Dataset node for evaluation and a Logic
eval/perplexitynode. Wire the model's output and eval data into it.
Running a hyperparameter sweep
Variants are the right tool for sweeps. Each combination of hyperparameters gets its own variant:
- Click + Variant, name it
lr=5e-5, edit only that one config value. - Repeat for
lr=1e-4,lr=2e-4,lr=5e-4. - Right-click your favorite candidate → Set as baseline.
- Run each variant in turn. Use Compare to diff results pairwise.
LoRA rank sweep
Same pattern — one variant per rank value. Because LoRA training is cheap, sweeping ranks 4, 8, 16, 32 in parallel is the fastest way to find the right tradeoff.
Managing checkpoints
Each successful run produces a checkpoint, recorded in the Model node's Run Data tab. Promote one with the Star action — starred checkpoints get a tag like baseline, candidate, or shipped that you can filter by from the Graphs Hub.
Exporting the winner
Open the starred checkpoint, click Export, and pick a destination — Hugging Face, S3, a local download, or a registered internal registry (if your workspace has one configured).
Tracking what you tried
Hidden variants stay in the project. Don't delete failed sweeps — hide them. Six months later when someone asks "did we try a higher LR?" the answer is one click away.