Fine-tuning

docs.scrimba.com

Fine-tuning means taking an existing model and continuing to train it on your own examples, so its default behaviour shifts toward what you want. This chapter is the working picture of what that changes, what it is good for, and when to reach for it instead of a prompt or retrieval.

What training actually does

To see what fine-tuning changes, it helps to know where a model comes from. A model is built in two broad stages.

The first is pretraining: the model reads an enormous amount of text and learns to predict the next token, over and over, until grammar, facts, and patterns of reasoning have settled into it as a side effect. What comes out is a base model, fluent but unfocused. It will happily continue your text, but it has not been taught to be a helpful assistant that answers questions.

The second stage is post-training: the base model is trained further on curated examples of good responses, along with human feedback about which answers are better. This is what turns a raw text predictor into the instruction-following assistant you actually talk to. The model you call from an API has already been through both.

Both stages work by the same mechanism: show the model examples and nudge its internal numbers, its parameters or weights, so its predictions move closer to those examples. Training is prediction practice that leaves the model permanently changed.

JunoWhat training actually does A model is built in two stages. Pretraining has it predict the next token across huge amounts of text, producing a fluent but unfocused base model. Post-training then trains it on curated good responses and human feedback, turning it into a helpful assistant. Both work the same way: adjusting the model's internal weights so its predictions match the examples it is shown.

A model reaches you after two training stages, and knowing the shape of each one tells you what fine-tuning can move.

Pretraining comes first: the model reads a vast pile of text and does one drill, predict the next token, billions of times. Grammar, world knowledge, and reasoning patterns settle into the weights as a side effect. The result is a base model, fluent at continuing text but not yet an assistant. It does not know to answer a question rather than extend it.

Post-training is the second stage, and it is the one fine-tuning imitates. The base model is trained further on curated examples of good responses plus human preference signals about which answer is better. That is what produces the instruction-following behaviour you call from an API.

Both stages run the same loop: feed the model an example, compare its prediction to the target, and adjust the weights to close the gap. Fine-tuning is that same loop, scoped down to your examples and run after the provider's two stages are done. Training shifts behaviour by editing weights, not by adding a lookup table, and that distinction is the whole reason fine-tuning is good at some jobs and bad at others.

JunoWhat training actually does Two stages get a model to you: pretraining drills next-token prediction into a fluent base model, then post-training shapes it into an assistant with curated examples and preference data. Both run the same loop, feed an example, adjust the weights toward it, and fine-tuning is that loop scoped to your data. It edits weights, it does not add a lookup table, which is why it changes behaviour well and stores facts poorly.

A model arrives after two training stages, and the boundary between them is where fine-tuning lives.

Pretraining is the expensive, once-per-model stage: enormous text corpora, the single objective of next-token prediction, and the weights (the model's tunable numbers) absorbing grammar, knowledge, and reasoning as a byproduct. The output is a base model, fluent but not steered toward being helpful. Post-training then aligns that base model into an assistant using curated demonstrations and human preference data. By the time you call an API model, both have run.

Fine-tuning is a continuation of post-training, scoped to your data and run on top of an already-aligned model. The mechanism is identical: present an example, measure how far the prediction sits from the target, adjust weights to reduce that gap. The consequence that matters operationally is that fine-tuning moves the same global parameters that hold everything else. You are not appending a module, you are nudging a shared surface, which is why a fine-tune can improve your target task and quietly degrade an unrelated capability at the same time.

That shared-surface property sets up every failure mode later in this chapter: overfitting, forgetting, drift. Fine-tuning reshapes behaviour, it never installs a fact store. Hold that and you will reach for it for the right reasons and skip it for the wrong ones.

JunoWhat training actually does Pretraining is the once-per-model heavyweight that drills next-token prediction into a base model, post-training aligns it into an assistant, and fine-tuning continues that second stage on your data. Same loop every time: present an example, adjust weights toward it. The catch is that you are nudging shared global parameters, not bolting on a module, so a fine-tune can lift your task and dent an unrelated one in the same run, which is where overfitting and forgetting come from.

What fine-tuning is

Fine-tuning is a small, targeted version of that second stage, run by you. You take a model that is already trained and continue training it on a set of your own examples, each one an input paired with the output you wish it produced. After enough examples, the model's weights shift so that this kind of response becomes its default, without you asking for it in the prompt every time.

The real distinction from prompting and RAG is where the change lives. Prompting and RAG leave the model frozen and steer it through the context you supply at the moment of the call. Fine-tuning changes the model itself, so the new behaviour is baked in and shows up even with a short prompt. You are not giving better instructions at runtime; you are shipping a model that already leans the way you want.

In practice you hand the provider a file of example conversations, it runs a training job, and it gives you back a new model id. You call that id exactly like any other model, except its answers now reflect what you trained on. Fine-tuning moves behaviour into the model, prompting and RAG steer a frozen one.

JunoWhat fine-tuning is Fine-tuning is a small version of post-training that you run: continue training an existing model on your own input-output examples until the behaviour you want becomes its default. Unlike prompting and RAG, which steer a frozen model through context, fine-tuning changes the model's weights, so the behaviour is baked in and shows up even with a short prompt. You hand over a file of examples and get back a new model id you call like any other.

Fine-tuning is post-training scoped to one task and run by you. You start from an already-trained model and continue training it on examples, each an input paired with the output you wish it had produced. Feed it enough of them and the weights settle so that response becomes the default, with no instruction in the prompt asking for it.

The line that matters is where the steering lives. Prompting and RAG leave the weights frozen and shape behaviour through context supplied at call time, so every call carries the instructions. Fine-tuning edits the weights once, up front, so the behaviour ships inside the model id and survives a short prompt. Runtime steering versus baked-in behaviour: that is the trade you are making.

The workflow is concrete. You assemble a file of example conversations, start a training job, and the provider returns a new model id. You call that id like any other model.

# Each example is a conversation in the exact shape you send at runtime,
# ending with the output you want the model to learn.
training_examples = [
    {"messages": [
        {"role": "system", "content": "Classify the ticket's urgency as low, medium, or high."},
        {"role": "user", "content": "I was charged twice this week and need it fixed today."},
        {"role": "assistant", "content": "high"},
    ]},
    {"messages": [
        {"role": "system", "content": "Classify the ticket's urgency as low, medium, or high."},
        {"role": "user", "content": "Is there a dark mode setting anywhere?"},
        {"role": "assistant", "content": "low"},
    ]},
    # ... hundreds to thousands more, covering the full range of inputs you expect
]
# The message shape and the training-job API vary by provider; the idea is the same.

JunoWhat fine-tuning is Fine-tuning is post-training scoped to your task: continue training a model on input-output examples until the behaviour you want is the default. Prompting and RAG steer a frozen model through context every call, fine-tuning edits the weights once so the behaviour rides inside the model id. You build a file of example conversations, run a training job, and call the returned id like any other model.

Fine-tuning is continued post-training on a dataset you control, producing a derived model behind a new id. Each training example is an input paired with a target output, and the job nudges the weights until your target responses become the model's default. The behaviour ships in the weights, so a short prompt is enough to evoke it.

Set it against the alternatives precisely, because the difference is operational, not philosophical. Prompting and RAG keep the weights frozen and inject behaviour through the context (the text the model reads at call time), which means you can change behaviour by editing a string and redeploying in seconds. Fine-tuning bakes behaviour into the weights, which means changing it requires a fresh training run, an eval pass, and a redeploy of a new model id. Runtime steering is editable in seconds, a fine-tune is editable only in a training cycle.

# A fine-tuned model is another id; the call shape is unchanged.
response = client.chat.completions.create(
    model="your-org/urgency-classifier-v1",  # pin this; treat it as part of your contract
    messages=[
        {"role": "system", "content": "Classify the ticket's urgency as low, medium, or high."},
        {"role": "user", "content": "The export button does nothing when I click it."},
    ],
)
# response.choices[0].message.content -> "low"  (the label only, the behaviour you trained in)
# Call shape shown is OpenAI-SDK style; it varies by provider.

That cost asymmetry is the thing to internalise before you commit. A fine-tuned id is a versioned artifact: you own its lifecycle, its evals, and its re-training every time the desired behaviour moves. Pin the id, version it, and budget for the fact that every change to what it learned is another job, not another commit.

JunoWhat fine-tuning is Fine-tuning is continued post-training on your data, yielding a derived model behind a new id whose behaviour lives in the weights. The contrast with prompting and RAG is operational: runtime steering is an editable string you redeploy in seconds, a fine-tune is a versioned artifact you change only through a training run, an eval pass, and a redeploy. Pin the id, treat it as a contract, and budget for the fact that every behaviour change is another job.

What it is good at, and what it is not

Fine-tuning earns its place when you need a consistent behaviour that a prompt struggles to pin down: a precise output format every time, a specific tone or house style, or a narrow task like classification done reliably across thousands of calls. It can also make calls cheaper and faster, because behaviour you would otherwise spell out in a long prompt is baked into the model, so your prompts get shorter.

Where it falls down is knowledge. Fine-tuning is poor at teaching a model new facts, and worse at facts that change. Training examples blur into general patterns rather than getting stored as exact, lookup-able entries, so a fine-tuned model still invents details and still goes stale the moment your information updates. Retraining every time a price or a policy changes is slow and expensive. When the problem is "the model does not know X", the answer is almost always RAG, not fine-tuning.

The rule of thumb: fine-tune to change form, not to add facts. Teach the model how to answer, and let retrieval handle what to answer with.

JunoWhat it is good at, and what it is not Fine-tuning is good for consistent behaviour: a fixed output format, a specific tone, or a narrow task done reliably, and it can shorten prompts so calls cost less. It is bad at knowledge: it does not store facts reliably, goes stale when they change, and is expensive to retrain. The rule is fine-tune to change form, not to add facts, and let RAG handle the facts.

Fine-tuning pays off for form: a precise output shape on every call, a fixed tone or house style, or a narrow task like classification held steady across thousands of requests. The win compounds when prompting alone keeps drifting, the model honours the format most of the time but not reliably enough to parse downstream. Bake the behaviour in and the format stops being something you hope for each call.

It also shortens prompts, and at scale that is a real cost lever. If you currently spell out the format and a few examples in every request, those instruction tokens get billed on every single call. Fold them into the weights and the per-call prompt shrinks. The arithmetic is worth doing: a fine-tune has a fixed training cost paid once, while a long prompt has a per-call cost paid forever, so prompt-shortening pays off when call volume is high enough that the saved tokens outrun the training bill.

Where it falls down is knowledge. Training examples blur into general patterns, not exact retrievable entries, so a fine-tuned model still invents details and still goes stale the moment a price or policy changes. Updating a fact means another training run. Fine-tune for form, use RAG for facts, and combine them. The strong pattern is both at once: fine-tune the model for your format and tone, then retrieve current facts at runtime and let the tuned model phrase them.

Before you climb to fine-tuning at all, confirm prompting actually failed. "Failed" means a clear instruction with a few well-chosen examples, tested across varied inputs, still misses the target rate, not that your first one-line prompt looked off. If you have not run that test, you are not ready to fine-tune.

JunoWhat it is good at, and what it is not Fine-tuning is for form: a fixed output shape, a set tone, a narrow task held steady, plus shorter prompts that save tokens on every call. The cost math is fixed-once training versus pay-forever prompt tokens, so prompt-shortening pays off at high volume. It is poor at facts, which blur into patterns and go stale, so pair a tuned model with RAG: form from the weights, facts from retrieval. And prove prompting actually failed, a real prompt with examples tested across inputs, before you climb.

Fine-tuning is reliable for form and unreliable for facts, and the reasons are mechanical. Form, output shape, tone, task conventions, is a behavioural regularity the weights can absorb across many examples. Facts are discrete and changeable, and the weights store them lossily as blended patterns, not as retrievable rows, so a tuned model still confabulates and still goes stale the instant the underlying fact moves.

Push past the headline and the real risks are the ways a fine-tune degrades. Overfitting is the model memorising your training set, including its quirks, instead of learning the general behaviour, so it nails your examples and fumbles inputs that differ. Catastrophic forgetting is the flip side: training hard on a narrow task can erode unrelated general ability the base model had, because you are moving shared weights, so your classifier gets sharper while its reasoning on anything else gets duller. Distribution drift is the slow one: a model tuned on last quarter's input distribution decays as real traffic shifts away from what it saw, and the decay is silent until you measure it.

Two consequences shape how you actually run this. First, you cannot ship a fine-tune on vibes; you need an eval set, a held-out batch of inputs with known-good answers, scored before and after, to prove the target improved and, no less important, that general ability did not regress. Second, every change to desired behaviour is another full job: re-curate data, retrain, re-eval, redeploy a new id. That operational drag is why parameter-efficient methods exist at all, ways to fine-tune by training a small set of added weights instead of the whole model, cheaper to run and store, though the decision of whether to fine-tune is unchanged by them. Either way, prove the fine-tune did not regress before you ship it.

JunoWhat it is good at, and what it is not Form absorbs into weights, facts do not: a fine-tune still confabulates and goes stale, so keep facts in retrieval. The risks are overfitting (memorising your set, fumbling new inputs), catastrophic forgetting (narrow training eroding unrelated ability since the weights are shared), and distribution drift as real traffic moves off your training data. So gate every fine-tune on a held-out eval scored before and after to prove no regression, and budget for the fact that each behaviour change is a full retrain-eval-redeploy cycle, which is the whole reason parameter-efficient methods exist.

Prompt, RAG, or fine-tune?

Most of the time you do not start here. The three techniques form a ladder, cheapest and fastest to change first:

1. Prompting. Always try this first. It is instant to iterate, costs nothing extra, and a clear prompt with a couple of examples solves more than people expect. If you have not made the prompt work, you are not ready to fine-tune.
2. RAG. Reach for this when the gap is knowledge: the model needs facts it does not have, or that change over time. RAG supplies them at question time without touching the model.
3. Fine-tuning. Reach for this when the gap is behaviour rather than knowledge and prompting cannot get it consistent enough, or when your prompts have grown so long that baking the instructions into the model is cheaper and faster at scale.

They are not exclusive, and the strongest systems combine them: fine-tune a model for your format and tone, then use RAG to feed it current facts at runtime. Behaviour from training, knowledge from retrieval.

So a quick test before you fine-tune. Can you get there with a better prompt? Then do that. Is the gap missing knowledge? Then it is RAG.

Climb the ladder; fine-tuning is the last rung, not the first. Only when the prompt is already right, the facts are already available, and the model still will not behave consistently enough does fine-tuning become the tool that fits.

JunoPrompt, RAG, or fine-tune? Treat the three as a ladder: prompt first because it is instant and cheap, add RAG when the model is missing knowledge, and fine-tune only when the gap is behaviour that prompting cannot make consistent, or to shorten long prompts at scale. They combine well: fine-tune for form, use RAG for facts. Fine-tuning is the last rung you reach for, not the first.

The three techniques form a ladder, ordered by how fast and cheap they are to change, and you climb it from the bottom.

1. Prompting. First, always. You iterate in seconds at no extra training cost, and a clear instruction with a few examples covers more ground than people expect. Until a real prompt has been tested and fallen short, the higher rungs are premature.
2. RAG. When the gap is knowledge, facts the model lacks or that change over time, retrieval supplies them at question time and leaves the weights alone. Updating a fact is updating your source, not retraining.
3. Fine-tuning. When the gap is behaviour that prompting cannot pin down, or when prompts have grown long enough that baking instructions into the weights is cheaper at scale.

The diagnosis that puts you on the right rung is naming the gap. A wrong format or tone is a behaviour gap, so prompt, then fine-tune. A missing or outdated fact is a knowledge gap, so retrieve. Misread the gap and you will fine-tune to fix a knowledge problem and watch it go stale, or stuff facts into a prompt that a retrieval step should have fetched.

They are not exclusive, and the combined pattern is the one to reach for in production: fine-tune for form, RAG for facts, together. A model tuned to your format and tone, fed current facts at runtime, gives you consistent shape and fresh content at once, neither of which the other technique would deliver alone.

Whether you can fine-tune at all depends on the model. Closed providers offer managed fine-tuning of their own models; an open-weight model you can fine-tune yourself on your own terms, which is one of the reasons to reach for open weights (Open and closed models).

JunoPrompt, RAG, or fine-tune? Climb the ladder from the cheapest rung: prompt first, add RAG when the gap is knowledge, fine-tune only when it is behaviour prompting cannot pin down or to shorten prompts at scale. The diagnosis is naming the gap, format or tone means behaviour, a missing or stale fact means retrieval. The production move is both together: fine-tune for form, RAG for facts, so you get consistent shape and fresh content at once.

The ladder, prompt then RAG then fine-tune, is ordered by iteration cost: how fast and cheap it is to change behaviour once shipped. A prompt is a string you edit and redeploy in seconds. A retrieval source is data you update without touching the model. A fine-tune is a training run, an eval pass, and a new model id. Climbing in that order means you only pay the higher iteration cost when the cheaper rungs provably cannot reach.

The real skill is diagnosing the gap correctly, because the failure mode is choosing the tool that does not address it. A behaviour gap (wrong shape, wrong tone, inconsistent task adherence) is what prompting and then fine-tuning fix. A knowledge gap (a fact the model lacks, or one that changes) is what retrieval fixes, and fine-tuning a knowledge gap is the classic expensive mistake: you train facts in, they are stale by the next update, and you cannot cite them.

Hold the operational asymmetry next to the diagnosis. RAG's recurring cost is retrieval quality and latency on every call, paid forever but adjustable live. Fine-tuning's recurring cost is a re-run on every change to learned behaviour, plus the regression risk each run carries.

So the combined architecture is not a compromise, it is the design: tune form into the weights, keep facts in retrieval, and version each independently. Form changes rarely and tolerates a slow update cycle; facts change constantly and demand a fast one. Splitting them lets each move at its own speed, and lets you re-eval the tuned model without re-indexing your corpus, or refresh the corpus without re-running a training job.

JunoPrompt, RAG, or fine-tune? The ladder is ordered by iteration cost: a prompt edits in seconds, a source updates without the model, a fine-tune is a full train-eval-redeploy. Diagnose the gap before you pick, behaviour goes to prompting then fine-tuning, knowledge goes to retrieval, and fine-tuning a knowledge gap is the costly mistake that ships stale, uncitable facts. The production architecture splits them on purpose: tune form into the weights, keep facts in retrieval, version each independently so form can move slowly and facts can move fast.

In practice

Fine-tuning data is nothing exotic. Each example is a short conversation in the exact shape you send at runtime, ending with the answer you wish the model had given:

# A handful shown here; a real dataset is hundreds to thousands of examples,
# covering the full range of inputs you expect at runtime.
training_examples = [
    {"messages": [
        {"role": "system", "content": "Classify the ticket's urgency as low, medium, or high."},
        {"role": "user", "content": "I was charged twice this week and need it fixed today."},
        {"role": "assistant", "content": "high"},
    ]},
    {"messages": [
        {"role": "system", "content": "Classify the ticket's urgency as low, medium, or high."},
        {"role": "user", "content": "Is there a dark mode setting anywhere?"},
        {"role": "assistant", "content": "low"},
    ]},
    # ...
]

You save these to a file, start a training job with the provider, and wait for it to finish. What comes back is a new model id, which you then call exactly like any other model:

# The fine-tuned model is another id; everything else is unchanged.
response = client.chat.completions.create(
    model="your-org/urgency-classifier-v1",
    messages=[
        {"role": "system", "content": "Classify the ticket's urgency as low, medium, or high."},
        {"role": "user", "content": "The export button does nothing when I click it."},
    ],
)
# response.choices[0].message.content -> "low"  (the label only, the behaviour you trained in)

The win is that the behaviour now lives in the model. The instructions can be shorter, the format holds across thousands of calls, and you did not have to grow the prompt to get there. The cost is everything around it: you need a quality dataset, a training run, and a fresh job each time you want to change what it learned, which is exactly why this sits at the bottom of the ladder. Upload examples, get a model id, call it like any other.

JunoIn practice Fine-tuning data is example conversations ending in the answer you want, a few hundred to a few thousand of them. You upload the file, run a training job, and get back a new model id you call like any other. The behaviour is then baked in, so prompts get shorter and the format stays consistent, at the cost of building the dataset and rerunning the job whenever it needs to change.

The workflow is upload, train, call, but the part that decides whether it works is the dataset, not the job. Quality and coverage matter more than raw count. Hundreds of clean, consistent examples that span the real range of inputs beat thousands of noisy or contradictory ones, because the model learns whatever regularity is actually in the data, including the mistakes.

Split your data before you train. Hold out a slice the model never sees during training, the validation set, and check its accuracy there rather than on the examples it trained on. A model can score near-perfectly on data it has already seen while doing poorly on anything new, so the held-out number is the one that tells you whether the behaviour generalises.

# Reserve a portion the training job never sees, to measure real performance.
split = int(len(training_examples) * 0.8)
train_set = training_examples[:split]        # the job trains on these
validation_set = training_examples[split:]   # you score the result on these
# The provider returns a new model id; you call it like any other model.
# Job API and split handling vary by provider; some take the validation file directly.

Two judgement calls sit on top of the mechanics. First, confirm prompting actually failed before spending a training run: a real instruction with a few examples, tested across varied inputs, not your first rough draft. Second, weigh the cost carefully. A fine-tune is a fixed training cost paid once against a long prompt's per-call cost paid forever, so prompt-shortening pays off only when call volume makes the saved tokens worth the job. And remember the combined pattern: a tuned model for form, RAG for the facts it should not try to memorise.

JunoIn practice The job is upload-train-call, but the dataset decides the outcome: a few hundred clean, varied examples beat thousands of noisy ones, since the model learns whatever is actually in the data. Hold out a validation slice and score there, not on the training examples, or a model that memorised will look better than it is. Confirm prompting really failed first, weigh fixed training cost against pay-forever prompt tokens, and pair the tuned model with RAG so it handles form while retrieval handles facts.

The pipeline is upload, train, evaluate, deploy, and the engineering lives in the dataset and the eval set, not the training call. Two split discipline carries it. A validation set, held out from training, tells you whether the behaviour generalised rather than got memorised. A separate held-out eval set, scored before and after, is what proves the fine-tune helped your target without regressing general ability, the catastrophic-forgetting check from the last section made concrete.

# Three disjoint slices: train on one, tune decisions on another, judge on a third.
train_set      = examples[:8000]        # the job trains on these
validation_set = examples[8000:9000]    # watch for overfitting during the run
eval_set       = examples[9000:]        # scored before AND after to prove no regression

# Score the base model and the fine-tune on the SAME eval_set, then compare.
base_score = run_eval(base_model_id, eval_set)
tuned_score = run_eval(fine_tuned_id, eval_set)
# Ship only if tuned_score beats base on the target AND holds on general-ability checks.
# Eval scoring and the training API vary by provider; the discipline does not.

Dataset quality is where overfitting and drift are won or lost. A narrow or quirky dataset trains a model that memorises your set's idiosyncrasies and fumbles inputs that differ, so coverage of the real input distribution matters more than count, and you re-measure as production traffic shifts away from what you trained on. Stale evals are how a quietly drifting model passes review and fails users.

The operational reality to design for is that every change to learned behaviour is a full re-run, not an edit. Re-curate, retrain, re-eval, redeploy a pinned new id, each time. That recurring cost, plus the regression risk on every run, is why you keep facts in RAG where they update live, reserve fine-tuning for behaviour that changes slowly, and treat each tuned id as a versioned artifact you can roll back to when a new run regresses.

JunoIn practice The pipeline is upload-train-eval-deploy, and the work is in the data splits, not the call: a validation set catches memorising, a held-out eval set scored before and after proves no regression. Dataset coverage of the real input distribution is how you beat overfitting and drift, so re-measure as traffic moves off your training data. And design for the recurring cost, every behaviour change is a full retrain-eval-redeploy, which is why facts belong in RAG and each tuned id is a versioned artifact you can roll back.

That is the last of the ways to shape what a model knows and how it answers, all of which leave it predicting text on a desk you arranged. The next chapters give it the ability to act: in Tool use the model calls your own functions, and then runs them in a loop as an agent.