LLM Fine-Tuning

Real-World Scenario

A support team wants a model to label incoming tickets with a category and an urgency flag. A large model with a dozen examples in the prompt does it well, but at 50,000 tickets a day the examples are most of the bill. Fine-tuning a smaller model to do the same job without the examples is the classic reason to fine-tune. Most of the work, and most of the failures, happen in the data before any training starts.

This lesson covers that data work and the decision, all in runnable Python. The training step itself runs in the LoRA and QLoRA lesson, which trains a LoRA adapter on a real model and measures the result. The JSONL produced here is the chat format that training code and hosted fine-tuning APIs expect.

When Fine-Tuning Is the Right Tool

The RAG vs fine-tuning vs prompting lesson runs that comparison on a real model. The rest of this lesson assumes the answer was “fine-tune”.

From Ticket Export to Training Examples

The raw data is an export of tickets with the label a human agent chose. Save the generator as tickets.py; it stands in for your helpdesk export and is seeded so everyone gets the same 60 rows.

import random

TEMPLATES = {
    "billing":  ["I was charged twice for {m}.", "My invoice for {m} is wrong.",
                 "Why did my bill go up in {m}?", "Please refund the {m} charge."],
    "outage":   ["The internet has been down since {t}.", "No connection at all since {t}.",
                 "Service keeps dropping every few minutes since {t}."],
    "account":  ["I can't log in to my account.", "Please change the email on my account.",
                 "How do I add my partner to the account?"],
    "hardware": ["The router light is solid red.", "The router is very hot and restarts.",
                 "One of the antennas snapped off."],
}
URGENT = {"outage"}

def export(n=60, seed=3):
    """A raw ticket export: text, the agent's label, and the customer who wrote it."""
    rng = random.Random(seed)
    rows = []
    for i in range(n):
        category = rng.choice(list(TEMPLATES))
        text = rng.choice(TEMPLATES[category]).format(m=rng.choice(["March", "April", "May"]),
                                                      t=rng.choice(["this morning", "yesterday"]))
        rows.append({"id": i, "customer": f"C{rng.randint(1, 25):02d}", "text": text,
                     "category": category, "urgent": category in URGENT})
    return rows

if __name__ == "__main__":
    rows = export()
    print(len(rows), "tickets;", rows[0])

$ python tickets.py
60 tickets; {'id': 0, 'customer': 'C12', 'text': 'Service keeps dropping every few minutes since this morning.', 'category': 'outage', 'urgent': True}

Each ticket becomes one chat-format example: the same system prompt you’ll use in production, the ticket as the user turn, and the exact JSON you want back as the assistant turn.

import json
from tickets import export

SYSTEM = "Classify the support ticket. Reply with JSON: category and urgent."

def to_example(row):
    label = {"category": row["category"], "urgent": row["urgent"]}
    return {"messages": [{"role": "system", "content": SYSTEM},
                         {"role": "user", "content": row["text"]},
                         {"role": "assistant", "content": json.dumps(label)}]}

if __name__ == "__main__":
    rows = export()
    with open("train.jsonl", "w") as f:
        for row in rows:
            f.write(json.dumps(to_example(row)) + "\n")
    print(open("train.jsonl").readline(), end="")
    print(sum(1 for _ in open("train.jsonl")), "lines written")

$ python to_jsonl.py
{"messages": [{"role": "system", "content": "Classify the support ticket. Reply with JSON: category and urgent."}, {"role": "user", "content": "Service keeps dropping every few minutes since this morning."}, {"role": "assistant", "content": "{\"category\": \"outage\", \"urgent\": true}"}]}
60 lines written

The assistant content is a string of JSON, not a nested object, because the model learns to produce text. It must be exactly the output you want in production, down to key order and spacing. The model will copy whatever formatting the examples use.

Validating Every Line

One malformed line can fail a training job, or worse, train in silently. This is what a hand-edited or merged export really looks like:

# The kinds of damage a hand-edited or merged export really picks up.
import json
from to_jsonl import SYSTEM

def example(*turns):
    return json.dumps({"messages": [{"role": r, "content": c} for r, c in turns]})

good = open("train.jsonl").read().splitlines()
lines = good[:2] + [
    good[2][:90],                                                    # cut off mid-line
    example(("system", SYSTEM), ("assistant", '{"category": "billing", "urgent": false}')),
    good[3],
    example(("system", SYSTEM), ("user", "Refund the April charge."), ("assistant", "Billing")),
    example(("system", SYSTEM), ("user", "My invoice is wrong."),
            ("assistant", '{"category": "Billing ", "urgent": "no"}')),
    example(("system", SYSTEM), ("user", " "), ("assistant", '{"category": "account", "urgent": false}')),
]
open("messy.jsonl", "w").write("\n".join(lines) + "\n")
print(len(lines), "lines in messy.jsonl")

$ python make_messy.py
8 lines in messy.jsonl

The validator checks structure, roles, empty turns and the label itself.

import json

CATEGORIES = {"billing", "outage", "account", "hardware"}

def problems(line):
    try:
        msgs = json.loads(line)["messages"]
    except (json.JSONDecodeError, KeyError) as e:
        return [f"not a valid example: {type(e).__name__}"]
    found = []
    if [m["role"] for m in msgs] != ["system", "user", "assistant"]:
        found.append(f"roles {[m['role'] for m in msgs]}")
    if any(not m["content"].strip() for m in msgs):
        found.append("empty message")
    try:
        label = json.loads(msgs[-1]["content"])
        if label.get("category") not in CATEGORIES or not isinstance(label.get("urgent"), bool):
            found.append(f"bad label {label}")
    except json.JSONDecodeError:
        found.append(f"assistant reply is not JSON: {msgs[-1]['content']!r}")
    return found

if __name__ == "__main__":
    for path in ["train.jsonl", "messy.jsonl"]:
        lines = open(path).read().splitlines()
        bad = {n: problems(line) for n, line in enumerate(lines, 1) if problems(line)}
        print(f"{path}: {len(lines)} lines, {len(bad)} bad")
        for n, found in bad.items():
            print(f"  line {n}: " + "; ".join(found))

$ python validate.py
train.jsonl: 60 lines, 0 bad
messy.jsonl: 8 lines, 5 bad
  line 3: not a valid example: JSONDecodeError
  line 4: roles ['system', 'assistant']
  line 6: assistant reply is not JSON: 'Billing'
  line 7: bad label {'category': 'Billing ', 'urgent': 'no'}
  line 8: empty message

The generated file is clean. The messy one has five bad lines out of eight, and each problem would have taught the model something wrong. An assistant reply of Billing teaches it to drop the JSON. A label of "Billing " with a trailing space and "urgent": "no" teaches it a fifth category and a string where a boolean belongs. The example with no user turn teaches it to answer without a question. Run a check like this before every upload, and make it strict about the label, because the label is the thing being learned.

Duplicates: Fewer Examples Than You Think, and Leakage

Helpdesk data repeats itself. Count what is actually distinct, then see what a random split does.

import random
from collections import Counter
from tickets import export

rows = export()
texts = Counter(r["text"] for r in rows)
print(f"{len(rows)} tickets, {len(texts)} distinct texts; most repeated:",
      texts.most_common(1)[0])

random.seed(0)                                     # the usual 80/20 random split
shuffled = random.sample(rows, len(rows))
cut = int(len(shuffled) * 0.8)
train, test = shuffled[:cut], shuffled[cut:]
seen = {r["text"] for r in train}
leaked = sum(r["text"] in seen for r in test)
print(f"random split: {leaked}/{len(test)} test tickets appear word for word in train")

unique = list({r["text"]: r for r in rows}.values())   # dedupe first, then split
random.seed(0)
shuffled = random.sample(unique, len(unique))
cut = int(len(shuffled) * 0.8)
train, test = shuffled[:cut], shuffled[cut:]
seen = {r["text"] for r in train}
print(f"dedupe then split: {sum(r['text'] in seen for r in test)}/{len(test)} leaked;"
      f" train {len(train)}, test {len(test)}")

$ python leakage.py
60 tickets, 20 distinct texts; most repeated: ('One of the antennas snapped off.', 9)
random split: 11/12 test tickets appear word for word in train
dedupe then split: 0/4 leaked; train 16, test 4

The 60-row export holds only 20 distinct texts; “One of the antennas snapped off.” alone appears 9 times. A random 80/20 split put 11 of the 12 test tickets into training word for word, so a model evaluated on that test set would be graded on examples it memorized, and the score would say nothing about new tickets. Deduplicating before splitting removes the exact copies, but leaves only 16 training and 4 test examples. That is the real size of this dataset. The export made it look three times bigger.

Exact deduplication isn’t the end of it: “charged twice for March” and “charged twice for April” are the same ticket for training purposes. Practice task 1 finds those.

Class Balance Changes After Cleaning

Check the label distribution after deduplication, not before.

from collections import Counter
from tickets import export

rows = export()
unique = list({r["text"]: r for r in rows}.values())
before, after = Counter(r["category"] for r in rows), Counter(r["category"] for r in unique)
print(f"{'category':9} {'raw':>9} {'deduped':>9}")
for cat in sorted(before):
    print(f"{cat:9} {before[cat]:3} ({before[cat] / len(rows):3.0%}) {after[cat]:3} ({after[cat] / len(unique):3.0%})")
print(f"{'total':9} {len(rows):3}        {len(unique):3}")

$ python balance.py
category        raw   deduped
account    17 (28%)   3 (15%)
billing    12 (20%)   8 (40%)
hardware   18 (30%)   3 (15%)
outage     13 (22%)   6 (30%)
total      60         20

In the raw export, hardware is the largest category at 30%. After deduplication it is 15%, and billing has doubled to 40%. The raw counts were inflated by a few tickets repeated many times. A model trained on the raw file would see “antennas snapped off” 9 times and learn that one sentence, not the hardware category. Balance, like size, only means something on distinct examples.

Does It Pay Back?

The point of this fine-tune is to stop paying for twelve examples in every prompt. Compare the two prompts and compute the break-even.

import json
from to_jsonl import SYSTEM

def tokens(text):
    return -(-len(text) // 4)                       # about 4 characters per token

examples = [json.loads(l)["messages"] for l in open("train.jsonl")][:12]
shots = "\n".join(f"Ticket: {m[1]['content']}\nAnswer: {m[2]['content']}" for m in examples)
few_shot_prompt = tokens(SYSTEM + shots) + 20       # + the new ticket
fine_tuned_prompt = tokens(SYSTEM) + 20

# Illustrative USD per million input tokens; check your provider's price list.
BASE, TUNED, TRAINING = 0.80, 1.60, 8.00
train_tokens = sum(tokens(l) for l in open("train.jsonl")) * 3          # 3 epochs
training_cost = train_tokens * TRAINING / 1e6
per_request_saving = (few_shot_prompt * BASE - fine_tuned_prompt * TUNED) / 1e6

if __name__ == "__main__":
    print(f"few-shot prompt:   {few_shot_prompt} tokens/request")
    print(f"fine-tuned prompt: {fine_tuned_prompt} tokens/request")
    print(f"training: {train_tokens:,} tokens -> ${training_cost:.2f}")
    print(f"saving per request: ${per_request_saving:.6f}")
    print(f"break-even after {training_cost / per_request_saving:,.0f} requests")

$ python cost.py
few-shot prompt:   318 tokens/request
fine-tuned prompt: 37 tokens/request
training: 12,129 tokens -> $0.10
saving per request: $0.000195
break-even after 497 requests

The few-shot prompt is 318 tokens against 37 for the fine-tuned model, even though the fine-tuned model is priced at twice the base rate here. Training on this small file costs ten cents, so it pays back after about 500 requests. At 50,000 tickets a day, the saving is about $10 a day on input tokens alone. The prices are placeholders with a realistic shape: fine-tuned models commonly cost more per token than their base models, and practice task 3 shows how much that ratio matters.

Two costs don’t appear in the arithmetic. The first is the time spent cleaning and labelling data, which usually dwarfs the training bill. The second is maintenance: a new ticket category means retraining, where a prompt change would have been one line.

Practice

1. Exact deduplication left “charged twice for March” and “charged twice for April” as separate examples. Normalize months and times to a placeholder, then check how many of leakage.py’s deduplicated test tickets still have a near-duplicate in training.

   Solution

   import random, re
   from tickets import export
   
   def template_key(text):
       return re.sub(r"\b(march|april|may|this morning|yesterday)\b", "<x>", text.lower())
   
   unique = list({r["text"]: r for r in export()}.values())
   if __name__ == "__main__":
       random.seed(0)
       shuffled = random.sample(unique, len(unique))
       cut = int(len(shuffled) * 0.8)
       train, test = shuffled[:cut], shuffled[cut:]
       seen = {template_key(r["text"]) for r in train}
       for r in test:
           print(f"{'LEAK ' if template_key(r['text']) in seen else 'clean'} {r['text']}")
       print(len({template_key(r["text"]) for r in unique}), "distinct templates in", len(unique), "tickets")

   $ python sol_near_dupes.py
   LEAK  I was charged twice for March.
   LEAK  Why did my bill go up in April?
   LEAK  The internet has been down since yesterday.
   clean I can't log in to my account.
   13 distinct templates in 20 tickets

   Three of the four “clean” test tickets still have a sibling in training that differs by one word. The 20 distinct texts come from only 13 templates. For evaluation purposes this test set is still mostly memorization, which is why the next task splits by template.

2. Split so that no template appears on both sides and every category is represented in the test set: group tickets by (category, template), and hold out one whole group per category.

   Solution

   import random
   from collections import defaultdict
   from sol_near_dupes import template_key, unique
   
   groups = defaultdict(list)                        # one group per template, per category
   for r in unique:
       groups[(r["category"], template_key(r["text"]))].append(r)
   
   random.seed(0)
   train, test = [], []
   for category in sorted({c for c, _ in groups}):
       keys = [k for k in groups if k[0] == category]
       random.shuffle(keys)
       test += groups[keys[0]]                       # one whole template per category held out
       for k in keys[1:]:
           train += groups[k]
   print(f"train {len(train)}, test {len(test)}")
   for r in test:
       print(f"  {r['category']:9} {r['text']}")
   print("template overlap:", {template_key(r["text"]) for r in test} & {template_key(r["text"]) for r in train})

   $ python sol_stratified.py
   train 14, test 6
     account   I can't log in to my account.
     billing   My invoice for April is wrong.
     billing   My invoice for March is wrong.
     hardware  One of the antennas snapped off.
     outage    Service keeps dropping every few minutes since this morning.
     outage    Service keeps dropping every few minutes since yesterday.
   template overlap: set()

   Every category now appears in the test set, and the template overlap is empty, so a correct test answer can’t come from memorizing a near-copy. The test set is still tiny, and with 13 templates in total no split can fix that. The real fix is more varied data: pull tickets from more months, or have agents write paraphrases, before training at all.

3. The break-even assumed the fine-tuned model costs twice the base price per token. Recompute it for fine-tuned prices of $1.60, $3.20, $6.40 and $8.00 per million tokens.

   Solution

   from cost import few_shot_prompt, fine_tuned_prompt, training_cost, BASE
   
   for tuned in [1.60, 3.20, 6.40, 8.00]:           # fine-tuned price per million input tokens
       saving = (few_shot_prompt * BASE - fine_tuned_prompt * tuned) / 1e6
       verdict = f"break-even after {training_cost / saving:,.0f} requests" if saving > 0 else "never pays back"
       print(f"fine-tuned at ${tuned:.2f}/M: {verdict}")

   $ python sol_price.py
   fine-tuned at $1.60/M: break-even after 497 requests
   fine-tuned at $3.20/M: break-even after 713 requests
   fine-tuned at $6.40/M: break-even after 5,513 requests
   fine-tuned at $8.00/M: never pays back

   The break-even stays in the hundreds of requests until the fine-tuned price nears the ratio of the two prompt lengths, then climbs steeply. At $6.40 it takes over 5,000 requests. At $8.00, 37 tokens at the fine-tuned price cost more than 318 tokens at the base price, so no volume pays back the training. Check the real price ratio for the model you would fine-tune before collecting any data.