## Why This Project Exists A support team at a Zomato-style company wants every chatbot response to match a specific brand voice: warm, concise, no corporate jargon, always acknowledging the customer's frustration before offering a solution. Prompting alone gets partway there, but the model drifts back toward generic phrasing over longer conversations, and RAG has nothing to contribute here, there is no missing fact to retrieve, the facts are already right, only the tone is wrong. This is the exact scenario fine-tuning exists for. This project fine-tunes a small open model with LoRA to reliably produce the target tone, tests it against the base model and against careful prompting, and produces a justified recommendation, not just a mechanically completed lab. ### What you are building * A tone-shifting dataset of before/after example responses * A LoRA fine-tune of a small open model on that dataset * A base-model comparison to prove the fine-tune actually changed behaviour * A catastrophic forgetting test on unrelated prompts * A three-way comparison: base model with prompting, RAG (to demonstrate why it does not apply here), and the fine-tuned model ### Architecture diagram Training Data: (generic_response, on_brand_response) pairs | v +------------------+ | Base Model | | (small open model)| +------------------+ | v +------------------+ | LoRA Fine-Tune | <-- only a small adapter is trained, | (PEFT) | base weights stay frozen +------------------+ | v +------------------+ +------------------+ +------------------+ | Base + Prompting | vs | RAG (no relevant | vs | Fine-Tuned Model | | | | facts to retrieve)| | | +------------------+ +------------------+ +------------------+ \ | / v v v +--------------------------------------+ | Tone Consistency + Forgetting Test | +--------------------------------------+ > 📌 **Engineering Decision:** LoRA/PEFT over full fine-tuning for almost every real case, and fine-tune only after confirming the problem is genuinely behaviour or tone, not missing facts, which would be a RAG problem instead. This project deliberately includes a RAG comparison specifically to demonstrate why RAG cannot fix a tone problem, reinforcing when each tool actually applies. ### Prerequisites This project assumes you have completed the LLM Fine-Tuning Lite module and are comfortable with the concept of LoRA adapters. You do not need a GPU-heavy setup, a small open model (1B to 3B parameters) fine-tunes reasonably on a single consumer GPU or a free-tier cloud notebook GPU. > 💡 **Tip:** Resist the urge to build a huge training dataset for this project. The Fine-Tuning Lite module deliberately keeps depth reduced, a focused 150 to 300 example dataset is enough to prove the concept and see a measurable tone shift, more data will not teach you more about the engineering decision this project is testing.
## Why Dataset Quality Matters More Than Dataset Size A fine-tuning dataset teaches the model a pattern by example, not by instruction. If your examples are inconsistent, some warm and concise, others still corporate and verbose, the model has no clean pattern to learn, and the fine-tune will produce an inconsistent result no matter how much data you feed it. ### Defining the target tone precisely Before writing a single example, write down the tone rules explicitly, this becomes your own quality checklist for every training example. ```text Target tone rules: 1. Acknowledge the customer's frustration or situation first, one sentence 2. State the resolution clearly, no hedging language ("we might be able to") 3. No corporate phrases: "per our policy", "we regret to inform you", "your satisfaction is our priority" 4. Maximum 3 sentences per response 5. End with a concrete next step, not an open-ended "let us know if..." ``` ### Writing paired examples ```json [ { "input": "My order from Punjab Grill arrived cold and the packaging was damaged.", "generic_response": "We regret to inform you of this inconvenience. Per our policy, we may be able to offer a partial refund pending review. Your satisfaction is our priority and we appreciate your patience during this process.", "target_response": "That's frustrating, food arriving cold defeats the whole point. I've issued a full refund to your original payment method, it'll reflect in 3 to 5 business days." }, { "input": "I've been waiting 45 minutes for my order and the app still says preparing.", "generic_response": "We sincerely apologize for any inconvenience this delay may have caused. Please be assured that we are looking into this matter and will update you accordingly.", "target_response": "45 minutes is way too long, I hear you. I've checked with the restaurant directly, it's out for delivery now and should reach you in the next 10 minutes." } ] ``` > **Note:** Each example includes both a `generic_response` and a `target_response`, not just the target alone. Having the contrast explicit makes it far easier to audit your own dataset for consistency, and some fine-tuning approaches can use the pair directly to sharpen the contrast the model learns. ### Generating enough examples efficiently Writing 200 examples by hand is slow. Use an LLM to generate draft pairs against your tone rules, then review and correct every single one by hand, do not skip the review step. ```python GENERATION_PROMPT = """Generate 10 customer support scenarios for a food delivery app, each with a generic corporate-sounding response and a target response following these rules: {tone_rules} Vary the complaint types: late delivery, wrong item, cold food, missing items, payment issues.""" ## Call an LLM with this prompt repeatedly to draft candidates, ## then a human reviews and edits every single one before it enters ## the training set - draft generation is not the same as final data ``` > 🔴 **Common Mistake:** Prioritising dataset size over quality. Two hundred inconsistent examples, some following the tone rules loosely and others not at all, teach the model a blurry, unreliable pattern. A hundred and fifty tightly consistent examples, each hand-reviewed against the explicit tone rules, teach a sharp one. Quality review time is not optional here. ### Guided practice Generate and hand-review 150 to 300 training pairs covering at least 5 distinct complaint categories. Before moving to Milestone 2, randomly sample 15 of your own examples and check each one against the 5 tone rules from earlier, if any fail, fix them now, a flawed example caught after training just means retraining.
## Why LoRA Instead of Full Fine-Tuning Full fine-tuning updates every weight in the model, which is expensive, slow, and carries a much higher risk of catastrophic forgetting since nothing constrains how far the weights can drift from their original values. LoRA (Low-Rank Adaptation) freezes the base model entirely and trains a small set of additional low-rank matrices that get added to specific layers, dramatically fewer parameters to train, much faster, and far less likely to damage the model's general capability. ### Setting up the fine-tuning environment ```bash ## Install PEFT (Parameter-Efficient Fine-Tuning) and dependencies pip install peft transformers datasets accelerate bitsandbytes ``` ### Preparing the dataset for training ```python from datasets import Dataset import json with open("tone_training_data.json") as f: raw_data = json.load(f) ## Format each example as a single training string the model learns to complete def format_example(example): return { "text": f"### Customer: {example['input']}\n### Support: {example['target_response']}" } formatted = [format_example(ex) for ex in raw_data] train_dataset = Dataset.from_list(formatted) ``` > **Note:** The `### Customer:` and `### Support:` markers are a simple prompt template the model learns to associate with "read a complaint, respond in the target tone." Consistency in this template between training and later inference matters, using a different format at inference time than what the model was trained on will noticeably hurt results. ### Configuring the LoRA adapter ```python from peft import LoraConfig, get_peft_model, TaskType from transformers import AutoModelForCausalLM, AutoTokenizer model_name = "meta-llama/Llama-3.2-1B" # small open model, fits on modest hardware tokenizer = AutoTokenizer.from_pretrained(model_name) base_model = AutoModelForCausalLM.from_pretrained(model_name) lora_config = LoraConfig( task_type=TaskType.CAUSAL_LM, r=8, # rank of the adaptation matrices, smaller = fewer trainable params lora_alpha=16, # scaling factor for the LoRA updates lora_dropout=0.05, target_modules=["q_proj", "v_proj"] # apply LoRA to attention projection layers ) model = get_peft_model(base_model, lora_config) model.print_trainable_parameters() ``` ```text trainable params: 1,703,936 || all params: 1,235,814,400 || trainable%: 0.14 ``` > 📌 **Engineering Decision:** `r=8` is a deliberately modest rank for a small, focused tone-shifting task. A higher rank trains more parameters and can capture more complex behaviour changes, but also increases both training cost and the risk of overfitting on a dataset this size. Start low, only increase rank if evaluation in Milestone 3 shows the adapter genuinely underfit the pattern. ### Running the training loop ```python from transformers import TrainingArguments, Trainer training_args = TrainingArguments( output_dir="./tone-lora-adapter", num_train_epochs=3, per_device_train_batch_size=4, learning_rate=2e-4, logging_steps=10, save_strategy="epoch" ) trainer = Trainer( model=model, args=training_args, train_dataset=tokenized_train_dataset # tokenized version of train_dataset ) trainer.train() model.save_pretrained("./tone-lora-adapter") ``` > 🔴 **Common Mistake:** Skipping the base-model comparison and assuming the fine-tune worked because training loss went down. A decreasing loss curve proves the model is fitting the training data, it does not prove the resulting behaviour actually generalizes to new complaints it did not see during training, that is exactly what Milestone 3 tests. ### Troubleshooting scenario After training, the fine-tuned model's responses on new test complaints still sound generic and corporate, unchanged from the base model. Before assuming the LoRA approach failed, check three things in order: did `get_peft_model` actually wrap the base model correctly (`print_trainable_parameters` should show a small nonzero trainable percentage), did the training loss actually decrease over the 3 epochs, and is the inference code loading the saved adapter weights at all, or accidentally running the base model alone.
## Proving the Fine-Tune Actually Changed Behaviour A fine-tune that only reproduces its exact training examples well but fails on new, unseen complaints has not learned the tone pattern, it has memorized the dataset. The real test is performance on complaints the model never saw during training. ### Building a held-out test set ```python ## These 20 complaints were NOT included in tone_training_data.json held_out_complaints = [ "My biryani order was missing the raita I paid extra for.", "The delivery partner left my order at the wrong building.", "I was charged twice for the same order.", # ... 17 more, covering complaint types both seen and unseen in training ] ``` > **Note:** Holding out test examples entirely from training is the same discipline as the train/test split from earlier in this roadmap, applied here to fine-tuning instead of classical ML. Evaluating only on training examples would be the fine-tuning equivalent of the data leakage mistake covered in Module 2, an inflated, meaningless success signal. ### Running both models on the same test set ```python def generate_response(model, tokenizer, complaint): prompt = f"### Customer: {complaint}\n### Support:" inputs = tokenizer(prompt, return_tensors="pt") outputs = model.generate(**inputs, max_new_tokens=80, temperature=0.7) return tokenizer.decode(outputs[0], skip_special_tokens=True) results = [] for complaint in held_out_complaints: base_response = generate_response(base_model, tokenizer, complaint) finetuned_response = generate_response(model, tokenizer, complaint) # LoRA model results.append({ "complaint": complaint, "base_response": base_response, "finetuned_response": finetuned_response }) ``` ### Scoring tone adherence with a rubric ```python TONE_JUDGE_PROMPT = """Score this customer support response against these tone rules on a scale of 0 to 5: 1. Acknowledges frustration first 2. States resolution clearly, no hedging 3. No corporate phrases (per our policy, we regret to inform you, etc) 4. 3 sentences or fewer 5. Ends with a concrete next step Response: {response} Give a score 0-5 and briefly explain what was missing, if anything.""" def score_tone(response): prompt = TONE_JUDGE_PROMPT.format(response=response) result = judge_client.messages.create( model="claude-sonnet-4-6", max_tokens=150, messages=[{"role": "user", "content": prompt}] ) return result.content[0].text ``` > 💡 **Tip:** Score all 20 base-model responses and all 20 fine-tuned responses with the same rubric and the same judge prompt, then compute the average score for each. This single number, average tone score before versus after, is the clearest evidence for whether the fine-tune worked. ### Concept check Before running the scoring, predict which of the 20 held-out complaints you expect the fine-tuned model to handle best, and which you expect it to still struggle with. Complaints closest in structure to your training examples should score higher, ones covering an edge case your training data under-represented should reveal the fine-tune's actual limits, not just its average performance.
## Why a Narrow Fine-Tune Can Break Unrelated Capability Catastrophic forgetting happens when fine-tuning on a narrow task degrades the model's performance on tasks it was never meant to change. A model fine-tuned exclusively on short, warm customer-support responses might start giving oddly terse or overly casual answers even to completely unrelated questions, like a general knowledge query, because its overall response style drifted, not just the specific support-tone behaviour you targeted. ### Building an unrelated-task test set ```python unrelated_prompts = [ "Explain how photosynthesis works in simple terms.", "What is the capital of Karnataka?", "Write a short paragraph about the history of chess.", "What are the main differences between TCP and UDP?", # ... prompts with no connection to customer support tone at all ] ``` > **Note:** These prompts are deliberately chosen to have nothing to do with the fine-tuning task. If the fine-tuned model's answers to these are noticeably worse, shorter, less coherent, or oddly styled compared to the base model's answers, that is catastrophic forgetting, the fine-tune has degraded general capability while chasing the narrow tone target. ### Running the comparison ```python forgetting_results = [] for prompt in unrelated_prompts: base_answer = generate_response(base_model, tokenizer, prompt) finetuned_answer = generate_response(model, tokenizer, prompt) forgetting_results.append({ "prompt": prompt, "base_answer": base_answer, "finetuned_answer": finetuned_answer }) ``` ### Scoring for degradation ```python FORGETTING_JUDGE_PROMPT = """Compare these two answers to the same general knowledge question. Has the second answer's quality, coherence, or relevance degraded compared to the first? Answer YES or NO and explain briefly. Question: {prompt} Answer A (base model): {base_answer} Answer B (fine-tuned model): {finetuned_answer}""" ``` > 🔴 **Common Mistake:** Narrow dataset causing catastrophic forgetting, then not testing for it because the fine-tune "worked" on the target task. A model that nails customer-support tone but has visibly degraded at explaining photosynthesis has a real problem, one that only shows up if you deliberately test for it, it will not appear in your tone-focused evaluation from Milestone 3 at all. ### Interpreting the results If forgetting is detected, the likely causes are training for too many epochs on too narrow a dataset, or a LoRA rank set too high for the dataset size, both push the adapter to overfit specifically to the training distribution's style at the expense of general capability. The fix is typically fewer epochs, a lower rank, or a somewhat more varied training set, not abandoning LoRA for full fine-tuning, which would carry an even higher forgetting risk for the same underlying cause. > 📌 **Engineering Decision:** If forgetting is detected, reduce epochs or LoRA rank and retrain before considering the adapter production-ready. A tone fine-tune that quietly damages the model's ability to handle any question outside customer support is not a net improvement, even if the target metric looks good in isolation. ### Guided practice Run the forgetting test against your own fine-tuned adapter from Milestone 2. If you detect degradation, retrain with `num_train_epochs=1` instead of 3, rerun both the tone evaluation and the forgetting test, and compare all four resulting scores (tone before/after, forgetting before/after) to see the tradeoff directly.
## Why RAG Genuinely Does Not Apply Here This milestone exists to make the RAG-versus-fine-tuning decision concrete rather than theoretical. RAG solves a missing-knowledge problem: the model does not have access to a fact it needs at generation time. The tone-shifting task in this project has no missing fact, the model already knows what a refund is and what a late delivery is, the only thing wrong is how it phrases the response. Retrieving more context about refund policy would not fix a response that still opens with "We regret to inform you." ### Demonstrating this directly ```python ## Attempt a RAG-based fix: retrieve "tone guideline" documents and ## include them in the prompt, to show concretely why this doesn't work TONE_GUIDELINE_DOC = """Support Tone Guidelines: Acknowledge frustration first. Keep responses under 3 sentences. Avoid corporate phrases. Always end with a concrete next step.""" rag_style_prompt = f"""Tone guidelines: {TONE_GUIDELINE_DOC} Customer: {complaint} Support:""" rag_style_response = generate_response(base_model, tokenizer, rag_style_prompt) ## Compare this against the actual fine-tuned model's response on the ## same complaint - the RAG-style prompt injection typically produces ## an inconsistent result that drifts back to generic phrasing over ## a longer conversation, while the fine-tuned model does not drift ``` > **Note:** This is functionally few-shot prompting with a tone guideline document included, dressed up as "retrieval." It is worth running this comparison explicitly, because it demonstrates the real distinction: stuffing instructions into context can nudge behaviour temporarily, but it does not durably change how the model generates by default the way a fine-tune does, especially as conversations get longer and the guideline text falls out of the effective attention window. ### The three-way comparison table Build this table from your actual Milestones 3 and this milestone's RAG-style test, using your own measured numbers, not placeholders. | Approach | Avg tone score (0-5) | Consistency across 20 test cases | Fixes a knowledge gap? | |:---|:---|:---|:---| | Base model, no help | Fill in from Milestone 3 | Fill in | No knowledge gap exists | | Base model + tone guideline in prompt | Fill in from this milestone | Fill in | No knowledge gap exists | | LoRA fine-tuned model | Fill in from Milestone 3 | Fill in | No knowledge gap exists | > 📌 **Engineering Decision:** Fine-tune only after confirming the problem is behaviour or tone, not missing facts. This project's three-way comparison exists to make that confirmation concrete: since no version of this task ever involved a missing fact, RAG was never a candidate solution here, and the comparison table should make that gap between prompting and fine-tuning's consistency visible in real numbers, not just assert it. ### Writing the final justification Your written justification should state, using your actual measured scores: how much more consistent the fine-tuned model was than prompting alone, whether any catastrophic forgetting was detected and how it was addressed if so, and an explicit statement of why RAG was never a viable option for this specific task, tied to the definition of what RAG actually solves. > 🔴 **Common Mistake:** Fine-tuning to fix a knowledge gap RAG would have solved faster. This project's task deliberately has no knowledge gap, but it is worth stating the inverse clearly in your write-up: if the underlying problem here had instead been "the bot gives outdated refund policy numbers," fine-tuning would have been the wrong tool entirely, that is a RAG problem, and no amount of retraining fixes a policy document that changed after training data was collected. ### Capstone lab checkpoint Complete the three-way comparison table with your real measured data, then write a one-page justification following the structure above. State explicitly what would have to change about this task for RAG or prompting alone to become the better choice instead of fine-tuning, this is what proves you understand the decision, not just the LoRA mechanics.
Why This Project Exists A support team at a Zomato-style company wants every chatbot response to match a specific brand ...
Why Dataset Quality Matters More Than Dataset Size A fine-tuning dataset teaches the model a pattern by example, not by ...
Why LoRA Instead of Full Fine-Tuning Full fine-tuning updates every weight in the model, which is expensive, slow, and c...
Proving the Fine-Tune Actually Changed Behaviour A fine-tune that only reproduces its exact training examples well but f...
Why a Narrow Fine-Tune Can Break Unrelated Capability Catastrophic forgetting happens when fine-tuning on a narrow task ...
Why RAG Genuinely Does Not Apply Here This milestone exists to make the RAG-versus-fine-tuning decision concrete rather ...
Final Verification 1. Dataset quality check Sample 15 training examples at random and confirm each one follows all 5 ton...
Aligns directly with DevOps, Site Reliability (SRE), and Platform Engineering job descriptions.