How to Fine-Tune an LLM: Methods, Frameworks & Step-by-Step Code [2026]

Fine-tuning an LLM means taking a pre-trained model and training it on your specific data so it performs your task better than any prompt could achieve. The barrier to entry has collapsed: QLoRA + Unsloth now let you fine-tune an 8B-parameter model on a 12 GB consumer GPU for under $1 in cloud costs.

This guide covers the complete journey, when to fine-tune (vs. RAG or prompt engineering), which method and framework to pick, how to prepare your dataset, a copy-paste Llama 3 walkthrough, real cost scenarios, and deployment.

This page owns the how-to workflow, from dataset preparation through deployment. If you already know the process and need to select infrastructure or software, compare the ten LLM fine-tuning tools we tested.

Fine-Tuning at a Glance

Before you commit to anything, here's the quick picture:

Now let's figure out if fine-tuning is actually the right move for your project.

When Should You Fine-Tune an LLM? (vs. RAG vs. Prompt Engineering)

This is the question most developers skip, and it costs them weeks of wasted effort. Fine-tuning is powerful, but it's not always the right tool. Here's a framework to decide.

The decision boils down to what you're trying to change. Here are real scenarios:

If you're choosing the right AI stack for your SaaS, this decision framework is step one. Many teams build complex RAG pipelines when a 500-example fine-tune would give them more consistent results at lower latency.

Verdict: Fine-tune when you need the model to consistently behave differently, not just know different things. If you only need new knowledge, RAG is cheaper and easier to maintain. If you need both, go hybrid.

How Does LLM Fine-Tuning Work? Full vs. LoRA vs. QLoRA

There are three main approaches, and they differ dramatically in hardware requirements, cost, and quality. Understanding the tradeoffs saves you from either over-investing or under-delivering.

Full Fine-Tuning (When Budget Is No Object)

Full fine-tuning updates every parameter in the model. It produces the best possible results but requires enormous resources, roughly 100+ GB of VRAM for a 7B model (you need to store the model, optimizer states, and gradients simultaneously). This is H100-cluster territory. Unless you're at a well-funded lab, skip this.

LoRA: The PEFT Revolution

LoRA (Low-Rank Adaptation) freezes the base model and adds small trainable matrices called adapters. Instead of updating a massive weight matrix W directly, LoRA decomposes the update into two small matrices A and B, where rank r is much smaller than the model dimension. The result: you train roughly 1-2% of the original parameters while retaining 98-99% of full fine-tuning quality.

The Hugging Face peft library is the standard implementation. LoRA adapters are typically 50-200 MB, tiny compared to the full model.

QLoRA: Fine-Tuning for Everyone

QLoRA takes LoRA one step further. It loads the base model in 4-bit precision using a special data type called NormalFloat4 (NF4), then applies LoRA adapters on top. The 4-bit quantization cuts VRAM usage by another ~25% compared to standard LoRA while preserving nearly identical quality.

This is what makes fine-tuning accessible. A 7B model that needs 100+ GB for full fine-tuning fits in 12 GB with QLoRA.

Verdict: For 90% of developers, QLoRA is the right choice. The quality difference from full fine-tuning is negligible for most tasks, and the hardware savings are massive. Start there and only scale up if your evaluation metrics demand it.

Which Fine-Tuning Framework Should You Use in 2026?

Picking a framework matters more than most people realize. The right one saves hours of setup and significantly speeds up training. Here's how the four major options compare.

Here's the quick recommendation:

• First fine-tune? Use Unsloth. Fastest training, easiest setup, free Colab notebooks to get started immediately.
• Need a web UI with zero code? Use LLaMA-Factory. Its LLaMA-Board GUI lets you configure and launch training from a browser.
• Doing alignment (RLHF, DPO, GRPO)? Use TRL. It's the Hugging Face standard for preference-based training, and v0.15.0 (March 2026) added native GRPO support.
• Running production pipelines on multi-GPU? Use Axolotl. YAML-based configs make experiments reproducible and auditable.

One useful trick: Unsloth and LLaMA-Factory can be combined. LLaMA-Factory supports Unsloth as a training backend, giving you the GUI convenience with Unsloth's speed optimizations. Teams building AI agents that use fine-tuned models often start with Unsloth for rapid iteration, then move to Axolotl for production reproducibility.

How Do You Prepare a Fine-Tuning Dataset?

Data quality is the single biggest factor in fine-tuning success. A well-curated 500-example dataset will outperform a noisy 10,000-example one almost every time.

Dataset Formats

The two dominant formats are chat (OpenAI-compatible) and instruction (Alpaca-style). Here's what each looks like in JSONL format:

Chat format (recommended for most use cases):

{"messages": [{"role": "system", "content": "You are a customer support agent for Acme Inc. Be helpful, concise, and always reference the user's order number."}, {"role": "user", "content": "I ordered a laptop 3 days ago (order #4821) and it still says processing."}, {"role": "assistant", "content": "I can see order #4821 is currently in our warehouse being prepared for shipment. Laptops typically ship within 3-5 business days. You should receive a tracking number by tomorrow. Would you like me to flag this for priority processing?"}]}
{"messages": [{"role": "system", "content": "You are a customer support agent for Acme Inc. Be helpful, concise, and always reference the user's order number."}, {"role": "user", "content": "Can I return my keyboard? Order #3019, bought it last week."}, {"role": "assistant", "content": "Absolutely! Order #3019 is within our 30-day return window. I'll email you a prepaid return label right now. Once we receive the keyboard, your refund will process within 2-3 business days."}]}
{"messages": [{"role": "system", "content": "You are a customer support agent for Acme Inc. Be helpful, concise, and always reference the user's order number."}, {"role": "user", "content": "My monitor arrived cracked. Order #5573."}, {"role": "assistant", "content": "I'm sorry about that! For order #5573, I'm initiating an immediate replacement shipment -- no need to return the damaged unit. The new monitor will ship today with express delivery. You'll get a tracking email within the hour."}]}

Instruction format (Alpaca-style):

{"instruction": "Summarize the following customer complaint in one sentence.", "input": "I've been waiting 2 weeks for my refund and nobody responds to my emails. This is the worst customer service I've ever experienced.", "output": "Customer is frustrated by a 2-week delay in receiving their refund and lack of email response from support."}
{"instruction": "Classify the sentiment of this review.", "input": "The product works fine but shipping took forever.", "output": "Mixed (positive product, negative shipping)"}

Dataset Size Guidelines

How much data do you actually need? It depends on task complexity:

1. 50-100 examples, proof of concept, enough to test if fine-tuning helps
2. 500-1,000 examples, useful for most single tasks (classification, extraction, formatting)
3. 5,000-10,000 examples, production-quality results for complex tasks
4. 10,000+ examples, diminishing returns unless your task has high variability

Data Quality Checklist

Before training, validate your dataset against these criteria:

• Consistent formatting across all examples (same system prompt, same output structure)
• Diverse examples covering edge cases and failure modes
• No contradictions (don't teach the model to say both "yes" and "no" to the same input pattern)
• Balanced distribution of output types (if doing classification, don't have 90% of examples in one class)
• Remove duplicate or near-duplicate entries

Pro tip: Use GPT-4 or Claude to generate initial synthetic training data, then refine with human review. 500 high-quality synthetic examples often outperform 5,000 noisy real examples. Meta's fine-tuning guide recommends this approach for bootstrapping datasets.

Step-by-Step: Fine-Tune Llama 3 8B with QLoRA and Unsloth

Here's the complete walkthrough. Every code block is copy-paste ready, you can run this in a free Google Colab notebook or on any machine with 12+ GB VRAM.

Step 1: Install Unsloth

pip install unsloth

That's it. Unsloth handles all the dependencies (transformers, peft, trl, bitsandbytes) automatically.

Step 2: Load the Base Model in 4-bit

from unsloth import FastLanguageModel

# Load Llama 3.1 8B in 4-bit quantization
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Meta-Llama-3.1-8B-bnb-4bit",
    max_seq_length=2048,
    load_in_4bit=True,
)

This downloads the 4-bit quantized model (~4 GB) and loads it into GPU memory. On an RTX 3060 (12 GB), you'll have plenty of headroom for training.

Step 3: Configure LoRA Adapters

# Add LoRA adapters to the model
model = FastLanguageModel.get_peft_model(
    model,
    r=16,                # LoRA rank -- 16 is the sweet spot for most tasks
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                     "gate_proj", "up_proj", "down_proj"],
    lora_alpha=16,       # Scaling factor (usually equal to r)
    lora_dropout=0,      # Unsloth optimizes for 0 dropout
    bias="none",
)

With r=16, you're training roughly 40 million parameters out of 8 billion, less than 0.5% of the model. That's the magic of LoRA.

Step 4: Load Your Dataset

from datasets import load_dataset

# Load your JSONL dataset from Hugging Face Hub or local file
dataset = load_dataset("json", data_files="train.jsonl", split="train")

# Format into chat template
def format_chat(example):
    text = tokenizer.apply_chat_template(
        example["messages"],
        tokenize=False,
        add_generation_prompt=False,
    )
    return {"text": text}

dataset = dataset.map(format_chat)

This takes the JSONL chat format from the previous section and applies Llama 3's chat template. The tokenizer handles all the special tokens (<|begin_of_text|>, <|eot_id|>, etc.).

Step 5: Configure and Run Training

from trl import SFTTrainer
from transformers import TrainingArguments
from unsloth import is_bfloat16_supported

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=2048,
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,    # Effective batch size = 8
        warmup_steps=5,
        max_steps=60,                     # Adjust based on dataset size
        learning_rate=2e-4,               # Standard for QLoRA
        fp16=not is_bfloat16_supported(),
        bf16=is_bfloat16_supported(),
        logging_steps=1,
        output_dir="outputs",
        seed=42,
    ),
)

# Start training
trainer.train()

Key hyperparameters to understand:

• Learning rate (2e-4): The standard for QLoRA. Go lower (2e-5) if you see the model forgetting general capabilities.
• Batch size (2) x gradient accumulation (4): Effective batch size of 8. Increase gradient_accumulation if your GPU runs out of memory.
• max_steps (60): For 500 examples, this is roughly 1 epoch. Start with 1-3 epochs and watch validation loss.
• Rank r (16): Lower (4-8) for simple tasks, higher (32-64) for complex tasks. 16 is a safe default.

Step 6: Save and Test

# Save the LoRA adapters (small -- ~50-200 MB)
model.save_pretrained("my-fine-tuned-model")
tokenizer.save_pretrained("my-fine-tuned-model")

# Quick inference test
FastLanguageModel.for_inference(model)
inputs = tokenizer(
    [tokenizer.apply_chat_template(
        [{"role": "user", "content": "I need to return order #7742"}],
        tokenize=False,
        add_generation_prompt=True,
    )],
    return_tensors="pt",
).to("cuda")

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

That's the entire pipeline. On an RTX 4090 with Unsloth, training 500 examples takes roughly 15-30 minutes. On a free Colab T4, expect 1-2 hours.

What About API-Based Fine-Tuning? (OpenAI, Google, Mistral)

Not everyone wants to manage GPUs. API providers let you fine-tune through a simple upload-and-train workflow. Here's how they compare to running it yourself.

When API fine-tuning makes sense: You need to iterate fast, your dataset is small, you don't want to manage infrastructure, or you specifically need a closed model like GPT-4o.

When local fine-tuning wins: Data privacy matters (healthcare, finance, legal), you're training frequently, you want to own and export the weights, or you're optimizing for cost at scale.

Verdict: API fine-tuning is the fastest path to a proof of concept. Local fine-tuning is the cheapest path to production. Most teams prototype on an API, then move to local Unsloth once they've validated the approach.

How Much Does It Cost to Fine-Tune an LLM?

The "fine-tuning is expensive" narrative is stuck in 2023. Here's what it actually costs today.

The cost curve flattens fast. A startup fine-tuning an 8B model on RunPod spends less on training than on a single cup of coffee. Even the 70B production scenario is under $30 -- that's one month of a junior developer's daily lunch budget.

Cloud GPU providers worth comparing: RunPod (best spot pricing), Lambda (reliable on-demand H100s), Vast.ai (cheapest but variable quality), and Modal (serverless, pay-per-second).

The chart above shows why QLoRA changed the game. A 7B model that required a multi-GPU cluster for full fine-tuning now fits on a laptop GPU. The 70B model drops from "cloud only" to a single A100.

Verdict: You can fine-tune a production-quality 8B model for under $1. The cost barrier to fine-tuning is gone. The real cost is dataset preparation time.

How Do You Evaluate a Fine-Tuned Model?

Training is only half the job. Without proper evaluation, you can't tell if your fine-tuned model actually improved, or if it just memorized your training data.

Automated Metrics

Track these during and after training:

• Training loss / perplexity: Should decrease steadily, then plateau. If it drops to near zero, you're overfitting.
• Task-specific metrics: Accuracy (classification), BLEU/ROUGE (summarization), exact match (extraction), F1 (multi-label). Pick the metric that matches your task.

Human Evaluation

Numbers don't capture everything. For generative tasks:

• A/B testing: Show base model vs. fine-tuned output side-by-side. Have 3-5 evaluators pick the better response across 50+ examples. Track win rate.
• Likert scale rating: Rate outputs on relevance (1-5), accuracy (1-5), and tone (1-5). Calculate average improvement over the base model.

Catastrophic Forgetting Check

This is the one most developers skip. After fine-tuning, run your model on a general benchmark like MMLU or HellaSwag. If scores drop more than 2-3 points, your model has lost too much general knowledge. The fix: lower your learning rate, reduce epochs, or switch to LoRA (which freezes the base weights).

Practical rule: Always hold out 10-20% of your dataset as a test set. Never evaluate on training data, that tells you nothing about real-world performance.

How Do You Deploy a Fine-Tuned Model?

Training is done. Now you need to serve it. Most guides skip this part entirely.

Step 1: Merge LoRA Adapters

If you used LoRA or QLoRA, merge the adapters back into the base model for inference:

# Merge adapters into base model
model.merge_and_unload()
model.save_pretrained("merged-model")
tokenizer.save_pretrained("merged-model")

Step 2: Choose Your Deployment Path

Local development and testing, Ollama:

# Convert to GGUF format (Ollama's native format)
python llama.cpp/convert_hf_to_gguf.py merged-model --outfile model.gguf --outtype q4_k_m

# Create an Ollama model
ollama create my-fine-tuned-model -f Modelfile
ollama run my-fine-tuned-model

Production serving, vLLM:

# Start an OpenAI-compatible API server
python -m vllm.entrypoints.openai.api_server \
    --model merged-model \
    --host 0.0.0.0 \
    --port 8000

Serverless (zero infrastructure): Upload your model to Together AI, Fireworks, or Modal. You get an API endpoint without managing servers. Cost scales with usage.

Advanced: Multi-Adapter Serving

Here's a pattern more teams should use: keep one base model loaded in memory and swap LoRA adapters per request. You could serve a customer-support adapter, a code-review adapter, and a summarization adapter, all from a single GPU. vLLM supports this natively with the --enable-lora flag.

Ready to implement? See our Best LLM Fine-Tuning Tools & Platforms [coming soon] for a deeper comparison of deployment options.

What Are the Most Common Fine-Tuning Mistakes?

After helping teams debug dozens of fine-tuning runs, these are the mistakes that come up over and over.

1. Overfitting on small datasets. You train for 10 epochs on 200 examples, training loss hits near zero, and the model parrots your training data verbatim. Fix: 1-3 epochs max, use a validation set, and watch for the gap between training loss and eval loss.

2. Catastrophic forgetting. The model nails your specific task but can't hold a basic conversation anymore. Fix: use LoRA/QLoRA (freezes base weights), keep learning rates low (2e-5 for full fine-tuning, 2e-4 for QLoRA), and evaluate on general benchmarks before deploying.

3. Garbage data quality. Inconsistent formatting, contradictions between examples, or duplicates. The model learns the noise. Fix: clean your data before training. Always. Spend more time on data curation than on hyperparameter tuning.

4. Starting with a model that's too large. Teams jump to 70B because "bigger is better," then can't afford the GPU costs. Fix: start with 8B. If 8B with good data can't solve your task, 70B with the same data probably won't either. Scale up the data quality first, then the model size.

5. No evaluation pipeline. Training without a held-out test set, then deploying based on vibes. Fix: split your data 80/10/10 (train/val/test) before you start. Compare against the base model on every test example.

6. Learning rate too high. Destroys the pre-trained knowledge in the first few steps. The model outputs gibberish. Fix: start at 2e-4 for QLoRA, 2e-5 for full fine-tuning. If outputs degrade, go lower.

How Techsy Approaches LLM Fine-Tuning

At Techsy, we follow a strict escalation path for every AI project: prompt engineering first, RAG second, fine-tuning only when the data proves it's needed. Most client projects don't actually require fine-tuning, well-crafted prompts or a RAG pipeline solve the problem at lower cost and complexity.

When fine-tuning is the right call, here's our process:

1. Dataset audit, We review the client's data for quality, coverage, and formatting. If we don't have enough examples, we help build a synthetic dataset using GPT-4 or Claude with human review.
2. Framework selection, Unsloth + QLoRA for 90% of startup projects. Axolotl for clients who need reproducible, multi-GPU production pipelines.
3. Training and evaluation, We always train with a held-out test set and benchmark against the base model. If the fine-tuned model doesn't measurably improve the target metric, we don't deploy it.
4. Deployment, vLLM for production serving, multi-adapter patterns when clients need multiple specialized models from a single GPU.

We've shipped fine-tuned models for startups that couldn't afford enterprise GPU budgets, QLoRA on RunPod keeps costs under $30 even for 70B models.

Need help fine-tuning an LLM for your use case? We help teams go from raw data to deployed model. Get a free consultation

Frequently Asked Questions About LLM Fine-Tuning

What is LLM fine-tuning?

LLM fine-tuning is the process of training a pre-trained language model on your own task-specific data so it performs that task better. You're essentially teaching the model new behaviors, formats, or domain expertise that generic prompting can't achieve reliably.

When should I fine-tune vs. use RAG?

Fine-tune when you need the model to behave differently, consistent output format, domain-specific language, particular tone. Use RAG when the model needs to know different things, especially if that knowledge changes frequently. For many production systems, a hybrid approach works best.

How much does it cost to fine-tune an LLM?

Anywhere from $0 to $540 depending on scale. Most individual developers spend under $1 using QLoRA on a RunPod RTX 4090 ($0.34/hr). A 70B production model on an A100 costs $17-27. Full fine-tuning on H100 clusters runs $270-540. API fine-tuning (OpenAI) costs $3-25 for 1,000 examples.

Can I fine-tune an LLM on my laptop?

Yes, if your laptop has a GPU with 12+ GB VRAM. An RTX 3060 laptop GPU handles 8B models with QLoRA. Apple Silicon Macs with 16+ GB unified memory can also fine-tune via MLX, though it's slower than CUDA. For larger models, you'll need cloud GPUs.

What is the difference between LoRA and QLoRA?

Both add small trainable adapter layers while freezing the base model. The difference: QLoRA also quantizes the base model to 4-bit precision (NF4 data type), reducing VRAM usage by ~25% compared to standard LoRA. Quality is nearly identical, QLoRA achieves 97-99% of full fine-tuning quality.

How many training examples do I need?

It depends on task complexity. 50-100 examples are enough for a proof of concept. 500-1,000 examples produce useful results for most single tasks. 5,000-10,000 examples deliver production quality for complex tasks. Beyond 10,000, you hit diminishing returns unless the task has extremely high variability.

Which base model should I fine-tune in 2026?

Llama 3.x for general-purpose tasks (best overall quality/size ratio). Mistral for European languages and efficient inference. Qwen 2.5 for multilingual and code tasks. Phi-4 when you need the smallest possible footprint. Gemma 2 for Google ecosystem integration.

What is GRPO and why does it matter?

GRPO (Group Relative Policy Optimization), introduced by DeepSeek, is a successor to RLHF for alignment training. The key advantage: it doesn't require training a separate reward model, which cuts the computational cost roughly in half. TRL v0.15.0 supports GRPO natively, making it accessible to anyone using the Hugging Face ecosystem.

How do I prevent catastrophic forgetting?

Use LoRA or QLoRA instead of full fine-tuning, they freeze the base model weights, which preserves general knowledge. Keep your learning rate low (2e-4 for QLoRA, 2e-5 for full). Train for as few epochs as needed (1-3 is usually enough). After training, run your model on general benchmarks (MMLU, HellaSwag) to verify it hasn't regressed.

Can I fine-tune and then use RAG together?

Absolutely, and many production systems do exactly this. Fine-tune for behavior and format consistency, then connect RAG for up-to-date knowledge retrieval. The fine-tuned model is better at using the retrieved context because it understands your domain's language and output requirements.

How long does fine-tuning take?

For most projects, 30 minutes to 8 hours. An 8B model with 500 examples on Unsloth + RTX 4090 finishes in 15-30 minutes. The same job on a free Colab T4 takes 1-2 hours. 70B models on A100s take 5-8 hours. Full fine-tuning on multi-GPU setups can take 20-40 hours.

Is OpenAI's fine-tuning API worth it?

For quick prototyping, yes. You can upload a JSONL file and have a fine-tuned GPT-4o-mini in 30 minutes with zero GPU setup. For production, local fine-tuning is usually better: you own the weights, control your data privacy, and costs are lower at scale. Most teams start with the API to validate the approach, then migrate to local.

Conclusion

Fine-tuning an LLM isn't the black magic it was two years ago. Here are the key takeaways:

1. Start with QLoRA + Unsloth, it handles 90% of use cases on consumer hardware
2. Good data beats a bigger model every single time. Spend your effort on dataset quality, not GPU upgrades.
3. The cost barrier is gone, fine-tune an 8B model for under $1 on cloud GPUs
4. Always evaluate against the base model before deploying. If it's not measurably better, don't ship it.
5. Consider RAG first, fine-tune only when you need the model to behave differently, not just know different things

Ready to implement? See our Best LLM Fine-Tuning Tools & Platforms [coming soon] for a detailed comparison of training frameworks and deployment options.

If you found this guide useful, check out our walkthrough on choosing the right AI stack for the broader architecture decisions around AI-powered products.

Sources

• Unsloth GitHub Repository, fine-tuning framework with 2-5x speed improvements
• Hugging Face TRL Documentation, SFTTrainer, DPO, and GRPO implementation
• Hugging Face PEFT Documentation, LoRA and parameter-efficient fine-tuning
• QLoRA Paper (Dettmers et al., 2023) -- 4-bit NormalFloat quantization research
• LoRA Paper (Hu et al., 2021), low-rank adaptation of large language models
• OpenAI Fine-Tuning API Documentation, API-based fine-tuning workflow
• RunPod GPU Cloud Pricing, cloud GPU cost reference
• vLLM Documentation, production LLM serving
• Meta Llama Fine-Tuning Guide, official Llama training recommendations
• DeepSeekMath Paper (GRPO), Group Relative Policy Optimization