Skip to content
Neural Soft
Contact us Español (View this page in Spanish)

Data, machine learning and deep learning

Language model fine-tuning course

Adapt a language model to your domain, tone or task, with judgement and method.

Request information

Fine-tuning lets you adapt a language model to a specific task, domain or style. Done well, it produces smaller, faster and cheaper models that match the big ones on that task. Done badly, it is an expensive way to make a model worse.

That is why this course starts with the key question: when fine-tuning pays off and when a good prompt or a RAG system is enough. Then we go through the complete process with open models: quality data, LoRA and QLoRA, preference tuning, rigorous evaluation and deployment.

What you will learn

  • Decide when fine-tuning beats prompting or RAG
  • Prepare high-quality instruction datasets
  • Fine-tune open models with LoRA and QLoRA
  • Understand preference tuning (DPO)
  • Evaluate and deploy the fine-tuned model

Syllabus

8 modules, from the fundamentals to the final project.

  1. When to fine-tune a model

    • Prompting, RAG and fine-tuning compared
    • Cases where it pays off
    • Cost and maintenance

  2. Data

    • Instruction and conversation formats
    • Dataset quality, diversity and size
    • Synthetic data generated with models

  3. Fine-tuning fundamentals

    • Full versus parameter-efficient fine-tuning
    • Tokenisers and chat templates

  4. LoRA and QLoRA

    • Low-rank adapters
    • 4-bit quantisation for training with less memory
    • Key hyperparameters

  5. Preference tuning

    • From RLHF to DPO
    • When to use it

  6. Fine-tuning on managed services

    • Fine-tuning on providers' platforms
    • Pros and cons

  7. Evaluation

    • Comparison with the base model
    • Catastrophic forgetting and regressions

  8. Deployment

    • Merging adapters and quantising
    • Serving the model with vLLM or Ollama

Tools you will practise with

  • Python
  • PyTorch
  • Hugging Face Transformers, PEFT and TRL
  • Unsloth
  • vLLM

Hands-on project

You fine-tune a small open model for a specific task (for example, classifying requests with your company's categories or writing in a particular style), evaluate it against the base model and against a good prompt, and serve it locally.

Requirements

  • Python and deep learning knowledge (the deep learning with PyTorch course is recommended).
  • GPU access (cloud alternatives are provided).

Other courses

Want this course for your team?

We adapt the content, length and format to your company.

Request information