Building with AI
Open and local AI models course
Generative AI on your own infrastructure, without data leaving the company.
Not every company can or wants to send its data to an external provider. Open AI models let you run generative AI on your own servers, with full control over the information, and their quality keeps improving.
In this course you will learn to choose the right model for each case, size the hardware, understand quantisation and run and serve models with Ollama, llama.cpp or vLLM. We finish by comparing their quality and cost with a commercial model, with data, so you can decide with judgement.
What you will learn
- Understand the open model ecosystem and its licences
- Choose the right model and hardware for each case
- Run models locally and serve them to a team
- Compare their quality and cost with commercial models
- Deploy them securely
Syllabus
7 modules, from the fundamentals to the final project.
-
Why open models
- Privacy, data sovereignty and cost
- Open models versus commercial APIs
- Model families and usage licences
-
Choosing a model
- Size, context, languages and tasks
- Leaderboards and your own tests
-
Hardware
- CPU, GPU and memory requirements
- Quantisation and its effect on quality
- Estimating the cost of an installation
-
Running locally
- Ollama and LM Studio
- llama.cpp and model formats
-
Serving a team
- vLLM and compatible APIs
- Internal chat interface
- Concurrency and performance
-
Applications
- RAG with local models
- Local embeddings
- Tool use
-
Security and operations
- Access control and logs
- Model updates
Tools you will practise with
- Ollama
- LM Studio
- llama.cpp
- vLLM
- Open WebUI
- Hugging Face
Hands-on project
You deploy an open model on a server or your own machine, serve it through a compatible API, connect it to a chat interface and compare its quality, speed and cost with a commercial model on your company's tasks.
Requirements
- Command-line skills.
- Some knowledge of Docker recommended.
Other courses
-
AI in apps and digital products course
AI features users understand, use and trust.
- Duration: 12 hours
- Level: Intermediate
-
AI-assisted programming course
Code faster with coding assistants and agents, without losing quality.
- Duration: 12 hours
- Level: Intermediate
-
Generative AI system evaluation course
Measure your AI application's quality with data, not impressions.
- Duration: 12 hours
- Level: Advanced
Want this course for your team?
We adapt the content, length and format to your company.