"Is my data private, and should I use a local model instead?" comes up again and again in my client calls, usually from people handling sensitive business, legal or production data. Setup and install questions came up in about 10 of my meetings in the last three months. So this guide covers both: why, and exactly how.
Why run AI locally?
- Privacy: for sensitive documents, a local model keeps the text on your machine.
- Cost: no monthly fee for the local model.
- Offline: it works without internet once downloaded.
- Learning: you understand what a "model" really is when you have one on your disk.
When cloud is still better: local models on an ordinary laptop are slower and less capable than ChatGPT, Claude or Gemini. Use local for private or simple tasks, cloud for hard ones.
This is not just a hobby trend. Even PewDiePie has announced Ajax, a personal assistant built on the open Qwen 3.5 model and designed to run on home PCs (it was not yet available to download as of late October 2026). And one of my own learners, a product manager with no technical background, ran a Gemma model locally with Ollama and built a small chatbot on top of it. If she can, you can.
Step 0: Check your laptop
- Windows: Windows 10 22H2 or newer. Ollama needs at least 4 GB of disk for the app, plus space for models (a few GB to tens of GB each).
- Mac: macOS 14 Sonoma or newer. Apple Silicon (M1 or newer) Macs work well because of unified memory. Intel Macs run on the CPU only, which is slower.
- Memory: more memory means bigger, smarter models. With 8 GB, stick to small models.
Not sure what you have? On Windows: Settings > System > About. On Mac: Apple menu > About This Mac.
Step 1: Download and install
Windows
- Go to ollama.com/download/windows and download OllamaSetup.exe.
- Double-click and install. You do not need administrator rights.
- Ollama now runs in the background. Look for the llama icon in the system tray (bottom right).
Comfortable with PowerShell? You can instead run: irm https://ollama.com/install.ps1 | iex
Mac
- Go to ollama.com/download and download the Mac app.
- Open it and move it to Applications. Launch it once.
On Linux, run curl -fsSL https://ollama.com/install.sh | sh in a terminal.
Step 2: Run your first model
- Open a terminal. Windows: press Win+R, type
cmd, press Enter. Mac: open the Terminal app. - Type:
ollama run gemma4:e2b- Wait for the download (about 4.6 GB, so use good Wi-Fi). Then you will see a
>>>prompt. - Type
helloand press Enter. The model replies. - Type
/byeto exit.
That's it. You are running AI on your own machine.
Step 3: Know it is working
- Type
ollama listto see the models you have downloaded. - Ollama also runs a local server at
http://localhost:11434. Apps on your laptop talk to the model through this address. You don't need it on day one, but it matters when you build your own chatbot later.
Step 4: Pick a model for your laptop
| Your laptop | Start with | Download size |
|---|---|---|
| 8 GB RAM, no graphics card | gemma4:e2b or qwen3.5:4b | About 4.6 GB or 3.3 GB |
| 16 GB RAM or Apple Silicon Mac | gemma4 or qwen3.5 | About 6.6 GB each |
| Gaming laptop with NVIDIA card, or a Mac with lots of memory | gemma4:12b | About 7.7 GB |
Sizes are as listed on ollama.com/library in October 2026. Browse there for more. My advice: start small, and only go bigger when the small one feels too limited.
Step 5: Want a chat window instead of a terminal?
- Ollama's own app on Windows and Mac includes a chat window, so you can pick a model and chat without the terminal.
- LM Studio (lmstudio.ai) is a free point-and-click alternative that downloads and runs models without Ollama.
Common problems
- "ollama is not recognized": close the terminal and open a new one, or restart the laptop.
- Very slow replies: the model is too big for your memory. Try a smaller one.
- Disk full: on Windows, you can move models to another drive by setting the
OLLAMA_MODELSenvironment variable, then restarting Ollama. - Strange squares in the progress bar on Windows 10: change the terminal font. It is cosmetic.
Local or cloud: my rule of thumb
Sensitive document, simple task: local. Hard reasoning, coding a full app, latest information: cloud. Many of my learners use both. Not sure which cloud plan? Read my take on which AI tool to pay for, and for cloud chats, see how to keep your data private in ChatGPT, Claude and Gemini.