How to Run Any LLM Locally on Mac with Ollama

TL;DR - The Quick Version

Short on time? Here is the cheat sheet.

Step	What You Do	How Long
Install Ollama	brew install ollama or grab the DMG from ollama.com	1 minute
Chat with your first AI	ollama run llama3.3 -- that's it, one command	2 minutes (+ download)
Pick your model	8GB Mac → 7B models, 16GB → 14B, 32GB → 32B, 64GB → 70B	1 minute
Connect to any tool	OpenAI-compatible API at localhost:11434 -- works with everything	5 minutes
VS Code coding assistant	Install Continue.dev, point it at Ollama, pick a model	5 minutes

The short version: Install Ollama, run one command, and you have a private, free, fully offline AI on your Mac. Works with 45,000+ models. No subscription, no API key, no data leaving your machine.

---

Your Mac Already Has Everything It Needs to Run a Local LLM

Here is something that would have sounded absurd three years ago: the MacBook sitting in front of you can run a self-hosted LLM that rivals what billion-dollar companies were deploying in 2023. No cloud subscription. No API key. No data leaving your machine. Just your Mac with Apple Silicon, a free open-source tool called Ollama, and about five minutes of setup.

I remember the first time I ran a 32B model locally on my M2 MacBook. I typed a question, watched the response stream in -- fast, coherent, genuinely useful -- and thought: "Wait, this is just... running on my laptop right now?" No internet. No monthly bill. Just my Mac thinking through a coding problem, completely offline. It felt like the first time I installed Linux. That same sense of "I own this."

By the end of this tutorial, you will have that same moment. We are going from zero to a fully working local AI setup: chatting with models, picking the right one for your hardware, using it as a coding assistant in VS Code, and knowing the gotchas to avoid. Whether you are a developer wanting private AI for your code, a writer tired of subscription fatigue, or just curious about running AI locally on Mac -- this guide has you covered.

What You Will Learn

• Install Ollama and talk to your first AI model in under 5 minutes
• Pick the right model size so your Mac stays snappy (not swapping to disk)
• Use the Ollama CLI and API like a pro
• Turn Ollama into a local coding assistant inside VS Code
• Avoid the security and performance mistakes that trip up most beginners

Time needed: about 30 minutes Level: Beginner -- no AI experience required What you need: A Mac with Apple Silicon (M1/M2/M3/M4), macOS Sonoma 14 or later, and at least 8GB of RAM

---

Why Bother Running LLMs Locally?

You might be wondering: if ChatGPT, Claude, and Gemini are so good, why would I run a model on my own machine?

Fair question. Here are four reasons that convinced me.

Your data stays yours. Every prompt you send to ChatGPT or Claude travels to a server, gets processed, and -- depending on the provider's policies -- may be used for training. When you run a model locally, nothing leaves your machine. If you work with client code, legal documents, medical records, or anything you would not want on someone else's server, this is a big deal.

It is free. No $20/month subscription. No per-token API costs. Once you download a model, you can use it as much as you want. The only cost is the electricity to run your Mac, which works out to roughly $0.01-0.03 per hour of active inference on Apple Silicon.

No internet required. On a plane? In a cabin in the mountains? Working in a coffee shop with spotty wifi? Your local model does not care. It works exactly the same offline as online.

It is fast in ways that matter. Cloud models have network latency -- that pause between hitting Enter and seeing the first word appear. Locally, time-to-first-token is typically 50-150ms on an M2 Pro. Cloud APIs, even under optimal conditions, add 200-800ms of network latency before the first token arrives. For the kind of rapid back-and-forth you do while coding, that responsiveness makes a real difference.

Now, the honest trade-off: cloud models like GPT-5.2 and Claude Opus 4.6 are still stronger for very complex reasoning and tasks requiring broad knowledge. But for coding, summarization, writing, and everyday questions? A good local model handles those beautifully.

Ollama has become the community's go-to tool for running LLMs locally on Mac. It has 162,000+ stars on GitHub, the r/LocalLLaMA subreddit (620,000+ members) treats it as the default starting point, and it gets updated constantly -- the latest release (v0.15.6) dropped just days ago.

Let's get started.

---

Step 1: Install Ollama on Your Mac

What we are doing: Getting Ollama installed so you can start running models.

You have two options, and both take under a minute.

Option A: Download Directly (I Recommend This)

1. Go to ollama.com
2. Download the Mac installer (it is a DMG file)
3. Open it, drag Ollama to your Applications folder, and launch it

That is it. Ollama adds itself to your terminal PATH automatically.

Option B: Use Homebrew

If you are already a Homebrew user:

brew install ollama

Make Sure It Worked

Open a fresh terminal window and type:

ollama --version

You should see something like ollama version is 0.15.6. If you do -- you are good to go.

Where Ollama Lives on Your Mac

You will want to know where things are stored, especially when models start eating disk space:

Location	What Is There
/Applications/Ollama.app	The application itself
~/.ollama/models/	Downloaded model weights (can grow to tens of GB)
~/.ollama/	Configuration and logs

Model files are large -- a single 7B model at Q4 quantization is about 4.5GB, and a 70B model is roughly 40GB. Make sure you have disk space to spare.

If Something Goes Wrong

• "command not found" after installing: Close your terminal and open a new one. The PATH update sometimes needs a fresh session. If the issue persists, verify that /usr/local/bin (Intel) or /opt/homebrew/bin (Apple Silicon) is in your PATH by running echo $PATH.
• Homebrew installation fails: Use the direct download (Option A) instead. The DMG installer handles PATH configuration automatically and tends to be more reliable.

---

Step 2: Run Your First Model

What we are doing: Downloading an AI model and having your first conversation with it. One command.

Ready? This is the fun part.

ollama run llama3.3

That single command does everything: it downloads Llama 3.3 (Meta's excellent 8B parameter model, about 5GB), loads it into memory using your Mac's GPU, and drops you into a chat session. The first time takes a few minutes for the download. After that, it starts in seconds.

You will see a >>> prompt. Just start typing:

>>> What makes Apple Silicon good for running AI models?

Watch the response stream in, word by word. That is running entirely on your machine. No server, no API call, no internet connection needed (after the initial download).

When you are done chatting, type /bye or hit Ctrl+D to exit.

How does it feel? If you are on a machine with 16GB+ of RAM, the speed should be genuinely impressive -- 15 to 25 tokens per second, which is fast enough to read comfortably as it generates.

If Something Goes Wrong

• Download seems stuck? Downloads resume automatically if interrupted. If your connection is slow, just let it run -- it will not start over from scratch.
• "out of memory" error? Llama 3.3 needs about 7-9GB of available RAM. If you are on an 8GB Mac with other apps open, try a smaller model first: ollama run phi4 or ollama run qwen2.5:7b. Close memory-heavy apps (browsers with lots of tabs are the usual culprit) and try again.
• Model loads but responses are extremely slow (< 1 token/sec)? Your Mac is likely swapping to disk. Try a smaller model -- this means the one you chose is too big for your available memory.

---

Step 3: Choose the Right Model for Your Mac

What we are doing: Figuring out which model gives you the best experience based on your specific hardware.

This is the step where most people either overshoot (downloading a model too big for their RAM, then wondering why their Mac sounds like a jet engine) or undershoot (sticking with tiny models when their hardware can handle much more).

Here is the simple rule: your RAM determines your model size. The quick formula: take the model's file size on disk, add about 2GB for overhead, and that is your minimum RAM requirement.

RAM-to-Model Mapping

Your Mac's RAM	Max Model Size	Disk Space Needed	RAM Used	Headroom for Apps
8GB	7B parameters	~4.5GB	~6-8GB	Minimal -- close other apps
16GB	14B parameters	~8GB	~10-12GB	Moderate
32GB	32-34B parameters	~18GB	~20-22GB	Comfortable
64GB	70B parameters	~40GB	~42-48GB	Generous

To put model sizes in perspective: a 7B model is your fast first-draft helper -- quick answers, solid coding suggestions, great for rapid iteration. A 32B model is a thoughtful senior colleague -- noticeably smarter, handles nuance well, takes a bit longer to respond. A 70B model approaches expert consultant territory -- frontier-quality output, but you are trading speed for capability.

Model Recommendations by Use Case

Now, which specific model should you try? Here are my recommendations:

If You Want...	Try This Model	Disk Size	Speed (M2 Pro)	Command
General chatting	Llama 3.3 8B	~5GB	~15-25 tok/s	ollama run llama3.3
Coding help	Qwen 2.5 Coder 7B	~4.5GB	~15-25 tok/s	ollama run qwen2.5-coder:7b
Deep reasoning	DeepSeek R1 7B	~4.5GB	~15-25 tok/s	ollama run deepseek-r1:7b
Image understanding	Gemma 3 12B	~7GB	~10-18 tok/s	ollama run gemma3:12b
Math and logic	Phi-4 14B	~8GB	~10-18 tok/s	ollama run phi4
Premium quality	Qwen 2.5 32B	~18GB	~5-10 tok/s	ollama run qwen2.5:32b
Premium reasoning	DeepSeek R1 32B	~18GB	~5-10 tok/s	ollama run deepseek-r1:32b
Frontier quality	Llama 3.3 70B	~40GB	~2-5 tok/s	ollama run llama3.3:70b

Understanding Quantization (Simpler Than It Sounds)

When you download a model through Ollama, it comes in a compressed format called GGUF with a specific quantization level. Quantization reduces the model's precision to save space and memory while keeping most of the quality. All you need to know: Q4_K_M is the default, and it is the right choice for almost everyone.

If you want the details:

Quantization	Quality Kept	Size vs Full Model	When to Use
Q2_K	~85%	~25%	Only if you are desperate for RAM -- noticeable quality drop
Q4_K_M	~95%	~35%	The default. Start here. Best balance for daily use
Q5_K_M	~97%	~42%	If Q4 produces odd outputs for your specific task
Q6_K	~99%	~50%	Diminishing returns for most people
Q8_0	~99.5%	~66%	Only if you have RAM to spare

If you are getting weird outputs or the model seems to struggle with nuance, try Q5_K_M before jumping to a larger model. But honestly, most people run Q4_K_M for months and never think about it.

Managing Your Models

# See what you have installed (and how much space each takes)
ollama list

# Delete a model you are not using anymore
ollama rm llama3.3

# Download a model in advance without starting a chat
ollama pull qwen2.5:32b

My advice: Start with a 7B model. Get comfortable with the workflow. Then try a 32B model and feel the quality difference. That "wow" moment when you go from 7B to 32B is something every local LLM user remembers.

---

Step 4: Master Essential Ollama Commands

What we are doing: Learning the commands that make daily use smooth and efficient.

You already know ollama run. Here is the full toolkit:

Command	What It Does	Example
ollama run <model>	Start chatting with a model	ollama run llama3.3
ollama pull <model>	Download a model (without starting a chat)	ollama pull qwen2.5:32b
ollama list	Show all installed models with sizes	ollama list
ollama ps	Show which models are currently loaded in memory	ollama ps
ollama rm <model>	Delete a model and free the disk space	ollama rm llama3.3
ollama show <model>	Display model metadata and configuration	ollama show llama3.3 --modelfile
ollama create <name>	Create a custom model from a Modelfile	ollama create mybot -f Modelfile
ollama cp <src> <dst>	Copy or rename a model	ollama cp llama3.3 my-llama

Create Your Own Customized Model

This is one of Ollama's best features. You can take any base model and create a personalized version with your own system prompt and settings. Here is an example -- a coding assistant that is opinionated about code quality:

Create a file called Modelfile:

FROM qwen2.5-coder:7b

PARAMETER temperature 0.2
PARAMETER num_ctx 8192

SYSTEM """
You are a pragmatic senior developer. You write clean, readable code.
You prefer simple solutions over clever ones. When reviewing code,
you focus on bugs and maintainability, not style preferences.
Always include brief comments explaining non-obvious logic.
"""

Then build and use it:

ollama create my-code-reviewer -f Modelfile
ollama run my-code-reviewer

Now you have a persistent coding personality that loads every time. I have three or four of these for different tasks -- a code reviewer, a writing editor, a brainstorming partner. They make local LLMs feel like having a team of specialists on call.

---

Step 5: Use the Ollama API

What we are doing: Connecting Ollama to other tools so it is not limited to terminal chat.

Ollama runs a local API server at http://localhost:11434 automatically. This is how you connect it to scripts, apps, and other developer tools.

The best part? Since early 2026, Ollama supports the OpenAI-compatible API format. That means any tool built for ChatGPT's API works with Ollama. You just change the URL. Ollama also has its own native API format at /api/chat, but the OpenAI-compatible endpoint at /v1/chat/completions is what you will use 99% of the time -- it is what the entire ecosystem speaks.

Quick Test with curl

curl http://localhost:11434/v1/chat/completions -d '{
  "model": "llama3.3",
  "messages": [
    {"role": "user", "content": "Explain the difference between a stack and a queue in one paragraph."}
  ]
}'

Python Example (Drop-In Replacement for OpenAI)

If you have ever used the OpenAI Python library, this will look very familiar:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"  # Required by the library, but Ollama doesn't check it
)

response = client.chat.completions.create(
    model="qwen2.5:32b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What are three underrated Python standard library modules?"}
    ]
)

print(response.choices[0].message.content)

See what happened there? Same OpenAI library, same code structure. Just a different base_url. You can prototype against a local model for free and switch to a cloud API later by changing one line. The OpenAI-compatible endpoint supports streaming, function calling, and structured JSON output -- the same feature set you would use with GPT-5.2 via OpenAI's API.

This is powerful. It means Ollama works with thousands of tools in the AI ecosystem -- anything that speaks the OpenAI API format. Automation scripts, Slack bots, internal tools -- they all just work.

---

Step 6: Set Up a Local Coding Assistant in VS Code

What we are doing: Turning Ollama into an AI pair programmer inside your editor, like GitHub Copilot but free and private.

If you are a developer, this is probably the section you have been waiting for. The tool we are using is Continue.dev -- an open-source AI assistant extension for VS Code that talks to Ollama natively.

Setup (5 Minutes)

1. Open VS Code
2. Go to Extensions and search for "Continue"
3. Install it
4. Open Continue's config (it will prompt you, or find it at ~/.continue/config.yaml)
5. Add your Ollama models:

models:
  - title: "Qwen Coder 7B (Fast)"
    provider: ollama
    model: qwen2.5-coder:7b

  - title: "Qwen Coder 32B (Quality)"
    provider: ollama
    model: qwen2.5-coder:32b

6. Restart VS Code. Done.

What You Get

• Code completions as you type -- like Copilot, but running on your machine
• A chat panel where you can ask "how does this function work?" or "write tests for this class"
• Inline editing -- select code, hit the shortcut, describe what you want changed
• Full privacy -- your code never leaves your laptop

I keep the 7B model loaded for fast completions while typing, and switch to the 32B model when I need help with something complex -- a gnarly refactor, debugging a tricky race condition, or designing an API. The quality of the 32B model on coding tasks is genuinely surprising.

For how this compares to cloud-based coding assistants, check our Cursor vs Copilot vs Claude Code comparison.

---

Common Mistakes (and How to Avoid Them)

I have made every single one of these. Let me save you the trouble.

1. Running Models on Battery

Your Mac throttles GPU performance on battery to save power. Inference speed can drop 40-60% compared to plugged-in operation. The fix is obvious but easy to forget: plug in before you start chatting. The r/LocalLLaMA community's number one tip is literally "always plug in for inference."

2. Trying a Model That Is Too Big

If you have 16GB of RAM and try to run a 70B model, your Mac will not crash -- but it will swap to disk, and inference will slow to a painful crawl (think: one word every few seconds). Check the RAM table in Step 3 before you download. Your Mac's Activity Monitor is also your friend -- keep an eye on memory pressure.

3. Accidentally Exposing Ollama to the Internet

This is a real security concern, and it deserves more than a passing mention. In January 2026, security researchers discovered 175,000 Ollama instances publicly accessible on the internet. Every single one was a user who had changed OLLAMA_HOST to 0.0.0.0 (which listens on all network interfaces) without setting up any authentication or firewall.

Here is why this matters: Ollama has no built-in authentication. None. If you bind it to a public interface, anyone who finds your IP can:

• Use your models (stealing your compute resources)
• Send arbitrary requests to the API
• Potentially exploit known vulnerabilities (a remote code execution vulnerability was discovered and patched in 2024 -- if you are not updating, you are exposed)

The safe approach: Keep the default localhost binding. This means only programs running on your Mac can reach Ollama. If you genuinely need network access (for example, to reach Ollama from another device on your home network), put a reverse proxy like nginx or Caddy with basic authentication in front of it. Never expose Ollama directly.

And keep Ollama updated: brew upgrade ollama or download the latest DMG from ollama.com.

4. Not Updating macOS

macOS Sonoma 14.4 and later include Metal framework optimizations that make inference 15-20% faster. That is free performance you get just by updating your OS. Check your version in the terminal:

sw_vers

If you are below 14.4, consider updating before spending time optimizing anything else.

5. Sticking with One Model Forever

Different models are better at different things. Qwen 2.5 Coder is excellent for code but mediocre at creative writing. DeepSeek R1 is great at reasoning but slower because it thinks out loud. Experiment. The beauty of Ollama is that trying a new model is one command away.

---

Advanced Tips

Try MLX for Even Faster Inference

Apple built their own ML framework called MLX, and it is optimized specifically for Apple Silicon's unified memory in ways that GGUF is not. MLX models run 30-50% faster on the same hardware thanks to zero-copy memory operations. The catch: the ecosystem is smaller, setup requires more Python knowledge, and you cannot just run one command like with Ollama.

If speed is your priority and you are comfortable with Python, check out the mlx-community models on Hugging Face. The MLX LM library (v0.30.6 as of February 2026) makes getting started reasonably straightforward.

Get a ChatGPT-Like Interface with Open WebUI

Not everyone wants to chat through a terminal. Open WebUI is a self-hosted web interface that connects to Ollama and gives you the full ChatGPT experience -- conversation history, model switching, file uploads, dark mode -- all running locally. Install it with Docker:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data --name open-webui \
  ghcr.io/open-webui/open-webui:main

Then visit localhost:3000 in your browser. If you do not have Docker installed, grab Docker Desktop for Mac first.

Optimize Your Setup

A few things that make a real difference:

• Close Chrome tabs. Seriously. Each tab eats RAM that your model could be using. A browser with 30 tabs open can consume 4-8GB.
• Only load one model at a time. Check with ollama ps -- idle models still consume memory. Ollama manages loading and unloading automatically, but if you are tight on RAM, make sure only one model is resident.
• Shorter context windows are faster. If you do not need 8K context, set num_ctx to 4096 in your Modelfile. This reduces memory usage and speeds up generation.
• Update macOS. Sonoma 14.4+ gives you 15-20% better inference speed for free.

Build a Personal AI Toolkit

Here is a workflow I have settled into on my 64GB Mac:

• Quick questions: Qwen 2.5 7B (loads in seconds, answers fast)
• Coding: Qwen 2.5 Coder 32B (the quality is worth the slower speed)
• Deep thinking: DeepSeek R1 32B (when I need the model to really reason through something)
• Writing: Llama 3.3 70B (when I want maximum prose quality and I am plugged in with time to spare)

Switch between them with ollama run <model-name> -- Ollama handles loading and unloading automatically based on available memory.

---

Summary

Let's take a step back and look at what you have accomplished.

You installed Ollama. You ran an AI model entirely on your own hardware. You learned how to pick the right model for your Mac, how to use the API to connect Ollama to anything, and how to set up a real coding assistant in VS Code. You know the security gotchas and the performance tricks.

That is not trivial. You now have something that costs $0/month, works without internet, and keeps your data completely private. A year ago, this setup required significant technical expertise. Today it takes 30 minutes and a single brew install.

Where to Go Next

1. Try a bigger model. If you have been playing it safe with 7B, go for 32B. The quality jump is real.
2. Build something. Use the API to connect Ollama to a project you are working on -- a Slack bot, a CLI tool, a RAG pipeline.
3. Level up your prompts. The same prompt engineering techniques that work with ChatGPT and Claude work with local models too.
4. Create custom Modelfiles. Tailor the model to your needs -- a writing editor, a code reviewer, a brainstorming partner.

More from Model Momentum

• ChatGPT vs Claude vs Gemini -- understand when cloud AI is the better call
• Cursor vs Copilot vs Claude Code -- the full coding assistant landscape
• Prompt Engineering 101 -- get better results from every model, local or cloud

---

Frequently Asked Questions

Is Ollama free?

Completely. Ollama is open-source under the MIT license, and every model you can download through it is free to use. No subscription, no usage limits, no hidden costs. You are running it on your own hardware, so the only expense is electricity (a few cents per hour).

How much RAM do I need to run local LLMs?

8GB is the minimum for small 7B models. 16GB lets you comfortably run 14B models, which are a significant step up in quality. 32GB opens the door to 32B models -- this is the sweet spot where local AI starts feeling genuinely competitive with cloud services. And 64GB lets you run 70B models, which approach frontier quality. Quick formula: take the model's file size on disk, add about 2GB for overhead, and that is the RAM you need available.

Is a local LLM as good as ChatGPT?

It depends on the task. For coding, summarization, translation, and everyday questions? A 32B model like Qwen 2.5 is surprisingly close. For complex multi-step reasoning, tasks requiring very broad knowledge, or cutting-edge capabilities? Cloud models like GPT-5.2 and Claude Opus 4.6 still have a meaningful edge. The smart approach is using both: local for private, frequent, fast tasks -- cloud for maximum capability when you need it.

Does Ollama use GPU on Mac?

Yes, and you do not have to configure anything. Ollama automatically uses Apple's Metal framework on all M-series Macs. Here is the nice thing about Apple Silicon: the GPU and CPU share the same pool of memory (unified memory architecture). So there is no "GPU VRAM" to worry about like on PC graphics cards. Whatever RAM your Mac has, both the CPU and GPU can access it for model inference.

Can I use Ollama as a coding assistant?

Absolutely. Install the Continue.dev extension in VS Code, point it at your local Ollama instance (localhost:11434), and pick a code-focused model. I recommend Qwen 2.5 Coder 7B for speed (great for inline completions) and Qwen 2.5 Coder 32B for quality (great for complex refactors and code review). Everything runs locally -- your code never touches an external server.

---

Tutorial current as of February 2026. Tested with Ollama v0.15.6 on macOS Sequoia 15.3 with Apple M2 Pro.