Privacy in local AI: what stays on your device

Cloud-hosted AI assistants have become the default. A prompt typed into a chat box is, in most cases, transmitted to a remote inference cluster, processed by a model the user cannot inspect, and stored in a log the user cannot read. The trade is convenience for visibility, and for many workflows that trade is fine. For others, it is not.

Local AI — running a model directly on your own hardware — reverses that trade. The model weights, the prompts, the responses, and the conversation history all live on the device. No round trip to a remote server, no central log, no model provider holding a copy of your input. The model is yours in the same sense a calculator is yours: the data you feed it never leaves the room.

Who this matters for:

• Journalists and their sources. Encrypted chat protects messages in transit; local AI protects the prompts that summarise, draft, or analyse them.
• Researchers working with unpublished data. Preprints, grant text, and review responses are not safe to send to a third-party inference endpoint under most institutional policies.
• Students in restrictive networks. Some campus and corporate networks block cloud AI entirely; local AI runs offline.
• Anyone whose threat model includes data breaches. Cloud providers get breached. Local-first setups do not have a remote target to breach.
• Anyone who simply prefers the property. Local inference is a structural choice, not a feature flag.

The rest of this post is a practical guide to what "local" actually means in an Ollama-based setup, what still leaves the device, and how to verify the boundary yourself.

Ollama is the most common runtime for local models on consumer hardware. The default install exposes an HTTP API on http://127.0.0.1:11434 and an OpenAI-compatible endpoint at /v1/chat/completions. A client application on the same machine — for example, a side-panel assistant — talks to that loopback port and nowhere else.

What stays on the device by default:

• Model weights. Once downloaded, the model is stored on disk and loaded into RAM or VRAM. No part of the weight file is sent anywhere.
• Prompts. The text you send to /api/chat or /v1/chat/completions is processed by the local inference engine. It is not transmitted to a remote endpoint.
• Responses. Token-by-token output is generated locally. The stream of tokens is delivered to the client on loopback and not forwarded.
• Conversation history. Anything the client application persists — message lists, system prompts, scratchpads — lives in the client's own storage. Ollama itself is stateless; the conversation context is reconstructed from client-side state on each turn.
• Embeddings. When the client uses /api/embeddings for vector search, the resulting vectors stay local. The text used to produce them is not transmitted.

The only network calls Ollama makes are:

• Model pull — when you run ollama pull <model>, the manifest and weight shards are downloaded from the Ollama registry (or a private registry you have configured). Subsequent loads use the local cache.
• Telemetry (configurable, off by default) — Ollama sends a small anonymous ping on the first run of a day. The payload contains the OS, the Ollama version, and a random UUID. Set the environment variable OLLAMA_TELEMETRY=false to disable it permanently.

No prompt, response, or conversation history leaves the device under a default Ollama install with telemetry disabled. The boundary is the loopback interface.

Local-first is not zero-network. Three categories of traffic are expected and worth listing explicitly so the boundary is auditable rather than implicit.

3.1 Model downloads

The first time you run ollama pull gemma3:4b, the model manifest and weight shards are downloaded over HTTPS from the Ollama registry. The download is a one-time event for each model; the cache lives in ~/.ollama/models (Linux and macOS) or %USERPROFILE%\.ollama\models (Windows). Subsequent ollama run commands do not re-download.

Custom registries are supported: set OLLAMA_HOST and OLLAMA_REGISTRY to point at a private mirror. Air-gapped setups can pre-stage the model directory and skip the registry entirely.

3.2 Documentation and help pages

If you open a "Help" link in a client application, the request goes to the project's documentation site. No user content is included in the request — it is a plain GET. The model name or local file path is not sent. Third-party documentation pages may set cookies or run analytics under their own privacy policies.

3.3 Source repository and stars

Update checks, bug reports, and "star this repo" buttons go to github.com. These calls are user-initiated and contain no prompt or response data. They are visible in the same packet capture as everything else.

3.4 What is NOT in this list

No analytics SDK is bundled with the model runtime. No crash reports are sent automatically. No A/B test framework phones home. None of these categories are opt-in or opt-out — they do not exist in the default install. If a client application adds them on top, that is a property of the client, not the runtime.

The full expected outbound set under a clean install is therefore: model downloads (one-time, cacheable), optional telemetry (off by default), and user-initiated links to docs or the source repository.

A practical threat model is more useful than a marketing claim. Below is the realistic scope of protection from running AI on your own hardware, and the boundary beyond which local-first stops helping.

What local AI protects against

• Mass surveillance of inference traffic. There is no remote endpoint to log your prompts.
• Provider data breaches. No remote store of your data means no remote store to be breached.
• Provider-side training on your input. The model weights you run are already final; nothing you send can retroactively train them.
• Provider lock-in. The model is a file on disk. You can run it with any compatible runtime.
• Region-based restrictions and account suspensions. A local install is not gated by an account or a billing status.

What local AI does NOT protect against

• Physical access to the device. An attacker with the laptop open and the disk unlocked sees everything you see.
• Malware on the device. A keylogger or a screen capture is unaffected by the model running locally. Local-first is a network property, not an endpoint security property.
• Side-channel attacks on the model. Inference leaks information through timing and power channels. Research-grade attacks, not consumer-relevant today, but real.
• Output of harmful or biased content. A local model inherits the bias of its training data. Running locally does not make a model "safer" in the policy sense.
• Poor operational security on the operator's side. Screenshots, screen sharing, shoulder-surfing, and cloud backups of the model cache are out of scope for the model itself.

Local AI is a network and data-sovereignty property. It does not turn a laptop into a fortress. Use it together with full-disk encryption, screen lock, and a sane threat model.

Trust is verified, not declared. Five concrete checks confirm that a local Ollama setup behaves as documented. None of them require trusting the model or the client — only the network and the logs.

1. Confirm Ollama only listens on localhost

   On Linux or macOS, list active connections on the Ollama port. The address should always be 127.0.0.1 or ::1:

   netstat -an | grep :11434

   On Windows PowerShell, the equivalent is:

   Get-NetTCPConnection -LocalPort 11434 | Select-Object LocalAddress, State, OwningProcess

   Any line showing a non-loopback address means the runtime has been bound to a public interface — investigate before continuing.

2. Capture raw traffic with Wireshark

   Start a packet capture on the loopback interface, send a prompt through the client, and stop the capture after sixty seconds. A useful display filter to isolate non-local traffic is:

   http.request and ip.dst != 127.0.0.0/8 and ip.dst != 192.168.0.0/16

   The expected result is zero requests. Any non-zero count deserves a manual review of the destination host.

3. Restart Ollama with debug logging

   Stop the running server and start it with verbose logging. Every inbound request and outbound connection is written to the log:

   OLLAMA_DEBUG=1 ollama serve

   Cross-reference each outbound host against the documented list (registry, optional telemetry, user-initiated links). Anything else is a configuration drift.

4. Confirm telemetry is off

   Read the environment file or service unit that starts Ollama. The variable OLLAMA_TELEMETRY should be set to false, or the variable should be absent. Since the v0.3.x release, Ollama's telemetry is off by default; explicit false makes the intent visible in the service definition.

5. Verify the client does not exfiltrate

   In the client application, disable any cloud fallback in settings. Then repeat the Wireshark capture with a long, varied conversation. The capture should remain empty outside the loopback interface. Tools like Pulse respect this boundary. Pulse sends no prompts or responses to external servers, only the LLM traffic stays between the local client and the local Ollama instance.

Run all five checks once after install, and again after any Ollama version upgrade. The behaviour contract is stable across patch releases; major version bumps occasionally add a new outbound endpoint and should be re-verified.