Blog ·
Local vs cloud AI assistants: what actually leaves your machine
Every AI tool has a privacy page, and almost all of them say the same reassuring nothing. The useful question is narrower and answerable: when you use the thing, which bytes leave your computer, where do they go, and who can read them?
For an assistant that sees your meetings and your screen, that question deserves a concrete answer rather than a badge.
Three setups, three answers
1. Vendor-hosted (the default almost everywhere). Your audio, screenshots and prompts go to the vendor’s servers, which forward them to a model provider. The vendor can technically read everything, holds the API relationship, and sets the retention policy. This is not sinister — it is how a free tier gets paid for — but it is the arrangement you are agreeing to.
2. Bring your own key (BYOK). You supply your own API key, and requests go from your machine straight to your own account at OpenAI, Google or whoever. The vendor is out of the data path entirely. Your data still leaves your computer, and the model provider’s policies still apply — but there is one fewer party, and you can see and audit every request in your own provider dashboard.
3. Local models. A model running on your own hardware, through something like Ollama. Nothing leaves the machine at any point. No API key, no network request, no third party. This works offline on a plane, and the ceiling is whatever your laptop can run.
The honest trade-off of running locally
Local models are not a free win, and anyone who tells you otherwise is selling something. A model small enough to run comfortably on a laptop is meaningfully less capable than a frontier hosted model, it competes with your other applications for memory, and on battery it is noticeably slower.
What you get in exchange is categorical rather than incremental: there is no data-handling policy to read, because there is no data leaving. For a conversation covering an unreleased product, a medical matter, or a client under NDA, that categorical answer is often worth more than the extra capability.
Questions worth asking any assistant
- Can I use my own API key? If not, the vendor is permanently in your data path.
- Is there a fully local mode? Not "we do not train on your data" — actually offline.
- Where does conversation history live? On the device, or in a cloud account you did not think about?
- Are keys and history encrypted at rest? And where is the key kept?
- What gets written to logs? Prompts and screenshots showing up in a log file is a common and quiet failure.
Why the log question matters more than people expect
Encryption at rest gets the attention, but the more common leak is duller: an application that logs prompts, file paths or API keys in plain text for debugging, and then ships that behaviour to production. The data was never sent anywhere — it just sits on disk, unencrypted, in a folder nobody thinks to look at, and gets swept up by whatever backs your machine up.
It is worth asking directly, because the answer is usually either "we never log that" or an uncomfortable pause.
Pick per conversation, not once
The setup does not have to be permanent. A reasonable default is a hosted model for everyday work where speed matters, and a local model for the specific conversations where nothing should leave the machine. The important thing is that switching is possible at all — plenty of tools give you exactly one option.
Arguium supports all three: hosted, your own key, or a local model through Ollama, chosen per session. The full detail is here.