Language models with open weights — Meta's Llama, the Mistral models, Alibaba's Qwen, DeepSeek, Google's Gemma family — have made a leap over the past two years that many decision-makers have missed. For everyday tasks, the gap to the commercial frontier models has become so small that a different question moves to the front: why are we still sending office data to somebody else's service?
What "open" means here
Open weights mean the model file sits with you. It runs on your hardware, without an internet connection, without an account with any provider, without a per-request fee. Nobody logs what your staff ask. Nobody changes the pricing overnight or retires the model your processes depend on.
This is not ideology; it is an operating model with concrete properties:
- Data protection by architecture. What never leaves the house needs no data processing agreement, no third-country transfer assessment, no debate with your data protection officer about US jurisdictions.
- Predictable costs. Hardware once instead of ongoing API fees. With regular use, that pays for itself faster than most people expect.
- No shutdown risk. A model you have downloaded cannot be taken away from you. It will work the same way in five years as it does today.
Where open models are strong today
The most rewarding application for smaller businesses is not free-form chat but working on your own document base: a model combined with a local search index (RAG, retrieval-augmented generation) answers questions from contracts, manuals, project files and email archives — with source references, and without a single page leaving the building.
From my own practice: I built exactly such a system for a cooperative organisation. Embeddings, index and model run on their own hardware; the knowledge base stays entirely within their own infrastructure. Staff ask questions in plain language and get answers that point back to the source document.
Also solid: summaries, drafts, translations, triaging incoming mail, structured data extraction from documents. That covers most of what offices actually use AI for.
Where I honestly advise against it
Open models are not level in every discipline. On long, deeply nested reasoning tasks, demanding software architecture, or research that needs current world knowledge, the commercial frontier models remain ahead. If that is your workload, a tiered approach works well: the sensitive material locally, the uncritical rest with the strongest available service.
And: operating them takes competence. Model selection, quantisation, updates, access rights, backups — none of it is rocket science, but it is work someone has to own. "Free" refers to the licence, not to the operation.
The realistic way in
Do not start with a flagship project; start with an afternoon. Install an open model on a machine you already have, feed it real (uncritical) work tasks, and judge the quality yourself. After that you can decide seriously what the next step is — a dedicated machine in the office, a RAG system on your document base, or nothing at all for now. That, too, is a legitimate outcome.
