Reading the AI research for government buyers, part 1.
Every AI pitch I see now comes in one of two flavors. Trust our fortress model. Or ride the open source wave. Both are incomplete, and both are designed to get you to pick a side before you have asked the questions that matter.
I want to cut through that for the people I work with: finance, HR and IT leaders in cities, counties, states and universities who answer to a council, a board and a public records request.
The gap is small, and it moves.
One recent analysis found that open weight models have come within roughly four months of the best publicly evaluated closed frontier models. In my view that gap is volatile, and it will keep moving.
There is a second gap that matters more to a public agency: availability. A model you are not allowed to use, or that a provider can restrict, does nothing for you no matter how well it scores.
Put those together and the conclusion is not “go open” or “go closed.” The conclusion is that model choice is now a live option, and the worst thing you can do is sign a contract that gives it away.
Open weights is not open
I think of it as a spectrum, not a switch. At one end are fully open models, where the training data and recipe are published along with the weights. In the middle are open weight models, where you get the weights and not much else. At the other end are closed models you can only reach through someone else’s service.
That matters for a government buyer because “open” gets used in sales conversations as a stand in for transparent, auditable and portable. An open weight model may be none of those. You get a file you can run. You do not necessarily get an explanation of what went into it.
My own read is that companies tend to open the layer they do not sell. Some of the biggest hardware makers invest in open weights, and it is easy to see why: more models running means more chips sold. None of that is sinister. It is just a reminder that “free” is a strategy, not a gift.
Cheap tokens carry the volume. Frontier tokens keep the money.
The market is splitting in two. At the top sit closed frontier models used for hard reasoning. At the bottom sits a large pool of cheap open weight models doing high volume work.
Leading organizations have already stopped choosing. What I see is that they use both: open models for customization and control, closed models for top capability. The working pattern looks like this: a frontier model plans and reviews, a cheaper model does the mechanical steps, and a harness holds the context, tools, permissions, fallbacks and evaluations.
That harness is the part you should own.
Government never bought a model
Here is the reframe I keep coming back to. Agencies do not buy models. They buy a system of work bolted to finance, HR, case management, permitting and identity. The model is a component inside that system, and it will be swapped more than once over the life of a ten year contract.
So stop buying a brand. Buy a posture. Before you choose open, closed or hybrid, answer five questions.
1. Data. Where do prompts and documents live? Who is the controller? Can you purge them? Self hosting an open model gives you the most control. A private cloud deployment of a closed model still leaves weights and logs with the provider. A public API gives you the least.
2. Evals you own. Build a test set from your own tickets, ordinances, policies and the quirks of your own ERP. A vendor demo is not an evaluation. If you cannot measure the model on your work, you cannot compare it to the next one.
3. Exit. Can you swap to a different model without rewriting the engagement? Fine tunes and derivative models create switching costs, and some open licenses tighten as usage grows. Read the license like a contract, because it is one.
4. Integration gravity. What actually sticks you: the API, the knowledge base you built for retrieval, the agent tools wired into your ERP, or the fine tune? That is your lock in. Keep the harness vendor neutral so the model stays replaceable.
5. Audit and public records. When the council asks why the bot said that, can you show the retrieved source, the policy and the human who approved the output? Prompts and outputs may become public records. Plan for it now.
In government, the knowledge that makes a model useful is how your staff actually runs payables, budget and HR. If you rent a model and pour that knowledge into someone else’s loop, they learn your agency. Keep the corrections, the evals and the data in your own loop. The learning loop compounds to whoever holds it.
What the industry conversation leaves out
A public buyer has extra obligations that most industry analysis, written for investors and enterprises, will not focus on: security authorizations, criminal justice data rules, public records and discovery, civil rights and the right to contest an automated decision, a model inventory with pinned versions, and total cost of ownership beyond the price per token. Those are not footnotes. They decide whether a pilot survives its first audit.
Do this in Phase Zero
None of this should be decided in a vendor demo. It belongs in Phase Zero, before the RFP is written. Inventory your systems, your data quality, your identity setup and your process owners. Build a simple decision matrix: closed APIs for speed on low risk work, open or self hosted where sensitivity and exit matter, and hybrid as the default. Write the five questions into the requirements.
Open versus closed is a civil war among vendors. Your job is not to pick a side. Your job is to pick a posture: data, evals, exit, integration and audit. Keep the harness. Route the models.
I also publish this on Substack.
Read Chapter 1 of Start at Zero free
Planning your next move? Book time with us.