The Balance of Power in Open Models Is Not a Leaderboard

The balance of power among open models got re-examined recently. Interconnects published a piece titled “The current balance of power in open models,” and it picked up 79 points and 25 comments on Hacker News. That’s not a viral number, but the audience for this kind of thread tends to be the people who actually make deployment decisions.

The phrase in the title is borrowed from international politics, and it fits better than a leaderboard would. The question isn’t who scores highest; it’s who holds which chips, and whose chips are hardest to replace.

“Open” is carrying too much weight

Grouping Llama, Mistral, Qwen, DeepSeek and Gemma under one label — “open models” — hides most of what matters. Downloadable weights are only the first layer. Below that sit questions about whether training data and training code are published, what the license permits and forbids, whether commercial use carries an active-user or revenue threshold, and whether a fine-tuned derivative can be redistributed.

Those terms decide whether a company can put a model inside its own product. A model with open weights and a restrictive license is a different object from one under Apache 2.0, at least for anyone shipping something. Skip that layer and the rest of the comparison turns into noise.

What benchmarks can’t measure

Benchmark scores are the easiest numbers to obtain and the easiest to misread. Contamination, prompt phrasing, a model’s uneven behavior across languages, and how well it holds up over long contexts can each move it several places on a given board. Worse, strong general question-answering doesn’t imply strong code generation, and strong code generation doesn’t imply reliable tool calling.

There’s also the question of what the model is actually being asked to do. A score on a general assistant eval says little about throughput under batch serving, or about how much prompt engineering a task needs before it works at all. Teams that have shipped both open and closed models usually end up tracking a handful of internal tasks rather than public boards, because the public boards don’t match their traffic.

For buyers, a second axis usually matters more: inference economics. Open weights mean you can host the model yourself; they also mean you own the ops, the GPUs and the latency. As closed API prices keep sliding, where that cost curve lands decides more procurements than a couple of benchmark points.

Who actually moves the needle

Three rough categories of actors are pushing. Large labs trade open weights for ecosystem, developer habits and recruiting leverage. Startups use openness as differentiation, since they can’t out-spend anyone on compute in the closed race. And state- or region-level investment is its own force — Chinese teams are especially active here, and the cadence of Qwen and DeepSeek releases is the obvious example.

None of these three groups is aligned with the others. A lab releasing weights to build goodwill is not the same as a startup trying to survive, and neither is the same as a national program measuring success by different criteria. The balance shifts when their interests happen to point the same direction.

The downstream layer gets overlooked: fine-tuning communities, inference frameworks, local deployment tooling. Whether a model gets used depends heavily on whether that tooling keeps up. Weights nobody can serve are barely different from weights that were never released.

Two shortcuts that keep showing up

Comment threads on this topic tend to collapse into two simplifications. One equates “weights are downloadable” with “open source,” ignoring licenses and training artifacts. The other treats a single model’s capability as the strength of the whole open ecosystem, ignoring tools, compute and talent.

Both distortions skew the read, and threads tend to reward whoever states a position first and most confidently rather than whoever has deployed the most models. Real shifts usually show up first in license language and toolchain support, not in leaderboard positions.

If you want to know which way the balance is tilting, waiting for the next benchmark release is the least useful option. Watch three things instead: whether licenses get looser, what gets released besides weights, and whether the surrounding tooling moves with it. When those three change, the balance has actually changed.