Small models are enough for the things you carry

The argument against running models on the device is always capability: a model that fits in your pocket cannot do what the big one does. True, and mostly irrelevant, because the work a carried device actually does is not the work that separates a frontier model from a small one. It is capture, extraction, classification, and deciding what deserves your attention — and small models are good at exactly those.

What a carried device is actually asked to do

Watch a real day and the tasks are repetitive and narrow.

Turn speech into text. Pull the four fields that matter out of a photographed document. Decide whether this message needs you now, later, or never. Attach what just happened to the right project. Answer a question you have asked in some form eleven times before.

None of that is reasoning at the edge of what is possible. It is perception and routing, done constantly, with low tolerance for delay — a workload profile that suits a small model far better than a large one.

The three properties that beat raw capability

Latency you can feel. A local model answers before you have finished looking up. A hosted one answers after a round trip that fails entirely when you are in a lift, a basement or on a train. For interactions that happen fifty times a day, the difference is not a preference; it changes which interactions you are willing to have.

Predictable cost. Inference on hardware you own is a fixed cost. Per-token pricing is a variable cost controlled by somebody else, on a volume that grows exactly as the device becomes more useful. That is a bad shape for a device you want people to use constantly.

Nothing leaves. The strongest privacy guarantee is architectural: the data never went anywhere. No policy, no region setting, no subprocessor list. On a device that hears your meetings and sees your documents, that is the argument.

Where small models genuinely fall down

Being honest about this matters more than the pitch.

Long-horizon reasoning over lots of context. Unusual, open-ended synthesis. Anything needing broad world knowledge the model was never given. Code of any real complexity. Push a small model into these and you get fluent, confident, wrong — the failure mode that costs trust fastest.

The answer is not to pretend otherwise. It is to route.

Routing is the actual design problem

A carried device should treat model choice the way a good operating system treats memory: automatically, with a policy you can inspect.

The local model handles the constant work. Anything bigger goes to KyberBot, the assistant that lives on the device, which decides where it runs: on your home node, on hardware you control in your own rack, or in KyberFusion, the cloud built to belong to the people who use it. The policy decides what may leave the device to get there. A rented frontier model, if used at all, is for work that is genuinely worth it and contains nothing you would mind quoting back.

The decision belongs in that one policy rather than in each application, because “may this leave the device” is exactly the kind of question that should have one answer in one place.

Why this gets easier every year

Two curves are moving in the right direction at once. Small models keep getting better at the narrow tasks, and the accelerators that fit in a carried device keep getting cheaper per unit of work.

The interesting consequence is not that the pocket eventually matches the datacentre — it will not. It is that the crossover point, where local is good enough for a given task, keeps moving up through the task list. Designing a device on the assumption that the crossover keeps rising is a very different bet from designing one that assumes a permanent connection to somebody else’s computer.