On-Device Text Classification With Tiny Models

Why Classification Is a Great Fit

Text classification — assigning input to one of a fixed set of categories — is among the most useful and most tractable tasks for a tiny model. The output space is small and known, success is easy to measure, and the task is narrow enough that a model with a few million parameters can do it well. It is a natural home for TinyLM.

Common Classification Jobs

The category list varies by application but the shape is the same: spam versus not-spam, positive versus negative sentiment, which support queue a message belongs in, which menu a typed request maps to, whether a field looks like an email or a phone number. Each is a small, closed set of labels — exactly what a tiny model handles best.

Running It On-Device

With TinyLM, the classifier runs in the browser on the device CPU. The input text goes into the WASM engine and a label comes out, with no network call. The model caches in IndexedDB after first load, so classification keeps working offline and the text being classified never leaves the device. You can see the local, instant behavior at https://ai.sprapp.com.

Specializing the Classifier

A generic model is rarely the best classifier for your specific labels. Fine-tune a base model like eeny or meeny with LoRA on examples of your categories, and accuracy on your task climbs. Because the model is tiny, gathering a few hundred or few thousand labeled examples and training is fast and cheap — and the resulting model is still small enough to ship everywhere.

Designing Around the Output

Treat the classifier's output as a label plus a confidence, not an oracle. For high-confidence predictions, act automatically; for low-confidence ones, fall back to a default, ask the user, or route to a larger model. This pattern lets you get the speed and privacy of on-device classification while handling the uncertain cases gracefully.

The Honest Accuracy Picture

A tiny classifier will not be perfect, and it will be worse on inputs unlike its training data. That is fine for many uses — a slightly imperfect, instant, private classifier is often more valuable than a marginally better one that costs money and sends data to a server. Match the accuracy bar to the stakes of the decision.

Where Classification Leads

Classification is a building block. Once you can reliably tag and route text on-device, you can build smarter interfaces: forms that adapt to input, search that understands intent, inboxes that pre-sort, and apps that respond differently to different kinds of text — all without connectivity or cloud cost.

The Takeaway

Of all the things you might ask a tiny model to do, classification is one of the safest bets. It plays to every tiny-model strength: narrow scope, small output space, easy validation, and a task that fits in a few million parameters. With TinyLM, you get that classification on-device, offline, private, and free to run.