Most demonstrations of large language models follow the same pattern: give the model a prompt and wait for it to generate text.
But text generation is only one way to use a language model.
A small model can also be trained or adapted to examine a state, evaluate a limited set of possible actions and choose one of them. In that role, the model starts to look less like a chatbot and more like a decision engine.
We built a simple demonstration of this idea using a 2B Jev-style Decision Model running locally with EuLLM.
The task is deliberately familiar: Snake.
The interesting part is not the game. It is what happens between two moves.
A decision in around 8 milliseconds
At every step, the model receives the current game state and evaluates the available actions.
It does not need to write an essay or produce hundreds of tokens. It needs to answer a much narrower question: What should I do next?
On an NVIDIA RTX 5070 Ti, the model in our test makes that decision in approximately 8 milliseconds.
The entire inference runs locally through EuLLM. There is no call to an external model provider and no remote API involved in the decision loop.
The demo also includes a deterministic evaluator that independently calculates the best available action from the current state.
Across more than a thousand decisions during a run, approximately 95% of the model's choices matched the evaluator's best move.
That distinction matters. The 95% figure is not a generic "accuracy score" and it is not an LLM judge evaluating another LLM. It measures agreement between the model's decision and a deterministic reference implementation for this specific task.
Why use an LLM for decisions?
A natural question is why anyone would use a language model for a task that can be solved with conventional code.
For Snake itself, there is no compelling reason. Snake is simply a visual environment where people can immediately understand whether a decision was good or bad.
The interesting applications begin when the state is harder to represent with a fixed ruleset.
Imagine an enterprise agent that has retrieved documents, inspected a customer request and called two tools. It now has to determine whether to answer, retrieve more evidence, invoke another tool or escalate the task to a human. That is a decision problem.
A security system might need to decide whether an event should be ignored, correlated with previous activity, investigated or escalated.
A RAG pipeline may need to decide whether the evidence currently retrieved is sufficient to answer a question.
An industrial system may have to select the next action from telemetry, operating conditions and textual instructions.
The output in each case can be extremely small even when the input is complex. That changes the economics of inference.
Small models don't always need to behave like small chatbots
The industry tends to evaluate small models by asking how closely they can imitate much larger general-purpose models. That is useful, but it is not the only way to use them.
A small model can instead be specialized around a narrow decision space. If the required output is one action among five possibilities, generating several paragraphs of natural language is unnecessary overhead.
This is particularly relevant for local AI. A 2B model is small enough to run on widely available hardware while still being large enough to interpret substantially richer inputs than a traditional classifier.
The useful comparison is therefore not always small model versus large model, but sometimes specialized model versus remote general-purpose inference for every decision.
For applications making hundreds or thousands of decisions, latency and infrastructure requirements quickly become significant.
Where EuLLM fits
EuLLM is not a model. It is an open platform designed around local and European AI infrastructure.
The project consists of three complementary parts.
EuLLM Engine provides the runtime used to execute models locally.
EuLLM Forge focuses on model specialization, verticalization and teacher-student workflows.
EuLLM Hub is being developed as a European model registry and distribution layer, including model and compliance information.
The Snake experiment concerns primarily the Engine side: a small quantized decision model running locally and responding with very low latency.
But the longer-term combination is more interesting. A larger teacher model can help produce specialized smaller models through Forge. Those models can then be executed by Engine close to the applications that use them.
That is particularly attractive when inference must remain private, predictable and inexpensive.
Decision models and AI agents
One of the most obvious applications is agentic AI.
Today's agents frequently send relatively simple control decisions to large remote models:
- Should the agent search again?
- Which tool should it call?
- Does the current evidence answer the user's question?
- Should the task be escalated?
These decisions often require understanding context, but they do not necessarily require a very large generative model.
A small local decision model can potentially act as part of the control plane, leaving expensive generative inference for the situations where it is actually necessary.
The same architecture can be applied to RAG pipelines. Retrieval does not have to be a fixed sequence of search, reranking and generation. A model can inspect retrieval quality and determine whether another search is justified, whether another source should be queried or whether confidence is sufficient to continue.
Again, the output is a decision rather than a document.
Local execution changes the latency budget
An 8 ms decision is not important simply because the number is small. It changes what applications are practical.
If every step of an automated workflow requires a remote HTTP request to an external inference service, latency accumulates. Network variability and service availability become part of the control loop.
Local inference removes that round trip. It also means that the underlying information does not need to leave the machine running the application.
For enterprise environments, those properties can be as relevant as model quality itself.
Snake is the visualization, not the objective
We chose Snake because the decision loop is visible. You can watch the model inspect the state, choose an action and immediately see the consequence.
The same mechanism becomes less visually exciting — but potentially much more valuable — when the actions are things such as:
retrieve_morecall_toolapproveescalateroute_to_departmentinvestigate_eventcontinue_workflow
That is where small decision models become interesting.
The experiment suggests a broader direction for EuLLM: local AI infrastructure does not have to be designed only around chat completion.
Sometimes the most useful output an LLM can produce is not a sentence.
It is the next action.
EuLLM is open source: github.com/eullm/eullm
Project: eullm.eu