Roadmap
Real-time status of every EuLLM component, the milestones we're hitting, and a full history of every release shipped.
Platform overview
EULLM Engine
v0.7.70Rust inference runtime. Multimodal vision + audio, drop-in Ollama replacement with OpenAI-compatible API and embedded chat UI on localhost:11435.
1,065 tok/s
Throughput
Vision+Audio
Multimodal
✓ tested
Windows
EULLM Forge
Model specialization pipeline. Components ready, end-to-end CLI integration in progress.
4B, 8B
Shipped sizes
GGUF
Export
Beta
Pipeline
EULLM Hub
Model registry with model and compliance cards. Self-hostable, with an EU-hosted region. Operational as prototype.
Prototype
Models
3 planned
Sectors
EU-hosted region
Hosting
Engine capabilities (v0.7.70)
Rust runtime · continuous batching · multimodal vision + audio · fully local on consumer GPUs
1,065 tok/s
Throughput
16 concurrent users, RTX 5070 Ti
Vision+Audio
Multimodal
OCR, scene, transcription
~2-4×
Quantized KV
context, Q4_0/Q5/Q8
--web
Web browsing
model-agnostic, any GGUF
What we're building
Phase 01: Foundation
Q1 2026
Core inference engine reaches production quality. Forge pipeline components built. Hub operational as prototype.
- Engine: standalone binaries (Linux x64, Windows x64)
- Multimodal vision + audio (Gemma 4)
- Continuous batching, 1,065 tok/s (16 users, RTX 5070 Ti)
- Quantized KV cache, Q4_0/Q5/Q8 (~2-4× context)
- OpenAI-compatible + Ollama drop-in API
- GPU: CUDA (tested), ROCm, Vulkan, Metal
- EU AI Act built-in audit logging
- Transparent web browsing (--web, model-agnostic)
- Interactive REPL: /temp, /maxtokens, /system
- Embedded chat UI, localhost:11435, ~29 KB in binary
- Forge: structural pruning + knowledge distillation
- Forge end-to-end pipeline CLI
- legal-it-4b and legal-it-8b published on Hugging Face
- Hub: Legal sector model (EU/Italian law)
- Next-gen legal model: administrative jurisprudence (Consiglio di Stato)
Phase 02: Ecosystem
Q2 2026
First production-ready Hub models go live. Forge stable CLI. Platform support expanded.
- Hub: Medical triage support model
- Hub: Finance & KYC compliance model
- AI Act compliance cards for all Hub models
- Forge: stable CLI + full documentation
- Multi-GPU inference
- Quantization wizard for consumer hardware
Phase 03: Enterprise
H2 2026
Enterprise-grade hardening: distributed inference, access control, Forge Studio visual UI.
- Multi-node distributed inference
- Kubernetes operator
- SSO / RBAC access control
- Forge Studio: visual fine-tuning UI
- Model versioning & rollback in Hub
- Certified EU data center partnerships
- SLA support tiers
Release history
- --moe-cache auto uses more of the leftover VRAM, up to ~69 tok/s on Qwen3.8-Flash-Next
- Multi-file GGUF models are now sized correctly for --fit
- New eullm finetune command: train a model's weights on your own text, locally
- Speculative decoding with the model's own MTP head: up to 61% faster code generation
- think: true returns reasoning in a separate field, matching Ollama's format
- New /v1/systemone endpoint: decisions as probabilities, without generating text
- Jev-Style decision models supported, including a 0.8B release
- Embeddings API: /api/embed and /v1/embeddings, any GGUF embedding model
- --embedding-model loads a reserved embedder alongside a generation model
- --keep-alive idle-unloads a model after a set duration
- Multimodal in the Chat UI: drop in an image or audio clip, fully local
- Vision + audio understanding stable (Gemma 4): OCR, scene description, transcription
- BOS token handling fix for multimodal prompts
- Multimodal vision launched: image OCR and scene description on consumer GPUs
- Audio understanding (experimental, CLI), transcription and in-content search
- Runs fully local, zero telemetry
- Math expression rendering in the Chat UI
- Quantized KV cache, Q4_0/Q5/Q8 for ~2-4× context on the same GPU
- Embedded chat UI on localhost:11435, ~29 KB in binary, zero CDN or external dependencies
- eullm -V now shows the active backend variant
- Standalone Windows binaries: CPU and CUDA
- Web tool calling: transparent URL fetching in conversation
- Legal-IT dataset preparation module
- GPU layer fitting improvements
- Drop-in Ollama replacement with continuous batching
- Quantized KV cache for larger context on 16 GB GPUs
- Transparent web browsing without function-call overhead
- EU AI Act audit logging built-in
- Interactive REPL: /temp, /maxtokens, /system commands
- Quantized KV cache quality/accuracy automatic recommendations
- Quantized KV cache math accuracy improvements
- 1% accuracy loss isolated to matrix operations only
- Default context window increased to 2 048 tokens
- Math accuracy benchmarking suite added
- Mixed KV cache type support
- Bug fixes
- Documentation updates
- Batch scheduler refinements
- Build pipeline stabilization
Shape the roadmap
Open an issue, vote on features, or contribute code. EuLLM is built in the open and every voice counts.