Typed decisions from a locally hosted model.

Three questions, one request

curl localhost:3000/v1/systemone \
  -H 'content-type: application/json' -d '{
  "state": "Ticket: Server room cooling failed, 31 degrees and rising.",
  "questions": [
    {"id": "urgent", "noul": "Is this urgent?"},
    {"id": "team", "choice": {"question": "Which team?",
                              "options": ["IT", "Facility", "HR"]}},
    {"id": "sev",  "score":  {"question": "How severe?",
       "levels": ["negligible", "minor", "moderate", "major", "critical"]}}
  ]}'

{"answers": {
   "urgent": {"type": "noul",   "noul": 0.9946, "label_mass": 0.999},
   "team":   {"type": "choice", "choice": "Facility", "index": 1,
              "confidence": 0.967, "label_mass": 0.999},
   "sev":    {"type": "score",  "score": 5, "expected_score": 4.90,
              "legend": "critical", "confidence": 0.807,
              "label_mass": 0.999}},
 "model": "gemma4:e2b-it-qat",
 "usage": {"input_tokens": 281, "questions": 3},
 "timing_ms": {"total": 85}}

Three questions, 85 ms, on a 4 GB model. label_mass is how much of the model’s own probability fell on the offered letters — the one sign that it was about to write something else.

How it works

A generative model answers by writing words, and then something has to parse the words back into a decision. cerno never lets it get that far.

1. Letter the options. Each question becomes a short prompt whose options are labelled A, B, C.
2. Ask for one token. The model generates exactly one, in one forward pass, whatever the number of options.
3. Read the distribution. Instead of taking the token, cerno reads P(A), P(B), P(C) from the logprobs.
4. Fold, floor, calibrate. Tokenizer variants of a label are combined, a label that fell out of the window is bounded and flagged, and temperature scaling makes the confidences mean something.

That is where the speed comes from — one token, not a sentence — and where the probabilities come from, since they are the model’s own.

What the measurements forced

Get started

The service and the terminal front end are on crates.io; installing needs a Rust toolchain and nothing else.

ollama pull gemma4:e2b-it-qat     # once, 4.3 GB
cargo install cerno-server cerno-tui

cerno-server                      # terminal 1: the service on 127.0.0.1:3000
cerno-tui                         # terminal 2: the terminal front end

The first request takes 5–10 s while Ollama loads the model; after that it is around 150 ms. http://localhost:3000/docs is the Swagger UI.

From code

Three clients, all published as cerno-sdk, written by hand against the same OpenAPI document and tested against the same conformance cases.

use cerno_sdk::Client;

let client = Client::new("http://localhost:3000")?;
let answers = client.systemone(text)
    .noul("urgent", "Is this urgent?")
    .choice("team", "Which team?", ["IT", "Facility"])
    .send().await?;

from cerno import Client

client = Client("http://localhost:3000")
answers = (client.systemone(text)
    .noul("urgent", "Is this urgent?")
    .choice("team", "Which team?", ["IT", "Facility"])
    .send())

import { Client } from "cerno-sdk";

const client = new Client("http://localhost:3000");
const answers = await client.systemone(text)
  .noul("urgent", "Is this urgent?")
  .choice("team", "Which team?", ["IT", "Facility"])
  .send();

The terminal front end

curl gives you the winner; cerno-tui shows you how close the runner-up was, which is the difference between “Facility” and “Facility, but it was nearly IT”. A run at a higher temperature next to one at 1.0 shows the calibration working directly.

Type a state, add questions with a, send with Ctrl+S. t and T step the calibration temperature, m cycles the models the service offers, ? lists the keys. The form is saved on exit and comes back on the next start.

Runs where your model runs

Every runtime gets the request fields that keep its distribution untruncated, and every host gets the same three primitives. Pick one with CERNO_HOST.

• Ollamadefault, native API
• vLLMOpenAI-compatible
• llama.cppllama-server
• LM StudioOpenAI-compatible
• OpenAI APIany compatible server
• Ollama System Oneauto for nimble, tev1

Which model to use is measured, not assumed: a 4 GB model matches the 26B reference’s accuracy and is better calibrated. See the model comparison.