sparks
A backend that turns any LLM into typed decisions (yes/no, choice, score) from a single forward pass, with no generated text to parse.
- Year
- 2026
- Built with
- Go, Python, Docker, LLM
- Links
Most LLM “decisions” are generated text that you parse and hope is valid. sparks skips the text. It exposes one endpoint, POST /v1/decide: send a state plus named, typed questions, and get back one calibrated answer per question, each with a full probability distribution. Nothing is generated beyond the answer label, which cuts output tokens by about 99%.
Question types
| Type | How it’s scored | Answer |
|---|---|---|
noul |
Yes/No, scored in both orders and averaged | P(yes), from 0 to 1 |
choice |
Options labeled A..Z, rotated cyclically and averaged to cancel position bias | The picked option |
score |
An ordered rubric mapped to labels in one forward pass | The expected rubric level |
How it works
Instead of sampling text, sparks reads the log-probabilities of the answer-label tokens (A, B, C, …) from a single forward pass against any OpenAI-style /v1/chat/completions backend that returns logprobs. Each distribution is temperature-scaled with a per-question calibration, and confidence is the probability of the best label.
The request and response format follows the Typesafe primitives (instructions + criteria).
Constraints
A backend has to pass a 30-second check: one logprobs call must return a distribution. Reasoning models that hide logprobs, or think before emitting the label, can’t be used.