The Decision Model
Some DocsGPT features need a quick, bounded judgement rather than generated text: is this message a prompt injection, does this chunk answer the question, did this conversation end with the user’s goal met. They ask the decision model, `arc53/arc-decide-0.4b-v0.2`, a small classifier that reads a state (a message, a chunk, a conversation) and answers typed questions with calibrated probabilities. It is not a chat model and never writes text, and it is unrelated to the decision journal of guardrails.
It runs on CPU with ONNX Runtime, which DocsGPT already ships for embeddings, so it adds no dependency; it can also run on a Mac’s GPU or, as `arc53/arc-decide-0.6b-v0.1`, on an NVIDIA GPU with vLLM. It is MIT-licensed, built on Qwen3 (Apache-2.0).
Everything ships off[](https://docs.docsgpt.cloud/Models/decision-model#everything-ships-off)
Every feature that uses the decision model is off by default, and nothing is downloaded or loaded while they all are. The operator turns each on with one switch in `.env`, `DECISION_INJECTION_ENABLED`, `DECISION_RELEVANCE_ENABLED` or `DECISION_ANALYTICS_ENABLED`; what then uses the model (which agents, sources, stages, thresholds and topics) is set in the UI. The first switch turned on downloads the model (about 0.5 GB) into `DECISION_CACHE_DIR`, and the API loads it when it starts. Until it is ready, features carry on without it. The switches are in Turning features on.
Two settings apply to the model as a whole:
``` DECISION_MODE=none # turn it off everywhere, whatever the feature settings say DECISION_PRELOAD=true # download, load and time it when the API starts, even with every feature off ```
Where it runs:
| Setup | Where the model runs | | --- | --- |
| Every install by default: Docker (`docsgpt up`, every Compose file), Kubernetes, `docsgpt up --native`, pip, a checkout | Inside the API process; Celery workers call it there and never load it. |
| Several API replicas or workers, or the model’s memory kept apart from the API’s | The `decider`, a container of its own that the API and the worker call. It is off until you turn it on. | | A Mac, fast | A native `decide-server` on the Mac’s GPU: see On a Mac’s GPU. | | An NVIDIA GPU | vLLM: see On an NVIDIA GPU. |
If a call cannot be answered (the model is off, loading, busy or unreachable), the feature falls back to what it does without it, so a missing model never fails a request.
Celery workers[](https://docs.docsgpt.cloud/Models/decision-model#celery-workers)
A Celery worker never loads the model: it sends its calls to the API process’s model. It reaches the API where it already hands over finished indexes, `WORKER_API_URL` or else `API_URL` (`http://backend:7091` in Compose, `http://docsgpt-api-service` in Kubernetes), at `/api/internal/decisions/v1/decide` (and `/v1/decide/batch`, `/v1/models`, `/v1/load`), with `INTERNAL_KEY`. The worker keeps only the 11 MB tokenizer, and its first call starts the API’s load as the API’s own first call would. A worker without `INTERNAL_KEY` gets no answers. With `DECISION_MODE=none` or `api`, the API answers these routes with 503 and loads nothing; in API mode (the decider included) the worker calls the decision server itself.
Turning features on[](https://docs.docsgpt.cloud/Models/decision-model#turning-features-on)
Each feature has its own switch, so you can turn them on one at a time:
| Switch (`.env`) | Default | What it lets use the model | | --- | --- | --- |
| `DECISION_INJECTION_ENABLED` | `false` | Prompt-injection scoring: agents’ and the floor’s `injection_model` guardrail controls (Admin → Guardrails), chat attachments once uploaded, and MCP tool descriptions while Admin → Guardrails warns on them |
| `DECISION_RELEVANCE_ENABLED` | `false` | Relevance of retrieved chunks, for the sources whose **Relevance** setting is on (Reorder, or Reorder and filter), or the admin’s defaults for new sources |
| `DECISION_ANALYTICS_ENABLED` | `false` | Labelling stored conversations in the background for Insights and docs gaps: conversations without an agent, and those of every agent whose **Collect insights** is on (the default) |
The injection check is described in Agent Guardrails, reviewing a source’s flagged chunks in Per-Source Configuration, relevance in Per-Source Configuration, and the labels in Analytics and Logs. Each is one switch, not a mode: there is no `monitor` or `auto` for it. Neither an agent nor a source can turn the model on by itself: while a switch is `false`, its settings do nothing and the UI hides them or marks them unavailable. While any is `true`, the API (or the decider) loads the model when it starts, so the first call does not fall back while it loads.
On a CPU, relevance is slow for an answer: about 27 s for 20 whole chunks on 4 vCPU, about 14 s with the 256-token window DocsGPT scores there, against the 2 s `DECISION_RELEVANCE_TIMEOUT`, after which the answer keeps the retriever’s order. The admin **Defaults** panel shows the predicted seconds per answer on your server once the model has loaded and warns when they exceed the timeout. On a CPU-only install, keep relevance for a few sources and fewer candidates, raise `DECISION_RELEVANCE_TIMEOUT` if a slower answer is acceptable, or run the model on a GPU or an API-mode server.
Prompt injection on a CPU[](https://docs.docsgpt.cloud/Models/decision-model#prompt-injection-on-a-cpu)
On a 4 vCPU server, `DECISION_INJECTION_ENABLED` costs:
- **A user message:** about 0.2 s per check, within the default 2 s stage deadline. A long message is read in 512-token windows of about 0.7 s each, end and start first, so only two or three windows fit the deadline.
- **A tool result:** the same, during the turn; a long result is mostly _not evaluated_ at the deadline.
- **Retrieved documents and attached files:** a DocsGPT-size chunk (1,150 tokens, three windows) takes about 1.7 s of compute, measured on 4 vCPU (AMD Zen 4) with the release model, so on a CPU `GUARDRAILS_INJECTION_LIVE_ITEMS=auto` scores none of them during the turn. They come from the score cache: a chunk a turn retrieves without a cached score is _not evaluated_ in that turn and scored in the background right after, so the next turn that retrieves it reads its score. The first turns after an agent adds a retrieval control are mostly unscored. Background scoring keeps the cores busy at most `DECISION_BACKGROUND_SHARE` (a quarter) of the time.
- **Every chat attachment**, scored in the background once uploaded: up to 64 windows (`GUARDRAILS_INJECTION_ATTACHMENT_MAX_WINDOWS`), about 30,000 tokens.
Admin → Guardrails and Admin → Defaults show the measured figure for your server once the model has loaded: _“Checking a user message adds about N s on this server”_. Background scoring runs where the model does, in the API process by default and beside its requests: the decider keeps that work out of the API process. To score retrieved documents during the turn, run the model on a GPU-backed API-mode server: it scores a chunk in a fraction of that time, so `auto` scores up to four of them within the stage deadline. On a CPU-only install, keep blocking controls failing open.
Conversation analytics[](https://docs.docsgpt.cloud/Models/decision-model#conversation-analytics)
`DECISION_ANALYTICS_ENABLED=true` lets the model label stored conversations for Insights and docs gaps. Which agents are labelled, and with which topics, is each agent’s own insights settings; the `ANALYTICS_*` settings in the Settings Reference only tune the sweep and never appear in the UI.
Nothing runs in the request path. A Celery beat task, `analytics-sweep`, runs every 5 minutes on the worker’s `analytics` queue: it reads finished messages, builds the states and writes the labels, and sends its model calls where the worker’s go (the API process, the decider or the decision server) at background priority. The worker never loads the model. See Background Jobs for the queue and the backfill command.
- **First use.** Turning it on downloads the model (about 0.5 GB), as any first feature does, and the API or the decider loads it when it starts. Until the model is ready, the sweep’s calls come back unavailable, it writes nothing and the next sweep tries again.
- **Cost on a CPU.** An analytics turn (a full 2,048-token state with the eight analytics questions) takes about 4.8 s on 4 vCPU (AMD Zen 4) with `arc53/arc-decide-0.4b-v0.2`. A typical turn is about 3,000 tokens over its states (the answer, the reply to it, the message, its share of the conversation), about 5 s of compute. At the default `DECISION_BACKGROUND_SHARE` (a quarter of the time), a 4-vCPU server labels about 190 turns an hour, fewer under chat load. A 30-day backfill of 10,000 turns is about 13 CPU-hours, about 53 hours of wall time.
- **On a GPU.** In API mode on an NVIDIA GPU (vLLM or `arcd serve`), the same backfill takes minutes: 290 short one-question requests per second were measured there; analytics-sized states were not.
- **Under chat load.** Chat comes first. A call someone waits on waits at most for one 256-token background pass, and background work keeps the cores busy at most a quarter of the time. Under steady chat load the sweep’s calls time out, the sweep stops early, and labels fall behind; nothing is lost, and the next sweep picks up where it left off. Messages take at most 70% of a sweep’s time while there is a backlog, so goal met and resolutions keep up with them. A conversation’s own call asks goal met and every topic that can overlap over one read of the conversation, about 5 s on 4 vCPU with 20 of them, so on a CPU-only server keep them few, or run the model on a GPU.
- **Failures.** A call the model answers none of writes nothing, stops the sweep and is tried again by the next one. A message (or conversation) the model fails on while answering the rest of the call is tried again by later sweeps, after the messages it has not failed on; after three failures, or at once when the model reports it as unreadable, it is counted as failed and left until the model or the questions change. A call about a single message that fails counts as one of those failures only when the model answers other calls in the same sweep, so a model that fails everything (for example, a misconfigured server) marks nothing as failed, and the messages are labelled once it is fixed. A few messages the model always fails on never hold up the others, and a message that keeps failing alone counts one failure for each day it does, even with nothing else to answer, so a model that stays broken for days marks the messages it tried as failed.
The sweep reads the stored conversation, the same text Logs shows, and its labels hold no text. In API mode that text goes to the decision server, as every feature’s does.
Offline and air-gapped installs[](https://docs.docsgpt.cloud/Models/decision-model#offline-and-air-gapped-installs)
The files are fetched from Hugging Face at a pinned revision, as real files under `DECISION_CACHE_DIR` (`models/decision` in the data home). To fetch them ahead of time:
`docsgpt decisions prefetch`
In a Docker stack, run it in the backend, which keeps the files in the `decision_models` volume it shares with the worker and the decider: `docker compose exec backend python -m docsgpt decisions prefetch` from the stack’s folder. In Kubernetes the API keeps the model in an `emptyDir` and downloads it again when its pod is recreated; for a cluster without internet access, mount a copy and set `DECISION_MODEL_PATH`, or use the decider, whose volume claim keeps it. For a machine with no internet access, run it on a connected machine and copy the directory, or point `DECISION_MODEL_PATH` at a copy of the model: nothing is downloaded then. `--runtime torch` fetches the weights a Mac’s GPU runs instead. `HF_HUB_OFFLINE=1` is honoured. A missing file or a failed download leaves the model unavailable, logs the fix, and is retried after `DECISION_LOAD_RETRY_SECONDS`. See Air-Gapped Deployment.
Running in the API process or the stack’s decider, the model sends no text anywhere: the text goes no further than the stack’s own network, and the only outbound call is that one-time download. API mode sends text to the server you point it at.
The decider[](https://docs.docsgpt.cloud/Models/decision-model#the-decider)
By default the model runs inside the API process. The `decider` runs it in a container of its own instead: the same `arc53/docsgpt` image running `docsgpt decide-server`, which the API and the worker call. Every Compose file and the Kubernetes manifests include it, turned off. Turn it on when:
- **You run several API replicas or many workers.** Each API process loads its own copy of the model (up to 1 GB of memory each), and every worker’s decisions run in an API process, beside its requests; one decider serves them all from one copy.
- **You want the model’s memory kept apart from the API’s.** The API process then stays at its usual size, and a decider that runs out of memory or is killed takes no requests down with it.
- **The API process recycles.** The image runs `gunicorn -w 1 --max-requests 5000`, so gunicorn replaces the API process after about 5,000 requests. The model inside it loads again each time, and decisions fail open for those seconds. The decider is not recycled.
To turn it on:
- **Compose.** Add three lines to `.env`, then run your `docker compose ... up -d` again:
``` COMPOSE_PROFILES=decider DECISION_BASE_URL=http://decider:7095 DECISION_KEY=<a long random string, e.g. from openssl rand -hex 32> ```
The first starts the container, the second points the backend and the worker at it, and the third is the key they send it; the decider refuses to start without one and logs why. A stack that already uses the `https` profile lists both: `COMPOSE_PROFILES=https,decider`. To turn it off, remove the first two lines, run `up -d` again and remove the container with the same `--env-file` and `-f` options you start the stack with, for example `docker compose --env-file .env -f deployment/docker-compose-hub.yaml --profile decider rm --stop --force decider` from a checkout.
- **`docsgpt up --decider`** writes the first two lines to the stack’s `.env`, and a random `DECISION_KEY` when it has none; every later `docsgpt up` keeps them. `docsgpt up --no-decider` removes the first two and the container, and keeps the key.
- **Kubernetes.** Uncomment `components: - optional-decider` in `deployment/k8s/kustomization.yaml` and apply it again with `kubectl apply -k deployment/k8s/`. The component adds the `docsgpt-decider` Deployment, its Service and the `docsgpt-decision-models` volume claim, and sets `DECISION_BASE_URL` on the API and the worker; see Kubernetes.
The decider idles at about 100 MB and loads nothing until a feature calls. The API and the worker call it at `DECISION_BASE_URL` (`http://decider:7095` in Compose, `http://docsgpt-decider-service:7095` in Kubernetes) with `DECISION_KEY`: in Compose the one `.env` sets, in Kubernetes `INTERNAL_KEY`, which the component sets for all three and which stays inside the cluster. The first feature turned on downloads the model into the decider’s volume (`decision_models` in Compose, the `docsgpt-decision-models` claim in Kubernetes), where it survives upgrades; the API and the worker then keep only the 11 MB tokenizer.
- **Memory and CPU.** Compose limits it to 2 GB (`DECISION_MEMORY` in `.env`), Kubernetes to 2Gi and 4 CPUs; the model peaks under 1 GB. Compose sets no CPU limit: interactive calls are short, and background work takes at most `DECISION_BACKGROUND_SHARE` of the cores.
- **Health.** The API never waits for it. A decider that is down, restarting or unreachable makes features carry on without the model (see the failure handling under API mode). `docsgpt up` and Kubernetes restart it; the checkout Compose files restart none of their services, so there it stays down until the next `docker compose up`. Its own `GET /health` answers while the process runs, loaded or not.
- **Routes.**`POST /v1/decide`, `POST /v1/decide/batch`, `GET /v1/models` (the model, its revision and calibration, its runtime, whether it is loaded and how fast it measured), `POST /v1/load` and `GET /health`. A request’s `X-Decision-Priority` (`hot` or `background`) picks its lane, and states not started within `X-Decision-Deadline-Ms` come back as `{"error": "timeout"}`. Its logs hold counts and timings, never text.
`docsgpt decide-server` runs the same server anywhere:
``` docsgpt decide-server # 127.0.0.1:7095, ONNX on the CPU docsgpt decide-server --host 0.0.0.0 # every interface: refuses to start without DECISION_KEY ```
A backend run on the host against `deployment/docker-compose-dev.yaml` can use one too: start it with `--profile decider`, and set `DECISION_KEY` and `DECISION_BASE_URL=http://127.0.0.1:7095` in `.env`.
On a Mac’s GPU[](https://docs.docsgpt.cloud/Models/decision-model#on-a-macs-gpu)
Docker on a Mac runs Linux with no access to the GPU, so the decider there runs on the CPU, where an injection check takes about 2.2 s on an M1, too slow for most features. Natively, PyTorch on the Mac’s GPU takes about 0.14 s, with about 1.2 GB of GPU memory. Run the server on the Mac itself:
``` uv tool install "docsgpt[decisions-torch]" # adds PyTorch and transformers; no image ever has them docsgpt decisions prefetch --runtime torch # the weights, 1.07 GiB docsgpt env set DECISION_BASE_URL=http://host.docker.internal:7095 DECISION_KEY=$(openssl rand -hex 32) docsgpt up # recreate the containers with the new address and key docsgpt decide-server --device mps # 127.0.0.1:7095, in the foreground ```
The containers reach the server through `host.docker.internal`. Installed this way, `docsgpt` uses the same data home as `docsgpt up` (`~/.docsgpt/server`), so the server reads the stack’s `.env`: its feature settings and the `DECISION_KEY` the containers send. For a stack in another folder, set `DOCSGPT_HOME` to it for these commands. For a native install (`docsgpt up --native`), set `DECISION_BASE_URL=http://127.0.0.1:7095` instead. If the stack ran the decider, `docsgpt up --no-decider` removes it and keeps this address. Starting the server with the Mac (a launchd unit) is not set up for you yet: keep it running in a terminal, or start it again after a restart.
PyTorch never loads in the API or the worker: it and faiss, the default vector store, each bring an OpenMP runtime, and macOS aborts a process that loads both. `DECISION_DEVICE=mps` is read by `decide-server` only. `docsgpt doctor` says where the decision model runs and, on a Mac with a feature on and the model in Docker, suggests this setup.
On an NVIDIA GPU[](https://docs.docsgpt.cloud/Models/decision-model#on-an-nvidia-gpu)
`deployment/optional/docker-compose.optional.decide-vllm.yaml` adds stock vLLM (`vllm/vllm-openai`, pinned) serving `arc53/arc-decide-0.6b-v0.1` at its pinned revision, and points the backend and the worker at it (`DECISION_PROTOCOL=vllm`):
``` docker compose --env-file .env -f deployment/docker-compose-hub.yaml \ -f deployment/optional/docker-compose.optional.decide-vllm.yaml up -d ```
It needs an amd64 host with the NVIDIA Container Toolkit. vLLM takes `DECISION_VLLM_GPU_SHARE` of the card’s memory (0.2 by default, about 4.8 GB of 24 GB), which leaves room for an LLM such as Ollama on the same card; if vLLM fails to start on a smaller card, raise it. Until vLLM answers, features carry on without the model. vLLM checks the same `DECISION_KEY` and logs no prompts; set it in `.env`, since without it every `docker compose` command that names the overlay stops with an error rather than start vLLM with no key. With `docsgpt up`, which runs its own Compose file only, run vLLM beside the stack as in API mode and set `DECISION_BASE_URL` and `DECISION_PROTOCOL=vllm` in its `.env`.
API mode[](https://docs.docsgpt.cloud/Models/decision-model#api-mode)
Instead of loading the model in the API process, DocsGPT can call a decision server. Set its address in `.env`; with `DECISION_BASE_URL` set, `DECISION_MODE` defaults to `api`, and both the API and the Celery worker call the server:
``` DECISION_BASE_URL=http://127.0.0.1:7095 # the server's root, without /v1 DECISION_KEY= # bearer key the server expects; unset sends none ```
Two kinds of server work:
- **A `/v1/decide` server** (the default `DECISION_PROTOCOL=decide`): `POST /v1/decide`, `POST /v1/decide/batch` and `GET /v1/models`, as `arcd serve`, the model’s research server, answers them. The server renders, cuts and calibrates; DocsGPT checks every answer against the question it asked. A server that does not report which model it runs is taken to run `arc53/arc-decide-0.6b-v0.1`; set `DECISION_MODEL` if it runs another.
- **Stock vLLM** (`DECISION_PROTOCOL=vllm`) on an NVIDIA GPU, serving `arc53/arc-decide-0.6b-v0.1` at its pinned revision. DocsGPT builds the prompt, cuts the state and applies the model’s calibration itself, and reads one label token per question:
``` vllm serve arc53/arc-decide-0.6b-v0.1 --revision a03cd6bc45fa7b75e05489454e2084bd3516db17 \ --logprobs-mode processed_logprobs --max-logprobs 16 --enable-prefix-caching --max-model-len 4096 \ --gpu-memory-utilization 0.2 ```
``` DECISION_BASE_URL=http://127.0.0.1:8000 DECISION_PROTOCOL=vllm ```
The two `logprobs` flags are required: without them the answers come back empty and every call fails open. `--gpu-memory-utilization 0.2` (about 4.8 GB of a 24 GB card) leaves room for an LLM on the same card. To let interactive decisions overtake background ones, start vLLM with `--scheduling-policy priority` and set `DECISION_VLLM_PRIORITY=true`.
In API mode DocsGPT downloads only the model’s tokenizer (11 MB, at the same pinned revision) into `DECISION_CACHE_DIR`, so it counts tokens and reports truncation exactly as the server cuts a state. On an air-gapped host, `docsgpt decisions prefetch` fetches it with the rest of the model.
**The server receives the text** of every message, chunk or conversation a feature asks about. Point `DECISION_BASE_URL` only at a server you trust with that data, as you would an LLM provider: it acts as a data processor for your users’ content. DocsGPT sends `DECISION_KEY` as a bearer key when it is set, and no key otherwise; it never sends `INTERNAL_KEY`, which guards its own internal API, to the decision server.
The Compose files pass `DECISION_KEY` from `.env` like any other setting and never set it from `INTERNAL_KEY`, so a server outside the stack gets no key until you set one. In Kubernetes, only the in-cluster `optional-decider` component sets `DECISION_KEY` from `INTERNAL_KEY`, on the API, the worker and the decider; DocsGPT itself never falls back to it.
When the server fails, features carry on without it:
- Each request’s timeout is the time left before the feature’s deadline. A connection error or a 502, 503 or 504 is retried once (`DECISION_API_RETRIES`), and only while 30% of that time remains; timeouts and other errors are not retried.
- After 3 consecutive failures (`DECISION_BREAKER_FAILURES`), calls are skipped for 30 seconds (`DECISION_BREAKER_COOLDOWN`); then one is let through, and its success resumes them.
- A server that refuses the key (HTTP 401 or 403) is logged once, and calls stop until a later check of the server succeeds. Set `DECISION_KEY` to the key the server expects.
- A server that answers one bad item with an error for the whole batch, as `arcd serve` does, has that batch sent again one item at a time.
States are sent in batches of `DECISION_BATCH_SIZE` (32). Up to `DECISION_API_CONCURRENCY` (4) requests are in flight for decisions someone waits on, and `DECISION_BACKGROUND_CONCURRENCY` (1) for background work, which never takes an interactive slot. DocsGPT reads the server’s `GET /v1/models` before the first call and every `DECISION_PROBE_SECONDS` (30).
For Measured speed, a server that reports its own timing on `/v1/models` is predicted from it, plus the round trip of each request. Any other server, `arcd serve` and vLLM included, is timed by sending the same few constant calls through it, when it is first used and again whenever it comes back after failing.
Memory and speed on CPU[](https://docs.docsgpt.cloud/Models/decision-model#memory-and-speed-on-cpu)
Measured on a 4 vCPU x86 server:
| | | | --- | --- | | Memory, loaded and idle | ~0.55 GB | | Memory, steady state | under 1 GB | | An injection check on a short message | ~0.15 s | | An analytics turn (a 2,048-token conversation, 8 questions) | ~4.8 s |
Before it loads, DocsGPT checks that the process has room: below `DECISION_MIN_FREE_MB` (1,200 MB by default) the load is refused and features carry on without the model. The model uses the cores the process may use (its CPU affinity, container quota and physical cores); set `DECISION_THREADS` to fix the count. One pass runs at a time: work someone waits on goes first, and background work (analytics, scans) keeps the CPU busy for at most `DECISION_BACKGROUND_SHARE` (a quarter) of the time.
Measured speed[](https://docs.docsgpt.cloud/Models/decision-model#measured-speed)
When the model loads, DocsGPT times a few fixed passes on this machine (a second or two on 4 vCPU) and predicts from them how long each feature’s calls take. The predictions are the costs the admin pages show, what `docsgpt doctor` and `GET /api/admin/decisions` compare with each feature’s budget, and what `GUARDRAILS_INJECTION_LIVE_ITEMS=auto` sizes itself by. They never turn a feature on or off: each follows its switch.
The predictions stay current. Real calls are compared with them, and when they drift outside 0.67 to 1.5 times the prediction the timing runs again (at most every 10 minutes). The CPU budget is re-read every minute, and a change rebuilds the model with the new thread count.
What is recorded[](https://docs.docsgpt.cloud/Models/decision-model#what-is-recorded)
Nothing a feature sends the model is stored. Every call leaves numbers only:
- **On the answer.** The assistant message’s metadata holds `decisions`: per call, the feature, its status, how long it waited, whether it acted on the answer (`applied` is `false` when the answer was only recorded, as by an injection control that flags or an agent in monitor mode) and the numbers it decided on, such as the injection check’s probability and thresholds, or at `retrieval` and `attachment` how many items were checked, read from the cache, scored during the turn, queued for later and withheld. A message keeps at most 64 entries and counts the rest in `dropped`. A turn that pauses for a tool approval keeps its entries, and the resumed turn adds its own in the next `round`.
- **In the trace.** Each call is a `decision` step in the request’s execution trace, with its states, answers, tokens and timings, and in the exported OpenTelemetry spans and metrics.
- **As daily counts.** The `decision_usage` table counts calls, states, answers and time per day, feature, mode, model version and status. Each process writes its counts once a minute, and a daily task keeps `DECISION_USAGE_RETENTION_DAYS` (90). Texts the prompt-injection check answered from its score cache are counted under status `cached`, beside those the model scored. They are the denominators for a flag-only rollout: how many messages and documents the injection check read, next to the guardrail events it raised. The feature labels are listed under Observability.
- **As conversation labels.** The analytics sweep’s answers become rows of `conversation_labels`: option names and probabilities, never text, and never on the answer or in a trace. Its calls are counted in `decision_usage` under `analytics_turn` (turns, replies and messages) and `analytics_conversation` (goal met and topics that can overlap), and each sweep logs one `analytics_sweep` line with its counts and milliseconds.
``` {"v": 1, "model": {"id": "arc53/arc-decide-0.4b-v0.2", "version": "arc53/arc-decide-0.4b-v0.2/p78cfb276f93a/cc3bc72ff6907", "runtime": "onnx-int8", "mode": "local"}, "entries": [{"feature": "injection_input", "round": 0, "status": "ok", "applied": false, "ms": 212.4, "outcome": {"p": 0.03, "flag_threshold": 0.5, "threshold": 0.9, "windows": [1, 1]}}, {"feature": "injection_retrieval", "round": 0, "status": "skipped", "applied": true, "ms": null, "outcome": {"items": 6, "cache_hits": 5, "live": 0, "lazy": 1, "max_p": 0.97, "flag_threshold": 0.5, "threshold": 0.9, "withheld": 1}}], "dropped": 0} ```
Decision calls never count toward users’ token limits: the model runs on your hardware and has no per-token price.
Checking it[](https://docs.docsgpt.cloud/Models/decision-model#checking-it)
`docsgpt decisions status` prints the settings, each feature’s mode, the pinned revision and whether the files are downloaded, without loading anything. In API mode it asks the decision server once, so the model, load status and speed it shows are the server’s (`"server": "no answer"` when it cannot reach it). `docsgpt doctor` adds a `decisions` line: where the model runs, what a decision server reports (its runtime, model version and load status), whether the files are here and the memory a load would have, and each feature’s mode. Relevance is listed as `rag_rerank on` or `off`, prompt-injection scanning as `injection_input on` or `off`, and conversation analytics as `analytics_turn on` or `off`; when the server reports its timing, a feature that is on shows its predicted call against its budget, for `rag_rerank` the `DECISION_RELEVANCE_TIMEOUT` (`rag_rerank on at 1.20s of 2s`), for `injection_input` the default 2 s guardrail stage deadline, for `analytics_turn` the `DECISION_BACKGROUND_TIMEOUT` (120 s).
A running server reports what its API process sees at `GET /api/admin/decisions` (admins only):
``` { "success": true, "available": true, "mode": "local", "status": "ready", "reason": null, "model": "arc53/arc-decide-0.4b-v0.2", "version": "arc53/arc-decide-0.4b-v0.2/p78cfb276f93a/cc3bc72ff6907", "runtime": "onnx-int8", "speed": {"threads": 4, "measured_at": "2026-10-15T09:12:03+00:00", "drift": 1.02, "guard_s": 0.152, "relevance_s": 1.41, "analytics_s": 4.1}, "features": { "injection_input": {"mode": "on", "effective": "on", "predicted_s": 0.15, "budget_s": 2.0, "reason": "configured"}, "rag_rerank": {"mode": "on", "effective": "on", "predicted_s": 14.2, "budget_s": 2.0, "reason": "configured"}, "analytics_turn": {"mode": "on", "effective": "on", "predicted_s": 4.1, "budget_s": 120.0, "reason": "configured"} } } ```
- `status` is `off` (`DECISION_MODE=none`), `idle` (not loaded), `loading`, `ready` or `unavailable`, and in API mode `degraded` while calls to a failing server are skipped. `reason` says why: `memory`, `download`, `offline`, `wrong_model` or `load_failed` when unavailable; in API mode also `circuit_open`, `auth` (the server refused the key) and `no_decider` (`DECISION_MODE=api` without `DECISION_BASE_URL`, or a worker without `INTERNAL_KEY`).
- `speed` is this machine’s timing: the predicted seconds of an injection check, of one relevance check on a whole 1,220-token chunk and of an analytics turn. In API mode they include the transport, and `speed` adds `round_trip_ms` and `measured_by` (`server` when the server reported its own timing, `client` when DocsGPT timed it).
- `runtime` is `onnx-int8` here; in API mode it is `vllm`, the runtime a server reports (`onnx-int8` for the decider, `torch-mps-fp16` on a Mac’s GPU, `torch-cuda-bf16` on an NVIDIA one), or `remote`. A worker calling the API process reports `api-process:onnx-int8` (`api-process` before its first call).
- `features` has, per feature, its configured `mode`, what it does now as `effective` (`off` or `on`), `predicted_s`, `budget_s` and `reason`. `rag_rerank` is `on` while `DECISION_RELEVANCE_ENABLED` is set, predicted for one call over 20 chunks (best-window passages where the model runs on a CPU, whole chunks elsewhere), and its `budget_s` is `DECISION_RELEVANCE_TIMEOUT`. `injection_input` is `on` while `DECISION_INJECTION_ENABLED` is set, predicted for one check on a user message, and its `budget_s` is the default guardrail stage deadline (2 s). `analytics_turn` is `on` while `DECISION_ANALYTICS_ENABLED` is set, predicted for one analytics turn, and its `budget_s` is `DECISION_BACKGROUND_TIMEOUT` (120 s).
The public `/api/config` carries four booleans and nothing about timings: `decision_available`, `decision_relevance_enabled` (`DECISION_RELEVANCE_ENABLED` and the model available), `decision_injection_enabled` (`DECISION_INJECTION_ENABLED` and the model available) and `decision_analytics_enabled` (`DECISION_ANALYTICS_ENABLED` and the model available), with `analytics_max_topics` and `analytics_max_overlapping_topics`, the limits the agent editor shows. The source settings show the relevance controls only while the second is true, and the agent editor and Logs show the analytics controls only while the fourth is; reading them never loads or times the model. Admin → Guardrails and Admin → Defaults show the prompt-injection model’s measured cost on this server (`predicted_input_s`), read the same way.