LLM Providers

Two provider slots, a model per stage, failover, and the spend ledger from provider down to a single episode. Portrait cut and the file itself in Videos.

Using Anthropic directly

Select Anthropic and save an API key under Settings > AI & Processing. Each stage can use its own model. MinusPod chooses a tool-call format that the selected model supports. Models that reject forced tool calls use automatic tool choice with a strict schema. A response without the requested tool result follows the normal bounded retry policy; its token usage still counts.

Temperature is not sent to Anthropic. API version 2026-10-21 removed the parameter on every model, so Anthropic stages run at the model default.

Reasoning settings

Under LLM Tunables, use Reasoning budget for models that accept manual thinking budgets, such as Claude 4.5 and 4.6. Use Reasoning effort for adaptive-only models, including Opus 4.7, Opus 4.8, and Claude 5 models. MinusPod sends the applicable setting for the model assigned to each stage. Both values persist when switching models or providers.

Leave effort blank to use the configured default; without an environment default, the provider decides. none disables thinking where supported; models that require thinking use low effort instead. A numeric budget is not converted into an effort level. Model restrictions follow Anthropic’s thinking API.

Prompt caching

MinusPod requests a 5-minute cache for static Anthropic system prompts. Transcript snippets and other per-call messages remain outside that cache boundary. Caching is automatic; there is no setting to enable it. Short prompts may not meet the provider’s minimum cache size, and a changed prompt or expired cache causes a miss. See Anthropic’s prompt caching guide.

Input token totals include cache reads and writes. Processing history reports these counts separately as subsets of input tokens. For direct Anthropic calls, estimated cost uses the model’s cache-read discount and the 5-minute cache-write rate. For OpenAI-compatible and OpenRouter calls, cached input tokens bill at the model’s cache read rate when the pricing catalog or an operator override supplies one, and at the full input rate otherwise. The Stats page shows the resulting cache hit rate (cache reads over input tokens) per model, per run and per contributing call. A provider-reported cost takes precedence; calls without known pricing remain unpriced.

Using Claude Code Wrapper (Max Subscription)

The Claude Code OpenAI Wrapper lets you run on a Claude Max subscription instead of API credits.

Quick Start:

1. Start the wrapper service:

   docker compose --profile wrapper up -d

2. Authenticate with Claude (first time only):

   docker compose --profile wrapper run --rm claude-wrapper claude auth login

3. Configure minuspod to use the wrapper by updating your .env:

   LLM_PROVIDER=openai-compatible
   OPENAI_BASE_URL=http://claude-wrapper:8000/v1
   OPENAI_API_KEY=not-needed

4. Restart minuspod:

   docker compose up -d minuspod

Other OpenAI-Compatible Endpoints:

The openai-compatible provider can work with other endpoints by configuring OPENAI_BASE_URL and OPENAI_API_KEY accordingly. Select models under Settings > AI & Processing > AI Models, or use OPENAI_MODEL to seed unset model settings.

Example .env for OpenAI-compatible mode:

# LLM Configuration (OpenAI-compatible)
LLM_PROVIDER=openai-compatible
OPENAI_BASE_URL=http://claude-wrapper:8000/v1
OPENAI_API_KEY=not-needed

# Server Configuration
BASE_URL=http://localhost:8000

OPENAI_MODEL is optional. It seeds any unset detection, verification, and chapters model; later Settings choices take precedence.

Using Ollama (Local or Cloud)

Ollama is an alternative to the Anthropic API. MinusPod supports both flavors:

• Local (http://host:11434): no auth, no API costs, nothing leaves the machine.
• Cloud (https://ollama.com/api): same OpenAI-compatible endpoints, just with Authorization: Bearer <key> on every request. Free tier works for this pipeline. Grab a key at ollama.com/settings/keys.

Configuration is identical either way: pick Ollama in Settings > LLM Provider, set the Base URL, and (for Cloud) paste the key. The key is stored encrypted. Leave it blank for local.

A Test connection button sits under the Base URL field for the Ollama and OpenAI Compatible providers. It hits the same /models route MinusPod uses to discover models (with the same automatic /v1 suffix for Ollama), works without an API key, and tells you whether the failure is an unreachable host, a wrong path, or a rejected key. You can test a URL before saving it; a saved API key is only sent when the tested URL matches the saved one.

Anthropic and OpenRouter get the same button. Their endpoints are fixed, so the test answers a different question: can this container reach the provider at all, and if not ok, is the problem the network or the saved key.

Heads up about Ollama Cloud model selection

Ollama Cloud’s /v1/models advertises the full Ollama library, including previews and local-only tags that Cloud won’t actually route. The dropdown shows whatever the endpoint returns, so entries like gemma4:31b, kimi-k2:1t, and gpt-oss:120b can appear but 404 when called.

If an episode processes with zero ads and the logs show {"type":"error","error":{"type":"not_found_error"}}, the model isn’t really on Cloud. The reliable list is at ollama.com/search?c=cloud. Cross-check the base name (before the :) before saving.

Setup

1. Install and start Ollama on your host machine
2. Pull a model (see recommendations below): ollama pull qwen3:14b
3. Update your docker-compose.yml:

environment:
  - LLM_PROVIDER=ollama
  - OPENAI_BASE_URL=http://host.docker.internal:11434/v1
  - OPENAI_MODEL=qwen3:14b

Linux users: host.docker.internal doesn’t resolve by default on Linux. Add extra_hosts: ["host.docker.internal:host-gateway"] to your Docker service definition.

The OPENAI_API_KEY variable is not required for Ollama. Token counts will still be tracked in the UI but cost will always show as $0.00, which is accurate since local inference is free.

---

Recommended Models

Note: LLMs are non-deterministic. The same prompt against the same model can yield different ads on different runs. The picks below are where I’d start, not where I’d stop tuning.

Cloud LLMs (benchmark-tested)

These come from the offline LLM benchmark included in this repo. The benchmark runs each candidate model against a corpus of human-verified episodes and scores accuracy (F0.5 and F1 at IoU >= 0.5), JSON compliance, latency, and per-episode cost. Full per-model breakdown (precision, recall, boundary accuracy, calibration, latency tail, token efficiency, cross-model agreement) is in benchmarks/llm/results/report.md. Want to expand the corpus or test more models? See benchmarks/llm/CONTRIBUTING.md.

The report ranks by F0.5, which weights precision twice as heavily as recall. The pipeline cuts the segments it flags, so a false positive (cutting real content) is a worse failure than a missed ad. F1 is shown next to it.

Use case	Model	F0.5	F1	Cost / episode	Why
Best accuracy overall	claude-haiku-4-5 (Anthropic direct)	0.908	0.920	$1.08	Tops the table with perfect JSON compliance and a clean pass on both no-ad controls. p50 latency is 24.2s, which is fine for MinusPod’s offline pipeline but rules out interactive use. The top tier also holds qwen/qwen3.5-plus-02-15, google/gemini-3.5-flash-lite, claude-sonnet-4-6, the Gemini Flash line, the Grok line, and mistralai/mistral-medium-3-5; they trade wins across episodes, so pick among them on speed and cost. Between the two Gemini Flash entries, 3.7 costs $0.64 per episode against 3.6’s $1.44 while statistically tied, so there is no reason to pick 3.6.
Fast and reliable	google/gemini-3.5-flash-lite (via OpenRouter)	0.860	0.851	$0.36	p50 0.7s, perfect JSON compliance, clean on both no-ad controls, and third overall, inside the top tier at a tenth of the frontier price. The production pick when you want speed and low cost without giving up accuracy.
Best Anthropic-direct at low latency	claude-sonnet-4-6	0.859	0.866	$3.24	Fourth overall, p50 4.1s versus Haiku’s 24.2s, perfect JSON compliance, clean on both no-ad controls. For Anthropic-key users who want faster turnaround than Haiku 4.5; both Opus 4.8 ($5.37/episode) and Opus 5 ($5.38/episode) score lower.
Cheapest viable	qwen/qwen3.7-flash (via OpenRouter)	0.767	0.774	$0.07	Best F0.5 per dollar (10.58) among models with no reliability flags. JSON compliance 0.96 and clean on both no-ad controls. p50 10.1s. A few cheaper models rank higher on raw F0.5/$, but all of them either false-positive on the no-ad controls or return brittle JSON.
Free while it lasts	stealth/ox-alpha* (via OpenRouter)	0.806	0.810	$0.00*	One tier below the leaders at no cost today, and clean on both no-ad controls. Two things to weigh against that: p50 latency is 66.6s, the slowest in its tier by roughly an order of magnitude, and it is verbose enough that 6.4% of calls hit the token ceiling, giving a 5.7% parse-failure rate and 0.94 JSON compliance. Worth trying if your pipeline is patient and you can absorb a few dropped windows.

* stealth/ox-alpha is a cloaked listing: an unannounced model published under a placeholder name, free during the preview. Treat the $0.00 as promotional, not structural. When it ships under its real name the slug disappears and pricing appears, so anything depending on it should have a fallback configured. Cloaked listings also generally allow provider-side prompt logging, which matters if your transcripts are sensitive.

Caveats:

• Corpus size, trial counts, statistical tiering, and how F0.5 and F1 are scored are covered in benchmarks/llm/results/report.md, linked above.
• The sweep ran against a frozen system prompt, benchmarks/llm/prompts/2026-08.txt, not the live one. Reproducing these numbers needs that file, via benchmark run --snapshot.
• Provider-side content moderation can take episodes out of a model’s reach entirely. One roster model was refused on 15.2% of its calls because the provider’s filter blocked explicit transcripts, losing a quarter of the ads in the two affected episodes. If your library includes explicit shows, check benchmarks/llm/results/parse-and-moderation.md before picking a model.

Local Ollama Models (by VRAM tier)

Note on benchmarking: the offline benchmark in benchmarks/llm/ covers cloud-hosted models (OpenRouter, Anthropic direct). Local Ollama runs are not in that sweep. Adding an Ollama provider would let contributors compare local quants apples-to-apples against the cloud numbers. The recommendations below come from author testing, not a structured benchmark, so verify them on your own content.

Models are loaded sequentially, not concurrently; VRAM requirements are not additive between passes.

Pass 1: First Pass Detection

Hardest task. Contextual reasoning, host-read ads, new sponsors. Use your best model here.

VRAM	Model	Quantization	Notes
8GB	qwen3:8b	Q4_K_M	Entry level. Handles standard sponsor reads well.
12GB	qwen3:14b	Q4_K_M	Best quality-to-VRAM ratio. Recommended.
16GB	qwen3:14b	Q5_K_M	Higher quality quant; use if you have headroom.
24GB	qwen3.5:27b	Q4_K_M	Strong contextual reasoning. 256K context.
24GB	qwen3.5:35b	Q4_K_M	Best quality under 40GB. 256K context.
40GB+	qwen3.5:122b	Q4_K_M	Author’s best local option for hard cases. Not yet measured in the cloud benchmark.

Verification Pass

Easier task. Looks for remnants in already-cut audio. Speed matters more than raw accuracy.

VRAM	Model	Quantization	Notes
8GB	qwen3:4b	Q8_0	Fast, good JSON compliance. Verification prompt is simpler.
12GB	qwen3:8b	Q5_K_M	Strong JSON compliance, faster than 14B.
16GB	mistral-nemo:12b	Q4_K_M	Excellent JSON reliability, fast inference.
24GB	qwen3:14b	Q5_K_M	Overkill for verification but uses available VRAM productively.

Chapters

Simplest task. Summarization only, no structured detection. Minimize VRAM usage and latency.

VRAM	Model	Quantization	Notes
Any	qwen3:4b	Q4_K_M	Sufficient for summarization. Fast.
Any	phi4-mini	Q4_K_M	Lean alternative, strong instruction following.
Any	llama3.2:3b	Q4_K_M	Smallest viable option if VRAM is tight.

Example split for 16GB VRAM: Pass 1 -> qwen3:14b Q5_K_M / Verification -> qwen3:8b Q5_K_M / Chapters -> qwen3:4b Q4_K_M

Avoid models under 7B for production use. JSON reliability drops sharply at smaller sizes, turning recoverable errors into silent detection failures. See JSON Reliability Risks.

---

Cloud vs. Local: What Changes

The LLM only sees host-read ads that blend into content, new sponsors not yet in the pattern database, and ambiguous mid-rolls without promo codes or URLs. Everything else (audio fingerprinting, text pattern matching, pre/post-roll heuristics, audio-signal enforcement) runs without an LLM and catches a substantial share of ads regardless of model.

Content type	Cloud-vs-local impact
Standard sponsor reads with promo codes / vanity URLs	Minimal: patterns and fingerprinting cover most of these without the LLM
Heavy host-read or conversational ad integrations	Noticeable: requires strong contextual reasoning
Network-inserted brand-tagline ads (no promo code, no URL)	Moderate: these are usually under 30 seconds, and short ads are the weakest detection bucket for most models in the cloud benchmark, so don’t expect local to outperform
New sponsors not in the pattern database	Moderate: depends heavily on model capability

qwen3:14b locally is fine for standard sponsor reads. The gap to cloud-frontier shows up on conversational ad reads that lack clear transitions. To measure the gap on your own content, capture the episode (see benchmarks/llm/) and compare predictions against your verified ground truth.

---

JSON Reliability Risks

MinusPod’s ad detection pipeline requires models to return structured JSON. The Anthropic API enforces this reliably. With Ollama or any open-weights serving, enforcement is model-dependent and failures are more likely.

Failure modes:

• Malformed JSON: missing brackets, trailing commas, unquoted keys. The parser tries direct parse, then markdown-fence extraction, then regex scan. Structurally broken JSON falls through all three.
• Truncated output: models under memory pressure or processing long transcript windows can cut off mid-response, leaving valid-looking but incomplete JSON.
• Preamble text: some models prefix their JSON with conversational text (“Sure, here are the ads I found:”). The parser usually strips this, but it adds fragility.

When a window fails to parse, those ads are silently missed. No UI error; the episode processes normally with gaps in detection coverage.

Cloud models vary widely on this. Benchmark JSON compliance ranges from 0.07 (openai/o4-mini, which buries JSON in reasoning chains) and 0.10 (qwen/qwen3-8b, which answers in prose) up to 1.00 (the Claude line, the Gemini 3.5 and 3.6 Flash models, Grok, Mistral Medium). See the JSON compliance chart in benchmarks/llm/results/report.md, and parse-and-moderation.md for what the failures actually look like.

Reducing the risk for local runs:

• Use a model of at least 7B parameters
• Prefer Qwen3 or Mistral families (consistently high JSON compliance in author testing)
• Don’t run other GPU workloads concurrently; memory pressure increases truncation risk
• Watch the extraction_method field in processing logs

Healthy run signal: extraction_method reads json_array_direct for most calls. Fallback methods (markdown_code_block, regex_*) mean the model isn’t returning clean JSON. Frequent fallback in production means you should upgrade the model.

Using OpenRouter

OpenRouter is a unified API that routes to 200+ models (Claude, GPT, Gemini, open-weights) with one API key. OpenRouter is supported as an LLM provider only: it does not support the /v1/audio/transcriptions endpoint required for Whisper transcription. For transcription without a GPU, use a remote Whisper backend such as Groq.

Setup

1. Get an API key from openrouter.ai/keys
2. Add these to your .env, then start with docker-compose.yml (GPU) or docker-compose.cpu.yml (no GPU):

LLM_PROVIDER=openrouter
OPENROUTER_API_KEY=sk-or-v1-your-key-here

Model Selection

No model ships by default: pick one in the Settings UI, or seed it with the OPENAI_MODEL env var; see its row in Environment Variables for how seeding works.

Any OpenRouter model ID works:

• anthropic/claude-sonnet-4-5: Claude Sonnet via OpenRouter
• openai/gpt-4o: GPT-4o via OpenRouter
• google/gemini-2.5-flash-preview: Gemini Flash via OpenRouter
• openrouter/free: router alias that picks a free model per request
• openrouter/auto: router alias that picks the best model for the prompt

The openrouter/free and openrouter/auto aliases are not in OpenRouter’s /api/v1/models list, so MinusPod adds them to the dropdown for you. Other unlisted model IDs can be typed straight into any model field (the “Type a model ID” link next to each dropdown), or seeded with OPENAI_MODEL.

All of these can be changed at runtime from the Settings UI. No container restart needed.

Using System One

Select TypeSafe or System One compatible under Settings > AI & Processing > LLM Provider. Each slot keeps its own credentials, endpoint, and tuning profiles. TypeSafe uses a fixed endpoint. Compatible endpoints require their own base URL; OPENAI_BASE_URL is not reused. See System One for setup and manual migration from the Jev proxy.

Detection, verification, and the ad category pass are supported. Review is experimental, including both passes when the reviewer inherits the pass route. Chapter generation and Pattern Cleanup require a chat provider. Disabled unsupported routes remain editable, but enabling or manually running them is blocked until the route is repaired. Failover supports chat providers only.

System One request limits

System One tuning retains four profiles: TypeSafe and compatible settings for each credential slot. The logical request deadline covers every adapter stage, queue wait, retry, and upstream request. Each HTTP request also uses the slot timeout. The default logical deadline is 75 seconds; the default HTTP timeout is 60 seconds. Blank protocol caps remove local limits, but do not prevent upstream truncation. Truncated responses are rejected.

The per-slot retry setting controls retries after the initial request; TypeSafe and compatible providers default to two retries. Retries use the shared backoff policy. The first retry can honor a rate-limit reset or Retry-After hint, capped by the selected profile’s maximum Retry-After wait. That cap defaults to 5 seconds and accepts zero, within the remaining logical deadline. Each actual HTTP attempt has its own usage record. Missing usage stays unknown.

The maximum concurrent operations setting defaults to 4 operations per process. Operations sharing a normalized endpoint and credential share capacity across slots and provider types. Conflicting profile limits use the stricter active or queued limit. This is not a cluster-wide limit. A local queue timeout sends no HTTP request and does not trigger the provider circuit breaker or chat failover.

System One preserves caller prompts, policy, and context in its structured questions. It ignores saved temperature, thinking budgets, token caps, JSON-schema settings, and Ollama context settings. Those values are retained for later chat routes. Use the dedicated System One controls for supported adapter tuning.

Per-Stage Providers

MinusPod supports up to two full provider configurations: Provider A, and one optional Provider B with its own type, base URL, and API key. See Provider B. Each pipeline stage then picks one of the two, not a provider type directly. Settings > AI & Processing > AI Models shows a slot selector next to each model selector:

• Ad Detection Provider - Default (Provider A) or Provider B. Provider B only appears in the list once it is enabled and configured.
• Verification Provider - Same as detection (default), Default (Provider A), or Provider B. Pointing it at the other slot is useful for cost control (a cheap pass-2 sanity check) or comparing two providers’ output on the same episode.
• Chapters Provider - Same as detection (default), Default (Provider A), or Provider B, independent of the other two.

The Ad Reviewer section has its own slot selector, described in Ad Reviewer. Same as pass inherits both the provider and model of whichever pass is being reviewed. Provider A or Provider B runs the reviewer on that slot instead, and enables the Review Model selector for its catalog.

Pattern Cleanup, under Settings > Experiments, also has its own provider and model selectors. Each can inherit detection’s choice. Choose a higher-quality model when reviewing patterns: an approved pattern can affect matches in many later episodes.

Switching a stage’s slot only changes that stage. Model discovery re-runs for the newly selected slot, so the model dropdown next to it shows that slot’s own catalog. Other stages keep their slot and model unless their provider account also changes.

When a provider type, custom endpoint, or stage route changes the account used by a stage, MinusPod clears that stage’s saved model. Empty fields show a reminder to choose a model and stay blank after reload until you choose a replacement. A stage still accepts a hand-typed model ID (the “Type a model ID” link) for models a slot’s catalog does not list. Changing only an API key keeps model selections. If a stage is set to Provider B and Provider B is later disabled, that stage falls back to Provider A until Provider B is turned back on.

Each slot needs its own API key and, where relevant, base URL configured under Settings > LLM Provider before a stage can use it; an unconfigured slot’s model list comes back empty until credentials are saved.

Routing versus failover

Provider A and Provider B are two independent accounts a stage can be pointed at. Picking one at configuration time is routing: it is not a chain, and nothing at runtime moves a call from one to the other on its own. When a call fails, it retries against the same slot and endpoint, then gives up; the episode does not spill onto the other provider through routing alone. Credentials can change if a key is rotated between attempts; see provider key rotation and active runs. A rate-limit pause behaves the same way: the episode waits in the queue for that account’s reset instead of rerouting.

The one time a stage’s slot changes on its own, outside failover, is at configuration time: a stage set to Provider B while Provider B is disabled or has no type saved resolves to Provider A, and logs a warning once. That is a fail-safe for an incomplete configuration, evaluated when the route is resolved, not a response to an error.

A separate feature, failover, does move work between providers at runtime: a dedicated standby provider (and transcriber) takes over a slot when the active one is unreachable, times out, or rejects the key, model, or billing, then switches back once it recovers. See Failover for triggers, mid-run behavior, health probes, and manual control.

System One routes do not participate in chat failover. Choose a supported chat provider for the standby account.

A run’s routes are frozen when it starts, and a rotated key is picked up without a restart; see Endpoint and model changes during an active run.

Staying under an account’s limits

Each slot has its own optional requests-per-minute, requests-per-day, and tokens-per-minute caps, counted per account rather than per provider type, so two accounts on the same provider throttle independently. The per-day counter resets at UTC midnight regardless of where you or the provider are. They are off by default. See Manual request-rate limits.

LLM Pricing

MinusPod tracks token usage and cost for every LLM call. The Settings page and GET /api/v1/system/token-usage show per-model breakdowns.

Custom model pricing

Settings > AI & Processing > AI Models shows pricing fields for configured routes, including enabled standby and supported manual chapter and Cleanup models. Enter input and output rates in USD per 1 million tokens. A third optional field sets the cached input rate; leave it blank to bill cached input at the input rate. An operator override wins over catalog pricing and appears as pricingSource: "operator". Leave both fields blank to use the catalog. Save both as 0 to mark a model explicitly free; missing pricing is different.

Inactive prices stay saved and reappear when those models are selected again. Recorded call costs stay unchanged.

Where pricing data comes from

Pricing is fetched automatically based on your configured provider:

Provider	Source	Method
Anthropic	pricepertoken.com	HTML scrape
OpenRouter	OpenRouter API (/api/v1/models)	JSON API
OpenAI, Groq, Mistral, DeepSeek, xAI, Together, Fireworks, Perplexity, Google	pricepertoken.com	HTML scrape
Ollama / localhost	N/A	Always $0
TypeSafe	Fixed official Jev pricing	$0.042 USD per 1 million input tokens; output tokens are free
System One compatible	Operator pricing override	Unknown unless configured

Pricing refreshes once every 24 hours in the background. You can also force a refresh with POST /api/v1/system/model-pricing/refresh, or view current rates with GET /api/v1/system/model-pricing; see API.

The OpenRouter API and the LiteLLM fallback also publish a cache read rate for some models. When the primary source has no cache rate for a model, the chain fills it from a later source. An operator override can set it as cacheReadCostPerMtok.

How model matching works

Different sources use different names for the same model. A normalization step strips provider prefixes, date suffixes, and punctuation so that claude-sonnet-4-5-20250929 (Anthropic API), anthropic/claude-sonnet-4-5 (OpenRouter), and Claude Sonnet 4.5 (pricepertoken.com display name) all resolve to the same pricing entry.

Offline / air-gapped installs

If the pricing fetch fails on startup and the database has no pricing data, MinusPod seeds a built-in table of Anthropic prices. A non-Anthropic model without a catalog row has unknown pricing and accrues $0 until a match or operator override exists. Existing cached pricing is kept when a fetch fails.

Pricing accuracy

Pricing data comes from third-party sources and may lag behind provider announcements. Check your provider’s billing dashboard for authoritative cost figures. MinusPod’s cost tracking is an estimate for convenience, not a billing system.

Reviewer Calibration Self-Test

The reviewer stage (the LLM pass that confirms, adjusts, or rejects candidate ad boundaries) can drift when you switch models or providers. tools.reviewer_calibration runs a small fixed corpus of 8 fictional transcripts (4 ad reads, 4 non-ads) through the same AdReviewer code path production uses, and reports how often the model’s verdict agrees with the expected label plus how often its response carries the structured is_ad field the contradiction guard relies on.

Run it manually against your configured provider:

PYTHONPATH=src python -m tools.reviewer_calibration

It prints a pipe-delimited table (case id, expected label, verdict, agreement) followed by a summary line with the model, provider, agreement fraction, and structured fraction. The command exits 1 when agreement falls below 0.75, so it can gate a CI or release check.

Agreement below 0.75 means the reviewer is disagreeing with the expected keep/drop call on 2 or more of the 8 corpus cases. That is a signal to check the model’s reasoning quality for this stage, not a hard failure, since the corpus is small and one model’s phrasing conventions can differ from another’s without being wrong. A low or dropping structured fraction on a model that used to report it consistently means the model has stopped emitting the is_ad field, which weakens the reviewer’s contradiction guard.

The self-test also runs automatically in a background thread whenever the reviewer model setting (reviewModel) changes via the Settings API, and stores its result under the reviewer_calibration_last setting. A failed or slow calibration run never blocks the settings save. Set reviewer_calibration_on_change to false (or the REVIEWER_CALIBRATION_ON_CHANGE environment variable) to disable the auto-run. Each auto-run costs 8 LLM calls, one per corpus case. Calibration runs go through the production reviewer, so they emit reviewer telemetry log lines tagged slug=reviewer-calibration; exclude that slug from reviewer metric queries so corpus cases do not skew real verdict counts.