AnythingLLM via Tokonomix

Set the Generic OpenAI provider in AnythingLLM to Tokonomix — power chat, RAG and agents with 130+ models and consensus on one key.

Generic OpenAI providerAll-in-one AI app (RAG + agents)MIT · open sourceDesktop · Docker · self-hosted

AnythingLLM — all-in-one AI app, via Tokonomix

What is AnythingLLM?

AnythingLLM (by Mintplex Labs) is an all-in-one AI application — chat over your documents (RAG), agents, and workspaces — that you run on the desktop or self-host. Its Generic OpenAI LLM provider points at any OpenAI-compatible endpoint, including Tokonomix.

Document chat (RAG), agent skills and multi-workspace organisation.
A Generic OpenAI LLM provider with a base URL, key and model name.
Configurable in the UI or by environment variable.
Open source, desktop or Dockerised.

Why route AnythingLLM through Tokonomix?

One provider, 130+ models — pick tokonomix-consensus for high-stakes answers over your docs.
Per-call spend caps bound RAG + agent usage.
Region / origin routing keeps document-grounded answers compliant.
Swap the model by changing one field.

Connect Tokonomix to AnythingLLM

In LLM Preference, choose Generic OpenAI and fill in the base URL, key, chat model and token limit (or set the matching environment variables).

# LLM Preference → Generic OpenAI  (or env)
GENERIC_OPEN_AI_BASE_PATH=https://tokonomix.ai/api/v1
GENERIC_OPEN_AI_MODEL_PREF=tokonomix-consensus
GENERIC_OPEN_AI_API_KEY=tok_live_your-key
GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT=200000

Verify it works

Save the Generic OpenAI settings (or restart with the env vars).
Ask a question in a workspace and confirm an answer comes back.
Confirm the call and token usage in your Tokonomix dashboard.

Model selection

Which model and endpoint should my agent use?

Pick by what your agent does, then swap the model id any time — every model in the catalog works through the same key.

What your agent does	Use this
Edits files / calls tools and streams responses (most coding agents)	An Anthropic model via the Anthropic endpoint (tools + streaming both work)
Calls tools but does not need streaming (any model)	The OpenAI-compatible endpoint with a single model id
Wants tools and streaming on a non-Anthropic model	Run non-streaming for now — streaming tool-calls on other providers is not enabled yet
Plain chat, no tools	Either endpoint; streaming is fine
Wants a cross-model verdict (consensus) on a high-stakes answer	Set model to tokonomix-consensus on the OpenAI-compatible endpoint — note: consensus rejects tool requests (no tool-calling)

Endpoint & API reference (technical)

anthropic-messages → https://tokonomix.ai/api/anthropic — tools ✓, streaming ✓ (use an Anthropic model). openai-completions → https://tokonomix.ai/api/v1 — tools ✓ (single-model passthrough), streaming with tools not yet enabled. Plain chat: streaming works on both. Consensus: openai-completions, model tokonomix-consensus, no tools. Swap the model id to any slug returned by GET /api/v1/models.

Browse all models →