Speed analysis
Latency measured across all benchmark runs. P50 (median) and P95 (95th percentile) give a realistic picture of response speed under normal and peak load.
Quality scores
How this model compares to the rest of the field on each prompt category, from a pairwise fit over the same prompts. The raw judge score sits underneath each number.
Win rate per category: how often this model beats a field-average model on a prompt from that category. 50% is average, not a failing grade. It is not a percentage of correct answers.
Pricing
What you pay per million tokens when you use this model on Tokonomix, plus an estimate for a typical conversation.
Tokens per second
Throughput in tokens per second, derived from measured P50 latency. Higher is better; fluctuations track provider-side load.
Estimated from P50 latency × 200 output tokens — the absolute number depends on this assumption; the trend is what matters.
Capabilities
Availability
Availability
How often this model answers when we call it — measured across real API requests and live tests over the last 30 days. This is separate from quality: these numbers only tell you whether the model responds, not how good the answer is.
Last 7 days
—
Last 30 days
100.0%
n=7
Median response time
1,091ms
n=7
Based on 387 measurements over the last 30 days.
Technical details
Only live API calls and live-test requests count — internal probes and benchmark runs are excluded.
Calls with a custom API key (BYOK) are excluded: those failures are key-specific, not a sign of model downtime.
Failed calls are NOT included in quality scores — quality is measured on successful responses only. Availability and quality are independent signals.
Median response time (p50) across successful calls with a recorded duration. Outliers (very slow or very fast calls) pull the median less than the average.
Total calls (30d)
7
OK responses (30d)
7
Total calls (7d)
0
OK responses (7d)
0
Tokonomix benchmark verdicts
Quality jumps to 84.5 with 42% faster latency, reasoning now perfect
Mistral-Nemo-Instruct-2407 demonstrates substantial improvements across multiple dimensions in this benchmark window. Overall quality increased by 6.7 points to reach 84.5, representing a meaningful step forward in model performance. Latency improved dramatically with a 42% reduction, bringing the p50 down from 7661ms to 4459ms, making the model noticeably more responsive for production workloads. The standout achievement is reasoning performance, which reached a perfect 100 score compared to not being measured in the previous window. Factual accuracy showed strong gains, rising from 50 to 91, indicating major improvements in knowledge retrieval and accuracy. Multilingual capabilities remained robust at 92, up slightly from 84, maintaining the model's strength in cross-language tasks. The most significant regression appears in coding performance, which dropped sharply from 100 to 55. This represents a 45-point decline and suggests recent changes may have impacted code generation quality. Users relying on coding capabilities should test thoroughly against their specific use cases. Despite this setback, the overall trajectory shows a model becoming faster and more capable across most domains, with the reasoning and factual improvements potentially offsetting coding limitations for general-purpose applications.
Quality
84.5
Latency p50
4,459 ms
Test runs
5