Speed analysis
Latency measured across all benchmark runs. P50 (median) and P95 (95th percentile) give a realistic picture of response speed under normal and peak load.
Quality scores
Evaluation results from judge-model scoring across diverse task categories. Scores reflect coherence, accuracy and instruction-following.
Pricing history
Direct provider rates per million tokens, plus a typical-conversation cost estimate.
Pricing over time
Input & output per 1M tokens · step-line = price changes
$0.1300
input / 1M
— stable
$0.1300
output / 1M
— stable
Tokens per second
Throughput in tokens per second, derived from measured P50 latency. Higher is better; fluctuations track provider-side load.
Estimated from P50 latency × 200 output tokens — the absolute number depends on this assumption; the trend is what matters.
Capabilities
Availability
Availability
No measurements yet
We haven't recorded enough API calls to show availability stats for this model. Data appears once the model starts receiving live traffic.
Tokonomix benchmark verdicts
Mistral-Nemo quality plummets 38 points to 46.8, latency up 43%
Mistral-Nemo-Instruct-2407 on OVH AI Endpoints has experienced a severe performance degradation in the current benchmark window. Overall quality dropped dramatically from 84.9 to 46.8, representing a 38.1 point decline that affects nearly all measured capabilities. The multilingual category saw the most significant collapse, falling from 97 to just 26. Creative performance dropped from 75 to 58, while the model now scores 50 in factual tasks and 53 in reasoning. These new categories replace the previously measured coding capability, which scored 83 in the last window. Latency has also deteriorated substantially, with p50 response times increasing 43% from 3051ms to 4372ms. This combination of quality collapse and slower response times suggests either a model version change, infrastructure issues, or configuration problems at the provider level. The stability between benchmark windows has clearly been compromised. Users should exercise caution and potentially consider alternative providers or models until performance stabilizes and returns to previously demonstrated levels.
Quality
46.8
Latency p50
4,372 ms
Test runs
5