Data Extraction Race — results

Software License Agreement — Acme & Northwind

$0.0108 total

Scoreboard

#1 Gemini 2.5 ProGoogle Gemini★ Won this round on this document
100/100
3730 ms · $0.0023
FieldExpectedModel answer
productEnterprise Analytics SuiteEnterprise Analytics Suiteexact
licenseeNorthwind Trading B.V.Northwind Trading B.V.exact
licensorAcme Industries LtdAcme Industries Ltdexact
signatoryMaria HendricksMaria Hendricksexact
annual_fee4800048000exact
term_months3636exact
license_typenon-exclusivenon-exclusiveexact
governing_lawState of DelawareState of Delawareexact
effective_date2026-06-042026-06-04exact
notice_period_days9090exact
#2 Cohere Command-AOpenRouter
97/100
2167 ms · $0.0020
FieldExpectedModel answer
productEnterprise Analytics SuiteEnterprise Analytics Suiteexact
licenseeNorthwind Trading B.V.Northwind Trading B.V.exact
licensorAcme Industries LtdAcme Industries Ltdexact
signatoryMaria HendricksMaria Hendricksexact
annual_fee48000$48,000.00close
term_months3636exact
license_typenon-exclusivenon-exclusiveexact
governing_lawState of DelawareState of Delawareexact
effective_date2026-06-042026-06-04exact
notice_period_days9090exact
#3 Llama-3.1-8B-InstructOVH AI Endpoints (GRA)
97/100
2762 ms · $0.0000
FieldExpectedModel answer
productEnterprise Analytics SuiteEnterprise Analytics Suiteexact
licenseeNorthwind Trading B.V.Northwind Trading B.V.exact
licensorAcme Industries LtdAcme Industries Ltdexact
signatoryMaria HendricksMaria Hendricksexact
annual_fee4800048000exact
term_months3636exact
license_typenon-exclusivenon-exclusiveexact
governing_lawState of DelawareState of Delawareexact
effective_date2026-06-04June 4, 2026close
notice_period_days9090exact
#4 gpt-oss-20bOVH AI Endpoints (GRA)
97/100
12333 ms · $0.0001
FieldExpectedModel answer
productEnterprise Analytics SuiteEnterprise Analytics Suiteexact
licenseeNorthwind Trading B.V.Northwind Trading B.V.exact
licensorAcme Industries LtdAcme Industries Ltdexact
signatoryMaria HendricksMaria Hendricksexact
annual_fee4800048000exact
term_months3636exact
license_typenon-exclusivenon-exclusiveexact
governing_lawState of DelawareState of Delawareexact
effective_date2026-06-04June 4, 2026close
notice_period_days9090exact
#5 Claude Opus 4.8Anthropic
94/100
2775 ms · $0.0064
FieldExpectedModel answer
productEnterprise Analytics SuiteEnterprise Analytics Suiteexact
licenseeNorthwind Trading B.V.Northwind Trading B.V.exact
licensorAcme Industries LtdAcme Industries Ltdexact
signatoryMaria HendricksMaria Hendricksexact
annual_fee48000$48,000.00close
term_months3636exact
license_typenon-exclusivenon-exclusiveexact
governing_lawState of DelawareState of Delawareexact
effective_date2026-06-04June 4, 2026close
notice_period_days9090exact

Source document

This Software License Agreement is entered into on June 4, 2026 between Acme Industries Ltd (the "Licensor") and Northwind Trading B.V. (the "Licensee"). The Licensor grants a non-exclusive license to the Enterprise Analytics Suite for an annual fee of $48,000.00, payable yearly in advance. The initial term runs for 36 months and renews automatically unless either party gives 90 days written notice. The governing law of this agreement shall be the State of Delaware. Liability is capped at the total fees paid in the preceding twelve months. The agreement was signed by Maria Hendricks on behalf of the Licensee. Extract the following fields as JSON: licensor, licensee, effective_date, annual_fee, term_months, notice_period_days, governing_law, product, license_type, signatory.

How scoring works

Every model received the identical prompt and field list and answered once at temperature 0. We compare each field to the ground truth: an exact match earns full credit, the same value in a different format earns partial credit, anything else earns none. No model judges another — the score is pure field matching. A win means this model scored highest on THIS document with THESE fields, not that it is better overall.