OpenAI
Sora 2
Legacy capture only, retained for provenance if access remains. Legacy configurations are reported separately from active Core AV.
Legacy capture Leaderboard Commissioning matrix
Listed configurations
Each row is a distinct benchmark subject with its own provider identity and requested settings. None has been generated or evaluated. Variants are never averaged into a single family number.
| Configuration | Tier | Provider model ID | Prompt key | Requested resolution | Audio request | Commissioning | Result |
|---|---|---|---|---|---|---|---|
| Sora 2 ref | Standard | sora-2 | sora-2 | 1280x720 | Not specified | Legacy capture if access remains | Not yet run |
| Sora 2 Pro | Pro | sora-2-pro | sora-2 | 1280x720 | Not specified | Legacy capture if access remains | Not yet run |
Requested generation settings, verbatim from the commissioning matrix. Audio request records the generation setting the commissioning matrix specifies for a configuration. It is not a verified model capability, and no configuration has been run.
- Sora 2 8s; 16:9 / 1280x720; preserve raw output
- Sora 2 Pro 8s; 16:9 / 1280x720; preserve raw output
Overall FARTScore
Results pending commissioning
No blinded evaluation has been locked for any Sora 2 configuration under Core v0.1.0. A composite score requires at least three independent raters per clip across 3 replicates.
Published when the FB-C02 smoke test is reviewed and the development pilot completes for this family.
Dimension profile
Fig. 1 Sora 2 — dimension profile
Rubric scale 0–4 per dimension
- A Acoustic 22%Pending
- T Timing 16%Pending
- L Attribution 16%Pending
- P Physical 14%Pending
- I Instruction 12%Pending
- S Social 10%Pending
- C Coherence 10%Pending
Rubric scale 0–4 · 0 failure · 1 poor · 2 mixed · 3 good · 4 excellent
Mean rater score on each of the seven weighted FARTScore dimensions, on the 0–4 rubric scale, with each dimension's weight shown beside it.
Source FARTBench Core v0.1.0 No locked evaluations
Test-by-test performance
| Test | Title | FARTScore | 95% CI | n | Result |
|---|---|---|---|---|---|
| FB-C01 | Basic Discreet Emission | — | — | 0 | Pending |
| FB-C02 | Multi-Agent Source Attribution | — | — | 0 | Pending |
| FB-C03 | Speech Interruption | — | — | 0 | Pending |
| FB-C04 | Sustained Tonal Emission | — | — | 0 | Pending |
| FB-C05 | Movement-Coupled Source | — | — | 0 | Pending |
| FB-C06 | Silent Condition | — | — | 0 | Pending |
| FB-C07 | Exact Event Counting | — | — | 0 | Pending |
| FB-C08 | Room-Acoustics Reverberation | — | — | 0 | Pending |
Observed strengths and weaknesses
No characterisation available
Statements about a model's failure surface are only published when locked evaluations support them. Nothing is inferred from vendor material, from other benchmarks, or from expectation.
Published when failure-code incidence is measurable across a family's replicates.
A published characterisation reads, for example: Observed weakness: elevated incidence of F14 macroscopic gas visualization during sustained-event tests. That sentence is not shown for any model until data supports it.
Cost, latency, and prompt sensitivity
Not captured
Generation cost and latency are recorded per sample in the generation manifest. Farts per Dollar is only reported once costs are captured consistently across configurations, and Prompt Sensitivity Index is deferred until a Raw track exists.
Raw samples
No published artifacts
Raw outputs are immutable benchmark evidence, hashed and preserved, but not published before the evaluations that reference them are locked.