FARTBench Core R01
Leaderboard
Sixteen AI video configurations, the same four scenes, four human reviewers per clip. Ranked by whether the requested thing happened and came from the right source; quality ratings are shown beside every score.
How the ranking works. Each configuration was asked for four scenes. It earns one point for every scene where at least 3 of the 4 reviewers agreed the requested event happened, and a second point only where they also agreed the source or cause was right — a Factual Score out of 8. Ties go to more scenes with every factual question right; anything still equal shares its rank. A refused scene is still one of the four: it earns nothing and is marked R, and it is coverage, not a bad video. A scene without a 3-of-4 consensus earns nothing and is never counted as a failure. Sound, timing and people ratings are shown beside the score and never break a tie.
| Rank | Configuration | Factual scoreScore | Event | Right source / causeCause | Every question | Sound | Timing | People | Reviewers | Coverage |
|---|---|---|---|---|---|---|---|---|---|---|
| 1= | Seedance 2.5 ByteDance Seedance | 8/8 | 4/4 | 4/4 | 4/4 | 3.9 n=16 · 0 N/A | 4.2 n=16 · 0 N/A | 3.8 n=16 · 0 N/A | 4/clip | 4/4 generated |
| 1= | Vidu Omni Pro 3.2a Vidu | 8/8 | 4/4 | 4/4 | 4/4 | 4.1 n=16 · 0 N/A | 4.3 n=16 · 0 N/A | 4.0 n=16 · 0 N/A | 4/clip | 4/4 generated |
| 3= | Wan 3 Alibaba Wan 3 | 6/8 | 4/4 | 2/4 | 2/4 | 3.2 n=16 · 0 N/A | 3.4 n=16 · 0 N/A | 3.6 n=16 · 0 N/A | 4/clip | 4/4 generated |
| 3= | FLUX 3 Draft Black Forest Labs FLUX 3 | 6/8 | 4/4 | 2/4 | 2/4 | 2.4 n=15 · 1 N/A | 3.1 n=15 · 1 N/A | 3.1 n=16 · 0 N/A | 4/clip | 4/4 generated |
| 3= | MiniMax H3 768p local MiniMax H3 | 6/8 | 4/4 | 2/4 | 2/4 | 2.7 n=15 · 1 N/A | 3.0 n=15 · 1 N/A | 3.1 n=16 · 0 N/A | 4/clip | 4/4 generated |
| 6 | FLUX 3 Full Black Forest Labs FLUX 3 | 5/8 | 4/4 | 1/4 | 0/4 | 3.1 n=15 · 1 N/A | 3.1 n=15 · 1 N/A | 3.1 n=16 · 0 N/A | 4/clip | 4/4 generated |
| 7 | Veo 3.1 Standard Google Veo 3.1 | 4/8 | 3/4 | 1/4 | 1/4 | 3.2 n=16 · 0 N/A | 2.6 n=16 · 0 N/A | 3.1 n=16 · 0 N/A | 4/clip | 4/4 generated |
| 8= | HappyHorse 1.1 HappyHorse | 3/8 | 2/4 | 1/4 | 1/4 | 1.6 n=10 · 6 N/A | 3.3 n=11 · 5 N/A | 3.5 n=12 · 4 N/A | 4/clip | 4/4 generated |
| 8= | Sora 2 Pro OpenAI Sora | 3/8 | 2/4 | 1/4 | 1/4 | 3.1 n=10 · 6 N/A | 3.3 n=10 · 6 N/A | 2.9 n=12 · 4 N/A | 4/clip | 4/4 generated |
| 10= | Kling VIDEO 3.0 Omni Kling 3 | 2/8 | 1/4 R | 1/4 | 1/4 | 3.0 n=4 · 8 N/A | 3.5 n=6 · 6 N/A | 3.1 n=9 · 3 N/A | 4/clip | 3/4 generated 1 refused |
| 10= | MiniMax H3 2K API MiniMax H3 | 2/8 | 1/4 | 1/4 | 1/4 | 2.4 n=7 · 9 N/A | 2.9 n=8 · 8 N/A | 3.0 n=11 · 5 N/A | 4/clip | 4/4 generated |
| 10= | Sora 2 OpenAI Sora | 2/8 | 1/4 | 1/4 | 1/4 | 2.3 n=11 · 5 N/A | 2.9 n=11 · 5 N/A | 2.8 n=11 · 5 N/A | 4/clip | 4/4 generated |
| 13= | Veo 3.1 Fast Google Veo 3.1 | 1/8 | 1/4 | 0/4 | 0/4 | 2.3 n=16 · 0 N/A | 1.7 n=14 · 2 N/A | 2.4 n=16 · 0 N/A | 4/clip | 4/4 generated |
| 13= | LTX-2.5 Fast LTX-2.5 | 1/8 | 1/4 | 0/4 | 0/4 | 1.0 n=6 · 10 N/A | 2.8 n=6 · 10 N/A | 2.7 n=10 · 6 N/A | 4/clip | 4/4 generated |
| 15= | LTX-2.5 Pro LTX-2.5 | 0/8 | 0/4 | 0/4 | 0/4 | 1.0 n=2 · 14 N/A | 3.5 n=2 · 14 N/A | 2.9 n=11 · 5 N/A | 4/clip | 4/4 generated |
| 15= | Grok Imagine Video 1.5 xAI Grok Imagine | 0/8 | 0/4 | 0/4 | 0/4 | 1.0 n=1 · 15 N/A | 3.0 n=1 · 15 N/A | 3.4 n=9 · 7 N/A | 4/clip | 4/4 generated |
= shared rank · R the provider refused that scene (coverage, not quality) · n = numeric ratings and N/A answers set aside; fewer than 8 shown grey. The frozen reviewer evidence behind every row is on each configuration's page and on the full results.