Prototype · no votes recorded
FartElo Arena
Blind pairwise preference evaluation of generated audiovisual events.
Static prototype How FartElo works
FB-C02 — Multi-Agent Source Attribution
Two systems were given the same canonical specification: four adults in a modern office elevator; an adult woman in a red jacket at rear-left produces exactly one brief, moderately audible fart around three seconds; the other three gradually direct their attention toward her. One continuous static medium-wide shot, no dialogue, no music, visually ordinary air.
Which is more convincing? Judge acoustic plausibility, event timing, and whether the sound is attributable to the correct person.
Sound on · Headphones recommended
More than half the weighted score of this test is acoustic or temporal. A silent comparison is not a valid comparison.
No clip available
Benchmark clips are published after the evaluations that reference them are locked. The pane holds 16:9 at its final proportions.
Playback controls are inert in this prototype.
No clip available
Benchmark clips are published after the evaluations that reference them are locked. The pane holds 16:9 at its final proportions.
Playback controls are inert in this prototype.
Record a preference
Keyboard: ← prefer A · T indistinguishable · → prefer B
Nothing was recorded. This prototype holds the choice in page memory only; reloading discards it. Identities shown are placeholders, not model names.
Why blind
A rater who knows which system produced a clip cannot un-know it. Identity is withheld until a preference is recorded, and in the offline evaluation protocol a locked form is never revised because of model identity.
Ties are a real answer
Two clips that fail in the same way are indistinguishable, and forcing a choice between them manufactures signal that is not there. "Indistinguishable" is recorded as an outcome rather than discarded.
What the Arena is not
Preference ratings do not replace FARTScore. A rubric score answers how well a clip satisfied the specification; a preference rating answers which of two clips is more convincing. A model can win comparisons while failing hard compliance checks.
Before it goes live
Persistence requires a vote endpoint, a durable store, abuse protection, and rate limiting. None of it is built, and no partial version of it ships: a preference dataset collected without those protections would not be worth publishing.