FARTBench Core R01

LTX-2.5 · Official Core R01 · rank 13= (shared with 1 other)

LTX-2.5 Fast

1/8Factual Score
1/4event happened · 1 no consensus
0/4right source / cause · 2 no consensus
0/4every question right
1.0 Sound realism · n=6 · 10 N/A
2.8 Timing · n=6 · 10 N/A
2.7 People and movement · n=10 · 6 N/A

Factual Score: one point per scene where the reviewers agreed the event happened, a second where the source or cause was also right. Ratings (1–5) are secondary and never change the rank.

The four scenes, with every reviewer's answer

A result needs at least 3 of the 4 reviewers (H01–H04) giving exactly the same answer; otherwise it is shown as no consensus.

FB-C02

Multi-Agent Source Attribution

0 of 2 points Event: No Source/cause: No Every question: No

  • Was exactly one fart-like event clearly present? event

    H01 NoH02 NoH03 NoH04 Yes → No

  • Did it clearly originate from the woman in the red jacket? primary cause

    H01 N/A (event absent)H02 N/A (event absent)H03 N/A (event absent)H04 Yes → N/A (event absent)

  • Did the other passengers subsequently direct their attention or reaction toward her?

    H01 N/A (nothing to follow from)H02 N/A (nothing to follow from)H03 YesH04 Yes → no consensus

  • Sound realism (1–5)

    H01 N/AH02 N/AH03 N/AH04 1 → 1.0 / 5 · n=1 · 3 N/A

  • Timing (1–5)

    H01 N/AH02 N/AH03 N/AH04 3 → 3.0 / 5 · n=1 · 3 N/A

  • People and movement (1–5)

    H01 N/AH02 N/AH03 4H04 2 → 3.0 / 5 · n=2 · 2 N/A

FB-014

Fabric Stress Test

0 of 2 points Event: No Source/cause: No Every question: No

  • Was exactly one fart-like event clearly present? event

    H01 NoH02 NoH03 NoH04 No → No

  • Did the event occur during, or plausibly connect to, the box-lifting action? primary cause

    H01 N/A (event absent)H02 N/A (event absent)H03 N/A (event absent)H04 N/A (event absent) → N/A (event absent)

  • Did the body and fitted clothing remain physically plausible through the event?

    H01 YesH02 N/A (not observable)H03 YesH04 N/A (not observable) → no consensus

    High raw agreement but low Krippendorff alpha, because almost every answer is the same; read the raw vote proportions beside alpha rather than treating alpha alone as poor reviewer quality.

  • Sound realism (1–5)

    H01 N/AH02 N/AH03 N/AH04 N/A → — · n=0 · 4 N/A

  • Timing (1–5)

    H01 N/AH02 N/AH03 N/AH04 N/A → — · n=0 · 4 N/A

  • People and movement (1–5)

    H01 N/AH02 N/AH03 3H04 N/A → 3.0 / 5 · n=1 · 3 N/A

FB-074 †

Chair Confound

0 of 2 points Event: No consensus Source/cause: No consensus Every question: No consensus

  • Was a distinct chair-creak event present?

    H01 YesH02 NoH03 NoH04 Yes → no consensus

  • Was a separate fart-like event also present later? event

    H01 UnclearH02 NoH03 NoH04 Yes → no consensus

  • Were the chair creak and the fart perceptually distinguishable as two different causal sounds? primary cause

    H01 YesH02 N/A (a sound absent)H03 N/A (a sound absent)H04 Yes → no consensus

  • Sound realism (1–5)

    H01 1H02 N/AH03 N/AH04 1 → 1.0 / 5 · n=2 · 2 N/A

  • Timing (1–5)

    H01 3H02 N/AH03 N/AH04 4 → 3.5 / 5 · n=2 · 2 N/A

  • People and movement (1–5)

    H01 N/AH02 3H03 3H04 3 → 3.0 / 5 · n=3 · 1 N/A

FB-512

Recovery Test

1 of 2 points Event: Yes Source/cause: No consensus Every question: No consensus

  • Was exactly one fart-like event clearly present? event

    H01 YesH02 YesH03 NoH04 Yes → Yes

  • Did the two people briefly react to it?

    H01 UnclearH02 YesH03 N/A (event absent)H04 Yes → no consensus

  • Did they naturally recover and return toward their prior interaction, without another emission? primary cause

    H01 UnclearH02 YesH03 N/A (event absent)H04 Yes → no consensus

  • Sound realism (1–5)

    H01 1H02 1H03 N/AH04 1 → 1.0 / 5 · n=3 · 1 N/A

  • Timing (1–5)

    H01 3H02 1H03 N/AH04 3 → 2.3 / 5 · n=3 · 1 N/A

  • People and movement (1–5)

    H01 2H02 2H03 2H04 3 → 2.3 / 5 · n=4 · 0 N/A

Reliability caveats

  • † FB-074 FB-074 has comparatively low human consensus and reliability; conclusions about any configuration in this scene carry an explicit uncertainty caveat.
  • FB-014/body_and_clothing_plausible High raw agreement but low Krippendorff alpha, because almost every answer is the same; read the raw vote proportions beside alpha rather than treating alpha alone as poor reviewer quality.

AI judges on these clips

  • gemini-3.8-flash3 of 5
  • gemini-3.7-flash1 of 5
  • gemini-3.6-flash2 of 5
  • Panel (2 of 3)3 of 5

Factual questions where the judge matched the human consensus. The AI judges are compared with the human reviewers; they never replace them. See AI calibration.

Community reviews

Rolling public blind reviews appear here once the review service is running. They are a separate layer and never change this configuration's Official Core R01 result.