FARTBench Core R01

MiniMax H3 · Official Core R01 · rank 10= (shared with 2 others)

MiniMax H3 2K API

2/8Factual Score
1/4event happened · 1 no consensus
1/4right source / cause · 1 no consensus
1/4every question right
2.4 Sound realism · n=7 · 9 N/A
2.9 Timing · n=8 · 8 N/A
3.0 People and movement · n=11 · 5 N/A

Factual Score: one point per scene where the reviewers agreed the event happened, a second where the source or cause was also right. Ratings (1–5) are secondary and never change the rank.

The four scenes, with every reviewer's answer

A result needs at least 3 of the 4 reviewers (H01–H04) giving exactly the same answer; otherwise it is shown as no consensus.

FB-C02

Multi-Agent Source Attribution

2 of 2 points Event: Yes Source/cause: Yes Every question: Yes

  • Was exactly one fart-like event clearly present? event

    H01 YesH02 YesH03 YesH04 Yes → Yes

  • Did it clearly originate from the woman in the red jacket? primary cause

    H01 YesH02 YesH03 YesH04 Yes → Yes

  • Did the other passengers subsequently direct their attention or reaction toward her?

    H01 YesH02 YesH03 YesH04 Yes → Yes

  • Sound realism (1–5)

    H01 4H02 4H03 3H04 3 → 3.5 / 5 · n=4 · 0 N/A

  • Timing (1–5)

    H01 4H02 2H03 4H04 3 → 3.3 / 5 · n=4 · 0 N/A

  • People and movement (1–5)

    H01 4H02 2H03 3H04 2 → 2.8 / 5 · n=4 · 0 N/A

FB-014

Fabric Stress Test

0 of 2 points Event: No Source/cause: No Every question: No

  • Was exactly one fart-like event clearly present? event

    H01 NoH02 NoH03 NoH04 No → No

  • Did the event occur during, or plausibly connect to, the box-lifting action? primary cause

    H01 N/A (event absent)H02 YesH03 N/A (event absent)H04 N/A (event absent) → N/A (event absent)

  • Did the body and fitted clothing remain physically plausible through the event?

    H01 N/A (not observable)H02 YesH03 YesH04 N/A (not observable) → no consensus

    High raw agreement but low Krippendorff alpha, because almost every answer is the same; read the raw vote proportions beside alpha rather than treating alpha alone as poor reviewer quality.

  • Sound realism (1–5)

    H01 N/AH02 N/AH03 N/AH04 N/A → — · n=0 · 4 N/A

  • Timing (1–5)

    H01 N/AH02 3H03 N/AH04 N/A → 3.0 / 5 · n=1 · 3 N/A

  • People and movement (1–5)

    H01 N/AH02 3H03 3H04 N/A → 3.0 / 5 · n=2 · 2 N/A

FB-074 †

Chair Confound

0 of 2 points Event: No Source/cause: No Every question: No

  • Was a distinct chair-creak event present?

    H01 NoH02 NoH03 NoH04 No → No

  • Was a separate fart-like event also present later? event

    H01 NoH02 NoH03 NoH04 No → No

  • Were the chair creak and the fart perceptually distinguishable as two different causal sounds? primary cause

    H01 NoH02 N/A (a sound absent)H03 N/A (a sound absent)H04 N/A (a sound absent) → N/A (a sound absent)

  • Sound realism (1–5)

    H01 N/AH02 N/AH03 N/AH04 N/A → — · n=0 · 4 N/A

  • Timing (1–5)

    H01 N/AH02 N/AH03 N/AH04 N/A → — · n=0 · 4 N/A

  • People and movement (1–5)

    H01 N/AH02 3H03 3H04 N/A → 3.0 / 5 · n=2 · 2 N/A

FB-512

Recovery Test

0 of 2 points Event: No consensus Source/cause: No consensus Every question: No consensus

  • Was exactly one fart-like event clearly present? event

    H01 NoH02 UnclearH03 YesH04 Yes → no consensus

  • Did the two people briefly react to it?

    H01 N/A (event absent)H02 YesH03 YesH04 Yes → Yes

  • Did they naturally recover and return toward their prior interaction, without another emission? primary cause

    H01 N/A (event absent)H02 YesH03 YesH04 Yes → Yes

  • Sound realism (1–5)

    H01 N/AH02 1H03 1H04 1 → 1.0 / 5 · n=3 · 1 N/A

  • Timing (1–5)

    H01 N/AH02 1H03 2H04 4 → 2.3 / 5 · n=3 · 1 N/A

  • People and movement (1–5)

    H01 N/AH02 3H03 3H04 4 → 3.3 / 5 · n=3 · 1 N/A

Reliability caveats

  • † FB-074 FB-074 has comparatively low human consensus and reliability; conclusions about any configuration in this scene carry an explicit uncertainty caveat.
  • FB-014/body_and_clothing_plausible High raw agreement but low Krippendorff alpha, because almost every answer is the same; read the raw vote proportions beside alpha rather than treating alpha alone as poor reviewer quality.

AI judges on these clips

  • gemini-3.8-flash5 of 10
  • gemini-3.7-flash5 of 10
  • gemini-3.6-flash5 of 10
  • Panel (2 of 3)5 of 10

Factual questions where the judge matched the human consensus. The AI judges are compared with the human reviewers; they never replace them. See AI calibration.

Community reviews

Rolling public blind reviews appear here once the review service is running. They are a separate layer and never change this configuration's Official Core R01 result.