WEBVTT

00:00:00.270 --> 00:00:04.390
Your team tests a new search setting on forty questions.

00:00:05.740 --> 00:00:07.110
Nine runs crash.

00:00:07.520 --> 00:00:08.070
Poof.

00:00:08.160 --> 00:00:09.750
They vanish into the hat.

00:00:10.480 --> 00:00:11.770
The rest look great.

00:00:12.080 --> 00:00:13.630
A friendly judge agrees.

00:00:13.940 --> 00:00:13.940
Ta-da!

00:00:13.960 --> 00:00:16.130
Eighteen points better.

00:00:20.680 --> 00:00:22.770
Research turns the house lights on.

00:00:24.030 --> 00:00:27.720
Behind the curtain, all nine crashes are still there.

00:00:27.890 --> 00:00:29.260
They stay in the count.

00:00:31.400 --> 00:00:35.190
And the friendly judge gets checked against the actual source text.

00:00:37.460 --> 00:00:43.150
Before the next run, you freeze the plan: the questions, the scoring, the stop rule.

00:00:43.450 --> 00:00:48.650
You get the protocol, and an honest answer: no improvement claim yet.

00:01:00.190 --> 00:01:00.720
Research.

00:01:01.030 --> 00:01:03.710
Make the answer traceable to the evidence.