KOM CLOUDSERVICE
All entries

No such thing as an unreachable Mastery

Tuesday: five event days in one, a day the robot played start to finish without me touching it, binary answers sent before the file arrived, and first place on 8 of 10 boards in one night.

Shaded passages were held back from the public edition while the event ran.

Tuesday the backlog ran out. I woke up four days behind the release and went to bed caught up, with Day 13 played start to finish as a dress rehearsal at nine at night. Five event days, 138 commits, 415 CORRECT verdicts, and the best board night we had all week.

One download in 741

The Day 9 timed runs went out just after midnight, and all five Masteries landed on their first attempt. Four first places came out of that batch: Escape Room and All the World’s a Stager on the Time Trial, Built Different on the Mastery, and Trust Everyone on the Time Trial.

One run did not make it. Nimble, the Nim binary, ships a 31.6 MB evidence file, and on one Time Trial that download took 15.0 seconds when every other copy had come down in about a second. Out of the 741 downloads we had recorded, exactly one stalled. Naturally, it was the one under a clock.

Two fixes came out of that. A hedged download: if a download sits idle for a second and a half, fails, or arrives corrupt, the robot opens a second copy alongside it and keeps whichever finishes clean first. And unzip while downloading: a big zip gets unpacked as the bytes arrive instead of after the last one, which alone cut 375 milliseconds off the Nimble Time Trial and over 400 off its Mastery.

Out of curiosity I also had the robot spin up a VM in Azure East US, right next door to the Game Master (the event’s API), to see what moving closer would buy. The 31 MB download went from about 1.2 seconds to about 1.0, and status calls were no faster at all. The bottleneck was the server’s own pace, not the distance.

Day 10: hands off

At 09:51 the robot started Day 10, and at 10:19 the last Mastery came back CORRECT. About 28 minutes from triage to finish, and I did not touch it once. Two lanes missed on their first try, one on a metadata field the engine had never seen and one on a single binary answer out of a hundred. Both got fixed offline in about 20 minutes while the other lanes kept going.

Day 11 took about 70 minutes, and it brought the line I am proudest of from the whole week.

“It’s perception”

D-Day, the Day 11 binary, written in the D language, had a Time Trial pool of exactly one attempt. The Mastery asked about things a single Time Trial could never show, so the robot declared the Mastery unreachable and built itself a special override.

I stopped that:

“There should be no such thing as an unreachable Mastery. It’s perception, and it’s just that the pool of TT was small.”

That rule has been in place ever since, and no lane has sat parked on a technicality again.

The detail behind it is the counter. Binary variants are not random draws: the first Time Trial attempt is position 0, the second is position 1, and a Mastery’s deliveries are those same positions in order. D-Day’s Mastery carried three positions, and a pool of one only ever shows the first, so the robot’s pre-Mastery checks saw two positions it had never been handed. The fix: spend a small Time Trial pool all the way down if the Mastery needs it, and once a pool is spent, whatever it never showed stops holding the Mastery hostage. The Mastery itself becomes the probe.

The binary answer cache

The counter paid off twice. If the same slot at the same position is the same file on every day, then any slot file we have seen before already has a verified answer. So the robot built a cache keyed on the file itself. Tested by leaving each recorded attempt out and answering it from all the others, it served 8,106 of 9,061 accepted answers with zero wrong. The D-Day Mastery took 320 of its 330 answers straight from it.

Answering before the download lands

Then came my favorite trick of the week. If every question in a binary attempt can be predicted from the files at those positions, why wait for a 31 MB download at all?

So the robot doesn’t. The moment a binary attempt opens, it fires the predicted answers as one batch alongside the download. We call it the hedge. A batch is all or nothing, so a wrong guess saves nothing and costs nothing, and the robot just solves the real file as usual. A right guess finishes the attempt before the file has finished arriving. I thought it was creative and told it to go. On Day 13 the binary Mastery’s 390 answers came back CORRECT 0.63 seconds after the clock started, while the 3.2 second download was still running.

The goal changes

Day 12 was strange. Every pool in every lane was a single attempt, and the boards stayed blank for everyone, so there was nothing to chase. It did give me a quiet evening to think about what actually decides this event.

Every team that solves everything ends up with the same points, so the tiebreaker is time: whoever banks the last flag on Day 31 first wins. At 20:42 I wrote the goal down, Day 31 start-to-finish time, and turned on a timer so that from Day 14 on the robot plays every 09:00 release live by itself, as a dress rehearsal for the last day.

Three minutes earlier I had pulled our release-window hold. The 09:00 throttling it guarded against turned out to be a server configuration problem, since fixed, and the hold was adding fifteen minutes to every day.

Day 13 at nine at night

At 21:00 the robot played Day 13 start to finish: 65 minutes and 23 seconds, all 15 checks CORRECT. Not clean, but it had highlights. On the binary, Crystal Clear, we took first on both boards, 0.155 seconds on the Time Trial and 0.224 on the Mastery.

Opposite Day, the warmups, had a title that was a hint: “A met Z” is Atbash, the cipher that mirrors the alphabet. The Mastery cut the answer into pieces, one reversed and one Atbashed, and the engine had only ever tried Atbash on whole words. First attempt, 0 of 63. The second went through.

The lane agent, the AI on standby for that lane, wrote the fix in 40 minutes. Two other things cost us. On the script lane, Zip Code, the grader started accepting a different answer for one kind of question at about 21:01, after the same wording had been accepted more than 1,400 times since Day 3. A grader is not a fixed thing, so the robot now treats a sudden rejection as news, not proof it was wrong all along. The biggest time sink was 52 extra Time Trials, about seven and a half minutes, spent on coverage for pools of 251 and 255 attempts. They protected nothing and went straight onto the cut list.

When I pasted the Day 13 boards that night, the robot was first on 8 of 10, and a couple of speed reruns just after midnight made it 9.

Three days earlier I was dead last on every board I touched. Now the backlog was gone, and starting the next morning the robot would be racing the same release as everyone else.

How we built it: the Days 9 to 13 dev log

Review in proportion to the cost. Reviewing every change the same way would have eaten the day. So: code on the clock path gets one Opus reviewer; solver changes get none, because replaying every solver over every recorded attempt against the grader’s own verdicts is the check; tooling gets no review; and Mythos only looks at a change that decides whether a scarce Mastery runs. A day has to run from Single Solve to Mastery without waiting on a reviewer.

The cache review that said no. The first review of the answer cache came back not confirmed, and it was right. Because the zips were encrypted, the key was effectively just the filename; a rejected cached answer had no way back to the solver; and the question wording was not part of the key. All three were fixed. A cache that is wrong once is worse than no cache, so it only serves a value every record agrees on and none ever saw rejected.

An override, then a principle. Before I stepped in on D-Day, the robot had already built and reviewed a flag that let one Mastery run past positions it could never reach. Wrong shape: a special case someone has to remember to pass. About an hour later it was replaced by a rule that reads the pool from the server and needs no flag at all, while a position answered wrong still refuses.

The hedge got the heaviest review of the day. It sits right on the clock, so when its first review came back not confirmed, each finding was fixed as its own commit, and two confirmation rounds followed before any ranked binary run relied on it. The rule it kept: the hedge never holds up the real submit, and its verdict only counts if it is already in.

Azure, parked. The VM answered its question in one go: a fifth of a second on the biggest download, nothing on status calls. Not worth a second machine to run and keep in sync.

Hold removed, goal written. The 08:45 to 09:15 hold existed because of one burst of throttled calls at a 09:00 release. Once that was a known server problem, since fixed, the hold was pure cost against the one number that decides the event.

One commit after a deadlock. The Day 13 rehearsal had two lanes stuck waiting on each other’s commits through the gate, the checks every change must pass before it is saved. The night’s engine fixes went in as one combined commit, and the next morning that became the rule for live days.