KOM CLOUDSERVICE
All entries

Trust, but verify the robot

Monday: Day 8 before breakfast, a full day spent making the robot prove itself, a 67 MB Swift binary hiding its secret in plain text, a 400 of 400 Mastery, and Day 9 started before midnight.

Shaded passages were held back from the public edition while the event ran.

Sunday was about speed. Monday was about trust. When the robot is answering in a third of a second and some Mastery pools hold a single ranked attempt, the scariest thing is not a hard challenge. It is a change I made last night that quietly broke something I solved last week.

Day 8 before breakfast

The robot was on Day 8 by 07:59. A little too eager, actually: it started the day before I had said go, so that turned into a rule on the spot. Before any new backlog day, the robot asks me first, and an announced plan I have not answered is not a yes.

Day 8 itself went well. Lost in Translation, the script lane, was a pile of polyglot files (PHP, HTA, WSF) with 615 questions, and the Single Solve went 615 for 615 in one round. Entity Relations in warmups was HTML character references like N, which Python’s standard library reads in one call, with one trap: ‐ is a Unicode hyphen, not the plain ASCII one. What’s in the Box? was about zip archives storing full folder paths, so two files can share a name and live in different rooms.

That script result was the lane engine paying off, the shared code each lane builds up a little more every day. The Day 7 engine answered 483 of the 615 on its very first try, before the AI agent on standby for the lane had read a byte.

09:00:20

At 09:00 Day 12 released, and at 09:00:20 the Game Master, the event’s API, refused five of our downloads with rate-limit errors. We had made 11 to 15 calls that minute, far under the limit we had measured the day before. A Swift Justice Time Trial attempt expired with nothing submitted.

We could not tell yet whether it was the server buckling under every team hitting the release at once or something on our side, so we did the cautious thing: no ranked clocks between 08:45 and 09:15. A day later word came back that it had been a server configuration problem, since fixed, and that rule came right back out. But on Monday morning all I knew was that the release minute had just eaten a run.

A day of proving it

Then I spent most of the day doing something that does not show up on any board. I made the robot prove itself. Four separate reviewers read the whole codebase cold and ranked what they found, and every change after that had to show it broke nothing we had already solved. The checks that guard a scarce Mastery got their own review, five rounds of it. Round five still came back with problems, and I said what I meant: “We can’t let it go unfixed.” So we fixed them.

Two things carried the day. The first was a new tool that replays every solver over every attempt we had ever recorded and refuses a change that moves an answer it did not say it would move: by the evening, 58,647 answers with zero failures. The second was a review that caught one of my own speed wins being a loss. Unzipping in-process was faster on the small archives and slower on Swift Justice’s big one, so big archives went back to 7-Zip. The rule that came out of it: one fix per commit, and every fix proves it breaks nothing we have already solved.

Swift Justice

The Day 8 binary was a good trick. The config blob the binary lane always hides was sitting inside a 67 MB statically linked Swift program, written out as base64 text. A raw scan for its magic header found nothing at all. The engine learned to search for the base64 and hex spellings of the header too.

At 21:21 the Mastery ran. 400 out of 400, CORRECT, 5.9 seconds of server time. 9th on the board. The leader was at 2.1 seconds, so there was plenty left to chase.

A count of clean runs is the wrong bar

Swift Justice looked ready after three clean Time Trials in a row. It was not. Every one of those runs had still been bringing something new. So the bar changed from “N clean runs” to “N runs that taught us nothing new”. That is the difference between the robot passing a test and the robot having seen the whole family of variants.

The counter

The best find of the day came out of staring at binary variants. They are not random draws. They come off a counter. Time Trial attempt one is position 0, attempt two is position 1, and so on. And the Mastery’s deliveries are those same positions: delivery 1 is position 0, delivery 2 is position 1.

That changes everything about a binary Mastery. The Swift Justice Mastery page said “Submit the flag after completing 5 new dynamic tasks in under 27 seconds.” We had recorded four of the five positions it would carry. For the fifth, the robot rebuilt it offline from slot files that were byte for byte identical to ones we had already seen, checked it against 76 recorded verdicts, and held the answers ready.

That 400 of 400 included the position we had never been handed. The counter meant that from then on every binary Mastery could be answered right the first time.

Calls I made on Monday

  • Call challenges by their names. Percent Chance, Node by Node, Routine Check. Not ids, not lane numbers. I want to read the logs like a scoreboard.
  • Once we hold 1st on a board, leave it alone. Reruns are for places we can still gain.
  • Time Trials prepare the Mastery. Big Time Trial pools are there to learn from. What gets protected is a correct, timed Mastery on the board.
  • And to the robot, on changes and builds: “you make the call on anything moving forward.”

At 23:07 it started Day 9. By 23:16 all five Single Solves were CORRECT. The script lane, All the World’s a Stager, went 695 for 695 on its very first try.

That was 9.6 seconds from start, without a lane agent touching it. That is what a lane engine looks like after a week of lessons.

How we built it: the Days 8 and 9 dev log

Why a day of proof instead of a day of play. My real fear was not a hard challenge. It was that the changes we made during a review would quietly break something. So the deal for every fix, in every tier, was simple: if any check disagrees, the fix does not land, and I hear why. The goal stayed the same, a correct, timed Mastery on every board of the backlog, and I kept the review to what protects the next real runs.

Why fix the slow path before picking a tool. When in-process unzip lost to 7-Zip, the easy answer was to switch back everywhere. Instead the slow part got fixed first, then both were timed on every zip shape we had recorded, with one condition: no lane may get slower. 7-Zip kept only the archives where it still won.

What we turned down. A full in-memory rewrite, so nothing touches the disk, was on the table. It would touch every engine at once, which was too much risk for the time it saved. A RAM-backed folder was tested instead: 5 to 25 milliseconds on the heavy lanes, noise on the light ones. I parked it for later. Hiding the token in printed output stayed as it was too, because it costs nothing on the clock either way.

Why no single speed limit. The first draft of the Mastery speed hold was 600 milliseconds, which was Swift Justice’s budget. Swift Justice downloads 67 MB, and a warmups file is a few lines of text. No single time fits every challenge, so each one’s budget now comes from its own recorded runs.

Why saturation beats a streak. Five clean runs in a row only tells you the solver did not trip. It does not tell you the variants have stopped changing. A new tool now says what each recorded attempt brought that no earlier one had, and preparation stops when five in a row bring nothing.

Why the hold at the end. After Day 9’s Single Solves I asked the robot to stop. The day had put a lot of solver changes in, and I wanted to look at them myself before any of Day 9’s timed runs spent a ranked attempt.