How the robot works

Huntress ran this year as an "AI Arena": 31 days, a new day at 09:00 ET, and the same five categories of challenge every day, plus a sixth, web, from Day 19. I call each category a lane. The organizers openly asked players to build deterministic solvers and let the robot play, which is about the best invitation I have ever received. The challenges come from the event's API, the Game Master, which hands out the files and grades the answers.

Every challenge comes in three modes, and each one unlocks the next:

• Single Solve: no clock. Download a file, answer its questions.
• Time Trial: a fresh variant of the file against a server clock. That clock was 30 s on Day 1, 6 to 10 s by Days 4 and 5, then 30 to 325 s from Day 15 on.
• Mastery: several variants at once, often in shapes the Time Trial never showed, against one shared clock.

One finished mode is a check, so five lanes times three modes makes 15 checks a day, 21 or 24 once web arrived. Day 30 threw in one extra: SHA1-Hulud, an old-fashioned reverse engineering challenge, Single Solve only and worth 5 points. Remember that one.

Three rules of the game shaped almost everything I built:

• One wrong answer sinks the whole batch. Send 1,500 answers together and a single bad value saves none of them. Send answers one at a time and a wrong one costs nothing.
• Ranked attempts run out. One Time Trial had 255 of them; one Mastery had 2. After that, only untimed practice is left.
• The grader's wording is the answer. Is that program ET_EXEC or EXEC, 64 or ELF64? Only the grader knows. If the thing a question asks about is missing, the answer is the literal word none.

On top of that, the server takes about 100 calls a minute from one team, every kind of call on the same counter. That sounds like plenty until a robot gets excited.

Then came the realisation that defined the whole project. There is no dynamic scoring. Every team that solves everything ends on the same points, so the order among them comes down to who banks their last flag first. Day 31 would decide it. From then on every morning was a dress rehearsal, judged on one number: seconds from the release to the last check correct. I call that stretch the race. The leaders were already doing all 15 checks in 45 to 60 seconds.

That is exactly how it ended. Fifty-six teams finished every check on 1,885 points. On Halloween morning the top four banked their last flags at 234.1, 236.4, 255.1 and 255.5 seconds. We were the 255.5.

The whole design is one bet. Most of what this event asks, the robot has seen some version of before, and for those it uses ordinary code: read the file, find the answer, submit. No AI model sits in that path. It is fast, it is the same every time, and it never refuses.

The AI comes in at the edges, where something is genuinely new. When the robot meets a question it cannot answer, it writes down exactly what it is stuck on, and AI agents waiting on standby take it from there. Some write a small fix to the code; others just answer that one question directly. A fix written by a model never touches the race until an automatic judge has tested it against every answer the grader has ever accepted. A refusal or a slow model can delay a fix. It can never stop the robot.

After each race the day's fixes become permanent code, each with a test, so tomorrow's first try starts further ahead than today's.

Two principles held across both generations of the robot. The lane is the unit of learning, not the challenge: the challenges are new every day, but the lanes repeat, so every lesson goes into that lane's shared engine. And right first, then fast, which later sharpened into something blunter: a miss only costs time. Once my board showed that a practice completion still earns the check mark, a miss stopped being fatal and became a delay.

Take Day 29. Release to the last of 21 checks took 72.5 seconds, nobody beat it that morning, and it put us 1st overall. Here is that morning from the robot's side, in the order things happen.

Before the bell

At 08:30 the robot gives itself a checkup before it is allowed to race. Can it reach the Game Master? Is its clock in step with the server's? Is there disk? Does every solver load? Does each lane still get its old answers right when it replays files the grader has already marked? Has the event's API changed overnight? Anything broken shows up here, not at 09:00.

At 08:50 the race starts up, with a watcher alongside it: an AI session that follows the race live and can step in. Every lane gets its own worker, already warmed up, solvers loaded, connections to the server already open, so nothing has to start up after the release. A minute or two before 09:00 the AI side gets ready too. The judge loads each lane's answer key, and the agents start and wait. On a morning with nothing new, they never get a job.

The bell

From 08:59:00 the robot starts asking whether the day is out. It never waits on one slow answer: it asks several ways at once, faster and faster in the last few seconds, and whichever signal comes back first wins. Warmups gets a shortcut. Its challenge name is predictable, so the robot can ask for it by name and start it before the day's list has even appeared.

The known stuff goes in seconds

Each lane downloads its file, unpacks it while it is still arriving, runs its solver on every question and submits. Answers the solver is sure of go in one batch. Answers it has to guess at go one at a time, because a wrong single costs nothing and a wrong value in a batch costs everything. The moment the Game Master says CORRECT, the lane starts the Time Trial, then the Mastery, the same way. On Day 29 every file lane, all fifteen of its checks, was done by 27.0 seconds. No model touched any of it.

The new stuff gets a ticket

When a solver has no answer, or the grader rejects the one it sent, the lane writes a gap ticket within 0.3 seconds: which question, which file, what it tried and what the grader said. The other lanes keep racing, and a timed attempt still inside its deadline simply stays open while help arrives. From Day 15 on the clocks were long enough for that to work.

The agents on that lane grab the ticket. A patch agent writes a small piece of code or data that teaches the solver the new shape, then hands it to the judge. The judge runs it locked in a sandbox against the lane's whole graded history, and lets it through only if it answers the ticket without changing a single answer the grader ever accepted. The running lane picks the patch up, re-solves its open questions and submits. Meanwhile a hand agent can work the question the way a person would and queue an answer for just that attempt. Whichever gets there first closes the question.

On Day 29 the last check was the second web challenge's Mastery. It came in after one hand answer, about 17 seconds of work, and one judged patch, 16 seconds. That was the morning: two small new things, handled while the clock ran, and 72.5 seconds on the board.

After the finish

Every race attempt goes into the graded record: every file and question the robot has ever been served, with the grader's verdicts. That record is the answer key every later change is tested against. Then comes the day close. The day's live patches get folded into proper solver code, the whole day is replayed with no agents at all, and nothing is saved until it passes the commit gate.

The race path

When a lane gets stuck

The gap ticket is the handoff, and keeping it small was one of the best decisions in the project. It holds only what the solver already knew: the question, the file, what it tried, what the grader said, and a one-word reason it got stuck. It ends with a work order: the shape of answer wanted, an earlier example, where to put the fix, and the command that judges it. An agent does not have to go exploring; it starts with everything on one page.

The agents. Every lane has three patch agents and an Opus hand agent waiting on it; web and the spare lanes also get a Sonnet hand agent. Patch agents write fixes. Hand agents answer the question directly for that one attempt, and when an answer is accepted, the method behind it can be handed to a short-lived derivation agent that turns it into a proper fix, so the next file gets solved, not just answered.

The judge is the reason I could let AI patch a race while it was running. It runs every fix in a locked box with no network and checks it against the lane's whole graded history. A fix passes only if it answers the stuck question and leaves every answer the grader ever accepted exactly where it was. That does not prove a new answer is right; only the live grader can say that. What it proves is that the fix broke nothing.

Hand answers are the one deliberate shortcut past the judge. A hand agent can queue a value for one specific attempt, and the robot sends it on its own, never inside a batch. A wrong hand answer can waste time, but it can never sink a batch of good ones.

Carry means reusing a fix or an accepted answer from one mode in the next mode of the same challenge. Carried code starts out untrusted, sends its answers one at a time, and earns trust only when the grader accepts it.

Backup and memory

Where the AI sits, and what fences it in

Security challenges look a lot like malware, because they are training versions of it, and AI safety filters do not always know the difference. On Day 5 a script specimen drew a false-positive refusal. On Day 20 all three web patch agents and the first fallback model were stopped while writing exploit steps; a replacement read the challenge prompt and solved it at 276 s. The real answer was structural: make every known solve plain code, so a refusal can delay a patch but can never stop the robot. A refused ticket goes straight to the fallback models.

Nothing from a challenge is ever run on my machine. Files are unpacked carefully, read and decoded, never executed; anything that truly has to run goes in a sandbox.

I wanted the failures in here, because most of the robot's rules trace back to one of them. Four mornings did most of the teaching.

Day 14, the first live morning: 36th. The robot took 658.9 s; the winner took 13.6. We spent 97 s on model work before the first clock even started, a warmups detour held up three lanes that were ready to go, and a new shortcut-file format sat waiting on the main session. The fix was the whole idea on this page: every lane in parallel, known shapes as plain code, no model in the path.

Day 19, the first web day: 62nd, and entirely my own doing. The poller expected the web challenge under Day 19, but web numbered its own days. So it kept asking for 66 minutes, 23,383 calls, two thirds of them resends, and the server started holding our account 20 to 50 seconds per call. The file lanes were done by 293 s. Web was solved by hand at 7,110 s.

Day 23, the worst morning on record: 20th. About 15% of the script answers were wrong but marked as certain, so they rode inside all-or-nothing batches and sank every good answer with them. Resends burned the clock, more than fourteen ranked attempts expired, and the script lane collected over 2,500 rate-limit refusals. Afterwards, one batch first, then smaller packs by question type, and nothing resent after a deadline.

Day 31, the one that cost the podium: 4th by four-tenths of a second. Five lanes were done by 76.9 s. The last web challenge, Framework Roulette, named the header it read its role from only in a Vary response header, and every tool the robot had read the body of a response, never its headers. About 228 of our 255.5 s went on that one Single Solve. The fix is small: read the headers too. It came one day too late.

And one that was not a bug at all: on Day 30, SHA1-Hulud, a classic crackme with no questions to open a ticket on, took 1 h 48 min by hand, and its 5 points dropped us from 1st overall to 10th that afternoon. Some things you just solve.

Every decision I made went into the instruction file with a date, and about two dozen of them were later reversed. Read in order, the reversals tell one story: the goal moving from "never waste a scarce attempt" to "seconds to all checks correct". A few of the big ones:

• One clock at a time (Oct 11) became parallel clocks in a release race (Oct 14).
• Protect a correct timed Mastery and prepare until nothing new shows up (Oct 12) became: a Mastery with more than 3 attempts goes as soon as possible (Oct 14), because a miss costs one attempt and nothing on the board.
• Review everything (Oct 12) became review in proportion to risk (Oct 13), because a day must run from Single Solve to Mastery without waiting on reviews.
• A self-imposed cap of 80 calls a minute became no cap in a race at all (Oct 14), only a guard against a runaway loop: never hold answers back on a guessed limit.
• v1 is the robot became v1 frozen as the manual fallback and v2 built from scratch (Oct 18), after Day 18's 92 s against a 45 s winner.

Everything here comes from the project's private repository: the instruction file and its dated decisions, the daily plan addenda, the design and contract documents, the session log, technique notes, lane playbooks, review files, git history, and the leaderboard record generated from the public scoreboard API. Line and test counts are from wc and grep, not from running the suites, and agent and judge timings come from the robot's own traces.

A few things are worth knowing when reading the numbers. Day 19's board time is 7,110.4 s because the web lane was solved by hand late; the slowest file lane finished at 293 s. Day 17's 1st place was at 674.7 s against 985.0 s, on a challenge the organizers later reissued. Speed-board counts change as other teams improve their best runs. The agent-director system exists only in parked work and was never part of the racing robot.

Left out on purpose: flags, the team token, challenge artifacts and answer values, and the details of the private infrastructure the robot runs on.

The day-by-day story is in the diary.