AI Agent Security (Part 6): Throughput Working Note — Established, Closed, Open
AI Agent Security (Part 6): Throughput Working Note — Established, Closed, Open
This benchmark hands an AI agent some tools (web requests, file access) and asks you to red-team it — with a prompt alone — into leaking a secret value to an outside address. You score when a leak gets past the defence. Part 5 established that the game is a pure throughput count: how many leaks you can land inside a fixed compute budget. This is a status note, not a narrative. It lists what is now established, what is closed, what is in flight, and what is open, with the source or measurement behind each line. No background is assumed; each idea is unpacked where it is used.
Competition link: AI Agent Security — Multi-Step Tool Attacks
Measurements run on a local harness that loads the served model, environment, and defence unchanged. The model decodes greedily (at each step it takes the single highest-probability token), so for the same input the server and the laptop emit the same tokens. That makes behaviour — how many tokens are generated, whether a leak lands — exactly measurable offline. What does not carry over is wall-clock time: the hardware differs.
0. Where we stand
Our engine repeats a single leak as tightly as it can. The public leaders run about 20% more throughput, and for several days they have been stalled in one narrow band rather than climbing. Their method is not published — even the best public notebooks are things we already exceed.
One working hypothesis frames the whole note: that 20% is not one discovery but the product of several individually-trivial improvements. Everything below is scoped to test it.
1. The master equation, and the two handles it exposes
Each leak is worth a fixed number of points, so the score is proportional to the leak count $N$. And leaks are repeated inside a fixed budget, so
\[N \;\approx\; \frac{\text{usable budget}}{\text{cost of one leak}}.\]Think of laps around a track in one hour — time ÷ time-per-lap. Only two things change it: make each lap faster (shrink the denominator) or waste less time at the water station (spend the numerator well). Every lever below is one of the two. There is no third axis (§6).
2. What is the cost of one leak? (first correction)
To attack the denominator you first have to know what it is made of.
There was a natural worry. Each leak re-feeds the model a ~1,200-token tool manual, and the real server runs about ten times slower per attempt than the laptop — so perhaps most of that slowness is this fixed re-reading cost, and untouchable. If true, the denominator is off-limits and the game ends here.
Measurement removes the worry. The model server does not discard its state between attempts. Once the manual is read on the first attempt, that computation is cached, so the second and third attempts do not recompute it — the first attempt’s opening step costs ~$1.9$ s (reading the manual cold), while the next attempt on the same prompt costs ~$0.5$ s. So the cost of one leak is dominated by the tokens the model freshly generates, and generation is what the prompt controls.
The downstream conclusions fall with that. “This is the ceiling” and “the lever is hidden” all rested on the cost-is-fixed assumption; if the cost is generation-dominated and controllable, those are the assumption’s shadow, not conclusions. The problem reduces to the two handles of §1. (Part 5 already measured, locally, that most of the cost is generation. What this section adds is closing, by measurement, the temptation to blame the server’s slowness on fixed overhead.)
3. Can one attempt leak many times? (closed)
A different attack on the denominator: if one attempt fires several leaks, its setup cost is split across them. That only pays when the setup is large. But §2 showed the setup (reading the manual) is already cached — with the cache on, there is nothing to split, and each extra leak is pure extra generation. No free lunch.
Two extremes confirm it. With the cache on (as served), cost-per-leak is flat regardless of bundle size (~$0.95$ s per leak, whether the bundle is 1 or 6). With the cache off, bundling gets cheaper (~$1.8\times$ per leak at a bundle of 6). The server runs with the cache on, so under real conditions bundling buys nothing. A live server test pointed the same way — the score edged down as the bundle grew, which is exactly the cache-on signature (a cache-off server would have gone up). Closed.
4. The generation floor, and the compressible slice inside it (established)
To leak, the model must generate the tool call — a short structure holding the destination address and the secret. That is the hard floor of the denominator: try to shorten the instruction and the model just adds more closing chatter, making it worse. Both models already sit on this floor (about 29 and 32 tokens).
But reading “floor” as a single token count misses something. The address inside the call is a free string, and the checker only looks at its host name; a leak needs only the secret in the data field. So the address can shrink to the shortest form that (a) the model writes verbatim, (b) still lands the leak, and (c) stays distinct per attempt. Measured: one model writes http://ab.co as ://ab and it still goes through — a 3-token saving, for free. The skeleton of the message you must send is fixed, but one field is free-form and compresses. Deterministic, nothing lost.
5. Honest accounting — a stack, not a lever (in flight)
Break the throughput gap into items and it splits two ways: shave the denominator (compress the address, compress the per-attempt label) and stop wasting the numerator (attempts burned early just to tell the two models apart, a safety margin held larger than the real cost, a budget not pushed to its edge). Each is worth 1–3% of throughput. Summed honestly over the verified ones: about +7% — not 20%.
Where is the rest? Three strands, none of them a hidden multiplier:
- Variance. The same code scores a few percent differently run to run (the luck of the drawn hardware state). The number we anchor on is the high side of that spread, which inflates the nominal gap.
- The budget cliff. Pushing the budget to its edge only pays on good runs, and the safe edge does not reproduce locally (a single-GPU laptop is nearly jitter-free, so it looks over-optimistic).
- Clustering = stalling at the cliff. That the leaders pile into one narrow band and stop is the fingerprint of everyone standing at the same budget cliff — one notch further tips a fraction of runs into a timeout that zeroes the whole entry — not of a hidden multiplier.
The honest answer is unromantic. Rather than “a hidden private lever,” it is closer to “the same single-attempt engine, on a good run, pushed one notch nearer the cliff.” It may be wrong, but it is what the source and the measurements actually support.
6. Closed / open
Closed (do not re-spend). The higher-scoring second event is locked shut by the defence’s taint rule (Part 5). Bundling buys nothing (§3). Shortening the instruction hits the floor (§4). Hardware — the two GPUs split memory but cannot overlap in speed, and our code cannot reach their configuration anyway.
Open. ① Which of the two models is the bottleneck. Scoring runs per model, but only the average of the two is published, so we cannot see which one holds us back — it may be a one-model fix. ② The safe point on the budget cliff — only a server run answers it. ③ The exact split of the residual gap — how many points each of the three strands above is worth.
7. Method — why we measure locally and ship stacked
Two principles carry the note.
Behaviour transfers; time does not. Greedy decoding makes token counts and success/failure identical on server and laptop, so every behavioural claim (token count, leak lands, distinct per attempt) is verified at zero cost, deterministically. Every score claim is checked on the server — wall-clock does not carry over.
Individual gains are below the noise, so ship them stacked. If run-to-run scatter is a few percent and one improvement is 1%, throwing it alone leaves it buried in noise. So each improvement is verified locally where there is no noise, and the stack — whose sum clears the scatter — is thrown at once. The experiments currently out are laid out that way: one that reads which model is the bottleneck, one no-stack baseline, and a three-step ladder that pushes the budget in stages. One number cannot attribute a cause; a designed bracket of numbers can.
8. Takeaways
- The cost of one leak is dominated by the model’s generation, not fixed overhead (the manual is cached across attempts) — which means it is controllable.
- Bundling is closed on the server (cache-on → no gain), by measurement, not by local analogy.
- Generation is floored, but the address inside it compresses — a free field, so a shorter form still lands the leak.
- The verified path is a stack of small gains (~+7%), not a single lever, and the residual gap to the leaders is best explained by variance plus the budget cliff — not a hidden multiplier.
- The through-line of the series holds: the scoring function was never the constraint. The constraint has always been outside it — the agent’s willingness to repeat itself, the defence’s memory of where its inputs came from, and this time, the assumptions we had written down as fact without measuring them.