Agent evaluation infrastructure
RL Gym is a statistically rigorous, contamination-resistant evaluation platform for AI agents. Verdicts come from N independent runs with a noise band, never a single run.
N runs
per verdict, with variance
Probe-gated
suites earn the right to count
VM-isolated
separate kernel per sandbox
Attested
to the exact suite and image
How it works
A single-run delta of 2 to 3 points can be pure evaluation noise. RL Gym is built so that by the time a number reaches you, noise, contamination, and forgery have each been dealt with explicitly.
Every evaluation runs your agent N independent times and reports the spread, not a cherry-picked score. The verdict comes with variance and a noise band derived from power analysis, so you know whether a delta is real before you act on it.
Before a suite’s numbers count, it has to hold up against a null agent that does nothing, a random agent, prompt injection, and state tampering. A suite a do-nothing agent can score on is measuring its own flaws, not your agent.
Agent code executes in Firecracker-class microVMs with their own kernel, fully isolated from the evaluator. Reference answers are never readable by the agent, so a score cannot be obtained by reading the answer key.
Each verdict is attested to the exact suite, backend, and image that produced it. When you compare two versions, you can prove both numbers came from the same trial, not two different ones.
Pre-launch
RL Gym is pre-launch. Join the waitlist and we will reach out as access opens.