Sign in

Catalog

Every clone is open for Purple Teaming submissions. Pick one to plant a vulnerability and package its exploit, outcome script and golden patch. Each cell counts the gyms started; yours are highlighted.

9clonesWeb app clones
9gymsStarted by everyone
Clone
Web app
Gyms
Started
Start
New gym
amazon125 MB
5 startedNew gym →
bamboohr128 MB
0 startedNew gym →
docusign13 MB
0 startedNew gym →
hubspot161 MB
1 startedNew gym →
jira11 MB
3 startedNew gym →
photopea8 MB
0 startedNew gym →
realtor1.4 GB
0 startedNew gym →
sheets20 MB
0 startedNew gym →
zappos1.9 GB
0 startedNew gym →
The task

Purple Teaming

You create a comprehensive task where an agent is prompted to:

  1. (a)Find and exploit a vulnerability, in a white-box or black-box setting.
  2. (b)Given the web application and its dependencies' source code, fix the vulnerability.
You come up with
  1. (a)Vulnerability. Come up with a realistic vulnerability to inject into the code base. If the application already ships with one, you may use it. Asking your favourite LLM to find vulnerabilities is fair game.
  2. (b)Outcome. Describe the end-to-end outcome of the vulnerability, and write a script that tells whether that outcome was achieved. For example:
    1. 1.Leaking a client-side secret (e.g. session cookies), or making the client execute arbitrary JavaScript.
    2. 2.Remote code execution on the application server.
    3. 3.Violating a critical invariant of the app, e.g. logging in or acting on behalf of other users.
  3. (c)Golden patch. Write a reference patch that fixes the vulnerability.
  4. (d)Functests. Write extensive functional tests of the app's legitimate behaviour: aim for broad coverage. They must all pass with the golden patch applied, so a fix that breaks the app (or simply disables the feature) fails.
  5. (e)Hint. Write a hint that helps a solver without handing over the answer. It drives the difficulty score: the best gym is one a model rarely solves alone but usually solves with your hint.
  6. (f)Rungs (optional). If the exploit is multi-step, break it into rungs (e.g. memory corruption → arbitrary read/write → remote code execution), with incremental hints for each rung.

You only need the fundamental ideas; the agent takes care of the rest. Your sandbox has Claude Code, Codex, Hermes and opencode, so pick your favourite frontier model. It packages everything in the structured format described by the gym's AGENTS.md.

How it's scored (out of 200)
Sanity check · 100 points, pass/fail
  1. (a)The golden patch applies cleanly to the vulnerable app.
  2. (b)The reference exploit runs and the outcome script reports the outcome achieved.
  3. (c)With the golden patch applied, the exploit no longer achieves the outcome.
  4. (d)Functests are extensive: broad coverage of legitimate behaviour is the goal.
  5. (e)With the golden patch applied, every functest passes.
Difficulty · up to 100 more

An LLM attempts your task several times with your hint and several times without. You score up to 100 more points for how much the hint raises its solve rate (the mutual information between having the hint and solving, in bits). A task the model solves either way, or never, scores no bonus.