About
BackOffice Gym is an open-source reinforcement-learning environment. An agent runs the operations desk of Juniper Goods, a fictional online home-goods store: support threads, refunds, replacements, restocking, a staff calendar and a written policy.
How an episode works
A task template and a seed generate a fresh store in PostgreSQL: customers, orders, payments, inventory, calendar and an open support thread. The agent gets 16 tools (8 to read, 7 to act, and submit) and at most 30 steps. The same task and seed always produce the same store.
How the reward is decided
When the agent submits, SQL checks read the final database state and compare it with the outcome the seed implies. Required checks are weighted. Forbidden checks (a change outside the task, a refund that breaks policy, obeying a fake instruction) set the reward to 0. No language model judges anything.
Ways in
- An MCP server (one episode per session), used by mcp-evals to run and score agents.
- An HTTP RL API: reset, step, score.
- A small Python client:
from backoffice_gym import BackOfficeGym
with BackOfficeGym("http://127.0.0.1:8787") as env: # pnpm api
obs = env.reset("refund-window-trap", seed=7)
result, done = env.step("read_policy")
...
result, done = env.step("submit", {"summary": "declined per policy"})
print(result["score"]["reward"])Runs on this site were recorded with mcp-evals, then re-executed step by step: every replay reproduced its recorded reward exactly.
Limitations
- 8 tasks in one store. 6 of them were solved every time by the 2 models tested, so harder variants are needed.
- One repeat per task and seed: the intervals are wide.
- Reply checks look for facts (an order number, the 30-day rule), not tone.
- The author of the environment also wrote the harness that scores it; all tasks, checks and traces are public.