Build. Scratch for Reinforcement Learning Environments

Problem: Building good RL environments is gatekept from non-technical people.

What we did: We built Scratch, but for RL environments.

Process: We developed through multiple rounds of real user interviews. We then used it to create a recursive self-improvement benchmark measuring how well models can build HUD environments.

The Twist:

• We used our product to create an RL environment that evaluates how good models are at creating RL environments

• We had GPT 5.5 and Opus 4.8 go head-to-head, each training its own Qwen 8B against a golden hidden test set (built on SC-bench).

• Our low-barrier design lets more humans generate tasks on the platform, making it economically useful.

• With the tweaks in our case study, it could hit state-of-the-art; it's nearly sellable, needing only more human-generated tasks and some robustness fixes.

• It's built on HUD and connects to real environment generation.

• We built this end to end, try the demo, and we highly recommend the slides, which walk through our whole journey.

• We also added a leaderboard and tested three Qwen models (two trained, one untrained); they tied, showing that today's frontier models still aren't very good at building RL environments.