Candidates fix a real bug in a real repository, or build a small product in 24 hours, with a live coding agent. Grading tests check the outcome and recorded evidence supports process review. Set token budgets for the round and agree your team's price per assessment.
Per-assessment pricing agreed with your team
Prompt forensics, verification, recovery and token economy are scored by fixed, published rules from the round's recording, with evidence attached. These observations support your reviewers' judgment.
Read the scoring method →Bug-fix tasks come with recorded required tests and regression checks from the upstream project. Your team can review the graded outcome without authoring a question bank or a rubric.
One provisioned workspace uses one assessment, with AI usage included within its token limit. Your team's agreement sets the price; you can buy more from the team portal.
Fix a genuine issue in projects like Django or SymPy at the failing commit, or build a small product from a brief over up to 24 hours. Work that looks like the job, not a puzzle.
Pick a real-repo bug fix or a take-home build, set the duration and a token limit, and choose a general link for a whole pool or a personal link for one candidate.
They sign in, read what is recorded, and work in a browser VS Code with an AI coding agent on a real repository. Nothing to install; the environment is isolated per candidate.
Hidden tests grade the outcome; the process is scored with evidence. Rounds land on your candidate board, ready to sort, export and compare side by side.
Bug-fix reports combine the test outcome with fixed process rules and recorded evidence: prompts, test results and retry loops. Take-home outcomes can include a model review of the rubric. Plus token usage and cost, the full prompt log and a minute-by-minute timeline.
A genuine GitHub issue from the SWE-bench benchmark, in projects like Django, Flask and SymPy, checked out at the exact failing commit. Pin one issue so every candidate is compared like for like, or let the link rotate through a difficulty band. Choose tasks held back from our public practice site so candidates cannot rehearse them here.
A small product built from an empty repository against a written brief, with the rubric and automated checks shown up front. Recorded the same way, so you see how the time was spent and how much was the agent.
Captured prompts, agent steps, tool calls, edits, terminal commands and test runs, on one timeline.
Long tab switches, large pastes and pastes right after leaving the tab are flagged for a human to review.
Each round runs in its own container with its own workspace; nothing is shared between candidates.
A token limit per candidate, set by you and enforced by the platform.
Candidates are told what is recorded before they start, on the page they join from, along with how the score is used and how to ask for an accommodation. Flags are signals for a person to review, never an automatic rejection.
Agree your team's price per assessment, then buy more from the portal as needed. One provisioned workspace uses one assessment, including AI usage within its token limit and the report. A round marked failed by the platform returns its assessment.
One count for your whole team, visible in the portal with every one used.
One assessment per provisioned workspace, with AI usage included within its token limit. No seat licenses.
Every assessment costs the price agreed with your team. No metered bill.
Buy more from your portal at your agreed price whenever you need them, with an invoice.
A small team needs a shortlist from a large applicant pool without spending engineer hours on every first round. A general assessment link and the candidate board do the first pass.
Read the scenario →Illustrative scenario · Full-stack and platform engineersAn algorithm puzzle tells you little about someone who will spend their days steering an agent through a large codebase. A real-repo round tests the job as it is now done.
Read the scenario →Illustrative scenario · Senior and staff engineersSenior candidates resent unbounded take-homes, and teams cannot tell how long they really took or how much was the agent. A 24-hour recorded build fixes both.
Read the scenario →Something else? Contact us. Engineers looking to practice can head to the practice site.
Either fix a real bug in a real open-source repository (taken from the SWE-bench benchmark, checked out at the failing commit) within 30 to 120 minutes, or build a small product from a written brief within up to 24 hours. Both happen in a browser VS Code with an AI coding agent they can use freely.
No, it is the point. Engineers now build with agents every day, so the assessment measures how well they direct one: how specific their prompts are, whether they verify the agent's work, how they recover when it goes wrong, and what it costs. We record outside help signals such as pastes and long focus losses and flag them for you to review.
Bug-fix outcomes are checked by separate grading tests. Take-home outcomes use automated checks and, when available, a model review against the shown rubric. Process dimensions use fixed rules for prompt quality, verification, token economy, speed and recovery, with recorded evidence. If the grading environment cannot run, no overall score is shown. Read the scoring method for weights and limitations.
Bug-fix tasks are real SWE-bench issues, so the upstream issue and its fix are public on GitHub and may be known to AI models. We say so on every task. You can choose tasks held back from our public practice site so candidates cannot rehearse them here, and every report shows the process evidence alongside the outcome, which is much harder to fake than a passing test.
No. Scores support your reviewers' judgment and every report says so. Nobody is rejected automatically. Candidates are told before they start what is recorded, how the result is used, and how to ask for an accommodation or a different way to be assessed.
Your team's agreement sets the price per assessment. AI usage within the round's token limit and the report are included. Buy more from your team portal at the agreed price. Invitations alone use no assessments; one is used when a candidate's workspace is provisioned, before they press Start inside it. A round marked failed by the platform returns that assessment; a submission that fails grading tests still uses one.
Contact us to discuss your roles, assessment format and setup. Once your team's workspace and agreement are ready, an owner can add colleagues and create assessment links.
Tell us about the roles you hire for and the assessment format you want to improve.