Method
You've seen AI-generated exercises. This isn't that.
The fastest way to earn that claim is to show you real quests from our own repositories — with the evidence they were grounded in and the answer keys they were graded against. So that's what this page is.
Grounded, or not generated.
Every quest starts from a scout candidate: a finding in your repo's recent history, anchored to a verbatim excerpt of your code. The excerpt is the contract — if a quest can't quote the code it's about, it is not generated. There is no fallback bank of synthetic exercises behind that rule. An unproductive repo day means fewer quests, not fabricated ones.
Structure is fixed. The model doesn't improvise.
Each of the five kinds is a fixed sequence of beats — locate predict explain fix — chosen by deterministic routing over your git history, not by a model's mood. A Reckon is always locate → predict; a Reclaim is always locate → predict → explain → fix. The LLM writes the narrative around the beats; it does not get to invent the test.
Graded against answer keys, not vibes.
Answer keys are fixed when the quest is composed — before you ever see it. locate and predict are scored purely mechanically: your click either lands on the keyed line or it doesn't; your choice either is the keyed answer or it isn't. explain is graded against a written rubric fixed at compose time. fix carries a fix contract — files it must touch, behavior it must preserve — scored deterministically and used as a ceiling on anything a model thinks of your diff.
Specimen A · Reclaim
The Welcome Bonus That Never Fires Twice
A real quest from our API repo, generated because an agent wrote the first-quest bonus logic and the git history shows we hadn't re-read it. Reproduced in full — including the answer key you never see while solving.
The evidence it's grounded in — quoted verbatim from the repo
const firstQuestBonus = progress.totalXp === 0 && input.outcome !== "needs-work" ? 40 : 0;01 · locate
Find the line in progressService.ts where firstQuestBonus is assigned its value.
Answer key: the exact line above. Your click either lands on it or it doesn't.
02 · predict
What value does firstQuestBonus evaluate to when progress.totalXp === 0 and input.outcome is "needs-work"?
- a. 40, because totalXp is 0 so the user is new
- b. 0, because the outcome check fails the ternary condition ✓
- c. 0, because totalXp is always non-zero by this point
- d. 40, because needs-work outcomes are excluded from totalXp
Answer key: b, fixed at compose time. Scoring is a string comparison — no model in the loop.
03 · explain
Explain how the firstQuestBonus line decides what value to assign — walk through both conditions and what the combined logic means.
The rubric your explanation is graded against:
- Identifies that the ternary requires BOTH conditions to be true simultaneously
- Explains that progress.totalXp === 0 checks whether the user has accumulated zero XP so far
- Explains that input.outcome !== "needs-work" checks the outcome of the current attempt
- Recognises that a user whose first attempt was "needs-work" will have non-zero totalXp on their next attempt, so the first condition fails
- Concludes that only a user with zero prior XP AND a passing outcome receives the 40-point bonus
04 · fix
Modify the guard so a user whose very first attempt was 'needs-work' can still qualify for the bonus on their first passing attempt.
A real bug, found by the quest pipeline in our own shipped code. The fix is graded against a contract — which files must change, which behavior must hold — before any model opinion is consulted.
Specimen B · Reckon
The Symbol That Isn't There
A two-beat Reckon on our quest-generation code itself — generated because the history said we'd lost touch with the file.
The evidence it's grounded in — quoted verbatim from the repo
export function groundedEvidenceFor(c: ScoutCandidate, file: string) {
return c.evidence.find(
(e) =>
(e.kind === "code-line" || e.kind === "code-comment") &&
e.file === file &&
(e.excerpt ?? "").trim().length > 0
);
}01 · locate
Find where this function decides an evidence entry is acceptable to return.
02 · predict
Which conditions must ALL be true for groundedEvidenceFor to return an evidence entry?
Four close-reading options; the key is the one that matches the code exactly (kind is code-line/code-comment, file matches, trimmed excerpt non-empty). The wrong options are each a plausible misreading — that's the point.
A third kind, Reconcile, works multi-file seams — a real one from our repo pairs measureComplexity with excerptSubstance, two functions that must agree on what "complexity" means, and asks you to make them agree.
And when it's wrong
We grade our own generator.
Every generated quest can be pulled into an internal review queue where we score it 1–5 and tag failures — including "hallucinated", the rejection we care most about. Those reviews become checks in the pipeline: each failure mode we observe gets a rule that mechanically blocks it from composing again. Slop is a process failure, and we treat it like one.
The alpha is invite-only. Free to start, on your own repo, in your own editor.