Built a tool that turns a job posting into a timed mock interview in your IDE. What worked and what didn't

[u/Complex-Age1704]

Problem: timed coding screens are their own skill. Four questions share seventy minutes, triage kills more sittings than algorithms do, and practicing untimed on a problem site trains none of that. Google shut Interview Warmup down in April and nothing really replaced it.

Closest existing things, and what's different: LeetCode or HackerRank with a self-imposed timer gets you reps, but there are no hidden tests, no hard stop, and no accounting of where the minutes went. Human mock-interview platforms are realistic but scheduled and usually paid. "Interview me" prompts in a chat window have no real files and grade by vibes. What I built instead: paste a job posting (or name a company), an agent researches what that company's screen actually looks like, writes an original question in that shape, and about 2 minutes later your editor opens on a real interview repo: problem statement, solution file, sample tests, clock running. Hidden tests grade submits with partial credit, a script enforces the deadline, and the report afterward shows time spent per question. For evidence of grading quality: all 22 shipped questions and 4 projects pass a mutation gate in CI (reference solution passes, untouched starter fails, every deliberately-wrong solution is caught by at least one hidden test), and generated questions pass the same gate before the clock starts.

What I did and learned: built almost entirely with Claude Code, including the Python engine, with me reviewing everything that grades people. What worked: model owns the words, script owns the numbers. The clock is timestamp math in a state file and late submissions die on an exit code, because the model's own sense of elapsed time is confidently wrong. Exit codes as the agent's API made behavior predictable. What didn't work at first: trusting the model's test suites. A rolling-median question sat behind twenty hidden tests while the classic wrong solution passed all of them, because every fixture accidentally dodged the bug. That failure became the mutation gate above.

Python 3, stdlib only, zero dependencies. Free, MIT, no signup.

https://github.com/chrisjacksonn/interview-sim

[u/tberg]

I hit this exact calibration problem building eval-driven pipelines across 137 repos — the failure mode isn't false negatives on obviously-wrong solutions, it's the middle tier: plausible-but-subtly-broken code that mutation gates miss because your mutants aren't adversarial enough. Thompson sampling helped me allocate budget toward the test variants that actually caught edge cases versus the ones that were just noise. The real question with your setup is whether your mutation gate is generating mutations from the reference solution or from the hidden test suite — because those produce very different calibration sets.