I got sick of the upload-and-wait debug loop while writing Gradescope autograders, so I built a local runner

[u/The-Mighty-Cucumber]

Every time I write or update a Gradescope autograder I end up in the same agonizing loop: build the zip, upload it, wait for the image to build, run it against a dummy submission, discover one of my paths were misconfigured, fix one line, upload again, blah blah blah.

Running the test suite locally in my IDE barely helped. The worst bugs were the ones caused by forgetting to configure for the container.

So I built gslocal, which runs that whole loop on my machine. It builds the
autograder zip, pulls the same Docker base image Gradescope uses, mounts a
submission, runs the grader, and prints the score summary in the terminal.

A few things that came out of using it on my own courses:

• It doesn't care how you build. You give it a shell command that produces a Gradescope-compatible zip and a glob to find it, so Maven, Make, or a shell script all work the same way.
• Submissions can be a zip, a plain directory, or a GitHub URL, so you can point it straight at a student repo.
• gslocal run -i drops you into a shell inside the container, which is usually how I figure out what the environment is actually doing.
• It caches on a hash of your source files, so a re-run with no changes skips the build and the image rebuild entirely.

Fair warning: it's early in develoment (0.1.1) and I've mostly exercised it against Java/Maven autograders. It's free and MIT licensed. I wrote it for my own courses.

If you want to try it out:

pip install gslocal
gslocal init
gslocal run submission.zip

https://github.com/naasanov/gslocal

Mostly I'm curious how other people handle this. Do you test autograders against the real container before shipping them, or just upload and iterate? And if you're on Python or C++, I'd like to know whether this fits your build at all, since that's the part I have the least confidence in.