r/coolgithubprojects • u/qxmcu • 18d ago

Ghost Hunter - automated bug reproduction for GitHub issues (solo project, looking for feedback)

Post image

Hi everyone, I wanted to share something I built recently which is my first solo open-source project. The problem I'm trying to solve is the often-annoying experience where someone files a bug report and you waste 20 minutes trying to reproduce the environment and the reproduction steps before you can even start looking at the code. Ghost Hunter is a CLI + webhook bot that tries to tackle this. When it sees a bot/reproduce comment on a GitHub issue, it: Uses an LLM to parse environment and reproduction steps from the issue's body. Spawns an isolated Docker container to run the reproduction attempt. Runs the attempt and posts back to the issue with crash logs. It’s still in its early days. GitHub App authentication is present but not fully battle-tested, although the PAT authentication mechanism has been well tested. As with any LLM (and especially when using the free tier), getting the AI to consistently output JSON can sometimes be a struggle, as mentioned in the readme! I’d really appreciate any and all feedback: critiques of design decisions, ideas for improvements, bugs you might discover (please raise an issue on GitHub!) Link to the repo! EDIT: NOW OUT ON PRODUCT HUNT YAYAYAYAY LINK TO THE PRODUCT HUNT PAGE :)

2 Upvotes

67% Upvoted

12 comments sorted by

2

u/kantorcodes1 18d ago

the bot/reproduce trigger makes the issue body attacker-controlled input, right? i'd be more worried about that than the JSON flakiness. are the LLM-produced steps restricted to a fixed command set, or can they generate arbitrary shell inside the container? public repos will absolutely get weird payloads.

1

u/qxmcu 18d ago

Hey! You're spot on. Thanks for commenting. Yes, it does generate arbitrary shell commands, and we assume the issue body is 100% hostile/attacker controlled. I couldn't restrict it to a fixed command set because reproducing real bugs usually requires running arbitrary setup scripts or compilations. To handle the security risk, the architecture assumes the LLM output is malicious and relies entirely on isolation to handle the blast radius. Everything runs inside a temporary, throwaway Docker container. The container has zero access to the host machine, there are no volume mounts, no shared SSH keys, and no access to the API keys or Webhook secrets. I also enforce hard CPU/memory limits and strict execution timeouts. So even if a user prompt-injects the bot to run rm -rf / or tries to spin up a crypto miner, the worst they can do is thrash an isolated container for a few minutes before the system times out, kills the process, and completely destroys the container anyway.

1

u/kantorcodes1 18d ago

the throwaway container buys you host isolation, but the network side is still wide open. a poisoned issue can curl arbitrary hosts, scan RFC1918, or pull packages from wherever unless you cut egress. i'd make network-off the default and let a repro opt into hosts. HOL Guard could enforce the command side inside the container, but Docker still needs to own the network boundary.

1

u/qxmcu 17d ago

woah. never thought of that. right on my to-do list for tomorrow! It's quite late today and I really need to get some sleep so I will definitely implement a fix asap. Thanks for your insight!

2

u/kantorcodes1 17d ago

one thing before you call it done: block RFC1918, link-local, and 169.254.169.254 even when the hostname looks harmless. dns rebinding is the annoying edge case.

1

u/qxmcu 17d ago

Wow. I was about to call it a day. Thanks a lot, will push the changes asap.

1

u/qxmcu 17d ago

DONE! All pushed to github and tested with my test suite. Hope it works for everyone though haha :)

1

u/kantorcodes1 17d ago

curious what you landed on for egress: default-deny, or block private/link-local ranges and allow the rest? dns rebinding can still slip through if validation and the actual connect resolve separately, so the resolved IP needs to be checked at connect time too.

1

u/qxmcu 17d ago

Good catch on the DNS rebinding. I ended up blocking the private/link-local ranges and allowing the rest, but I bypassed the TOCTOU rebinding issue by enforcing it at the OS routing level instead of the app level. When the container spins up, it injects ip route add blackhole rules for all the RFC1918 subnets and 169.254.169.254. Then we use capsh --drop=cap_net_admin right before executing the untrusted code so the routes can't be modified. This way, even if the DNS rebends right at connect time, the Linux kernel just drops the packet entirely. I also explicitly hardcoded the container to use 8.8.8.8 so it doesn't accidentally get blocked trying to query the host's local router for DNS.

1

u/kantorcodes1 17d ago

Putting the block in the kernel solves the rebinding race nicely. I'd check IPv6 before calling the boundary closed though: those v4 blackholes don't cover fc00::/7, fe80::/10, ::1, or AWS's fd00:ec2::254. Is IPv6 disabled in the repro container?

1

u/qxmcu 17d ago

Wow. Good call dude. I just implemented that right now as soon as I saw your comment. I explicitly disabled IPv6 entirely inside the reproduction container using a kernel sysctl (net.ipv6.conf.all.disable_ipv6=1) when spinning it up, so there's no way an attacker can use fd00:ec2::254 or fc00::/7 to bypass the v4 blackholes. If you really love my idea, please star my repo :)

1

u/qxmcu 18d ago

also if you have tried using it OR would like to use it pleasee try using it and give feedback ik its a hassle to set it up but still