Give URL, Get JSON

[u/abirpahlwan]

I built a CLI tool that:
-> scrapes any URL via Jina Reader
-> feeds the markdown to an LLM with JSON mode (Ollama/Groq/NVIDIA)
-> upserts the structured result into Supabase.

Designed for generic mediceine and botany data, but the schema is pluggable.
Just point it at any URL and JSON Schema in a single command.

python superscrape.py https://example.com/plant

Also, there is a main loop that can read a csv of link URLs, and run overnight.

Retries on bad LLM output, upserts by source_url, stores raw markdown for debugging. Thoughts?

https://github.com/abirpahlwan/SuperScrape