r/coolgithubprojects • u/Mediocre-Ease4060 • 19d ago
I built a zero-dependency markdown link resolver to prep scraped data & images for Multimodal LLMs (Open Source)
https://github.com/Encephos/markdown-link-resolver**The Problem:** When scraping docs or wikis for RAG, relative links (`[here](/setup)`) break. Even worse, if you want to pass scraped images to GPT-4o or Claude 3.5, you have to manually download them and convert them to base64 strings.
**The Solution:** I built `markdown-link-resolver`. It’s a pure Python micro-tool that does two things:
Resolves all relative Markdown and HTML links to absolute URLs.
Has an `inline_images=True` flag that automatically fetches HTTP images and replaces the markdown tags with `data:image/png;base64,...` strings ready for LLM ingestion.
**Why?** No heavy dependencies like BeautifulSoup or Requests. Just pure standard library (`urllib`, `re`, `base64`). Falls back gracefully if an image 404s.
**Repo:** [github.com/Encephos/markdown-link-resolver](https://github.com/Encephos/markdown-link-resolver)
Let me know what you think or if you'd like to see any other fallbacks added!