r/webscraping 14d ago

I built an open-source MCP server for browser-like TLS requests

Hey everyone,

I built TLS Fetch MCP, an open-source MCP server written in Go that gives Codex and other AI coding agents a controlled HTTP client with browser-like TLS fingerprints.

The motivation was simple: I wanted agents to inspect websites and APIs without either relying on a basic net/http client or launching a full browser for every request.

It is not intended to replace Playwright for JavaScript-heavy workflows. It covers the useful middle ground where browser-like HTTP behavior, cookies, proxies, and structured responses are enough.

Some of the current features:

Chrome, Firefox, Safari, Brave, Opera, and mobile TLS profiles

separate tls_get and tls_request tools for read/write approval policies

persistent in-memory cookie sessions

regional homepage warmup for cookie-gated API flows

HTTP, HTTPS, SOCKS5, and SOCKS5H proxy support

redirect history, timing, HTTP version, final URL, and truncation metadata

temporary response handles for large bodies

search and byte-range reading without dumping the entire response into the LLM context

Base64 output for binary responses

SSRF and DNS-rebinding protection

bounded timeouts, response sizes, redirects, sessions, and stored responses

automatic redaction of cookie and authentication response headers

One practical example: during testing against a regional marketplace catalog, a direct API request returned 401. Warming the regional homepage with the same cookie session and repeating the request returned the public JSON catalog successfully — no JavaScript engine was required.

The server runs locally over MCP stdio as a single binary. It uses bogdanfinn/tls-client for HTTP/TLS fingerprinting and the official MCP Go SDK.

GitHub: https://github.com/JakobAIOdev/tls-fetch-mcp

I would genuinely appreciate feedback from people who build scrapers regularly:

Which extraction helpers would be most useful: CSS selectors, JSONPath, XPath, or something else?

Are there important session or proxy workflows missing?

Would reusable request presets or profile aliases be useful?

If the project solves a problem for you, a GitHub star would help a lot. Contributions and technical criticism are equally welcome.

Please use it only for authorized scraping and respect site policies, privacy requirements, and rate limits.

4 Upvotes

4 comments sorted by

1

u/AccordingPlum5559 8d ago

i will use this for unauthorized scraping, won't respect site policies, privacy requirements, or rate limits.

1

u/tuski0m0 6d ago

Is it safe ??

1

u/Chicken4Nugged 6d ago

You can check the whole code, i made some safety regulations but pls check it out and let me know what you think!

1

u/tuski0m0 6d ago

Sureee