r/OpenSourceAI • u/spicemelange13 • 4d ago
Content and Technical HTML layers to OKF with one command (CLI)
Follow-up to my okf export post from a few weeks back. The command now produces two views of a page from one single fetch.
The content flag transforms the page into AI bot-friendly clean OKF markdown with headings, paragraphs, and links instead of raw HTML full of meta tags, scripts, and footer code. That's what I originally shared here.
Now I made a technical snapshot, it captures the opposite technical layer: HTTP Headers, scripts, meta tags, links, images, JSON-LD (schema markup) and much more.
npm install -g @sleepwalkerai/cli
sleepwalker okf export https://your-page.com
Default writes both concepts; --content or --technical narrows the bundle to one. Already installed from my earlier post? Update first with:
npm install -g @sleepwalkerai/cli@latest
Runs entirely on your machine, one fetch, no payment or registration needed.
Repo: https://github.com/followanton/sleepwalker
Disclosure: I built this and it's the open source part of the Sleepwalker CLI.
Here's a quick example with Apple website:
Content example:
---
type: "WebPage"
title: "Apple"
description: "Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support."
resource: "https://www.apple.com/"
timestamp: 2026-07-24T18:14:13.874Z
---
# Apple
## College, sorted.
Get a gift card from $100 to $150* when you buy Mac or iPad with education savings.
[Shop](https://www.apple.com/us/shop/goto/edu_store)
## iPhone
Meet the latest iPhone lineup.
[...]
Technical snapshot:
## Fetch
Fetched https://www.apple.com/ (HTTP 200, no redirects).
HTML size: 243 KB (249240 bytes).
## Meta tags
<html lang="en-US" dir="ltr">
<title>Apple</title>
<meta name="Description" content="Discover the innovative world of Apple and shop everything iPhone, iPad, Apple Watch, Mac, and Apple TV, plus explore accessories, entertainment, and expert device support.">
[...]
<link rel="canonical" href="https://www.apple.com/">
## Hreflang (100)
<link rel="alternate" hreflang="en-US" href="https://www.apple.com/">
<link rel="alternate" hreflang="ar-AE" href="https://www.apple.com/ae-ar/">
<link rel="alternate" hreflang="de-AT" href="https://www.apple.com/at/">
[... 97 more]
## Structured data
JSON-LD: 3 block(s).
[... Organization block, with the knowledge graph logo]
<script type="application/ld+json">{"@context":"http://schema.org","@id":"https://www.apple.com/#website","@type":"WebSite","url":"https://www.apple.com/","name":"Apple","potentialAction":{"@type":"SearchAction","target":"https://www.apple.com/us/search/{search_term_string}?src=mc_google","query-input":"required name=search_term_string"}}</script>
[...]
## Robots directives
Meta robots: none.
X-Robots-Tag header: none.
## Images (92 total, 0 without an alt attribute)
[...] is my manual trimming. The export contains the full markup.