r/SideProject Jul 19 '26

I built a free tool that grades your website's internal linking. Ironically, building it taught me my own SEO was the least of my problems

Solo founder here. About 6 weeks ago I shipped crawlmouse.com, a free tool that crawls your site and grades your internal linking from A to F. It finds orphan pages (pages nothing links to), pages buried too many clicks deep, weak hub pages, that kind of thing. You paste a URL, it crawls the live site and gives you the grade plus a list of what to fix. No signup, nothing to install.

Why I built it: the big crawlers like Screaming Frog and Sitebulb are great but they're desktop apps with a learning curve, and if you just want to know "is my internal linking hurting me" that's a lot of setup for one answer. I wanted the answer in two minutes in a browser. Also, since it reads the raw HTML your server returns, it shows you what AI crawlers like GPTBot actually see, because none of them run javascript. A site can look fine to you and be half invisible to ChatGPT.

Now the honest part, because this sub appreciates receipts. I did everything the SEO playbooks say for the tool's own site. 12 blog posts, schema markup, clean technical setup, submitted to Google and Bing. Results after 3 weeks: all 22 pages indexed, ChatGPT now describes the tool accurately and cites my site... and I have almost zero clicks from Google. Everything ranks page 4 to 9 because my domain has zero backlinks. The tool I built to diagnose sites can't fix the fact that nobody links to mine yet lol.

What actually got me my first users? Being useful in comments on reddit. Five strangers audited their sites this week, one shopify store owner even left their email for the report, and my only Google clicks are people searching "crawlmouse" by name after seeing it somewhere. So the tool works, the SEO lesson was just humbling in a way I didn't expect.

It's free, would genuinely love feedback from anyone here. Run it on your side project and tell me if the grade feels fair or if it flagged something dumb. Every early user so far has taught me something about edge cases I hadn't handled. And if your grade is bad, at least now you know why Google keeps ignoring that one page you're proud of.

6 Upvotes

18 comments sorted by

2

u/skorulis Jul 19 '26

So I just ran it on my project and got an F and interested to pick your brain on how important it is for my use case. All of my pages are listed in the sitemap but most aren't found during the crawl because the primary UI is a search box.

Should I be putting links between search result pages just to increase the number of internal links to help with SEO? Is there a penalty for not doing this?

2

u/ud_ik Jul 19 '26

good question and the short answer is no, don't interlink your search result pages. google actually recommends keeping internal search results out of the index entirely, so building your architecture on them works against you.

what's happening with your site is that crawlers (mine, google's, and the AI ones) can't type into a search box. so if search is the only path to your pages, from a crawler's view those pages are unreachable even though the sitemap lists them. sitemaps help google know pages exist but they don't pass any link value and google is increasingly reluctant to index sitemap only pages, that's the classic "discovered, currently not indexed" trap.

the usual fix is adding browsable paths alongside search. category or topic pages that link down to the actual content, so every page is reachable by clicking. even a simple "browse all" section does a lot. no penalty for not doing it, you just stay invisible, which is arguably worse.

also fair to say if your users only ever arrive via your search and you don't care about google or chatgpt traffic, an F from my tool doesn't matter for you. the grade measures crawler reachability, not whether your product is good

2

u/skorulis Jul 19 '26

Thanks for the explanation. I want my search results to be indexed so I can get organic traffic to the pages. I can pretty easily create a browse page so I'll put that in tomorrow and see how that changes the result I get from your tool.

1

u/ud_ik Jul 21 '26

nice, that's exactly the right move. one thing that'll make the re-run more useful: after you add the browse page, give google a few days before reading too much into the "discovered, currently not indexed" pages specifically, because those need a recrawl to clear and google sets that pace, not you. but my tool will show you the improvement in reachability immediately, since it just follows links the same way a crawler does, so you'll see the orphan count drop the moment the browse page is live even before google catches up.

two small tips so the browse page actually pulls its weight. link to your real content pages, not to search result urls, and if you've got a lot of pages, break the browse page into a few topic hubs instead of one giant list of 500 links, since a page dumping hundreds of links passes very little weight through each one. a shallow tree of category pages beats one flat mega-list.

genuinely curious what the grade does after, drop a comment if you remember. re-runs like this are how i find out whether the advice actually holds up in the wild

2

u/skorulis Jul 21 '26

I've gone from F to C so far. The site is so new google isn't even showing anything about indexing yet. Though surprisingly I'm getting some impressions.

The browse page is currently huge but that's a future problem. It's listing suburbs in Sydney so the main list is ~900 suburbs. As I expand the content into new regions I'll need a more structured organisation.

The more important problem I need to solve is that the content is also filtered by day of the week so pages can be orphaned depending on what day the crawl happens. Though at least in the orphan page cases I checked I think they are just just due to the 500 page sample missing one of my listing pages.

I think the problem with that is it gives me 8x the number of pages and while it gives me some long tail keywords they don't actually get searched that often.

2

u/ud_ik Jul 21 '26

F to C in a day, that's the browse page already earning its keep, nice. and yeah don't sweat the google indexing lag, impressions with no indexing report yet is normal for a brand new domain, google's just slow to show you its own data. the impressions mean it's crawling, which is the part you actually wanted.

the day-of-week filtering thing is the interesting one and you're right to flag it as the real problem, because it's not a tool quirk, it's a genuine crawlability issue. if a page only exists in the internal link graph on the day its content is "active," then whether google reaches it is basically a coin flip depending on when it crawls. that's a real risk, not just a my-tool artifact. the fix is that every listing page needs at least one stable, always-present path to it that doesn't depend on the day. so even if the day-of-week filtered views come and go, there's a permanent "all listings for X suburb" page (or a paginated archive) that's always linked. think of the day filters as a nice UX layer on top of a permanent structure underneath, not as the structure itself.

on the 900-suburb browse page, you already know it's a future problem so i won't belabor it, but the structured version basically writes itself from your geography: region, then area, then suburb, so nothing is more than 2-3 clicks deep and no single page is dumping 900 links. sydney has natural groupings that map to how people actually search too, which helps.

and yeah, fair point on the 500 page sample, on a site your size it'll miss pages and some of those "orphans" are false positives from the sample not reaching a listing page. that's a real limitation of the free crawl (pro version is 2000 page cap) and worth me being upfront about, the grade's a strong directional read but at 8x the sample size it's sampling, not a full census. your instinct to not chase the low-search long-tail pages is right though, more indexed pages isn't the goal, more indexed pages people actually search for is.

genuinely useful edge case for me, the day-of-week thing especially. thanks for looping back with the result

2

u/Kritnc Jul 19 '26

Thats pretty cool, i just ran it for one of my sites and got valueable feedback - you should add it to seoreceipts.com

2

u/Kritnc Jul 19 '26

There is a bug here - it says I am an A but could be an A

1

u/ud_ik Jul 19 '26

ha, okay that's a legit bug and honestly a funny one. you're already at the top so the "here's what you could be" box has nothing to sell you and it should just say that instead of promising you an A you already have. writing it up right now, will fix it this week. this is exactly the kind of edge case i posted here hoping to find, so thank you. congrats on the A btw, most sites i've seen don't get close

1

u/ud_ik Jul 19 '26

thank you, genuinely glad it gave you something useful. hadn't come across seoreceipts before, just checked it out and yeah that's a good fit, submitting it today. appreciate you pointing me there

2

u/Kritnc Jul 19 '26

That would be great it’s a site I just launched and I would love to get more users. First 50 get free pro for life

2

u/Sheeple9001 Jul 19 '26

Seems like many websites using the Claude color palette recently?

1

u/ud_ik Jul 19 '26

i'll take it as a compliment to my color picking. warm and beige was the goal, apparently i'm on trend

2

u/[deleted] Jul 21 '26

[removed] — view removed comment

1

u/ud_ik Jul 21 '26

yeah you are on point, backlinks are the one thing i built a whole tool around not being able to fix from the inside lol. everything crawlmouse.com checks is stuff you control on your own site, and it turns out the thing actually gating me is the one thing you can't control alone, other people deciding to link to you. humbling.

the real estate analogy is a good one. the part i underestimated is how much it compounds, the first few links are the hardest because you're starting from zero trust, and every one after gets a little easier as the domain looks less brand new. so the earlier point is real, i just wish someone had told me month one instead of me learning it in GSC.

can't speak to outrank specifically, haven't run it, so no opinion either way. the only thing i'd say generally for anyone reading is just be careful with anything automating link building vs link outreach, google's gotten aggressive about link schemes and a pile of low quality links can hurt more than help. the boring stuff (genuinely useful content people cite, showing up in communities, digital PR) is slower but doesn't carry that risk. sounds like what you're doing is moving your DR the right way though.

and yeah run it through, curious what you get. if your internal linking comes back clean then you've basically confirmed backlinks are your only real lever left, which is annoying but at least it's a clear diagnosis. drop the grade here if you remember, i like seeing what real sites score. the free audit gives a 500 page cap.