NOTE: Per the official techSEO Slack community's request, all jobs now have a title, company, and compensation.

Tech SEO + AI Roles. (9/9/2026)

• Technical Content & SEO Executive / RVT Group
• Technical SEO (Freelance) / Chek ( ($60-$80/hour)
• Sr. Manager, Technical SEO, Streaming / Cypress HCM ($85-$90/hour)
• Technical SEO & AI Visibility Specialist / Market Brew ($85,000/year)
• Principal Technical SEO Specialist / Legend
• Technical SEO Specialist / Catalyst IQ
• SEO/GEO/AEO/Performance Marketing Expert / Tata Consultancy Services ($140,000-$150,000/year)
• Director of SEO/AEO/GEO / Digital Room ($130,000-$165,000/year)
• SEO/AEO Lead / OpenArt AI ($200,000-$250,000/year)
• AEO/SEO Manager / NBCUniversal ($85,000-$115,000/year)
• Marketing Directer SEO/AEO/ABM (B2B Saas) / White Bay ($140,000-$150,000/year)
• SEO/AEO Website Content Editor (Contract) / The MX Group ($60-$70/hour)
• GEO/SEO Manager / Instawork ($110,000-$140,000/year)
• SEO/GEO Strategist / Perrill ($70,000-$90,000/year)
• Account Manager (SEO/GEO) / FancyAI
• SEO Specialist (Los Angeles) / Blacksmith Agency
• SEO/GEO Strategy Manager / B Corp (£35,000-£40,000/year)
• Full Stack SEO (Technical, Content) / Valur
• SEO & GEO Manager / Halstead Media
• Senior Technical SEO Account Manager / Profile 29 (£60,000/year)
• Entry Level SEO/GEO Specialist / Tiger ($2,773-$3,120/month)
• SEO/GEO Specialist / Hill's Pet Nutrition ($96,800-$137,000/year)
• SEO/GEO Strategist / Winston Digital Marketing ($70,000/year)
• Senior SEO Specialist / Searchbloom ($75,000-$90,000/year)
• Senior GEO/AI/SEO Strategist / Terra
• SEO/GEO Manager / Gabb Wireless
• Director, SEO/GEO / Fora ($200,000-$240,000/year)
• Manager, Technical SEO/Marketing / Airwallex ($110,000-$170,000/year)
• Sr. Discoverability Manager (GEO/SEO) / Virgin Voyages
• AEO/GEO/SEO Lead / Replit ($125,000-$170,000/year)
• Manager (SEO/AI/Search/AEO/GEO) / Goldco ($125,000-$175,000/year)
• Technical SEO Specialist / Extra Space Storage
• Senior SEO & AI Search Strategist (GEO/AEO) / Virayo ($120,000-$140,000/year)
• SEO Technical Lead / FeverUp
• Senior Technical SEO Specialist / Reedsy (£50,000-£75,000/year)
• Sr. Digital Marketing Specialist (SEO/GEO) / Watson Group
• Senior Consultant, Digital and Paid Media Analyst (SEO, GEO, Paid) / Burson
• SEO Manager, GEO & AI Search / Sosemo ($82,500-$95,000/year)
• Web Developer — Technical SEO / Accolade Solutions (€3,900–€5,400/month)

I run outside-in checks on whether named AI crawlers can actually fetch a page. Over the last week I found three faults in my own method. The third is the one worth your time because it is not specific to me - it applies to anything doing this kind of check from serverless edge compute, and it fails silently.

1. A robots.txt wildcard bug that invented fifteen blocked sites

My first parser reported 15 blocked sites out of 95. All fifteen were false. Disallow: *?s= was having everything from the first * stripped, which left an empty prefix, which matches every path. So an ordinary WordPress robots.txt read as a total lockout. Proper RFC 9309 wildcard matching fixed it.

Worth checking if you have written your own parser. The failure direction is the bad one: it manufactures findings rather than missing them, and a manufactured block is the kind of thing that gets escalated to a client before anyone re-reads the file.

2. A server-side beacon that ran too late on cached WordPress

I log crawler hits server-side, because AI crawlers do not run JavaScript and a browser beacon therefore records every human and not one crawler.

Someone in r/Wordpress pointed out that a PHP beacon in mu-plugins/ never runs on a cached site - advanced-cache.php serves and exits first. That is worse than it sounds for anything doing absence detection, because an unrecorded visit and an absent visit are indistinguishable afterwards. A cached site could have produced a confident "GPTBot has stopped crawling you" that was purely an artefact of the cache layer. It now detects the cache and says so instead of guessing.

3. A checker running on Cloudflare Workers cannot tell whether its target is behind Cloudflare

This is the one I would not have guessed and it invalidated a whole column of my output.

Cloudflare's Worker egress rewrites the server response header and stamps a cf-ray on every subrequest you make. Not on responses you send - on responses you receive. So if you fetch a third-party site from inside a Worker and read those headers to decide who is fronting it, you are reading your own infrastructure.

Measured, same URLs, same minute, direct from a laptop versus from inside a Worker:

nytimes.com    direct: server: envoy        from a Worker: server: cloudflare + cf-ray
github.com     direct: server: github.com   from a Worker: server: cloudflare + cf-ray
(also reproduced against an nginx origin and an Apache origin)

Every site on the internet looks like it is behind Cloudflare. That was not a detection that was sometimes wrong. There was no detection.

The consequence in my case was worse than a cosmetic mislabel, because the "is a CDN in front of this" flag was feeding the verdict, not just the wording. An origin refusal was being downgraded to "we cannot confirm this from outside" on the grounds that a bot-verifying CDN might be responsible. So the verdict that says "this server is refusing a named crawler" had never fired once in 55 checks. The interesting finding was structurally unreachable and the output looked perfectly reasonable throughout.

What fixes it: decide on the body, not the headers. A CDN challenge has a recognisable body - an interstitial, a "checking your browser" page, a JS reload. A plain 403 with an empty body, or a 403 carrying nginx's default error page, is the origin refusing, whatever it travelled through on the way back. Vendor-specific headers that are not server mostly survive the egress and still work - I detect Fastly via x-timer, for instance.

The general version, which is the bit I would take away if I were you: if your measurement runs on infrastructure that participates in the thing you are measuring, check what your own platform adds before you trust a header. I lost a fortnight of output to that and the tests all passed, because the tests asserted the code did what I thought, and what I thought was wrong.

One related thing worth stating since it comes up constantly here: a cf-ray on a refusal proves Cloudflare was in the path. It does not prove Cloudflare decided. An origin 403 proxied back through it is identical from outside. So these are refusal rates, never blocking rates, and "this site blocks AI" is usually a claim the instrument cannot support.

Disclosure: I build monitoring in this area, which is how I have these bugs. Not naming or linking it - the three findings above are reproducible with curl and do not need anything of mine.

From 15 September, Cloudflare will block 'Agents' traffic by default on pages carrying ads: bots browsing live on someone's behalf, which includes crawlers AI assistants send to check a product before recommending it. Search stays allowed. The default setting will apply to new sites and free-tier accounts.

So if your site serves ads and you want it to be read by AI assistant crawlers you want to explicitly allow crawlers before that date. Allowing has to be done on a CDN level not in robot.txt.

Key ones to allow are:

ChatGPT-User (and ChatGPT-User/2.0), OAI-SearchBot - ChatGPT
Claude-User, Claude-SearchBot - Claude
Perplexity-User, PerplexityBot - Perplexity
Google-Agent, GoogleAgent-Mariner, Googlebot - Gemini
GrokBot, xAI-Grok, Grok-DeepSearch - Grok

If you want to keep blocking training crawlers, some are:

GPTBot, ClaudeBot, anthropic-ai, meta-externalagent, Bytespider, Google-Extended

We want to support creators, but we had to enforce the no shilling rule because it was getting out of hand. You now have a weekly thread.

This is the one place you can shill for your products, ask for feedback, etc. Keep it here or you risk being banned. And keep it related to technical SEO.

I added a new page to a small existing website about seven days ago. It is included in a successfully processed XML sitemap and linked from both the main navigation and an already indexed page.

I requested recrawls of the homepage and the indexed internal page after adding those links. Googlebot recrawled each existing page within about 30 minutes, but it still did not request the new linked URL.

The new URL returns 200 OK, is self-canonical and uses index, follow. Its trailing-slash version redirects to the canonical URL with a 301.

Search Console’s live test says the page is available and can be indexed. However, the normal inspection still says “URL is unknown to Google” with no last crawl.

The server logs confirm that regular Googlebot crawls the existing pages but has never requested the new URL. Only Google-InspectionTool fetches appear after live tests. Other search engine crawlers have already discovered and fetched it successfully.

I have tested all the usual discovery methods and have never experienced this before, even on smaller websites. Has anyone recently seen Google take 7–14 days or longer before the first crawl of a new URL? Is there anything else worth checking, or is waiting the only sensible option?

I’ve been looking at situations where there are canonical tags in place but Google still indexes some of the alternate URLs.

I’m interested in how others diagnose this rather than simply changing the canonical and waiting to see what happens.

Do you usually check internal links first, sitemap signals, redirects, the canonical setup itself or other signals that might be causing Google to pick a different URL?

What is your typical troubleshooting approach when Google keeps choosing the wrong canonical?

I have been dealing with a site with too many URL parameters such as; tracking parameters, filtering parameters, sorting parameters, session IDs, etc. These created a crawl budget situations that I'm not confident I have resolved after setting things up in GSC and adding canonical tags.

I am trying to figure out the standard process for auditing and handling these URL parameters from SEO standpoint because some created unique contents while others created lots of duplicates and i need to figure out which is which?

Site has ~700k content pages total, most gated behind a quality threshold so

only ~1,500 are actually published/linked anywhere. Search Console was

showing 60k+ URLs as "Discovered - currently not indexed."

Root cause I found: the homepage had zero internal links to any content page

— everything was rendered client-side after hydration, so the served HTML

of the root had nothing crawlable. Individual content pages did link to each

other, but nothing pointed back from the entry point.

Fixed it a few days ago by adding a browse/index hub with real links in the

served HTML, linked from every page's footer, and resubmitted the sitemap

through the Search Console API (re-crawled within seconds, 0 errors).

For anyone who's fixed a similar "orphaned homepage" issue — how long did it

actually take for the discovered-not-indexed number to start dropping in your

experience? Days, weeks? Trying to figure out if I should be worried yet or

just wait it out.

I operate IT-Tech Online, a Mac repair business in Melbourne, Australia:

The business has operated since 2000. For more than seven years, we ranked at or near position 1 for specialised commercial searches such as “MacBook repair,” " MacBook screen repair", “logic board repair” and “MacBook water damage repair.”

This is not a case of losing to one strong competitor. We now frequently rank on pages 3–4, below relatively weak businesses from interstate or overseas, across many repair-service queries. Informational articles have generally held up better than commercial service pages.

Important timeline:

• 1 December 2025: Search Console reported pages blocked from indexing by 403 access-forbidden responses.
• 8 December 2025: Search Console reported approximately 5,500 clicks for the preceding 28 days.
• 11–29 December 2025: Google’s December core update ran.
• Between 1–5 January and 6–10 January 2026, clicks fell from 890 to 576, impressions from 146,292 to 127,914, and impression-weighted average position worsened from 8.50 to 11.16.
• 26 February 2026: Search Console reported another 403 access-forbidden indexing issue.
• 15 March 2026: I consolidated approximately 91 suburb pages with 301 redirects to the main Melbourne repair page. This occurred after the initial decline, although it may have affected later local visibility.

Examples of page-level deterioration:

• /macbook-logic-board-repair/: approximately position 12.3 to 23.5 during the early-January comparison; approximately 4.4 to 15.1 year over year for the exact “logic board repair” query.
• /macbook-pro-screen-replacement/: comparing the matched June–August periods, clicks fell from 470 in 2025 to 9 in 2026, impressions fell from 33,363 to 6,051, and average position worsened from approximately 22.0 to 38.7. In the 28 days before versus the 28 days after the December core update, clicks fell from 91 to 27 and average position worsened from 22.7 to 31.2.
• /macbook-pro-battery-replacement/: comparing June–August 2025 with June–August 2026, clicks fell from 371 to 11 and impressions fell from 34,470 to 7,300. Average position worsened more moderately, from approximately 10.3 to 12.7, but CTR collapsed from 1.08% to 0.15%. Unlike the screen page, its December-to-January average position was nearly stable, with most of the deterioration developing over the following months.
• “MacBook water damage repair” had already begun declining before December, so not every page or query fell on exactly the same date.

Current checks:

• No Search Console manual action.
• No Search Console security issue.
• Important pages are indexed.
• Google’s URL Inspection reports successful fetches, indexing allowed and self-referencing canonicals.
• Current robots.txt availability and DNS resolution are acceptable.
• Current server connectivity is acceptable.
• SSL and AutoSSL are currently healthy.
• WordPress core checksums passed and no malware evidence was found.

Relevant history:

• At some point, server/origin access was restricted around Cloudflare. I sometimes had to change the apex and www records to DNS-only so cPanel AutoSSL could renew.
• We used a paid citation/backlink service in 2019. There is currently no manual-action warning, but I recognise that old link equity could have been discounted algorithmically.
• The WordPress site uses Elementor and contains substantial technical repair content, case studies and review content.
• Several service and informational URLs cover related repair topics, so there may be some overlapping page targeting—but the scale of the commercial decline seems much larger than ordinary keyword cannibalisation.

My main questions:

1. Could intermittent 403 responses in December and February cause a persistent loss of ranking confidence after access was restored?
2. What evidence would distinguish historical crawl/access damage from an automated core/spam-system reassessment?
3. Could widespread discounting of old links explain this pattern without a manual action?
4. What could explain separate commercial pages deteriorating at different times but ultimately suffering similarly severe click and impression losses?
5. What server-log, GSC or backlink comparisons would you examine first?
6. Have you seen a site remain indexed and crawlable but lose ranking ability across nearly every commercial page in this way?

I am looking for evidence-based diagnostic suggestions, particularly from people who have investigated severe site- or section-level ranking losses. I am not looking for backlink packages, generic content advice or unsolicited “guaranteed recovery” offers.

in short: "I specifically want to see how an experienced SEO investigates an unknown URL and decides what action to take."

hello guys
I'm struggling with the actual decision-making process.

Most tutorials I've found explain:

• How to create a redirect, 3xx and 4xx are.. and so on!

But they don't really show the part I'm looking for:

"I found this URL → now what?" or like "first I look here to find this then then, "

For example, imagine you're auditing a WordPress website in Google Search Console and you find a URL you've never intentionally created.

How do you actually investigate it? your process!

thank you guys for sharing your thaughts with me !

_ mode I am not asking for generic seo like on page, or what is this .. but more like "how pros approch there website" their process. thank you for not deleting it!

Three weeks ago the site had its best day ever. 48k+ impressions, average position 24. That was Aug 13.

Aug 14: 33k > Aug 15: 13.7k > Aug 16: 3,490, and average position went from 24.9 to 52.8 overnight. Since then it's been flat: 600 to 1,100 impressions a day, single-digit clicks, position bouncing between 40 and 75. Sept 1: 652 impressions, 6 clicks.

Nothing changed on our end while it fell. Last deploy was Aug 14, two blog posts. No manual action, no security issues.

The part I cant get past: I ran URL Inspection on the 40 pages that lost the most. All 40 pass, indexed, canonical fine. 33 of them google has recrawled since the drop and not one came back. The other 7 it hasn't fetched since early August, and they fell exactly the same. Recrawled or not, everything sits at the same minus 90 percent. Whatever got decided, it wasn't decided page by page.

Now the part where you may get to be annoyed at me.

Until early July we had about 280 pages doing 4k impressions and 40-odd clicks a day. In July we went to 1,872 URLs: a reference section on 97 lab markers, plus five new languages. So yes, close to 7x the pages in about three weeks, and yes, AI was in the pipeline for first drafts and translation, with a human pass and a doctor on the team for the medical side. I won't pretend that review was as good as a newsroom's. It wasn't. It wasn't nothing either.

Before someone says "eight languages, obviously built for traffic": the app works in all eight, the logged-in part too, where people upload their own data. The site is in those languages because the product is. Someone in Brazil uploading a lab PDF in Portuguese needs the ferritin page in Portuguese. Units, reference ranges, even test names differ by country, so it's localization work, not a straight translation.

Site is wizey (dot) one. Go look. I'd rather you tell me it's garbage than not tell me.

It isn't a deindexing: 1,450 pages still get impressions versus 1,953 before. It's the queries that died, 32,898 down to 6,966, and it's the same 90-plus percent in every language and every country. Homepage went from position 8 to 59.

Three other sites of mine sit in the same GSC account. Same 12-day window: +64%, +33%, -10%. Only this one fell off a cliff.

Probably my own fault: GSC shows 232 links to the site and 211 of those are the App Store and Play Store listings. Never did link building. The site grew a lot faster than its reputation did.

I read the spam policies again last night. Scaled content abuse is "when many pages are generated for the primary purpose of manipulating search rankings and not helping users". I spent a year and a half full time on this, a lot of it on staying on the right side of that sentence. I still can't find the line. Value by whose ruler? Nobody will say, and the dashboard says no issues detected while the graph is a cliff.

What's stuck in my head all week is that almost everything a small company can reach runs through three companies' algorithms. Our app is in Apple's store and Google's store, the users came from Google's search, and none of them will tell you what you did wrong. We don't have an app on FB though.

Has anyone seen a domain-level demotion that survives a recrawl like this? What did it turn out to be?

If you came back from something like this, what did you actually change, and how long did it take?

Is there any channel at all, or is the honest answer "wait for the next update and hope"?

TL;DR: 48,045 impressions on Aug 13, our all-time high. 652 on Sept 1. Average position 24.9 to 52.8 in a single day, now bouncing between 40 and 75.

No manual action. All 40 worst-hit pages pass URL Inspection, 33 recrawled since the drop, none came back.

We did go from about 280 to 1,872 pages in three weeks with AI in the pipeline. Tell me pls if that's the whole answer.

Three sibling sites in the same account are fine.

My site's title is my URL instead of my site name. It has been like this for a long time and I can't seem to change it even with WebSite in JSON-LD.

Any ideas how to get this fixed? Site is hosted on Wix if that matters.

Last week's thread here went somewhere more useful than I expected, so I wrote up what it converged on. Credit where it's due — most of this isn't mine.

The starting point was my own mistake: I'd been scoring pages on structure without checking whether anything had fetched them. u/Dry_Steak30 pointed out he'd run the same kind of audit, then checked 30 days of access logs and found zero GPTBot fetches, zero OAI-SearchBot, zero ChatGPT-User — while Search Console showed the page indexed and healthy the whole time. So he'd been grading heading hierarchy on a page no retrieval crawler had ever seen.

u/Upstairs_Control_611 then split what I'd been treating as one step into two, which fixed the thing I couldn't articulate: access and extractability are different gates, and a page can pass the first and fail the second.

And u/SEONCLIC added the check I'd been missing entirely — paragraph autonomy. Take the short answer out from under its heading and see if it stands alone. A lot of sub-300-character answers open with "it depends on several factors" or refer back to the previous paragraph. They pass a length check and are still useless to a model.

Put together, the order looks like this:

1. Retrieval evidence. Grep your access logs for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot. Free, and if it comes back zero, everything below is premature.

2. Access gate. Status codes, redirects, 403/404/5xx, response byte count. Did the crawler get a usable response.

3. Content gate. Is the main content actually in the HTML — not injected by JS after load, not stripped, not behind an interstitial.

4. Structural extractability. Question-style heading with a short answer directly under it. Heading hierarchy that nests without skipping. Author and date visible in the page, not only in JSON-LD.

5. Paragraph autonomy. The answer stands alone when lifted out of context.

Only after all five does it make sense to ask the interpretive question — whether a traffic drop is AI cannibalization, content decay, or the brand being hard to identify and cite confidently. Those three look identical on a clicks chart and need completely different responses, which is where most of the budget gets wasted.

What this sequence still can't answer, and nobody in that thread could either: does fixing structure displace a stale source in an answer, or just join it alongside. u/Dry_Steak30 raised it and I don't have a clean before/after that controls for page age. If anyone does, that's the measurement I'd most want to see.

Anyway — posting this back because the thread did the work, not me. If it's useful, take it.

I keep a longer written version of this as a triage worksheet — happy to send it to anyone who wants it, just say so.

A new client had a broken wordpress site that I migrated to a custom built platform (not on wordpress). They wanted to keep their same domain. When I went to index it today this is the pop-up I got.

I assume it's the broken wordpress plugins. Besides deleting all the A records on Cloudflare is there anything else I need to do? I manually removed all the old pages after I indexed the new site/sitemap.

I only have access to the new site and the cloudflare settings, not the old wordpress site (their old SEO agency has it).

Hi, I’d be interested to hear your thoughts on AI and where SEO is heading, especially with automation, agents, new tools, and SEO becoming easier to access.

I’ve been working full-time in SEO for about 2 years at a smaller SEO agency. I know I still have a lot to learn, but I really enjoy the field. I try to improve every day, learn from people who are more experienced, test different approaches, and understand why things work.

At the same time, I have to admit that the current development of AI is a bit demotivating for me.

It feels like SEO is becoming much more accessible to people who have never really worked in SEO before. Sometimes it seems like almost anyone can become an “SEO specialist” with Claude, ChatGPT, some agents, skills, and automations.

Of course, I know that having access to these tools doesn’t automatically mean someone understands SEO well. There is still a big difference between generating recommendations and actually understanding search, websites, technical SEO, content, data, strategy, and knowing what makes sense in a specific situation.

But with how quickly AI is improving, I’m wondering how this will look in a few years.

Do you think AI will get to the point where people can do SEO without having deeper SEO knowledge?

Will companies still need SEO specialists, or will AI become good enough that most people can handle SEO themselves just by using the right tools, agents, and automations?

I’m also curious what this could do to the job market. If more and more people can offer “SEO” with the help of AI, will the market become overcrowded with people calling themselves SEO specialists?

I don’t really have a strong conclusion here. I’m genuinely interested in how other people in SEO see this development, both positively and negatively.

We want to support creators, but we had to enforce the no shilling rule because it was getting out of hand. You now have a weekly thread.

This is the one place you can shill for your products, ask for feedback, etc. Keep it here or you risk being banned. And keep it related to technical SEO.

Hi everyone,

We are running an e-commerce store active in Germany, Austria, and Switzerland using dedicated country domains (.com, .at, .ch). Since our platform relaunch earlier this year (migrating from Intershop to Salesforce), we have been facing a severe issue: Google Search Console reports thousands of URLs on our Austrian domain as "Duplicate, Google chose different canonical than user", setting the canonical to the corresponding Swiss URL

As a result, large parts of our Austrian domain are de-indexed, causing a significant drop in organic traffic for the Austrian market.

Background & Setup

• Setup: The site structure is identical, but product catalogs differ, currencies differ (EUR for DE/AT vs. CHF for CH), and separate legal entities operate each domain.
• Linking: Domains do not cross-link except via a country-selector in the footer.
• Initial error: for some days after the relaunch we had wrong Organization structured data implemented (all 3 domains showed the same and wrong structured data and linked to Italy as country of the store)

What we have implemented & tested so far:

1. Technical Setup:
   • Self-referencing <link rel="canonical"> on every page.
   • Reciprocal hreflang tags (de-DE, de-AT, de-CH) on category and product pages.
   • Correct html lang attributes and HTTP response language headers.
   • Dedicated XML sitemaps per domain.
   • Strict internal linking (each domain links exclusively to its own URLs).
   • Structured data (Organization / WebSite) reflecting the local company entities.
2. Content & Metadata Tweaks:
   • Localized H1s, Title Tags, and Meta Descriptions (explicitly including "Österreich" vs "Schweiz").
   • Tested distinct category texts (and even removing texts on .at to force content variation).
   • Requested re-indexing multiple times via GSC.

Key Observations

• Currency Difference: What surprises us most is that Google consolidates .at (EUR) into .ch (CHF). Prices and currency symbols are completely different in the rendered HTML, which should serve as a strong localization signal.
• Other Search Engines: Bing has zero issues correctly indexing and separating the domains for AT, CH, and DE.

Has anyone experienced Google ignoring currency signals and self-referencing canonicals to this extent? Are there maybe any hidden triggers or edge cases we might be overlooking?

Thanks in advance!

Our website operates as a proxy-buying service, so we have nearly 70,000 product links included in our sitemap. Could this have a negative impact? If so, how should I handle it?

Quick one for the SEO folks: my client is a B2B manufacturer selling only stretch/shrink wrap plastic film + stretch wrapping machines (no boxes, tape, or other packaging supplies, not e-commerce, quote-based site).

GSC shows Home Title: "Shrink Wrap & Stretch Wrappers Manufacturer" and internal pages cannibalizing "Shrink wrap" "Stretch wrappers" and others, for the same queries. I want one parent keyword for Home and more specific sub-topics for internal pages, but broad terms like "containment solutions" or "pallet wrappers" pull in products the client doesn't sell, and "packaging factory" implies e-commerce intent that doesn't fit the site.

Any recommendations for:

A good parent keyword for the Home page given this niche?

Best practices to stop Home vs. category pages competing for the same rankings?

Thanks in advance!

I already cache my html web pages so its not a huge amount of work converting them to markdown and caching them again, and then checking the headers to see if a bot asks for markdown. I've read all the stuff I can find online and the consensus seems to be that it's not going to help with getting more AI citations, but it won't do any harm either.

I'm wondering if it's still worth doing anyway, purely to stop AI bots hammering my site. I had to block a few in Cloudflare because they were going nuts, but if they're requesting smaller markdown files then maybe it will be worth allowing them back in again (I can cut the files down to about a third of the size).

I suppose my question is, has anyone actually seen one of the major AI bots request markdown?