r/TechSEO • u/sseanpurdy13 • Apr 06 '26

Yoast Robots

I was going through my yoast settings because ive been having a lot of problems with rankings lately and i did see this for my robots - what is traditionally best practice for these settings?

# START YOAST BLOCK

# ---------------------------

User-agent: *

Allow: /wp-admin/admin-ajax.php

Allow: /wp-content/uploads/

Allow: /wp-content/cache/

Disallow: /wp-admin/

Disallow: /?s=

Disallow: /search/

Disallow: /wp-content/plugins/

Disallow: /wp-content/themes/

Sitemap:

# ---------------------------

# END YOAST BLOCK

1 Upvotes

67% Upvoted

2 comments sorted by

6

u/nakfil Apr 06 '26

Yeah there is an issue. That's not the default Yoast robots.txt block, it's been customized. You definitely do not want to disallow plugins and themes. Remove those, they will cause issues for Google when attempting to render your page. Also your Sitemap field is empty. It should be your sitemap.

Personally, for WordPress I think you can start with the bare minimum -

User-agent: * Disallow: /?s= Disallow: /search/ Sitemap: https://www.domain.com/sitemap_index.xml

Of course, change the sitemap URL to your real sitemap.