Tested

Review robots.txt for rules that block important resources or sections

A rule-by-rule review that flags blocking rules and gives the corrected lines.

Channels

  • Google Search
  • Website

Inputs

  • The live robots.txt file
  • A list of key paths, CSS and JS resources that must stay crawlable