Google: Robots.txt Can Be In Any Content Type

Jun 2, 2020 • 7:55 am | comments (0) by twitter | Filed Under Google Search Engine Optimization
 

John Mueller of Google said that your robots.txt content-type can be anything really. He said "any content type is fine," adding "the important part is the content itself."

He also said he has "seen html pages that included valid robots txt lines among a valid html page."

If you are concerned, test it with Google's robots.txt validator.

Oh, the Penguin runner asked this question:

Forum discussion at Twitter.

Previous story: Google & Bing Black Lives Matter Statements
  blog comments powered by Disqus