Custom Robots Header Tags for Blogger, X-Robots tags

The robots header tags allow you to control how your website should display in search results. You can apply the robots tags using meta tags and X-robots tags in the response header. Blogger allows you to set both types of robots tag. In this article, we will understand the best practice of implementing Robots Header tags to boost the SEO of the Blogger website.

Recently, Blogger made many changes in its interface to improve the user experience. From the SEO point of view, the major update was on new themes. Blogger now supports HTML5 themes that drastically boost the SEO of the webpage. But along with the theme and content, you need to configure the X-Robots-Tag setting properly.

Please do not confuse Robots Header tags with custom robots.txt of Blogger. The robots.txt file serves as a site-wide guide for web spiders, residing in the root directory of a website. It instructs user agents on which sections of the site to crawl and which to avoid. This file applies globally to the entire website.

In contrast, the Robots Header tags are page-specific directives placed within the HTML head section of individual pages. They communicate to web spiders how to index or noindex a particular page, influencing the page’s visibility in search engine results. Unlike robots.txt, which governs the entire site, Robots Header tags operate page-by-page, providing targeted instructions for each webpage.

What is X-Robots-Tag or Robots Header tag and Robots Meta Tag?

X-Robots-Tags or Robots Header Tags: X-Robots-Tags are part of the HTTP header response sent by the server when a web page is accessed. These tags convey page-level settings and directives to search engine bots. Webmasters can control how search engines index their content by including X-Robots tags in the HTTP header. For example, an X-Robots-Tag might specify that a certain page should not be indexed or should have its links followed, influencing the search engine’s behavior for that specific page.

Robots-Tags Meta Tags: On the other hand, they are typically meta tags embedded within the HTML code of individual pages. These meta tags serve a similar purpose to X-Robots-Tags by providing page-level instructions to search engine bots. Webmasters can include meta tags like “noindex” or “nofollow” to specify whether a particular page should be excluded from search engine indexes or if links on that page should not be followed.

NOODP Robots Header tag is no longer supported.

We talked about the custom robots header tags available with Blogger. Many websites and YouTube channel guides regarding the robots header tags for the Blogger blog suggest enabling robots header tags noodp for the pages and posts. But that is an entirely wrong practice, then what is right?

Search engines like Google no longer support the noodp tag. So don’t follow the wrong practice. It may spoil the On-Page SEO. I hope that Blogger will soon remove this option from its settings.

Who should enable custom robots header tags in the Blogger blog?

Before making any changes in the Blogger blog settings, we’ve to understand who should enable this custom robots header tag setting and who should not.

Who shouldn’t enable this tag option?

• If you’re running a simple blog, then no need to enable this feature: This feature allows you to alter the default index status of Blogger posts or pages. The other ways to improve SEO of your blog are –
  • Implement the best practice of robots.txt and robots meta tags combination,
  • add meta tags to the Blogger Theme,
  • Add Schema Structure Data to Blogger.
  • Create SEO-friendly content.

Who should enable and use these tags?

• You are an advanced user who understands which pages to index and which not.
• You are dealing with time-bound content that needs to be removed from search results after a specific time.
• You have such complex written content that is readable in one language only. If translated, their meaning may be completely different. So you can stop translation on some pages.
• You don’t want to highlight snippets.

How we enable custom robots header tags?

Log in to the Blogger dashboard to enable these X-robots header tags.

1. Open the settings of the blog,
2. Scroll to crawlers and indexing option,
3. Switch on the “Enable custom robots header tags” option,

4. Home page tags: Click on this, and the option shown in the above image will appear. You’ve to click on all, and save the settings,
5. Archive and search page tags: from the options provided above, you’ve to select noindex. 
6. Posts and Pages type: To index all the pages, you’ve to click on enable “all”. And then save the settings.

As we already discussed, enabling the noodp tag is unnecessary.

Enabling custom robots header tags will allow you to set custom header tags for every page and post. So you can decide on the suitable robots header tags.

The above image displays the settings of the Blogger blog post. As you can see, now there is an option available for custom robots tags available in the post’s sidebar.

If you enabled custom robots header settings for the simple blog, leave all these settings.

You can also check the HTTP response of the webpage to check all the settings.

Final Words

In this article, we understood the best practice of custom robots header tags for the Blogger blog. What should we choose, and what should we ignore? The best approach, in this case, is don’t touch this setting and make changes in the Robots meta tag of the blog.

I hope you liked this article. In case of any doubt, query, or feedback, comment in the comment section provided below. Thank You!