How to Manage RSS Feed URLs in Google Search Console

Are you encountering a bunch of  “Crawled – not currently indexed” statuses with RSS feed URLs in your Google Search Console? Don’t panic. These are not errors but rather RSS feeds that facilitate subscription through various news aggregator software. 

However, if you wish to manage them to optimise your crawl budget or for other reasons, here’s what you need to know and how you can take action.

What Are RSS Feed URLs?

RSS feed URLs are simply URLs that serve as RSS feeds, providing information about new posts on your website. They are not problematic per se but can consume your crawl budget unnecessarily.

An example of what an RSS Feed URL look like:

https://aubreyyung.com/seo/feed/

If you are using WordPress as your CMS, you will likely see these individual RSS feeds as WordPress automatically creates them.

How RSS Feed URLs Impact Indexing Status

RSS feed URLs provide updates about new posts on your website, but it is not necessary for search engine indexing. 

When Google’s crawlers spend resources on crawling these URLs without indexing them, it can impact your website’s crawl budget. 

This allocation of resources to non-essential URLs might affect the indexing of more important pages on your site, potentially impacting your site’s overall visibility in search results.

If you notice there are too many URLs not being indexed because of “Crawled – not currently indexed”, consider disable them.

RSS Feed URLs showing up in Google Search Console Page indexing report

How to Manage RSS Feed URLs

1. Editing the robots.txt File

You can directly manage RSS feed URLs by accessing your website’s root folder and locating the robots.txt file. By adding a Disallow directive for RSS feed URLs, you can instruct search engine crawlers not to index these pages.

Here’s how you can do it in the robots.txt file:

User-agent: *
Disallow: /*/feed/

Simply paste this directive into your robots.txt file to prevent crawlers from accessing RSS feed URLs.

2. Using a WordPress Plugin

If you are using WordPress as your CMS, managing RSS feed URLs is even easier. There are different plugins that help you disable your RSS feed or edit them.

For example, you can install “Disable Everything” to disable the RSS feed option. and disable it. Or alternatively, many popular SEO plugins (such as Yoast, RankMath) offer similar control over the crawling.

Refer to the documentation or support resources provided by the plugin developer for detailed instructions on how to disable the RSS feed setting.

3. Setting a Noindex Tag for Your RSS Feed

Another effective method to manage RSS feed URLs is by setting a noindex tag specifically for your feed pages. This approach instructs search engine crawlers not to index these pages, thereby preventing them from appearing in search results.

<meta name="robots" content="noindex, follow">

Is an RSS feed considered duplicate content?

Feed URLs themselves are not typically considered duplicate content in the traditional sense. They serve as a mechanism for delivering updates about new posts or content on your website to users who have subscribed to your RSS feed through feed readers or news aggregators.

However, in some cases, if the content of your feed URLs mirrors the content of your website pages exactly, it could potentially be flagged as duplicate content by search engines. This scenario is rare because feed URLs usually contain excerpts or summaries of your content rather than the full content itself.

To avoid any potential issues with duplicate content, ensure that the content in your feed URLs is distinct from the content on your website pages. Providing unique excerpts or summaries in your feed can help prevent any duplicate content concerns.

Conclusion

RSS feed URLs are not a cause for concern, but managing them can enhance your website’s performance. 

Whether through direct editing of the robots.txt file or using WordPress plugins, you can efficiently handle RSS feed URLs and improve your website’s SEO.

Remember, while it’s optional, optimizing your crawl budget is always a good practice in maintaining a healthy website.

Aubrey Yung

Aubrey Yung

Aubrey is an SEO Consultant with 5+ years of B2B and B2C marketing experience.