-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Description
Search console has been reporting that a lot of the links on my website have the rel="noindex" tag, and is not indexing them.
Truth is, all the pages on my website are currently marked to be indexed, and there is no rel="noindex" tag on the pages.
I checked the http headers too, using this tool.
When I tried recrawling one of the problematic pages in Search Console, it still showed the issue, and it showed Referring page: my-yourls-link.com.
I checked the headers of that URL using the same tool as above, and surely the yourls link contains a X-Robots-Tag => noindex.
While it seems to make sense to have this tag on the Yourls link, I'm not sure why Google is not indexing the main URL.
I checked a Bit.ly link and ran it through the header checker tool, and they don't have any noindex tags.
Same with TinyURL. No noindex tags.
Seems like the noindex tag should be removed. None of the industry standard URL shorteners have it.
Perhaps we should undo this commit.