google-sitemap-generator
Opened 7 years ago
Last modified 7 years ago
#2776 new enhancement
Sitemaps have "Too Many URLs" if too many posts of a given type are created within same month
| Reported by: |
|
Owned by: |
|
|---|---|---|---|
| Priority: | normal | Severity: | normal |
| Plugin: | google-sitemap-generator | Keywords: | Sitemaps, URLs, chunks |
| Cc: |
Description
Currently, the plugin will break down sitemap files by post type, and then post year and month.
This is perfect for the vast majority of sites, however, when there are too many posts published during the same one-month timeframe, Google's diagnostic tools get angry and report that you have "Too Many URLs."
An example of where this might happen (and my real-world scenario) is when importing a large data set. In this case, all the posts have the same publish date within the timeframe the import took to complete, which could be just a few minutes.
I would love for the plugin to have a setting for "max URLs per file" that the user can modify.
Then, in months where there are more URLs than that the "max URLs per file" setting, the plugin would chunk up the sitemap files to ensure no one file has too many URLs.
In terms of a default for this setting, while Google publishes recommendations of 50,000 URLs or less, it's my understanding that limiting the sitemaps to 10,000 URLs will make them easier for Google to crawl and minimize the chance of a timeout.
A simple approach for file naming here might look like this...
https://www.mysite.com/sitemap-pt-article-2019-04.xml
https://www.mysite.com/sitemap-pt-article-2019-04~chunk-02.xml
I did my best to try and see how / where you might do this in the code to help it along, but couldn't make 100% sense of a good way.
It does seem like at minimum some array chunking on $posts variable would need to happen inside of the Index() and BuildPosts() functions within sitemap-builder.php, though there might be a "higher-level" spot to do this I'm not aware of.
I appreciate your consideration for this request, it's an otherwise perfect plugin for a wide range of needs.