In Progress: Remove post limit on sitemaps by nbloomf · Pull Request #5394 · Automattic/jetpack

nbloomf · 2016-10-21T20:09:04Z

Addresses #3314: Remove 1000 post limit on sitemaps.

Changes proposed in this Pull Request:

Replace sitemap generating code.

Testing instructions:

Create a few test posts and upload some images.
Activate the sitemaps module of Jetpack.
Go to "example.com/news-sitemap.xml".
Go to "example.com/sitemap.xml". (It may take a refresh or two the first time.)

Note that sitemap.xml links to two other files; one is the root of a tree of sitemaps for pages, the other is the root of a tree of sitemaps for images.

These changes allow for arbitrarily large sitemap trees per the spec. To test this functionality without having to create an arbitrarily large site, adjust the constant SITEMAP_MAX_ITEMS in modules/sitemaps.php, (5 is a good value.) This constant determines the maximum number of URLs on a single sitemap, so that if it's set to 5 and we have 6 posts then the tree structure kicks in.

The constant SITEMAP_INTERVAL is the minimum number of seconds between rebuilds; it's currently set to 60 for testing.

We reuse the filter jetpack_sitemap_post_type to allow plugin authors to include custom post types in the sitemap.

Proposed changelog entry for your changes:

[no] Make sure your changes respect WordPress' coding standards.
[NA] Did you make changes, or create a new .js file? If Gulp is installed on your testing environment, run gulp js:hint before to commit your changes. It will allow you to detect errors in Javascript files.
[NA] Did you create a new action or filter? Add inline documentation to help others understand how to use the action or the filter.
[no] Create unit tests if you can. If you're not familiar with Unit Testing, you can check this tutorial.

…le.com/jetpack-sitemap/sitemap and example.com/jetpack-sitemap/sitemapN

…edundant DB queries.

nbloomf · 2016-11-01T03:49:30Z

Hmm... the WordPress 404 means that the URL intercept callback isn't working. My dev environment has PHP 7.0.11; tomorrow I will try using 5.6 and lower to see if I can reproduce the error.

The permalink settings shouldn't have an effect - the module hijacks the usual routing process on sitemap URLs.

…ist class

This reverts commit 534d2d2, reversing changes made to eaf0463.

nbloomf · 2016-11-02T01:57:12Z

@eliorivero : I think I found the reason why the module wasn't working for you. When you get a chance, see if it's working in the latest commit.

~~BTW, I think I messed up the commit history by merging Automattic/jetpack:master back into this branch and then reverting the merge.~~ Never mind I don't know what I'm doing

Conflicts: class.jetpack.php tests/php/test_class.jetpack.php

nbloomf · 2016-11-03T16:57:58Z

I have messed something up while trying to merge commits back from Automattic/jetpack:master and now this repo is broken. I will open a new PR with a fresh fork. :(

nbloomf added 30 commits October 9, 2016 14:27

Add (empty) Jetpack_Sitemap_Manager class

d72a2f3

Register custom post type jetpack_sitemap

f7c712f

Route example.com/zsitemap.xml and example.com/zsitemapN.xml to examp…

caf2125

…le.com/jetpack-sitemap/sitemap and example.com/jetpack-sitemap/sitemapN

Fix URI routing, return correct header type (XML).

0c9c5e2

Fix whitespace.

753d220

Add skeleton of code to generate sitemap data.

88fa49e

Add function to generate all sitemaps.

791abef

Fix whitespace.

07a4e29

Fix whitespace.

6265e32

Call get_permalink() on a post, rather than a post ID, to eliminate r…

2a1e85c

…edundant DB queries.

Abstract function get_contents_of_post_by_title_and_type.

a20a926

Refactor: rearrange code.

757db04

Refactor: name regex constants.

16f7137

Refactor: get rid of singleton pattern

36265cf

Add code to generate all vanilla sitemap indices.

d55a2f1

Escape entities before storing, unescape when retrieving.

c48828b

Encapsulate old sitemap code in a class; preparing for removal

0ff593f

Fix sitemap names

7eee655

Correct magic numbers

b4b6da5

Factor constructor into helper functions.

4b9b4d1

Factor out serve_raw_and_die in URL handler

f02fe79

Refactor: change some function names

13bfab9

Include link to main page

71c730e

Delete old sitemaps on regeneration

b3ac85d

Remove delete_all_sitemaps function

620f77a

Add comments

6ae4c26

Refactor: Change variable names

b1bf171

Add comments

2e673ab

Refactor: Rename variables.

3855312

Edit comments.

bddc626

nbloomf added 3 commits October 31, 2016 20:57

Remove unused variable

c3d11ab

Remove anonymous functions

c13cd98

Flip debug flag

88b900c

nbloomf added 8 commits November 1, 2016 09:47

Coding Standards: Fix spacing of array arguments

e3558be

Move static XSL generating functions to separate Jetpack_Sitemap_Styl…

31fdeec

…ist class

Remove anonymous function

6e312c5

Add PHPDoc for inner functions

eaf0463

Merge branch 'master' of https://github.com/Automattic/jetpack

534d2d2

Revert "Merge branch 'master' of https://github.com/Automattic/jetpack"

475dce7

This reverts commit 534d2d2, reversing changes made to eaf0463.

Replace self:: with explicit class name

fb32d83

Use home_url() instead of site_url()

fe8de74

nbloomf added 13 commits November 2, 2016 17:08

Add new functions to manipulate sitemap post_type list option

a1eb9a1

Use jetpack_sitemap_post_type filter hook to allow custom post types

530e1fd

Remove unneeded function

58e7b18

Refactor: Move code

d9f739c

Add inline documentation for apply_filters

f00f5af

Edit comments

c221800

Re-add deleted files to resolve merge conflict (???)

02cf4ce

Only log debug messages when WP_DEBUG is defined and true

972e440

Remove old comments

a00cc35

Resolve merge conflict

22accf9

Edit comment

74a81ee

Resolve merge conflict

6953649

Merge remote-tracking branch 'upstream/master'

8321366

Conflicts: class.jetpack.php tests/php/test_class.jetpack.php

nbloomf closed this Nov 3, 2016

matticbot removed the [Status] In Progress label Nov 3, 2016

nbloomf mentioned this pull request Nov 3, 2016

Sitemaps: remove post limit on sitemap #5503

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

In Progress: Remove post limit on sitemaps#5394

In Progress: Remove post limit on sitemaps#5394
nbloomf wants to merge 106 commits intoAutomattic:masterfrom
nbloomf:master

nbloomf commented Oct 21, 2016 •

edited

Loading

Uh oh!

nbloomf commented Nov 1, 2016

Uh oh!

nbloomf commented Nov 2, 2016 •

edited

Loading

Uh oh!

nbloomf commented Nov 3, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

nbloomf commented Oct 21, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes proposed in this Pull Request:

Testing instructions:

Proposed changelog entry for your changes:

Uh oh!

nbloomf commented Nov 1, 2016

Uh oh!

nbloomf commented Nov 2, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nbloomf commented Nov 3, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nbloomf commented Oct 21, 2016 •

edited

Loading

nbloomf commented Nov 2, 2016 •

edited

Loading