Fix support for feeds with XML preample + DTD, entities by Alkarex · Pull Request #915 · simplepie/simplepie

Alkarex · 2025-06-23T22:57:23Z

Alternative of #914 with support for XML feeds using HTML entities

Simple feeds with XML preample + DTD are crashing (see example in test) due to regression from 162a3d3
Its implementation is buggy, as it creates a second DOCTYPE declaration even when there is an existing one, which is completely invalid.

Example of feed: https://blog.plover.com/index.rss
Downstream Issue: FreshRSS/FreshRSS#7514

Note that the culprit feature added support for html entities in xml seems to come from a misunderstanding of
https://www.rssboard.org/rss-encoding-examples
HTML entities are only allowed in XML when there is a DTD declaring them. SimplePie is even allowing Atom documents with undeclared HTML entities - which I am not sure is on purpose.

Downstream issue: FreshRSS/FreshRSS#7687

Alternative of simplepie#914 with support for XML feeds using HTML entities Simple feeds with XML preample + DTD are crashing (see example in test) due to regression from simplepie@162a3d3 Its implementation is buggy, as it creates a second DOCTYPE declaration even when there is an existing one, which is completely invalid. Example of feed: https://blog.plover.com/index.rss Downstream Issue: FreshRSS/FreshRSS#7514 Downstream PR: #35 Note that the culprit feature *added support for html entities in xml* seems to come from a misunderstanding of https://www.rssboard.org/rss-encoding-examples HTML entities are only allowed in XML when there is a DTD declaring them. SimplePie is even allowing Atom documents with undeclared HTML entities - which I am not sure is on purpose. Downstream issue: FreshRSS/FreshRSS#7687

Upstream: simplepie#915 Alternative of simplepie#914 with support for XML feeds using HTML entities Simple feeds with XML preample + DTD are crashing (see example in test) due to regression from simplepie@162a3d3 Its implementation is buggy, as it creates a second DOCTYPE declaration even when there is an existing one, which is completely invalid. Example of feed: https://blog.plover.com/index.rss Downstream Issue: FreshRSS/FreshRSS#7514 Note that the culprit feature *added support for html entities in xml* seems to come from a misunderstanding of https://www.rssboard.org/rss-encoding-examples HTML entities are only allowed in XML when there is a DTD declaring them. SimplePie is even allowing Atom documents with undeclared HTML entities - which I am not sure is on purpose. Downstream issue: FreshRSS/FreshRSS#7687

fix FreshRSS#7687 FreshRSS/simplepie#37 Upstream: simplepie/simplepie#915 Partial revert of FreshRSS#7515 HTML entities are normally only allowed in XML when there is a DTD declaring them. SimplePie is even allowing Atom documents with undeclared HTML entities - which I am not sure is on purpose.

Alkarex · 2025-06-24T07:24:54Z

tests/Unit/ItemTest.php

+<?xml version="1.0"?>
+<!DOCTYPE rss PUBLIC "-//Netscape Communications//DTD RSS 0.91//EN" "http://www.rssboard.org/rss-0.91.dtd">
+<rss version="0.91">
+	<channel>
+		<item>
+			<title>Item Title</title>
+		</item>
+	</channel>
+</rss>


This simple feed could not be parsed

fix #7687 FreshRSS/simplepie#37 Upstream: simplepie/simplepie#915 Partial revert of #7515 HTML entities are normally only allowed in XML when there is a DTD declaring them. SimplePie is even allowing Atom documents with undeclared HTML entities - which I am not sure is on purpose.

jtojnar

Thanks, this looks reasonable and the tests are great, just a single nit.

src/Parser.php

jtojnar · 2025-06-29T02:06:56Z

tests/Unit/ItemTest.php

+                ,
+                'Item Title',
+            ],
+            'Test RSS 0.91 XML and DTD and external HTML entity' => [


HTML entity

The RSS 0.91 DTD actually copies the entity list from HTML 3.2 and appears to be a subset of declare_html_entities.

jtojnar · 2025-06-29T02:09:35Z

tests/Unit/ItemTest.php

+<rss version="0.91">
+	<channel>
+		<item>
+			<title>caf&eacute;</title>


Weirdly, the entities from declare_html_entities appear to be substituted, even though the DOCTYPE within uses html as root element. 🤷‍♀️

Yes, I was also surprised this worked at all

I changed to rss

src/Parser.php

jtojnar · 2025-06-29T02:33:03Z

tests/Unit/ItemTest.php

+XML
+                ,
+                'café',
+            ],


In theory, the following should work as well but it is probably to niche to care about:

Suggested change

],

],

'Test RSS 0.91 DTD and external HTML entity and internal entity' => [

<<<XML

<!DOCTYPE rss PUBLIC "-//Netscape Communications//DTD RSS 0.91//EN" "http://www.rssboard.org/rss-0.91.dtd" [

<!ENTITY adios "Adiós">

]>

<rss version="0.91">

<channel>

<item>

<title>&adios; Adiós</title>

</item>

</channel>

</rss>

XML

,

'Adiós Adiós',

],

Indeed, but merging DTDs do not look easy as they may contain more complicated syntax

jtojnar · 2025-06-29T21:41:38Z

Thanks.

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [freshrss/freshrss](https://freshrss.org/) ([source](https://github.com/FreshRSS/FreshRSS)) | minor | `1.26.3` -> `1.27.0` | --- ### Release Notes <details> <summary>FreshRSS/FreshRSS (freshrss/freshrss)</summary> ### [`v1.27.0`](https://github.com/FreshRSS/FreshRSS/blob/HEAD/CHANGELOG.md#2025-08-18-FreshRSS-1270) [Compare Source](FreshRSS/FreshRSS@1.26.3...1.27.0) - Features - Implement support for HTTP `429 Too Many Requests` and `503 Service Unavailable`, obey `Retry-After` [#7760](FreshRSS/FreshRSS#7760) - Add sort by category title, or by feed title [#7702](FreshRSS/FreshRSS#7702) - Add search operator `c:` for categories like `c:23,34` or `!c:45,56` [#7696](FreshRSS/FreshRSS#7696) - Custom feed favicons [#7646](FreshRSS/FreshRSS#7646), [#7704](FreshRSS/FreshRSS#7704), [#7717](FreshRSS/FreshRSS#7717), [#7792](FreshRSS/FreshRSS#7792) - Rework fetch favicons for fewer HTTP requests [#7767](FreshRSS/FreshRSS#7767) - Add more unicity criteria based on title and/or content [#7789](FreshRSS/FreshRSS#7789) - Automatically restore user configuration from backup [#7682](FreshRSS/FreshRSS#7682) - API add support for states in `s` parameter of `streamId` [#7695](FreshRSS/FreshRSS#7695) - Improve sharing via Print [#7728](FreshRSS/FreshRSS#7728) - Redirect to the login page from bookmarklet instead of 403 [#7782](FreshRSS/FreshRSS#7782) - Clean local cache more often, when refreshing feeds [#7827](FreshRSS/FreshRSS#7827) - Security - Implement reauthentication (*sudo* mode) [#7753](FreshRSS/FreshRSS#7753) - Add `Content-Security-Policy: frame-ancestors` [#7677](FreshRSS/FreshRSS#7677) - Ensure CSP everywhere [#7810](FreshRSS/FreshRSS#7810) - Show warning when unsafe CSP policy is in use [#7804](FreshRSS/FreshRSS#7804) - Fix access rights when creating a new user [#7783](FreshRSS/FreshRSS#7783) - Improve security of form for user details [#7771](FreshRSS/FreshRSS#7771), [#7786](FreshRSS/FreshRSS#7786) - Disallow setting non-existent theme [#7722](FreshRSS/FreshRSS#7722) - Regenerate cookie ID after logging out [#7762](FreshRSS/FreshRSS#7762) - Require current password when setting new password [#7763](FreshRSS/FreshRSS#7763) - Add missing access checks for feed-related actions [#7768](FreshRSS/FreshRSS#7768) - Strip more unsafe attributes such as `referrerpolicy`, `ping` [#7770](FreshRSS/FreshRSS#7770) - Remove unneeded execution permissions [#7802](FreshRSS/FreshRSS#7802) - Bug fixing - Fix redirections when scraping from HTML [#7654](FreshRSS/FreshRSS#7654), [#7741](FreshRSS/FreshRSS#7741) - Fix multiple authentication HTTP headers [#7703](FreshRSS/FreshRSS#7703) - Fix HTML queries with a single feed [#7730](FreshRSS/FreshRSS#7730) - WebSub: only perform a redirection when coming from WebSub [#7738](FreshRSS/FreshRSS#7738) - Include enclosures in entries’ hash [#7719](FreshRSS/FreshRSS#7719) - Negative side-effect: users of the option to *automatically mark updated articles as unread* will once have some articles with enclosures re-appear as unread - Fix cancellation of slider exit UI [#7705](FreshRSS/FreshRSS#7705) - Honor *disable update* on update page [#7733](FreshRSS/FreshRSS#7733) - Fix no registration limit setting [#7751](FreshRSS/FreshRSS#7751) - Fix XML encoding of sharing functions [#7822](FreshRSS/FreshRSS#7822) - SimplePie - Fix propagation of HTTP error codes [#7670](FreshRSS/FreshRSS#7670) - Fix support for XML feeds with HTML entities [#7689](FreshRSS/FreshRSS#7689), [simplepie#915](simplepie/simplepie#915) - Fix feeds encoded in UTF-16LE [#7691](FreshRSS/FreshRSS#7691), [simplepie#916](simplepie/simplepie#916) - Various upstream contributions [simplepie#917](simplepie/simplepie#917), [simplepie#924](simplepie/simplepie#924), [simplepie#926](simplepie/simplepie#926), [simplepie#932](simplepie/simplepie#932), [simplepie#933](simplepie/simplepie#933) - Sync upstream [#7706](FreshRSS/FreshRSS#7706), [FreshRSS/simplepie#45](FreshRSS/simplepie#45), [#7775](FreshRSS/FreshRSS#7775), [FreshRSS/simplepie#50](FreshRSS/simplepie#50), [#7824](FreshRSS/FreshRSS#7824), [#7825](FreshRSS/FreshRSS#7825), - Fix regex *Backtrack limit was exhausted* in `clean_hash()` [#7813](FreshRSS/FreshRSS#7813), [FreshRSS/simplepie#48](FreshRSS/simplepie#48) - Deployment - Docker default image (Debian 12 Bookworm) updated to PHP 8.2.29 [#7805](FreshRSS/FreshRSS#7805) - Docker alternative image updated to Alpine 3.22 with PHP 8.4.11 and Apache 2.4.65 [#7740](FreshRSS/FreshRSS#7740), [#7740](FreshRSS/FreshRSS#7740), [#7803](FreshRSS/FreshRSS#7803) - Start supporting PHP 8.5+ [#7787](FreshRSS/FreshRSS#7787), [#7826](FreshRSS/FreshRSS#7826) - Docker Alpine dev image `:newest` updated to PHP 8.5-alpha and Apache 2.4.65 [#7773](FreshRSS/FreshRSS#7773) - Docker: interpolate `FRESHRSS_INSTALL` and `FRESHRSS_USER` variables [#7725](FreshRSS/FreshRSS#7725) - Docker: Reduce how much data needs to be chown/chmod’ed on container startup [#7793](FreshRSS/FreshRSS#7793) - Test for database PDO typing support during install (relevant for MySQL / MariaDB with obsolete driver) [#7651](FreshRSS/FreshRSS#7651) - Extensions - Add API endpoint for extensions [#7576](FreshRSS/FreshRSS#7576) - Expose the reading modes for extensions [#7668](FreshRSS/FreshRSS#7668), [#7688](FreshRSS/FreshRSS#7688) - New extension hook `before_login_btn` [#7761](FreshRSS/FreshRSS#7761) - UI - Improve *mark as read* request showing popup due to `onbeforeunload` [#7554](FreshRSS/FreshRSS#7554) - Fix lazy-loading for `<video poster="...">` and `<image>` [#7636](FreshRSS/FreshRSS#7636) - Avoid styling `<code>` inside of `<pre>` [#7797](FreshRSS/FreshRSS#7797) - Improve confirmation logic with `data-auto-leave-validation` [#7785](FreshRSS/FreshRSS#7785) - Update `chart.js` to 4.5.0 [#7752](FreshRSS/FreshRSS#7752), [#7816](FreshRSS/FreshRSS#7816) - Various UI and style improvements: [#7616](FreshRSS/FreshRSS#7616), [#7811](FreshRSS/FreshRSS#7811) - I18n - Show translation status in README [#7715](FreshRSS/FreshRSS#7715) - Improve Indonesian [#7654](FreshRSS/FreshRSS#7654), [#7721](FreshRSS/FreshRSS#7721) - Improve Persian [#7795](FreshRSS/FreshRSS#7795) - Misc. - Improve PHP code [#7642](FreshRSS/FreshRSS#7642), [#7665](FreshRSS/FreshRSS#7665), [#7761](FreshRSS/FreshRSS#7761), [#7781](FreshRSS/FreshRSS#7781), [#7794](FreshRSS/FreshRSS#7794) - Update dev dependencies [#7708](FreshRSS/FreshRSS#7708), [#7709](FreshRSS/FreshRSS#7709), [#7710](FreshRSS/FreshRSS#7710), [#7711](FreshRSS/FreshRSS#7711), [#7776](FreshRSS/FreshRSS#7776), [#7777](FreshRSS/FreshRSS#7777) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://gitea.alexlebens.dev/alexlebens/infrastructure/pulls/1253 Co-authored-by: Renovate Bot <renovate-bot@alexlebens.net> Co-committed-by: Renovate Bot <renovate-bot@alexlebens.net>

pull-request-size bot added the size/M label Jun 23, 2025

Alkarex force-pushed the fix-xml-dtd-entities branch from a0add8f to df3b0dd Compare June 23, 2025 23:00

Alkarex mentioned this pull request Jun 23, 2025

Merge: Fix support for feeds with XML preample + DTD, entities FreshRSS/simplepie#37

Merged

Alkarex mentioned this pull request Jun 24, 2025

Fix support for feeds with XML preample + DTD #914

Closed

Alkarex mentioned this pull request Jun 24, 2025

Fix support for XML feeds with HTML entities FreshRSS/FreshRSS#7689

Merged

Alkarex commented Jun 24, 2025

View reviewed changes

jtojnar reviewed Jun 29, 2025

View reviewed changes

src/Parser.php Outdated Show resolved Hide resolved

jtojnar added this to the 1.9.0 milestone Jun 29, 2025

jtojnar reviewed Jun 29, 2025

View reviewed changes

Extract as helper method

0133dff

pull-request-size bot added size/L and removed size/M labels Jun 29, 2025

jtojnar merged commit 0340309 into simplepie:master Jun 29, 2025
10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix support for feeds with XML preample + DTD, entities#915

Fix support for feeds with XML preample + DTD, entities#915
jtojnar merged 2 commits intosimplepie:masterfrom
FreshRSS:fix-xml-dtd-entities

Alkarex commented Jun 23, 2025 •

edited

Loading

Uh oh!

Alkarex Jun 24, 2025

Uh oh!

jtojnar left a comment •

edited

Loading

Uh oh!

Uh oh!

jtojnar Jun 29, 2025

Uh oh!

jtojnar Jun 29, 2025

Uh oh!

Alkarex Jun 29, 2025

Uh oh!

Alkarex Jun 29, 2025

Uh oh!

Alkarex Jun 29, 2025

Uh oh!

Uh oh!

jtojnar Jun 29, 2025

Uh oh!

Alkarex Jun 29, 2025

Uh oh!

Uh oh!

jtojnar commented Jun 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-            ],
+            ],
+            'Test RSS 0.91 DTD and external HTML entity and internal entity' => [
+<<<XML
+<!DOCTYPE rss PUBLIC "-//Netscape Communications//DTD RSS 0.91//EN" "http://www.rssboard.org/rss-0.91.dtd" [
+    <!ENTITY adios "Adi&oacute;s">
+]>
+<rss version="0.91">
+    <channel>
+        <item>
+            <title>&adios; Adi&oacute;s</title>
+        </item>
+    </channel>
+</rss>
+XML
+                ,
+                'Adiós Adiós',
+            ],

Conversation

Alkarex commented Jun 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jtojnar left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jtojnar commented Jun 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Alkarex commented Jun 23, 2025 •

edited

Loading

jtojnar left a comment •

edited

Loading