New unicity policies for feeds with bad GUIDs by Alkarex · Pull Request #4487 · FreshRSS/FreshRSS

Alkarex · 2022-08-04T12:56:18Z

fix #4485
fix #5210
fix #5257
fix #5410
fix #5909
fix #6658

Frenzie

Maybe there's another one we should toss in there as well, like published?

eht16 · 2022-09-10T15:14:32Z

I would second the idea to include "published" and/or some aliases of it.

Recently, I stumbled upon https://status.otc-service.com/rss/?mt=Europe%20%28Germany%29 which does not have a "guid" and the links in there are not unique, only the "time" field seems usable to distinguish the posts.

Alkarex · 2024-09-28T14:34:04Z

Let's first make a PR to add a new feed-level option to override the ID logic (e.g. force use of link)

Frenzie · 2024-10-04T13:23:28Z

app/i18n/en/sub.php

 		'title_add' => 'Add an RSS feed',
 		'ttl' => 'Do not automatically refresh more often than',
+		'unicityPolicy' => array(
+			'_' => 'Articles unicity',


Maybe something like:

Suggested change

'_' => 'Articles unicity',

'_' => 'Article uniqueness criteria',

Or:

Suggested change

'_' => 'Articles unicity',

'_' => 'Article duplicate matching criteria',

Which is a long way of saying perhaps unicity isn't the most common word.

Interesting ChatGPT's feedback 🤖

Indeed not the most common word but not so rare either, is it ? And probably "guessable". I do not really mind, but unicity is slightly more precise here.

It is important to be understandable indeed. On the other hand, this option should not be too tempting to fiddle with :-P

Let's see what our testers say, if any

Either way I would make it article (singular) unicity. ^_^

Do not fall back to hash of content automatically, as this seems to often create too many duplicates

Alkarex · 2024-10-11T12:51:36Z

I have observed several examples of feeds with temporary invalid GUIDs (non-unique) due to e.g. temporary duplicated articles.
This leads to premature changes of unicity criteria and unwanted duplicates.

So here is a new tolerance parameter. I will test some more.

$invalidGuidsTolerance (default 0.05) The maximum ratio (rounded) of invalid GUIDs to tolerate before degrading the unicity criteria.
Example for 0.05 (5% rounded): tolerate 0 invalid GUIDs for up to 9 articles, 1 for 10, 2 for 30, 3 for 50, 4 for 70, 5 for 90, 6 for 110, etc.
The default value of 5% rounded was chosen to allow 1 invalid GUID for feeds of 10 articles, which is a frequently observed amount of articles.

Frenzie · 2024-10-11T15:21:56Z

Temporarily duplicated? Interesting. Sounds good! 👍

Alkarex · 2024-10-12T08:29:49Z

Example right now https://www.academie-sciences.fr/fr/?option=com_obrss&task=feed&id=2:fil-d-actualite-de-l-academie-des-sciences&format=feed

Error on a single entry http://www.academie-sciences.fr/fr/Communiques-de-presse/communique-de-presse-prix-nobel-physiologie-ou-medecine.html out of 30

feed.xml

<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:media="http://search.yahoo.com/mrss/">
	<channel>
		<title>Les derniers articles publiés sur le site de l'Académie des sciences</title>
		<description></description>
		<link>https://www.academie-sciences.fr/</link>
		<lastBuildDate>2024-10-12T09:40:14+02:00</lastBuildDate>
		<generator>FeedCreator 1.8.3 (obRSS 3.4.2)</generator>
		<atom:link href="https://www.academie-sciences.fr/fr/?option=com_obrss&amp;task=feed&amp;id=2:fil-d-actualite-de-l-academie-des-sciences&amp;format=feed" rel="self" type="application/rss+xml" />
		<language>fr-FR</language>
		<item>
			<title>Communiqué de presse | Prix Nobel Physiologie ou médecine</title>
			<link>http://www.academie-sciences.fr/fr/Communiques-de-presse/communique-de-presse-prix-nobel-physiologie-ou-medecine.html</link>
			<description></description>
			<pubDate>2024-10-11T12:06:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Communiques-de-presse/communique-de-presse-prix-nobel-physiologie-ou-medecine.html</guid>
		</item>
		<item>
			<title>Communiqué de presse | Prix Nobel Chimie</title>
			<link>http://www.academie-sciences.fr/fr/Communiques-de-presse/communique-de-presse-prix-nobel-physiologie-ou-medecine.html</link>
			<description></description>
			<pubDate>2024-10-10T12:06:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Communiques-de-presse/communique-de-presse-prix-nobel-physiologie-ou-medecine.html</guid>
		</item>
		<item>
			<title>Affleck Ian</title>
			<link>http://www.academie-sciences.fr/fr/In-memoriam/affleck-ian.html</link>
			<description></description>
			<pubDate>2024-10-04T13:00:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/In-memoriam/affleck-ian.html</guid>
		</item>
		<item>
			<title>Ghislain de Marsily</title>
			<link>http://www.academie-sciences.fr/fr/In-memoriam/ghislain-de-marsily.html</link>
			<description></description>
			<pubDate>2024-04-21T14:47:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/In-memoriam/ghislain-de-marsily.html</guid>
		</item>
		<item>
			<title>Communiqué de presse | Prix Nobel de Physique </title>
			<link>http://www.academie-sciences.fr/fr/Communiques-de-presse/communique-de-presse-prix-nobel-de-physique.html</link>
			<description></description>
			<pubDate>2024-10-10T11:45:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Communiques-de-presse/communique-de-presse-prix-nobel-de-physique.html</guid>
		</item>
		<item>
			<title>Rencontre avec Jocelyne Troccaz - Des automates aux robots chirurgicaux</title>
			<link>http://www.academie-sciences.fr/fr/Colloques-conferences-et-debats/rencontre-avec-jocelyne-troccaz-des-automates-aux-robots-chirurgicaux.html</link>
			<description></description>
			<pubDate>2024-11-05T13:06:00+01:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Colloques-conferences-et-debats/rencontre-avec-jocelyne-troccaz-des-automates-aux-robots-chirurgicaux.html</guid>
		</item>
		<item>
			<title>Rencontre avec Bérengère Dubrulle  - Les mystères de la turbulence - Cycle Rencontre à l'Académie des sciences</title>
			<link>http://www.academie-sciences.fr/fr/Seances-publiques/rencontre-avec-berengere-dubrulle-les-mysteres-de-la-turbulence-cycle-rencontre-a-l-academie-des-sciences.html</link>
			<description></description>
			<pubDate>2024-10-29T10:40:00+01:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Seances-publiques/rencontre-avec-berengere-dubrulle-les-mysteres-de-la-turbulence-cycle-rencontre-a-l-academie-des-sciences.html</guid>
		</item>
		<item>
			<title>Une Babel des sciences ? Traductions et sciences ouvertes </title>
			<link>http://www.academie-sciences.fr/fr/Colloques-conferences-et-debats/une-babel-des-sciences-traductions-et-sciences-ouvertes.html</link>
			<description></description>
			<pubDate>2024-10-02T14:34:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Colloques-conferences-et-debats/une-babel-des-sciences-traductions-et-sciences-ouvertes.html</guid>
		</item>
		<item>
			<title>Michael Hall, lauréat de la Grande médaille de l’Académie des sciences</title>
			<link>http://www.academie-sciences.fr/fr/Actualites-scientifiques/seance-2024-de-remise-de-la-grande-medaille-de-l-academie-des-sciences.html</link>
			<description></description>
			<pubDate>2024-09-17T12:06:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Actualites-scientifiques/seance-2024-de-remise-de-la-grande-medaille-de-l-academie-des-sciences.html</guid>
		</item>
		<item>
			<title>La Médaille d'Or du CNRS remise à Edith Heard </title>
			<link>http://www.academie-sciences.fr/fr/Communiques-de-presse/la-medaille-d-or-du-cnrs-remise-a-edith-heard.html</link>
			<description></description>
			<pubDate>2024-10-02T16:44:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Communiques-de-presse/la-medaille-d-or-du-cnrs-remise-a-edith-heard.html</guid>
		</item>
		<item>
			<title>Section Biologie humaine et sciences médicales</title>
			<link>http://www.academie-sciences.fr/fr/Listes-par-section-scientifique/section-biologie-humaine-et-sciences-medicales.html</link>
			<description></description>
			<pubDate>2015-01-12T12:54:51+01:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Listes-par-section-scientifique/section-biologie-humaine-et-sciences-medicales.html</guid>
		</item>
		<item>
			<title>Marc Fontecave, lauréat de l'ENI Energy Transition Award 2024</title>
			<link>http://www.academie-sciences.fr/fr/Membres-a-la-une/marc-fontecave-laureat-de-l-energy-transition-award-2024.html</link>
			<description></description>
			<pubDate>2024-09-19T12:18:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Membres-a-la-une/marc-fontecave-laureat-de-l-energy-transition-award-2024.html</guid>
		</item>
		<item>
			<title>Séance de remise de la Grande médaille de l’Académie des sciences 2022 à Terence Tao</title>
			<link>http://www.academie-sciences.fr/fr/Colloques-conferences-et-debats/ceremonie-grande-medaille-2022-terence-tao.html</link>
			<description></description>
			<pubDate>2023-03-21T13:06:00+01:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Colloques-conferences-et-debats/ceremonie-grande-medaille-2022-terence-tao.html</guid>
		</item>
		<item>
			<title>Appel à candidature : Grand Prix Lamonica de neurologie 2024</title>
			<link>http://www.academie-sciences.fr/fr/Appel-a-candidature/prix-lamonica-neurologie.html</link>
			<description></description>
			<pubDate>2024-07-19T08:00:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Appel-a-candidature/prix-lamonica-neurologie.html</guid>
		</item>
		<item>
			<title>Appel à candidature : Grand prix Sicard – Académie des sciences 2024</title>
			<link>http://www.academie-sciences.fr/fr/Appel-a-candidature/appel-a-candidature-grand-prix-sicard-academie-des-sciences-2024.html</link>
			<description></description>
			<pubDate>2024-07-29T09:47:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Appel-a-candidature/appel-a-candidature-grand-prix-sicard-academie-des-sciences-2024.html</guid>
		</item>
		<item>
			<title>Michèle Vergne, lauréate de la médaille Weyl-Wigner</title>
			<link>http://www.academie-sciences.fr/fr/Membres-a-la-une/michele-vergne-laureate-de-la-medaille-weyl-wigner.html</link>
			<description></description>
			<pubDate>2024-08-20T12:18:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Membres-a-la-une/michele-vergne-laureate-de-la-medaille-weyl-wigner.html</guid>
		</item>
		<item>
			<title>Pierre Tiollais</title>
			<link>http://www.academie-sciences.fr/fr/In-memoriam/hommage-a-pierre-tiollais.html</link>
			<description></description>
			<pubDate>2024-08-05T12:06:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/In-memoriam/hommage-a-pierre-tiollais.html</guid>
		</item>
		<item>
			<title>Section Biologie intégrative</title>
			<link>http://www.academie-sciences.fr/fr/Listes-par-section-scientifique/section-biologie-integrative.html</link>
			<description></description>
			<pubDate>2015-01-12T12:54:51+01:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Listes-par-section-scientifique/section-biologie-integrative.html</guid>
		</item>
		<item>
			<title>Campagne d'appels à candidature</title>
			<link>http://www.academie-sciences.fr/fr/Prix-et-medailles/campagne-d-appels-a-candidatures.html</link>
			<description></description>
			<pubDate>2014-07-03T14:34:15+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Prix-et-medailles/campagne-d-appels-a-candidatures.html</guid>
		</item>
		<item>
			<title>Mini-organes et embryons précoces in vitro : Quels enjeux ?</title>
			<link>http://www.academie-sciences.fr/fr/Colloques-conferences-et-debats/mini-organes-et-embryons-precoces-in-vitro-quels-enjeux.html</link>
			<description></description>
			<pubDate>2024-11-12T13:06:00+01:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Colloques-conferences-et-debats/mini-organes-et-embryons-precoces-in-vitro-quels-enjeux.html</guid>
		</item>
		<item>
			<title>Appel à candidature : prix Irène Joliot-Curie 2024</title>
			<link>http://www.academie-sciences.fr/fr/Appel-a-candidature/appel-a-candidature-prix-irene-joliot-curie.html</link>
			<description></description>
			<pubDate>2024-07-19T08:47:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Appel-a-candidature/appel-a-candidature-prix-irene-joliot-curie.html</guid>
		</item>
		<item>
			<title>Appel à candidature : Grand Prix Lamonica de cardiologie 2024</title>
			<link>http://www.academie-sciences.fr/fr/Appel-a-candidature/prix-lamonica-cardiologie.html</link>
			<description></description>
			<pubDate>2024-07-19T08:00:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Appel-a-candidature/prix-lamonica-cardiologie.html</guid>
		</item>
		<item>
			<title>Comité des Scientifiques pour l'Ukraine (CSU)</title>
			<link>http://www.academie-sciences.fr/fr/Comites-thematiques/comite-scientifiques-ukraine.html</link>
			<description></description>
			<pubDate>2022-04-01T09:30:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Comites-thematiques/comite-scientifiques-ukraine.html</guid>
		</item>
		<item>
			<title>Stéphane Mallat, IEEE Fourier Award for Signal Processing</title>
			<link>http://www.academie-sciences.fr/fr/Membres-a-la-une/stephane-mallat-ieee-fourier-award-for-signal-processing.html</link>
			<description></description>
			<pubDate>2024-04-29T12:00:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Membres-a-la-une/stephane-mallat-ieee-fourier-award-for-signal-processing.html</guid>
		</item>
		<item>
			<title>Lydéric Bocquet, lauréat de la médaille de l’innovation du CNRS 2024</title>
			<link>http://www.academie-sciences.fr/fr/Membres-a-la-une/lyderic-bocquet-laureat-de-la-medaille-de-l-innovation-du-cnrs-2024.html</link>
			<description></description>
			<pubDate>2024-07-09T12:18:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Membres-a-la-une/lyderic-bocquet-laureat-de-la-medaille-de-l-innovation-du-cnrs-2024.html</guid>
		</item>
		<item>
			<title>L’infiniment petit, avenir de la transition énergétique - Lydéric Bocquet - L'HUMANITÉ MAGAZINE</title>
			<link>http://www.academie-sciences.fr/fr/Articles-en-partenariat/l-infiniment-petit-avenir-de-la-transition-energetique-lyderic-bocquet-l-humanite-magazine.html</link>
			<description></description>
			<pubDate>2024-07-04T07:10:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Articles-en-partenariat/l-infiniment-petit-avenir-de-la-transition-energetique-lyderic-bocquet-l-humanite-magazine.html</guid>
		</item>
		<item>
			<title>Communiqué de presse | Dimanche, la vigilance républicaine s’impose</title>
			<link>http://www.academie-sciences.fr/fr/Communiques-de-presse/communique-de-presse-dimanche-la-vigilance-republicaine-s-impose.html</link>
			<description></description>
			<pubDate>2024-07-03T10:25:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Communiques-de-presse/communique-de-presse-dimanche-la-vigilance-republicaine-s-impose.html</guid>
		</item>
		<item>
			<title>Patrick Flandrin, lauréat du prix de l'éducation Carl Friedrich Gauss 2023</title>
			<link>http://www.academie-sciences.fr/fr/Membres-a-la-une/patrick-flandrin-laureat-du-prix-de-l-education-carl-friedrich-gauss-2023.html</link>
			<description></description>
			<pubDate>2024-04-29T12:00:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Membres-a-la-une/patrick-flandrin-laureat-du-prix-de-l-education-carl-friedrich-gauss-2023.html</guid>
		</item>
		<item>
			<title>Un poisson pour comprendre les cellules souches du cerveau - Laure Bally-Cuif - L'HUMANITÉ MAGAZINE</title>
			<link>http://www.academie-sciences.fr/fr/Articles-en-partenariat/un-poisson-pour-comprendre-les-cellules-souches-du-cerveau-laure-bally-cuif-l-humanite-magazine.html</link>
			<description></description>
			<pubDate>2024-06-06T07:10:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Articles-en-partenariat/un-poisson-pour-comprendre-les-cellules-souches-du-cerveau-laure-bally-cuif-l-humanite-magazine.html</guid>
		</item>
		<item>
			<title>Patrick Couvreur, lauréat de la &quot;Biomaterialia Golden Medal » et le « Controlled Release Society Founder Award » 2024</title>
			<link>http://www.academie-sciences.fr/fr/Membres-a-la-une/patrick-couvreur-laureat-de-la-biomaterialia-golden-medal-2024.html</link>
			<description></description>
			<pubDate>2024-05-28T12:00:00+02:00</pubDate>
			<guid isPermaLink="false">http://www.academie-sciences.fr/fr/Membres-a-la-une/patrick-couvreur-laureat-de-la-biomaterialia-golden-medal-2024.html</guid>
		</item>
	</channel>
</rss>

Alkarex · 2024-10-14T14:13:12Z

Another example from https://www.php.net/feed.atom
5 invalid GUIDs out of 741 (why!?) entries

<entry xmlns="http://www.w3.org/2005/Atom" xml:base="entries/2021-12-17-1.xml">
  <title>PHP 8.1.1 Released!</title>
  <id>https://www.php.net/archive/2021.php#2021-12-17-1</id>
  <published>2021-12-17T17:31:25+01:00</published>
  <updated>2021-12-17T17:31:25+01:00</updated>
  <link href="https://www.php.net/index.php#id2021-12-17-1" rel="alternate" type="text/html"/>
  <link href="https://www.php.net/archive/2021.php#2021-12-17-1" rel="via" type="text/html"/>
  <category term="releases" label="New PHP release"/>
  <category term="frontpage" label="PHP.net frontpage news"/>
  <content type="xhtml">
    <div xmlns="http://www.w3.org/1999/xhtml"><p>The PHP development team announces the immediate availability of PHP 8.1.1. This is a bug fix release.</p>

<p>All PHP 8.1 users are encouraged to upgrade to this version.</p>

<p>For source downloads of PHP 8.1.1 please visit our <a href="https://www.php.net/downloads.php">downloads page</a>,
Windows source and binaries can be found on <a href="https://windows.php.net/download/">windows.php.net/download/</a>.
The list of changes is recorded in the <a href="https://www.php.net/ChangeLog-8.php#8.1.1">ChangeLog</a>.
</p>    </div>
  </content>
</entry>
  <entry xmlns="http://www.w3.org/2005/Atom" xml:base="entries/2021-12-16-2.xml">
  <title>PHP 8.0.14 Released!</title>
  <id>https://www.php.net/archive/2021.php#2021-12-17-1</id>
  <published>2021-12-16T23:33:54+00:00</published>
  <updated>2021-12-16T23:33:54+00:00</updated>
  <link href="https://www.php.net/index.php#id2021-12-16-2" rel="alternate" type="text/html"/>
  <link href="https://www.php.net/archive/2021.php#2021-12-16-2" rel="via" type="text/html"/>
  <category term="releases" label="New PHP release"/>
  <category term="frontpage" label="PHP.net frontpage news"/>
  <content type="xhtml">
    <div xmlns="http://www.w3.org/1999/xhtml"><p>The PHP development team announces the immediate availability of PHP 8.0.14. This is a bug fix release.</p>

<p>All PHP 8.0 users are encouraged to upgrade to this version.</p>

<p>For source downloads of PHP 8.0.14 please visit our <a href="https://www.php.net/downloads.php">downloads page</a>,
Windows source and binaries can be found on <a href="https://windows.php.net/download/">windows.php.net/download/</a>.
The list of changes is recorded in the <a href="https://www.php.net/ChangeLog-8.php#8.0.14">ChangeLog</a>.
</p>    </div>
  </content>
</entry>

Frenzie · 2024-10-14T14:15:15Z

Strange, I wonder how that happened. The 8.0.14 is actually https://www.php.net/archive/2021.php#2021-12-16-2. A human copy/paste mistake?

Edit: looks like I'm wrong(ish) :-) The ID is 2021-12-17-1 so the link/id in the feed is correct, but in the href it gives 2021-12-16-2.

<a id="2021-12-17-1" href="https://www.php.net/archive/2021.php#2021-12-16-2" rel="bookmark" class="bookmark">PHP 8.0.14 Released!</a>

fix FreshRSS#6897 Regression from FreshRSS#4487

* Fix add feed with MariaDB / MySQL fix #6897 Regression from #4487 * Type fix

math-GH · 2024-12-07T21:13:01Z

app/i18n/es/sub.php

+			'_' => 'Article unicity criteria',	// TODO
+			'forced' => '<span title="Block the unicity criteria, even when the feed has duplicate articles">forced</span>',	// TODO
+			'help' => 'Relevant for invalid feeds.<br />⚠️ Changing the policy will create duplicates.',	// TODO
+			'id' => 'Standard ID (default)',	// TODO


@Alkarex
I need some understanding for Standard ID:
Is it the GUID (Global Unique ID)?

Yes, except that it is not a GUID but a unique ID within a given feed, and varies depending on the version of RSS / Atom.

See:

RSS 2.0 guid https://www.rssboard.org/rss-specification#ltguidgtSubelementOfLtitemgt

Atom id https://www.rfc-editor.org/rfc/rfc4287#page-19

RSS 1.0 item URI https://web.resource.org/rss/1.0/spec#s5.5

Would ID (default) also be a good String?
It is difficult to translate Standard and default in the same String because both could be translated to something like Standard ID (Standardeinstellung). Too much Standard in 1 String :D

Yes, I think this is fine enough

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [freshrss](https://github.com/FreshRSS/FreshRSS) | minor | `1.24.3` -> `1.26.0` | --- ### Release Notes <details> <summary>FreshRSS/FreshRSS (freshrss)</summary> ### [`v1.26.0`](https://github.com/FreshRSS/FreshRSS/blob/HEAD/CHANGELOG.md#2025-02-23-FreshRSS-1260) [Compare Source](FreshRSS/FreshRSS@1.25.0...1.26.0) - Features - Add order-by options to sort articles by received date (existing, default), publication date, title, link, random [#7149](FreshRSS/FreshRSS#7149) - Allow searching in all feeds, also feeds only visible at category level with `&get=A`, and also those archived with `&get=Z` [#7144](FreshRSS/FreshRSS#7144) - UI accessible from user-query view - Add search operator `intext:` [#7228](FreshRSS/FreshRSS#7228) - New shortcuts for adding user labels to articles [#7274](FreshRSS/FreshRSS#7274) - New *About* page with system information [#7161](FreshRSS/FreshRSS#7161) - Bug fixing - Fix regression denying access to app manifest [#7158](FreshRSS/FreshRSS#7158) - Fix unwanted feed description updates [#7269](FreshRSS/FreshRSS#7269) - Ensure no PHP buffer for SQLite download (some setups would first put the file in memory) [#7230](FreshRSS/FreshRSS#7230) - Fix XML encoding regression in HTML+XPath mode [#7345](FreshRSS/FreshRSS#7345) - Improve cURL proxy options and fix some constants [#7231](FreshRSS/FreshRSS#7231) - Fix UI of global view unread articles counter [#7247](FreshRSS/FreshRSS#7247) - Hide base theme in carrousel [#7234](FreshRSS/FreshRSS#7234) - Deployment - Reduce superfluous Docker builds [#7137](FreshRSS/FreshRSS#7137) - Docker default image (Debian 12 Bookworm) updated to PHP 8.2.26 and Apache 2.4.62 - Docker alternative image (Alpine 3.21) updated to PHP 8.3.16 - UI - Add footer icons to reader view [#7133](FreshRSS/FreshRSS#7133) - Remove local reference to font *Open Sans* to avoid bugs with some local versions [#7215](FreshRSS/FreshRSS#7215) - Improve stats page layout [#7243](FreshRSS/FreshRSS#7243) - Smaller *mark as read* button in mobile view [#5220](FreshRSS/FreshRSS#5220) - Add CSS class to various types of notifications to allow custom styling [#7287](FreshRSS/FreshRSS#7287) - Various UI and style improvements: [#7162](FreshRSS/FreshRSS#7162), [#7268](FreshRSS/FreshRSS#7268) Security - Better authorization label for OIDC in the UI [#7264](FreshRSS/FreshRSS#7264) - Allow comments in `force-https.txt` [#7259](FreshRSS/FreshRSS#7259) - I18n: - Improve German [#7177](FreshRSS/FreshRSS#7177), [#7275](FreshRSS/FreshRSS#7275), [#7278](FreshRSS/FreshRSS#7278) - Improve Japanese [#7187](FreshRSS/FreshRSS#7187), [#7195](FreshRSS/FreshRSS#7195), [#7332](FreshRSS/FreshRSS#7332) - Misc. - Improve PHP code [#7191](FreshRSS/FreshRSS#7191), [#7204](FreshRSS/FreshRSS#7204) - Upgrade to PHPStan 2 [#7131](FreshRSS/FreshRSS#7131), [#7164](FreshRSS/FreshRSS#7164), [#7224](FreshRSS/FreshRSS#7224), [#7270](FreshRSS/FreshRSS#7270), [#7281](FreshRSS/FreshRSS#7281), [#7282](FreshRSS/FreshRSS#7282) - Update to CssXPath 1.3.0 (no change) [#7211](FreshRSS/FreshRSS#7211) - Update dev dependencies [#7165](FreshRSS/FreshRSS#7165), [#7166](FreshRSS/FreshRSS#7166), [#7167](FreshRSS/FreshRSS#7167), [#7279](FreshRSS/FreshRSS#7279), [#7280](FreshRSS/FreshRSS#7280), [#7283](FreshRSS/FreshRSS#7283), [#7284](FreshRSS/FreshRSS#7284), [#7285](FreshRSS/FreshRSS#7285), [#7347](FreshRSS/FreshRSS#7347) - Update GitHub Actions to Ubuntu 24.04 [#7207](FreshRSS/FreshRSS#7207) ### [`v1.25.0`](https://github.com/FreshRSS/FreshRSS/blob/HEAD/CHANGELOG.md#2024-12-23-FreshRSS-1250) [Compare Source](FreshRSS/FreshRSS@1.24.3...1.25.0) - Features - Add support for [regex search (regular expressions)](https://freshrss.github.io/FreshRSS/en/users/10\_filter.html#regex) [#6706](FreshRSS/FreshRSS#6706), [#6926](FreshRSS/FreshRSS#6926) - ⚠️ Advanced regex syntax for searches depends on the database used (SQLite, PostgreSQL, MariaDB, MySQL), but FreshRSS filter actions such as auto-mark-as-read and auto-favourite always use [PHP PCRE2 syntax](https://php.net/regexp.introduction). - Allow dynamic search operator in user queries, like `search:UserQueryA date:P1d` [#6851](FreshRSS/FreshRSS#6851) - New feed mode *HTML+XPath+JSON dot notation* (JSON in HTML) [#6888](FreshRSS/FreshRSS#6888) - Better HTTP compliance with support for HTTP response headers `Cache-Control: max-age` and `Expires` [#6812](FreshRSS/FreshRSS#6812), [FreshRSS/simplepie#26](FreshRSS/simplepie#26) - Support custom HTTP request headers per feed (e.g. for `Authorization`) [#6820](FreshRSS/FreshRSS#6820) - New unicity policies and heuristic for feeds with bad article IDs [#4487](FreshRSS/FreshRSS#4487), [#6900](FreshRSS/FreshRSS#6900) - Fallback to GUID if article link is empty [#7051](FreshRSS/FreshRSS#7051) - New option to automatically mark new articles as read if an identical title already exists in the same category [#6922](FreshRSS/FreshRSS#6922) - New reading view option to display unread articles + favourites [#7088](FreshRSS/FreshRSS#7088) - And corresponding new filter state `&state=96` (no UI button yet) - Add ability to remove content from articles with CSS selectors, also when not using full content [#6786](FreshRSS/FreshRSS#6786), [#6807](FreshRSS/FreshRSS#6807) - Update `phpgt/cssxpath` library with improved CSS selectors [#6618](FreshRSS/FreshRSS#6618) - Support for `:last-child`, `:first-of-type`, `:last-of-type`, `^=`, `|=` - New condition option to selectively retrieve full content of articles [#33fd07f6f26310d4806077cc87bcdf9b8b940e35](FreshRSS/FreshRSS@33fd07f), [#7082](FreshRSS/FreshRSS#7082) - Allow parentheses in quoted search [#7055](FreshRSS/FreshRSS#7055) - New UI feature to download a user’ SQLite database or a database SQLite export (to be produced by CLI) [#6931](FreshRSS/FreshRSS#6931) - New button to delete errored feeds from a category [#7030](FreshRSS/FreshRSS#7030) - Better import of Inoreader user labels [#6791](FreshRSS/FreshRSS#6791) - Rebuild feed favicon on cache clear [#6961](FreshRSS/FreshRSS#6961) - New sharing with Bluesky [#7116](FreshRSS/FreshRSS#7116) - New sharing with Telegram [#6838](FreshRSS/FreshRSS#6838) - Bug fixing - Fix searches with a parenthesis before an operator like `("a b")` or `(!c)` [#6818](FreshRSS/FreshRSS#6818) - Fix auto-read tags [#6790](FreshRSS/FreshRSS#6790) - Fix CSS selector for removing elements [#7037](FreshRSS/FreshRSS#7037), [#7073](FreshRSS/FreshRSS#7073), [#7081](FreshRSS/FreshRSS#7081), [#7091](FreshRSS/FreshRSS#7091), [#7083](FreshRSS/FreshRSS#7083) - Fix redirection error after creating a new user [#6995](FreshRSS/FreshRSS#6995) - Fix favicon error in case of wrong URL [#6899](FreshRSS/FreshRSS#6899) - Use cURL to fetch extensions list (allows e.g. IPv6) [#6767](FreshRSS/FreshRSS#6767) - Fix XML encoding in cURL options [#6821](FreshRSS/FreshRSS#6821) - Fix initial UI scroll for some browsers [#7059](FreshRSS/FreshRSS#7059) - Fix menu for article tags in some cases [#6990](FreshRSS/FreshRSS#6990) - Fix share menu shortcut [#6825](FreshRSS/FreshRSS#6825) - Fix HTML regex pattern during install for compatibility with `v` mode [#7009](FreshRSS/FreshRSS#7009) - More robust creation of user data folder [#7000](FreshRSS/FreshRSS#7000) - API - Fix API for categories and labels containing a `+` [#7033](FreshRSS/FreshRSS#7033) - Compatibility with FocusReader - Supported by [Capy Reader](https://github.com/jocmp/capyreader) (Android, open source) [capyreader#492](jocmp/capyreader#492) - Improved UI for API [#7048](FreshRSS/FreshRSS#7048) - Allow adding multiple feeds to a category via API [#7017](FreshRSS/FreshRSS#7017) - API support edit multiple tags [#7060](FreshRSS/FreshRSS#7060) - API return all categories also those without any feed [#7020](FreshRSS/FreshRSS#7020) - Compatibility - Require PHP 8.1+ (drop PHP 7.4) [#6711](FreshRSS/FreshRSS#6711) - Improved support of PHP 8.4+ [#6618](FreshRSS/FreshRSS#6618), [phpgt/CssXPath#227](phpgt/CssXPath#227), [#6781](FreshRSS/FreshRSS#6781), [#4374](FreshRSS/FreshRSS#4374) - Require PostgreSQL 10+ (drop PostgreSQL 9.5) [#6705](FreshRSS/FreshRSS#6705) - Require MariaDB 10.0.5+ (drop MariaDB 5.5) [#6706](FreshRSS/FreshRSS#6706) - Require MySQL 8+ (drop MySQL 5.5.3) [#6706](FreshRSS/FreshRSS#6706) - Deployment - Docker: dev image `freshrss/freshrss:oldest` updated to Alpine 3.16 with PHP 8.1.22 and Apache 2.4.59 [#6711](FreshRSS/FreshRSS#6711) - Docker alternative image updated to Alpine 3.21 with PHP 8.3.14 and Apache 2.4.62 [#5383](FreshRSS/FreshRSS#5383) - Update Dockerfiles to newer key-value format [#6819](FreshRSS/FreshRSS#6819) - Docker minor improvement of entrypoint [#6827](FreshRSS/FreshRSS#6827) - SimplePie - Refactor [our embedding](lib/README.md) of SimplePie [#4374](FreshRSS/FreshRSS#4374) - Our fork is maintained in its [own repository](https://github.com/FreshRSS/simplepie/tree/freshrss). - Remove HTTP `Referer` [#6822](FreshRSS/FreshRSS#6822), [FreshRSS/simplepie#27](FreshRSS/simplepie#27) - If some sites require it, add `Referer: https://example.net/` to the custom HTTP headers of the feed [#6820](FreshRSS/FreshRSS#6820) - Upstream fixes [simplepie#878](simplepie/simplepie#878), [simplepie#883](simplepie/simplepie#883) - Sync upstream [#6840](FreshRSS/FreshRSS#6840), [#7067](FreshRSS/FreshRSS#7067) - Security - Apache protect more non-public folders and files [#6881](FreshRSS/FreshRSS#6881), [#6893](FreshRSS/FreshRSS#6893), [#7008](FreshRSS/FreshRSS#7008) - Add privacy settings on extension list retrieval [#4603](FreshRSS/FreshRSS#4603), [#7132](FreshRSS/FreshRSS#7132) - Fix login in unsafe mode when using a password with special XML characters [#6797](FreshRSS/FreshRSS#6797) - Fix login in e.g. Brave browser by avoiding synchronous XHR [#7023](FreshRSS/FreshRSS#7023) - Fix invalid login message [#7066](FreshRSS/FreshRSS#7066) - Modernise `windows.open noopener` (to avoid flash of white page in dark mode) [#7077](FreshRSS/FreshRSS#7077), [#7089](FreshRSS/FreshRSS#7089) - UI - Searchable *My Labels* field [#6753](FreshRSS/FreshRSS#6753) - Add subscription management button to reading view [#6946](FreshRSS/FreshRSS#6946) - New option for showing label menu in article row [#6984](FreshRSS/FreshRSS#6984) - Move to next unread label on mark as read [#6886](FreshRSS/FreshRSS#6886) - Improved article footer for small / mobile screens [#7031](FreshRSS/FreshRSS#7031) - Improve Web accessibility: fix `aria-hidden` bug, and use HTML5 `hidden` [#6910](FreshRSS/FreshRSS#6910) - Default styles for `<pre>` and `<code>` [#6770](FreshRSS/FreshRSS#6770) - Refactor the sharing menu to use a `<template>` instead of duplicated HTML code [#6751](FreshRSS/FreshRSS#6751), [#7113](FreshRSS/FreshRSS#7113) - Refactor the label menu to use a `<template>` [#6864](FreshRSS/FreshRSS#6864) - Rework UI for authors [#7054](FreshRSS/FreshRSS#7054) - Avoid Unicode escape of authors in HTML UI [#7056](FreshRSS/FreshRSS#7056) - Improved subscription management page [#6816](FreshRSS/FreshRSS#6816) - Improve user query management page [#7062](FreshRSS/FreshRSS#7062) - Restore JavaScript form validation compatibility with Web browsers using older engines (SeaMonkey) [#6777](FreshRSS/FreshRSS#6777) - Reorganise some options [#6920](FreshRSS/FreshRSS#6920) - New shortcut `?` to show shortcut page and help [#6981](FreshRSS/FreshRSS#6981) - Use of consistent colours in statistics [#7090](FreshRSS/FreshRSS#7090) - Various UI and style improvements [#6959](FreshRSS/FreshRSS#6959) - Extensions - New extension hook `simplepie_after_init` [#7007](FreshRSS/FreshRSS#7007) - I18n - Add Finnish [#6954](FreshRSS/FreshRSS#6954) - Improve English [#7049](FreshRSS/FreshRSS#7049), [#7053](FreshRSS/FreshRSS#7053) - Improve German [#6847](FreshRSS/FreshRSS#6847), [#7068](FreshRSS/FreshRSS#7068), [#7128](FreshRSS/FreshRSS#7128) - Improve Italian [#6872](FreshRSS/FreshRSS#6872), [#7069](FreshRSS/FreshRSS#7069), [#7086](FreshRSS/FreshRSS#7086) - Improve Spanish [#6894](FreshRSS/FreshRSS#6894), [#6908](FreshRSS/FreshRSS#6908) - Improve Turkish [#6960](FreshRSS/FreshRSS#6960) - Misc. - Better cache name for JSON feeds [#6768](FreshRSS/FreshRSS#6768) - Fix inversed encoding logic in `Minz_Request::paramArray()` [#6800](FreshRSS/FreshRSS#6800) - Pass PHPStan `booleansInConditions` [#6793](FreshRSS/FreshRSS#6793) - Rename PHPStan configuration file to `phpstan.dist.neon` to allow custom configuration in `phpstan.neon` [#6892](FreshRSS/FreshRSS#6892) - Code improvements [#6800](FreshRSS/FreshRSS#6800), [#6809](FreshRSS/FreshRSS#6809), [#6983](FreshRSS/FreshRSS#6983) - Makefile improvements [#6913](FreshRSS/FreshRSS#6913) - Fix PHPCS `ControlSignature` [#6896](FreshRSS/FreshRSS#6896) - Update *PHPMailer* [#6968](FreshRSS/FreshRSS#6968), [#7046](FreshRSS/FreshRSS#7046) - Code updates to PHP 8.1 syntax [#6748](FreshRSS/FreshRSS#6748) - Update dev dependencies [#6780](FreshRSS/FreshRSS#6780), [#6964](FreshRSS/FreshRSS#6964), , [#6965](FreshRSS/FreshRSS#6965), [#6966](FreshRSS/FreshRSS#6966), [#6967](FreshRSS/FreshRSS#6967), [#6970](FreshRSS/FreshRSS#6970), [#7042](FreshRSS/FreshRSS#7042), [#7043](FreshRSS/FreshRSS#7043), [#7044](FreshRSS/FreshRSS#7044), [#7045](FreshRSS/FreshRSS#7045), [#7047](FreshRSS/FreshRSS#7047), [#7052](FreshRSS/FreshRSS#7052) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://gitea.sh4ke.rocks/lickler/freshrss/pulls/29 Co-authored-by: Michael Wittig <michael.wittig@posteo.de> Co-committed-by: Michael Wittig <michael.wittig@posteo.de>

Use hash of link+title fallback for bad GUIDs

c9ffa2c

Alkarex added the Feed problem 🗞️ Feeds that have issues while loading/reading in FreshRSS label Aug 4, 2022

Alkarex added this to the 1.20.0 milestone Aug 4, 2022

Alkarex mentioned this pull request Aug 4, 2022

[BUG] feed is missing newest items #4485

Closed

Frenzie approved these changes Aug 4, 2022

View reviewed changes

Merge branch 'edge' into better-bad-guids

38f62bb

Alkarex modified the milestones: 1.20.0, 1.21.0 Aug 31, 2022

Alkarex modified the milestones: 1.21.0, 1.22.0 Feb 6, 2023

Alkarex mentioned this pull request Mar 14, 2023

[BUG] Duplicate posts appearing from a WordPress blog #5151

Closed

Alkarex mentioned this pull request Mar 27, 2023

[BUG] old article appears again, which were already marked as read #5210

Closed

Alkarex mentioned this pull request Apr 8, 2023

[BUG] ITems showing up that don't show up in RSS source #5257

Closed

Alkarex modified the milestones: 1.22.0, 1.23.0 Aug 13, 2023

Alkarex modified the milestones: 1.23.0, 1.24.0 Nov 24, 2023

Alkarex modified the milestones: 1.24.0, 1.25.0 Apr 1, 2024

math-GH added the BLOCKED ❌ merge conflict or failing tests that block the PR label Jun 14, 2024

Alkarex mentioned this pull request Jul 24, 2024

[Bug] Can't update rss flux on 2 web sites H2 and CERT #6658

Closed

Alkarex added 3 commits July 24, 2024 17:19

Merge branch 'edge' into better-bad-guids

7310644

Merge branch 'edge' into better-bad-guids

b87ea61

Merge branch 'edge' into better-bad-guids

027b913

Minor syntax

28acccb

Alkarex mentioned this pull request Sep 29, 2024

[BUG] Medium.com RSS feeds have new articles every day, even if those articles are old #5909

Closed

Alkarex linked an issue Sep 29, 2024 that may be closed by this pull request

[BUG] Medium.com RSS feeds have new articles every day, even if those articles are old #5909

Closed

Alkarex removed the BLOCKED ❌ merge conflict or failing tests that block the PR label Oct 2, 2024

Frenzie reviewed Oct 4, 2024

View reviewed changes

Alkarex added 3 commits October 4, 2024 22:26

Slight rewording

7c1caa0

Rename variables to unicityCriteria

156797b

Merge branch 'edge' into better-bad-guids

3f1b1a0

This was linked to issues Oct 6, 2024

[BUG] old article appears again, which were already marked as read #5210

Closed

[BUG] ITems showing up that don't show up in RSS source #5257

Closed

Alkarex added 6 commits October 6, 2024 11:46

Change automatic criteria degrading logic

bf29a8f

Do not fall back to hash of content automatically, as this seems to often create too many duplicates

Typo

77756ed

unicityCriteriaForced

952fa45

label title

d8fbf7e

Merge branch 'edge' into better-bad-guids

094e1e7

invalidGuidsTolerance

7dd84f1

Forgotten parameter

72440d7

Merge branch 'edge' into better-bad-guids

1443d3f

Alkarex merged commit 256dcc2 into FreshRSS:edge Oct 14, 2024

Alkarex deleted the better-bad-guids branch October 14, 2024 07:34

Alkarex mentioned this pull request Oct 15, 2024

[Bug] Adding feeds is broken (MariaDB / MySQL) #6897

Closed

Alkarex added a commit to Alkarex/FreshRSS that referenced this pull request Oct 15, 2024

Fix add feed with MariaDB / MySQL

4a560aa

fix FreshRSS#6897 Regression from FreshRSS#4487

Alkarex mentioned this pull request Oct 15, 2024

Fix add feed with MariaDB / MySQL #6900

Merged

Alkarex added a commit that referenced this pull request Oct 15, 2024

Fix add feed with MariaDB / MySQL (#6900)

f77b456

* Fix add feed with MariaDB / MySQL fix #6897 Regression from #4487 * Type fix

Frenzie mentioned this pull request Nov 21, 2024

[Bug] Unable to Retrieve All Feed Items #7012

Closed

Alkarex mentioned this pull request Nov 28, 2024

[BUG] FreshRSS duplicating entries in very rare, mysterious circumstances #5410

Closed

math-GH reviewed Dec 7, 2024

View reviewed changes

Vynce mentioned this pull request Dec 7, 2024

[WirecutterDealsBridge] Add bridge RSS-Bridge/rss-bridge#4359

Merged

	'_' => 'Articles unicity',
	'_' => 'Article uniqueness criteria',

	'_' => 'Articles unicity',
	'_' => 'Article duplicate matching criteria',

Uh oh!

Conversation

Alkarex commented Aug 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Frenzie left a comment

Choose a reason for hiding this comment

Uh oh!

eht16 commented Sep 10, 2022

Uh oh!

Alkarex commented Sep 28, 2024

Uh oh!

Frenzie Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

Alkarex Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

Frenzie Oct 4, 2024

Choose a reason for hiding this comment

Uh oh!

Alkarex commented Oct 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Frenzie commented Oct 11, 2024

Uh oh!

Alkarex commented Oct 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Alkarex commented Oct 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Frenzie commented Oct 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

math-GH Dec 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Alkarex Dec 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

math-GH Dec 7, 2024

Choose a reason for hiding this comment

Uh oh!

Alkarex Dec 7, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Alkarex commented Aug 4, 2022 •

edited

Loading

Alkarex commented Oct 11, 2024 •

edited

Loading

Alkarex commented Oct 12, 2024 •

edited

Loading

Alkarex commented Oct 14, 2024 •

edited

Loading

Frenzie commented Oct 14, 2024 •

edited

Loading

math-GH Dec 7, 2024 •

edited

Loading

Alkarex Dec 7, 2024 •

edited

Loading