Skip to content

Implement custom order-by#7149

Merged
Alkarex merged 61 commits intoFreshRSS:edgefrom
Alkarex:order-by
Jan 6, 2025
Merged

Implement custom order-by#7149
Alkarex merged 61 commits intoFreshRSS:edgefrom
Alkarex:order-by

Conversation

@Alkarex
Copy link
Member

@Alkarex Alkarex commented Dec 27, 2024

Add option to sort results by received date (existing, default), publication date, title, URL (link), random.

fix #1771
fix #2083
fix #2119
fix #2596
fix #3204
fix #4405
fix #5529
fix #5864
fix FreshRSS/Extensions#161

URL parameters:

  • &sort=id (current behaviour, sorting according to newest received articles)
  • &sort=date (publication date, which is not indicative of how new an article is)
  • &sort=title
  • &sort=link
  • &sort=rand (random order - which disables infinite scrolling, at least for now)

combined with &order=ASC or &order=DESC

image

Implementation notes

The sorting criteria by received date (id), which is the default, and which was the only one before this PR, is the one that has the best sorting characteristics:

  • uniqueness: no entries have the exact same received date
  • monotonicity: new entries always have a higher received date
  • performance: this field is efficiently indexed in database for fast usage, including for paging (indexing could also be done to other fields, but with lower effective performance)

In contrary, sorting criteria such as by publication date, by title, or by link are neither unique nor monotonic. In particular, multiple articles may share the same publication date, and we may receive articles with a publication date far in the future, and then later some new articles with a publication date far in the past.

To understand why sorting by publication date is problematic, it helps to think about sorting by title or by link, as sorting by title and by publication date share more or less the same characteristics.

Problem 1: new articles

New articles may be received in the background after what is shown on screen, and before the next user action such as mark all as read. Due to the lack of monotonicity when sorting by e.g. publication date or title, users risk marking as read a batch of articles containing some fresh articles without seeing them.

Mitigation: A parameter idMax tracks the maximum ID related to a batch of actions such as mark all as read to exclude articles received after those that are displayed.

Problem 2: paging / pagination

When navigating articles, only a few articles are displayed, and a new "page" of articles needs to be received from the database when scrolling down or when clicking the button to show more articles. When sorting by e.g. publication date or title, it is not trivial to show the next page without re-showing some of the same articles, and without skipping any. Indeed, views are often with additional criteria such as showing only unread articles, and users may mark some articles as read while viewing them, hereby removing some articles from the previous pages. And like for Problem 1, new articles may have been received in the background. Consequently, it is not possible to use OFFSET to implement pagination (so the patches suggested by a few users were wrong due to that, in particular).

Mitigation: idMax is also used (just like for Problem 1) and a Keyset Pagination approach is used, combining an unstable sorting criterion such as publication date or title, together with id to ensure stable sorting. (So, 2 sorting criteria + 1 filter criteria)

See e.g. https://www.alwaysdeveloping.net/dailydrop/2022/07/01-keyset-pagination/

Problem 3: performance

Sorting by anything else than received date (id) is doomed to be slow(er) due to the combination of 3 criteria (see Problem 2). An OFFSET approach (which is not possible anyway as explained) would be even slower. Furthermore, we have no SQL index at the moment, but they would not necessarily help much due to the multiple sorting criteria needed and involving some OR logic which is difficult to optimise for databases.

The nicest syntax would be using tuples and corresponding indexes, but that is poorly supported by MySQL https://bugs.mysql.com/bug.php?id=104128

Mitigation: a compatibility SQL syntax is used to implement Keyset Pagination

Problem 4: user confusion

Several users have shown that they do not fully understand the difference between received date and publication date, and particularly not the pitfalls of publication date.

Mitigation: the menus to mark-as-read before 1 day and before 1 week are disabled when sorting by anything else than received date. Likewise, the separation headers Today and Yesterday and Before yesterday are only shown when sorting by received date.

Again here, to better understand why, it helps to think about sorting by title or by link, as sorting by title and by publication date share more or less the same characteristics.

  • We should write a Q&A and/or documentation about the problems associated to sorting by publication date: risks of not noticing new publication, of inadvertently marking them as read, of having some articles with a date in the future hanging at the top of the views (vice versa when sorting in ascending order), performance, etc.

Problem 5: APIs

Sorting by anything else than received date breaks the guarantees needed for a successful synchronisation via API.

Mitigation: sorting by received date is ensured for all API calls.

Alkarex and others added 2 commits January 6, 2025 15:56
Co-authored-by: maTh <1645099+math-GH@users.noreply.github.com>
@Alkarex Alkarex merged commit 1f466d7 into FreshRSS:edge Jan 6, 2025
@Alkarex Alkarex deleted the order-by branch January 6, 2025 15:00
@Fayyaadh
Copy link

Add option to sort results by received date (existing, default), publication date, title, URL (link), random.

fix #1771 fix #2083 fix #2119 fix #2596 fix #3204 fix #4405 fix #5529 fix #5864 fix FreshRSS/Extensions#161

URL parameters:

  • &sort=id (current behaviour, sorting according to newest received articles)
  • &sort=date (publication date, which is not indicative of how new an article is)
  • &sort=title
  • &sort=link
  • &sort=rand (random order - which disables infinite scrolling, at least for now)

combined with &order=ASC or &order=DESC

Hi there

Thank you so much for all your work on this.

Can you perhaps elaborate a bit on how to use this?

I went to the main feed page and added the query parameter for &sort=date to the URL but nothing happened.

Is the feature not yet released in the latest Docker image?

Thanks for the help.

@Alkarex
Copy link
Member Author

Alkarex commented Jan 20, 2025

@Fayyaadh What Docker image are you running, more precisely?
Make sure you are using our rolling release and are using one of our official images such as freshrss/freshrss:edge or freshrss/freshrss:alpine-edge

@Fayyaadh
Copy link

@Fayyaadh What Docker image are you running, more precisely? Make sure you are using our rolling release and are using one of our official images such as freshrss/freshrss:edge or freshrss/freshrss:alpine-edge

My docker compose file just has freshrss/freshrss without any tag.

I guess I'll need to change that?

@Frenzie
Copy link
Member

Frenzie commented Jan 20, 2025

Yes, this is only in edge.

@Fayyaadh
Copy link

Ok I pulled the new docker image and the feature now works!

Is there any way to get it to persist though?

So if I click away from the main feed and then go back, the query parameter &sort=date is no longer there.

Sh4kE added a commit to Sh4kE/freshrss-helm-chart that referenced this pull request Feb 24, 2025
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [freshrss/freshrss](https://freshrss.org/) ([source](https://github.com/FreshRSS/FreshRSS)) | minor | `1.25.0-alpine` -> `1.26.0-alpine` |

---

### Release Notes

<details>
<summary>FreshRSS/FreshRSS (freshrss/freshrss)</summary>

### [`v1.26.0`](https://github.com/FreshRSS/FreshRSS/blob/HEAD/CHANGELOG.md#2025-02-23-FreshRSS-1260)

[Compare Source](FreshRSS/FreshRSS@1.25.0...1.26.0)

-   Features
    -   Add order-by options to sort articles by received date (existing, default), publication date, title, link, random [#&#8203;7149](FreshRSS/FreshRSS#7149)
    -   Allow searching in all feeds, also feeds only visible at category level with `&get=A`, and also those archived with `&get=Z` [#&#8203;7144](FreshRSS/FreshRSS#7144)
        -   UI accessible from user-query view
    -   Add search operator `intext:` [#&#8203;7228](FreshRSS/FreshRSS#7228)
    -   New shortcuts for adding user labels to articles [#&#8203;7274](FreshRSS/FreshRSS#7274)
    -   New *About* page with system information [#&#8203;7161](FreshRSS/FreshRSS#7161)
-   Bug fixing
    -   Fix regression denying access to app manifest [#&#8203;7158](FreshRSS/FreshRSS#7158)
    -   Fix unwanted feed description updates [#&#8203;7269](FreshRSS/FreshRSS#7269)
    -   Ensure no PHP buffer for SQLite download (some setups would first put the file in memory) [#&#8203;7230](FreshRSS/FreshRSS#7230)
    -   Fix XML encoding regression in HTML+XPath mode [#&#8203;7345](FreshRSS/FreshRSS#7345)
    -   Improve cURL proxy options and fix some constants [#&#8203;7231](FreshRSS/FreshRSS#7231)
    -   Fix UI of global view unread articles counter [#&#8203;7247](FreshRSS/FreshRSS#7247)
    -   Hide base theme in carrousel [#&#8203;7234](FreshRSS/FreshRSS#7234)
-   Deployment
    -   Reduce superfluous Docker builds [#&#8203;7137](FreshRSS/FreshRSS#7137)
    -   Docker default image (Debian 12 Bookworm) updated to PHP 8.2.26 and Apache 2.4.62
    -   Docker alternative image (Alpine 3.21) updated to PHP 8.3.16
-   UI
    -   Add footer icons to reader view [#&#8203;7133](FreshRSS/FreshRSS#7133)
    -   Remove local reference to font *Open Sans* to avoid bugs with some local versions [#&#8203;7215](FreshRSS/FreshRSS#7215)
    -   Improve stats page layout [#&#8203;7243](FreshRSS/FreshRSS#7243)
    -   Smaller *mark as read* button in mobile view [#&#8203;5220](FreshRSS/FreshRSS#5220)
    -   Add CSS class to various types of notifications to allow custom styling [#&#8203;7287](FreshRSS/FreshRSS#7287)
    -   Various UI and style improvements: [#&#8203;7162](FreshRSS/FreshRSS#7162), [#&#8203;7268](FreshRSS/FreshRSS#7268)
        Security
    -   Better authorization label for OIDC in the UI [#&#8203;7264](FreshRSS/FreshRSS#7264)
    -   Allow comments in `force-https.txt` [#&#8203;7259](FreshRSS/FreshRSS#7259)
-   I18n:
    -   Improve German [#&#8203;7177](FreshRSS/FreshRSS#7177), [#&#8203;7275](FreshRSS/FreshRSS#7275), [#&#8203;7278](FreshRSS/FreshRSS#7278)
    -   Improve Japanese [#&#8203;7187](FreshRSS/FreshRSS#7187), [#&#8203;7195](FreshRSS/FreshRSS#7195), [#&#8203;7332](FreshRSS/FreshRSS#7332)
-   Misc.
    -   Improve PHP code [#&#8203;7191](FreshRSS/FreshRSS#7191), [#&#8203;7204](FreshRSS/FreshRSS#7204)
        -   Upgrade to PHPStan 2 [#&#8203;7131](FreshRSS/FreshRSS#7131), [#&#8203;7164](FreshRSS/FreshRSS#7164), [#&#8203;7224](FreshRSS/FreshRSS#7224),
            [#&#8203;7270](FreshRSS/FreshRSS#7270), [#&#8203;7281](FreshRSS/FreshRSS#7281), [#&#8203;7282](FreshRSS/FreshRSS#7282)
    -   Update to CssXPath 1.3.0 (no change) [#&#8203;7211](FreshRSS/FreshRSS#7211)
    -   Update dev dependencies [#&#8203;7165](FreshRSS/FreshRSS#7165), [#&#8203;7166](FreshRSS/FreshRSS#7166), [#&#8203;7167](FreshRSS/FreshRSS#7167),
        [#&#8203;7279](FreshRSS/FreshRSS#7279), [#&#8203;7280](FreshRSS/FreshRSS#7280), [#&#8203;7283](FreshRSS/FreshRSS#7283),
        [#&#8203;7284](FreshRSS/FreshRSS#7284), [#&#8203;7285](FreshRSS/FreshRSS#7285), [#&#8203;7347](FreshRSS/FreshRSS#7347)
    -   Update GitHub Actions to Ubuntu 24.04 [#&#8203;7207](FreshRSS/FreshRSS#7207)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xNzYuNCIsInVwZGF0ZWRJblZlciI6IjM5LjE3Ni40IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.sh4ke.rocks/lickler/freshrss/pulls/32
Co-authored-by: Michael Wittig <michael.wittig@posteo.de>
Co-committed-by: Michael Wittig <michael.wittig@posteo.de>
Sh4kE added a commit to Sh4kE/freshrss-helm-chart that referenced this pull request Feb 24, 2025
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [freshrss](https://github.com/FreshRSS/FreshRSS) | minor | `1.25.0` -> `1.26.0` |

---

### Release Notes

<details>
<summary>FreshRSS/FreshRSS (freshrss)</summary>

### [`v1.26.0`](https://github.com/FreshRSS/FreshRSS/blob/HEAD/CHANGELOG.md#2025-02-23-FreshRSS-1260)

[Compare Source](FreshRSS/FreshRSS@1.25.0...1.26.0)

-   Features
    -   Add order-by options to sort articles by received date (existing, default), publication date, title, link, random [#&#8203;7149](FreshRSS/FreshRSS#7149)
    -   Allow searching in all feeds, also feeds only visible at category level with `&get=A`, and also those archived with `&get=Z` [#&#8203;7144](FreshRSS/FreshRSS#7144)
        -   UI accessible from user-query view
    -   Add search operator `intext:` [#&#8203;7228](FreshRSS/FreshRSS#7228)
    -   New shortcuts for adding user labels to articles [#&#8203;7274](FreshRSS/FreshRSS#7274)
    -   New *About* page with system information [#&#8203;7161](FreshRSS/FreshRSS#7161)
-   Bug fixing
    -   Fix regression denying access to app manifest [#&#8203;7158](FreshRSS/FreshRSS#7158)
    -   Fix unwanted feed description updates [#&#8203;7269](FreshRSS/FreshRSS#7269)
    -   Ensure no PHP buffer for SQLite download (some setups would first put the file in memory) [#&#8203;7230](FreshRSS/FreshRSS#7230)
    -   Fix XML encoding regression in HTML+XPath mode [#&#8203;7345](FreshRSS/FreshRSS#7345)
    -   Improve cURL proxy options and fix some constants [#&#8203;7231](FreshRSS/FreshRSS#7231)
    -   Fix UI of global view unread articles counter [#&#8203;7247](FreshRSS/FreshRSS#7247)
    -   Hide base theme in carrousel [#&#8203;7234](FreshRSS/FreshRSS#7234)
-   Deployment
    -   Reduce superfluous Docker builds [#&#8203;7137](FreshRSS/FreshRSS#7137)
    -   Docker default image (Debian 12 Bookworm) updated to PHP 8.2.26 and Apache 2.4.62
    -   Docker alternative image (Alpine 3.21) updated to PHP 8.3.16
-   UI
    -   Add footer icons to reader view [#&#8203;7133](FreshRSS/FreshRSS#7133)
    -   Remove local reference to font *Open Sans* to avoid bugs with some local versions [#&#8203;7215](FreshRSS/FreshRSS#7215)
    -   Improve stats page layout [#&#8203;7243](FreshRSS/FreshRSS#7243)
    -   Smaller *mark as read* button in mobile view [#&#8203;5220](FreshRSS/FreshRSS#5220)
    -   Add CSS class to various types of notifications to allow custom styling [#&#8203;7287](FreshRSS/FreshRSS#7287)
    -   Various UI and style improvements: [#&#8203;7162](FreshRSS/FreshRSS#7162), [#&#8203;7268](FreshRSS/FreshRSS#7268)
        Security
    -   Better authorization label for OIDC in the UI [#&#8203;7264](FreshRSS/FreshRSS#7264)
    -   Allow comments in `force-https.txt` [#&#8203;7259](FreshRSS/FreshRSS#7259)
-   I18n:
    -   Improve German [#&#8203;7177](FreshRSS/FreshRSS#7177), [#&#8203;7275](FreshRSS/FreshRSS#7275), [#&#8203;7278](FreshRSS/FreshRSS#7278)
    -   Improve Japanese [#&#8203;7187](FreshRSS/FreshRSS#7187), [#&#8203;7195](FreshRSS/FreshRSS#7195), [#&#8203;7332](FreshRSS/FreshRSS#7332)
-   Misc.
    -   Improve PHP code [#&#8203;7191](FreshRSS/FreshRSS#7191), [#&#8203;7204](FreshRSS/FreshRSS#7204)
        -   Upgrade to PHPStan 2 [#&#8203;7131](FreshRSS/FreshRSS#7131), [#&#8203;7164](FreshRSS/FreshRSS#7164), [#&#8203;7224](FreshRSS/FreshRSS#7224),
            [#&#8203;7270](FreshRSS/FreshRSS#7270), [#&#8203;7281](FreshRSS/FreshRSS#7281), [#&#8203;7282](FreshRSS/FreshRSS#7282)
    -   Update to CssXPath 1.3.0 (no change) [#&#8203;7211](FreshRSS/FreshRSS#7211)
    -   Update dev dependencies [#&#8203;7165](FreshRSS/FreshRSS#7165), [#&#8203;7166](FreshRSS/FreshRSS#7166), [#&#8203;7167](FreshRSS/FreshRSS#7167),
        [#&#8203;7279](FreshRSS/FreshRSS#7279), [#&#8203;7280](FreshRSS/FreshRSS#7280), [#&#8203;7283](FreshRSS/FreshRSS#7283),
        [#&#8203;7284](FreshRSS/FreshRSS#7284), [#&#8203;7285](FreshRSS/FreshRSS#7285), [#&#8203;7347](FreshRSS/FreshRSS#7347)
    -   Update GitHub Actions to Ubuntu 24.04 [#&#8203;7207](FreshRSS/FreshRSS#7207)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xNzYuNCIsInVwZGF0ZWRJblZlciI6IjM5LjE3Ni40IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Reviewed-on: https://gitea.sh4ke.rocks/lickler/freshrss/pulls/33
Co-authored-by: Michael Wittig <michael.wittig@posteo.de>
Co-committed-by: Michael Wittig <michael.wittig@posteo.de>
Sh4kE added a commit to Sh4kE/freshrss-helm-chart that referenced this pull request Feb 24, 2025
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [freshrss](https://github.com/FreshRSS/FreshRSS) | minor | `1.24.3` -> `1.26.0` |

---

### Release Notes

<details>
<summary>FreshRSS/FreshRSS (freshrss)</summary>

### [`v1.26.0`](https://github.com/FreshRSS/FreshRSS/blob/HEAD/CHANGELOG.md#2025-02-23-FreshRSS-1260)

[Compare Source](FreshRSS/FreshRSS@1.25.0...1.26.0)

-   Features
    -   Add order-by options to sort articles by received date (existing, default), publication date, title, link, random [#&#8203;7149](FreshRSS/FreshRSS#7149)
    -   Allow searching in all feeds, also feeds only visible at category level with `&get=A`, and also those archived with `&get=Z` [#&#8203;7144](FreshRSS/FreshRSS#7144)
        -   UI accessible from user-query view
    -   Add search operator `intext:` [#&#8203;7228](FreshRSS/FreshRSS#7228)
    -   New shortcuts for adding user labels to articles [#&#8203;7274](FreshRSS/FreshRSS#7274)
    -   New *About* page with system information [#&#8203;7161](FreshRSS/FreshRSS#7161)
-   Bug fixing
    -   Fix regression denying access to app manifest [#&#8203;7158](FreshRSS/FreshRSS#7158)
    -   Fix unwanted feed description updates [#&#8203;7269](FreshRSS/FreshRSS#7269)
    -   Ensure no PHP buffer for SQLite download (some setups would first put the file in memory) [#&#8203;7230](FreshRSS/FreshRSS#7230)
    -   Fix XML encoding regression in HTML+XPath mode [#&#8203;7345](FreshRSS/FreshRSS#7345)
    -   Improve cURL proxy options and fix some constants [#&#8203;7231](FreshRSS/FreshRSS#7231)
    -   Fix UI of global view unread articles counter [#&#8203;7247](FreshRSS/FreshRSS#7247)
    -   Hide base theme in carrousel [#&#8203;7234](FreshRSS/FreshRSS#7234)
-   Deployment
    -   Reduce superfluous Docker builds [#&#8203;7137](FreshRSS/FreshRSS#7137)
    -   Docker default image (Debian 12 Bookworm) updated to PHP 8.2.26 and Apache 2.4.62
    -   Docker alternative image (Alpine 3.21) updated to PHP 8.3.16
-   UI
    -   Add footer icons to reader view [#&#8203;7133](FreshRSS/FreshRSS#7133)
    -   Remove local reference to font *Open Sans* to avoid bugs with some local versions [#&#8203;7215](FreshRSS/FreshRSS#7215)
    -   Improve stats page layout [#&#8203;7243](FreshRSS/FreshRSS#7243)
    -   Smaller *mark as read* button in mobile view [#&#8203;5220](FreshRSS/FreshRSS#5220)
    -   Add CSS class to various types of notifications to allow custom styling [#&#8203;7287](FreshRSS/FreshRSS#7287)
    -   Various UI and style improvements: [#&#8203;7162](FreshRSS/FreshRSS#7162), [#&#8203;7268](FreshRSS/FreshRSS#7268)
        Security
    -   Better authorization label for OIDC in the UI [#&#8203;7264](FreshRSS/FreshRSS#7264)
    -   Allow comments in `force-https.txt` [#&#8203;7259](FreshRSS/FreshRSS#7259)
-   I18n:
    -   Improve German [#&#8203;7177](FreshRSS/FreshRSS#7177), [#&#8203;7275](FreshRSS/FreshRSS#7275), [#&#8203;7278](FreshRSS/FreshRSS#7278)
    -   Improve Japanese [#&#8203;7187](FreshRSS/FreshRSS#7187), [#&#8203;7195](FreshRSS/FreshRSS#7195), [#&#8203;7332](FreshRSS/FreshRSS#7332)
-   Misc.
    -   Improve PHP code [#&#8203;7191](FreshRSS/FreshRSS#7191), [#&#8203;7204](FreshRSS/FreshRSS#7204)
        -   Upgrade to PHPStan 2 [#&#8203;7131](FreshRSS/FreshRSS#7131), [#&#8203;7164](FreshRSS/FreshRSS#7164), [#&#8203;7224](FreshRSS/FreshRSS#7224),
            [#&#8203;7270](FreshRSS/FreshRSS#7270), [#&#8203;7281](FreshRSS/FreshRSS#7281), [#&#8203;7282](FreshRSS/FreshRSS#7282)
    -   Update to CssXPath 1.3.0 (no change) [#&#8203;7211](FreshRSS/FreshRSS#7211)
    -   Update dev dependencies [#&#8203;7165](FreshRSS/FreshRSS#7165), [#&#8203;7166](FreshRSS/FreshRSS#7166), [#&#8203;7167](FreshRSS/FreshRSS#7167),
        [#&#8203;7279](FreshRSS/FreshRSS#7279), [#&#8203;7280](FreshRSS/FreshRSS#7280), [#&#8203;7283](FreshRSS/FreshRSS#7283),
        [#&#8203;7284](FreshRSS/FreshRSS#7284), [#&#8203;7285](FreshRSS/FreshRSS#7285), [#&#8203;7347](FreshRSS/FreshRSS#7347)
    -   Update GitHub Actions to Ubuntu 24.04 [#&#8203;7207](FreshRSS/FreshRSS#7207)

### [`v1.25.0`](https://github.com/FreshRSS/FreshRSS/blob/HEAD/CHANGELOG.md#2024-12-23-FreshRSS-1250)

[Compare Source](FreshRSS/FreshRSS@1.24.3...1.25.0)

-   Features
    -   Add support for [regex search (regular expressions)](https://freshrss.github.io/FreshRSS/en/users/10\_filter.html#regex) [#&#8203;6706](FreshRSS/FreshRSS#6706), [#&#8203;6926](FreshRSS/FreshRSS#6926)
        -   ⚠️ Advanced regex syntax for searches depends on the database used (SQLite, PostgreSQL, MariaDB, MySQL),
            but FreshRSS filter actions such as auto-mark-as-read and auto-favourite always use [PHP PCRE2 syntax](https://php.net/regexp.introduction).
    -   Allow dynamic search operator in user queries, like `search:UserQueryA date:P1d` [#&#8203;6851](FreshRSS/FreshRSS#6851)
    -   New feed mode *HTML+XPath+JSON dot notation* (JSON in HTML) [#&#8203;6888](FreshRSS/FreshRSS#6888)
    -   Better HTTP compliance with support for HTTP response headers `Cache-Control: max-age` and `Expires` [#&#8203;6812](FreshRSS/FreshRSS#6812), [FreshRSS/simplepie#26](FreshRSS/simplepie#26)
    -   Support custom HTTP request headers per feed (e.g. for `Authorization`) [#&#8203;6820](FreshRSS/FreshRSS#6820)
    -   New unicity policies and heuristic for feeds with bad article IDs [#&#8203;4487](FreshRSS/FreshRSS#4487), [#&#8203;6900](FreshRSS/FreshRSS#6900)
    -   Fallback to GUID if article link is empty [#&#8203;7051](FreshRSS/FreshRSS#7051)
    -   New option to automatically mark new articles as read if an identical title already exists in the same category [#&#8203;6922](FreshRSS/FreshRSS#6922)
    -   New reading view option to display unread articles + favourites [#&#8203;7088](FreshRSS/FreshRSS#7088)
        -   And corresponding new filter state `&state=96` (no UI button yet)
    -   Add ability to remove content from articles with CSS selectors, also when not using full content [#&#8203;6786](FreshRSS/FreshRSS#6786), [#&#8203;6807](FreshRSS/FreshRSS#6807)
    -   Update `phpgt/cssxpath` library with improved CSS selectors [#&#8203;6618](FreshRSS/FreshRSS#6618)
        -   Support for `:last-child`, `:first-of-type`, `:last-of-type`, `^=`, `|=`
    -   New condition option to selectively retrieve full content of articles
        [#&#8203;33fd07f6f26310d4806077cc87bcdf9b8b940e35](FreshRSS/FreshRSS@33fd07f), [#&#8203;7082](FreshRSS/FreshRSS#7082)
    -   Allow parentheses in quoted search [#&#8203;7055](FreshRSS/FreshRSS#7055)
    -   New UI feature to download a user’ SQLite database or a database SQLite export (to be produced by CLI) [#&#8203;6931](FreshRSS/FreshRSS#6931)
    -   New button to delete errored feeds from a category [#&#8203;7030](FreshRSS/FreshRSS#7030)
    -   Better import of Inoreader user labels [#&#8203;6791](FreshRSS/FreshRSS#6791)
    -   Rebuild feed favicon on cache clear [#&#8203;6961](FreshRSS/FreshRSS#6961)
    -   New sharing with Bluesky [#&#8203;7116](FreshRSS/FreshRSS#7116)
    -   New sharing with Telegram [#&#8203;6838](FreshRSS/FreshRSS#6838)
-   Bug fixing
    -   Fix searches with a parenthesis before an operator like `("a b")` or `(!c)` [#&#8203;6818](FreshRSS/FreshRSS#6818)
    -   Fix auto-read tags [#&#8203;6790](FreshRSS/FreshRSS#6790)
    -   Fix CSS selector for removing elements [#&#8203;7037](FreshRSS/FreshRSS#7037), [#&#8203;7073](FreshRSS/FreshRSS#7073),
        [#&#8203;7081](FreshRSS/FreshRSS#7081), [#&#8203;7091](FreshRSS/FreshRSS#7091), [#&#8203;7083](FreshRSS/FreshRSS#7083)
    -   Fix redirection error after creating a new user [#&#8203;6995](FreshRSS/FreshRSS#6995)
    -   Fix favicon error in case of wrong URL [#&#8203;6899](FreshRSS/FreshRSS#6899)
    -   Use cURL to fetch extensions list (allows e.g. IPv6) [#&#8203;6767](FreshRSS/FreshRSS#6767)
    -   Fix XML encoding in cURL options [#&#8203;6821](FreshRSS/FreshRSS#6821)
    -   Fix initial UI scroll for some browsers [#&#8203;7059](FreshRSS/FreshRSS#7059)
    -   Fix menu for article tags in some cases [#&#8203;6990](FreshRSS/FreshRSS#6990)
    -   Fix share menu shortcut [#&#8203;6825](FreshRSS/FreshRSS#6825)
    -   Fix HTML regex pattern during install for compatibility with `v` mode [#&#8203;7009](FreshRSS/FreshRSS#7009)
    -   More robust creation of user data folder [#&#8203;7000](FreshRSS/FreshRSS#7000)
-   API
    -   Fix API for categories and labels containing a `+` [#&#8203;7033](FreshRSS/FreshRSS#7033)
        -   Compatibility with FocusReader
    -   Supported by [Capy Reader](https://github.com/jocmp/capyreader) (Android, open source) [capyreader#492](jocmp/capyreader#492)
    -   Improved UI for API [#&#8203;7048](FreshRSS/FreshRSS#7048)
    -   Allow adding multiple feeds to a category via API [#&#8203;7017](FreshRSS/FreshRSS#7017)
    -   API support edit multiple tags [#&#8203;7060](FreshRSS/FreshRSS#7060)
    -   API return all categories also those without any feed [#&#8203;7020](FreshRSS/FreshRSS#7020)
-   Compatibility
    -   Require PHP 8.1+ (drop PHP 7.4) [#&#8203;6711](FreshRSS/FreshRSS#6711)
    -   Improved support of PHP 8.4+ [#&#8203;6618](FreshRSS/FreshRSS#6618), [phpgt/CssXPath#227](phpgt/CssXPath#227),
        [#&#8203;6781](FreshRSS/FreshRSS#6781), [#&#8203;4374](FreshRSS/FreshRSS#4374)
    -   Require PostgreSQL 10+ (drop PostgreSQL 9.5) [#&#8203;6705](FreshRSS/FreshRSS#6705)
    -   Require MariaDB 10.0.5+ (drop MariaDB 5.5) [#&#8203;6706](FreshRSS/FreshRSS#6706)
    -   Require MySQL 8+ (drop MySQL 5.5.3) [#&#8203;6706](FreshRSS/FreshRSS#6706)
-   Deployment
    -   Docker: dev image `freshrss/freshrss:oldest` updated to Alpine 3.16 with PHP 8.1.22 and Apache 2.4.59 [#&#8203;6711](FreshRSS/FreshRSS#6711)
    -   Docker alternative image updated to Alpine 3.21 with PHP 8.3.14 and Apache 2.4.62 [#&#8203;5383](FreshRSS/FreshRSS#5383)
    -   Update Dockerfiles to newer key-value format [#&#8203;6819](FreshRSS/FreshRSS#6819)
    -   Docker minor improvement of entrypoint [#&#8203;6827](FreshRSS/FreshRSS#6827)
-   SimplePie
    -   Refactor [our embedding](lib/README.md) of SimplePie [#&#8203;4374](FreshRSS/FreshRSS#4374)
        -   Our fork is maintained in its [own repository](https://github.com/FreshRSS/simplepie/tree/freshrss).
    -   Remove HTTP `Referer` [#&#8203;6822](FreshRSS/FreshRSS#6822), [FreshRSS/simplepie#27](FreshRSS/simplepie#27)
        -   If some sites require it, add `Referer: https://example.net/` to the custom HTTP headers of the feed [#&#8203;6820](FreshRSS/FreshRSS#6820)
    -   Upstream fixes [simplepie#878](simplepie/simplepie#878), [simplepie#883](simplepie/simplepie#883)
    -   Sync upstream [#&#8203;6840](FreshRSS/FreshRSS#6840), [#&#8203;7067](FreshRSS/FreshRSS#7067)
-   Security
    -   Apache protect more non-public folders and files [#&#8203;6881](FreshRSS/FreshRSS#6881), [#&#8203;6893](FreshRSS/FreshRSS#6893), [#&#8203;7008](FreshRSS/FreshRSS#7008)
    -   Add privacy settings on extension list retrieval [#&#8203;4603](FreshRSS/FreshRSS#4603), [#&#8203;7132](FreshRSS/FreshRSS#7132)
    -   Fix login in unsafe mode when using a password with special XML characters [#&#8203;6797](FreshRSS/FreshRSS#6797)
    -   Fix login in e.g. Brave browser by avoiding synchronous XHR [#&#8203;7023](FreshRSS/FreshRSS#7023)
    -   Fix invalid login message [#&#8203;7066](FreshRSS/FreshRSS#7066)
    -   Modernise `windows.open noopener` (to avoid flash of white page in dark mode) [#&#8203;7077](FreshRSS/FreshRSS#7077), [#&#8203;7089](FreshRSS/FreshRSS#7089)
-   UI
    -   Searchable *My Labels* field [#&#8203;6753](FreshRSS/FreshRSS#6753)
    -   Add subscription management button to reading view [#&#8203;6946](FreshRSS/FreshRSS#6946)
    -   New option for showing label menu in article row [#&#8203;6984](FreshRSS/FreshRSS#6984)
    -   Move to next unread label on mark as read [#&#8203;6886](FreshRSS/FreshRSS#6886)
    -   Improved article footer for small / mobile screens [#&#8203;7031](FreshRSS/FreshRSS#7031)
    -   Improve Web accessibility: fix `aria-hidden` bug, and use HTML5 `hidden` [#&#8203;6910](FreshRSS/FreshRSS#6910)
    -   Default styles for `<pre>` and `<code>` [#&#8203;6770](FreshRSS/FreshRSS#6770)
    -   Refactor the sharing menu to use a `<template>` instead of duplicated HTML code [#&#8203;6751](FreshRSS/FreshRSS#6751), [#&#8203;7113](FreshRSS/FreshRSS#7113)
    -   Refactor the label menu to use a `<template>` [#&#8203;6864](FreshRSS/FreshRSS#6864)
    -   Rework UI for authors [#&#8203;7054](FreshRSS/FreshRSS#7054)
        -   Avoid Unicode escape of authors in HTML UI [#&#8203;7056](FreshRSS/FreshRSS#7056)
    -   Improved subscription management page [#&#8203;6816](FreshRSS/FreshRSS#6816)
    -   Improve user query management page [#&#8203;7062](FreshRSS/FreshRSS#7062)
    -   Restore JavaScript form validation compatibility with Web browsers using older engines (SeaMonkey) [#&#8203;6777](FreshRSS/FreshRSS#6777)
    -   Reorganise some options [#&#8203;6920](FreshRSS/FreshRSS#6920)
    -   New shortcut `?` to show shortcut page and help [#&#8203;6981](FreshRSS/FreshRSS#6981)
    -   Use of consistent colours in statistics [#&#8203;7090](FreshRSS/FreshRSS#7090)
    -   Various UI and style improvements [#&#8203;6959](FreshRSS/FreshRSS#6959)
-   Extensions
    -   New extension hook `simplepie_after_init` [#&#8203;7007](FreshRSS/FreshRSS#7007)
-   I18n
    -   Add Finnish [#&#8203;6954](FreshRSS/FreshRSS#6954)
    -   Improve English [#&#8203;7049](FreshRSS/FreshRSS#7049), [#&#8203;7053](FreshRSS/FreshRSS#7053)
    -   Improve German [#&#8203;6847](FreshRSS/FreshRSS#6847), [#&#8203;7068](FreshRSS/FreshRSS#7068), [#&#8203;7128](FreshRSS/FreshRSS#7128)
    -   Improve Italian [#&#8203;6872](FreshRSS/FreshRSS#6872), [#&#8203;7069](FreshRSS/FreshRSS#7069), [#&#8203;7086](FreshRSS/FreshRSS#7086)
    -   Improve Spanish [#&#8203;6894](FreshRSS/FreshRSS#6894), [#&#8203;6908](FreshRSS/FreshRSS#6908)
    -   Improve Turkish [#&#8203;6960](FreshRSS/FreshRSS#6960)
-   Misc.
    -   Better cache name for JSON feeds [#&#8203;6768](FreshRSS/FreshRSS#6768)
    -   Fix inversed encoding logic in `Minz_Request::paramArray()` [#&#8203;6800](FreshRSS/FreshRSS#6800)
    -   Pass PHPStan `booleansInConditions` [#&#8203;6793](FreshRSS/FreshRSS#6793)
    -   Rename PHPStan configuration file to `phpstan.dist.neon` to allow custom configuration in `phpstan.neon` [#&#8203;6892](FreshRSS/FreshRSS#6892)
    -   Code improvements [#&#8203;6800](FreshRSS/FreshRSS#6800), [#&#8203;6809](FreshRSS/FreshRSS#6809), [#&#8203;6983](FreshRSS/FreshRSS#6983)
    -   Makefile improvements [#&#8203;6913](FreshRSS/FreshRSS#6913)
    -   Fix PHPCS `ControlSignature` [#&#8203;6896](FreshRSS/FreshRSS#6896)
    -   Update *PHPMailer* [#&#8203;6968](FreshRSS/FreshRSS#6968), [#&#8203;7046](FreshRSS/FreshRSS#7046)
    -   Code updates to PHP 8.1 syntax [#&#8203;6748](FreshRSS/FreshRSS#6748)
    -   Update dev dependencies [#&#8203;6780](FreshRSS/FreshRSS#6780), [#&#8203;6964](FreshRSS/FreshRSS#6964), , [#&#8203;6965](FreshRSS/FreshRSS#6965),
        [#&#8203;6966](FreshRSS/FreshRSS#6966), [#&#8203;6967](FreshRSS/FreshRSS#6967), [#&#8203;6970](FreshRSS/FreshRSS#6970),
        [#&#8203;7042](FreshRSS/FreshRSS#7042), [#&#8203;7043](FreshRSS/FreshRSS#7043), [#&#8203;7044](FreshRSS/FreshRSS#7044),
        [#&#8203;7045](FreshRSS/FreshRSS#7045), [#&#8203;7047](FreshRSS/FreshRSS#7047), [#&#8203;7052](FreshRSS/FreshRSS#7052)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS45MC4zIiwidXBkYXRlZEluVmVyIjoiMzkuMTc2LjQiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbXX0=-->

Reviewed-on: https://gitea.sh4ke.rocks/lickler/freshrss/pulls/29
Co-authored-by: Michael Wittig <michael.wittig@posteo.de>
Co-committed-by: Michael Wittig <michael.wittig@posteo.de>
@mdemoss
Copy link
Contributor

mdemoss commented Feb 25, 2025

Possibly this either complements or obviates #4280 which I requested for my implementation of a random order-by which I called "shuffle."

To implement a random order, do pagination, and not have items "jump around" I think it may be necessary to track which items have been marked read since the view of items was requested. Recording the time when an item is marked read may still be the best way to do that.

@Alkarex
Copy link
Member Author

Alkarex commented Feb 25, 2025

which I requested for my implementation of a random order-by which I called "shuffle."

Quick idea to do that: you can also define a hash function (e.g. md5()) with a salt, apply it to the items ID, and you will have a new, stable, randomized order. If you want to reshuffle, just change the salt.

@mdemoss
Copy link
Contributor

mdemoss commented Feb 25, 2025

In my first implementation of the concept I used a cryptographic hash (sha1) but in the second, current implementation I use a "multiplicative hash."

Should be faster and more portable SQL.

@CRYPTO-GENIK
Copy link

I just wanted to chime in with a huge thank you to everyone that helped put this feature together!
It absolutely should be the default setting to sort by publication date and this is a huge feature to the app so
Again -- Thank you! Thank you!

Let's get persistence of last sort or a default config option added and pushed out to everyone!

Alkarex added a commit to Alkarex/FreshRSS that referenced this pull request May 10, 2025
Case when processing was faster than 1 second.
fix FreshRSS#7412
Regression from FreshRSS#7149
Alkarex added a commit that referenced this pull request May 10, 2025
* Fix newest articles not shown
Case when processing was faster than 1 second.
fix #7412
Regression from #7149

* Simplify uTimeString()
PHPStan has become a bit smarter
@brevilo
Copy link

brevilo commented Jul 1, 2025

Let's get persistence of last sort or a default config option added and pushed out to everyone!

This! Is there a separate issue to track?

Other than that, a huge thank you - as always.

@shadowlocked
Copy link

Is there any existing way to address the default setting in config? For journalists and researchers, anything other than sorting by publication date is not a good default.

@Alkarex
Copy link
Member Author

Alkarex commented Oct 23, 2025

When possible, please either open a new question or use an existing open issue or question.

Is there any existing way to address the default setting in config?

@shadowlocked Yes there is, and more than one way. Since this PR, by changing the corresponding preference in your config file. Alternatively, thanks to the sorting criteria being kept while you navigate between articles / feeds / categories, you can just change once to your favourite sorting criteria. This means that you can even bookmark your FreshRSS by including the sorting criteria in the URL.

See:

For the first approach, check the sort order in your data/users/shadowlocked/config.php on the model of:

'sort' => 'id',

For journalists and researchers, anything other than sorting by publication date is not a good default.

Publication date is highly misunderstood by many users, which is also a reason why it has not been promoted more. Articles may have publication dates far in the past or in the future (or no date at all), either for legitimate reasons or due to source bugs. A consequence is that the sorting view is not stable when using publication date, since new articles will often be inserted in the middle of existing articles, which is quite confusing. This makes it a poor and error-prone default.
The only stable and reliable criteria for sorting by freshness is the received date, which very often will be quite similar to the publication date.

But with that been said, yes, the default sorting criteria can nevertheless already be changed, and there are issues open to contributors for additional UI and options.

@Alkarex
Copy link
Member Author

Alkarex commented Dec 11, 2025

Work in progress:

Tests welcome

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment