Skip to content

[Bug] Slow query from EntryDAO::sqlListWhere() with large amount of entries #8514

@pe1uca

Description

@pe1uca

Describe the bug

When scrolling through the UI the last entries take a long time to load.

To Reproduce

  1. Set up an instance.
  2. Have millions of entries.
  3. Select a feed with few unread entries.
  4. Scroll through the "Normal view" and see a long wait time when fetching the last unread entries of a feed (either on first load, or when scrolling and the spinner at the bottom appears again).

Expected behavior

Not have to wait 1 or 2 minutes to read the last entries of a feed.

FreshRSS version

1.27.2-dev

System information

  • Database version: PostgreSQL 17
  • PHP version: 8.4.16
  • Installation type: Docker
  • Device: PC
  • OS: Fedora 43
  • Browser: Firefox 147.0.3

Additional context

The query generated by this function takes around 1.5 minutes when the entry table has millions of rows.

private function sqlListWhere(string $type = 'a', int $id = 0, int $state = FreshRSS_Entry::STATE_ALL, ?FreshRSS_BooleanSearch $filters = null,
string $id_min = '0', string $id_max = '0', string $sort = 'id', string $order = 'DESC',
string $continuation_id = '0', array $continuation_values = [], int $limit = 1, int $offset = 0,
string $secondary_sort = 'id', string $secondary_sort_order = 'DESC'): array {

I think in my tests it was when used from EntryDAO::listWhereRaw().

Here's the query

SELECT e.id FROM "pe1uca_entry" e INNER JOIN "pe1uca_feed" f ON f.id = e.id_feed WHERE e.id_feed=$1  AND (e.is_read=0) AND e.id <= $2 ORDER BY e.id DESC LIMIT 21

And here's where it's being used

SELECT e0.id, e0.guid, e0.title, e0.author, e0.content, e0.link,
e0.date, e0."lastSeen", e0."lastUserModified", encode(e0.hash, 'hex') AS hash, e0.is_read, e0.is_favorite, e0.id_feed, e0.tags, e0.attributes
FROM "pe1uca_entry" e0 INNER JOIN (SELECT e.id FROM "pe1uca_entry" e INNER JOIN "pe1uca_feed" f ON f.id = e.id_feed WHERE e.id_feed=$1  AND (e.is_read=0) AND e.id <= $2 ORDER BY e.id DESC LIMIT 21) e2 ON e2.id=e0.id ORDER BY e0.id DESC

(I tried this last query by replacing the second query with the direct list of ids, and it took 5ms)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Bug (unconfirmed)issues that could not be reproduced yet

    Type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions