Skip to content

perf: parallelize per-item Plex watch history calls #2936

@stormshaker

Description

@stormshaker

Background

The main rule execution bottlenecks can be addressed by bulk-fetching Sonarr/Radarr libraries and caching Plex watchlists/collections (one fetch per resource instead of one per media item). After those fixes, Plex watch history remains the only significant per-item bottleneck with no clean bulk equivalent.

The problem

PlexApiService.getWatchHistory(ratingKey) is called once per media item for properties including lastViewedAt, seenBy, sw_lastWatched, sw_viewedEpisodes, sw_amountOfViews, sw_allEpisodesSeenBy, and sw_watchers.

For a rule processing 50,000 episodes, each needing lastViewedAt, that's 50,000 sequential await calls. At 50ms per round-trip on a local network, that's ~2,500 seconds. The event loop awaits each one before starting the next — fast hardware doesn't help sequential waiting.

There is no Plex bulk endpoint for watch history. /status/sessions/history/all without a filter is unbounded and not suitable for per-item lookup at scale.

Proposed approach

Add a configurable concurrency pool to the rule executor's item processing loop using p-limit or similar. Processing N items at concurrency=10 reduces wall time by ~10× for watch-history-dependent rules without hammering the Plex server.

Key considerations:

  • Default concurrency should be conservative (5–10) to avoid Plex rate limiting
  • Error handling needs care: a single item failure shouldn't abort the whole batch
  • Shared mutable state in the executor loop needs review before parallelising
  • Concurrency limit could be user-configurable in settings

Expected impact

For a 50,000-episode rule relying on lastViewedAt:

  • Sequential (current): ~2,500s
  • Concurrency=10: ~250s
  • Concurrency=20: ~125s

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions