Skip to content

EN: Improved movie playback performance for MovieStim#7264

Merged
mdcutone merged 27 commits intopsychopy:devfrom
mdcutone:dev-en-movie-readers
Jun 11, 2025
Merged

EN: Improved movie playback performance for MovieStim#7264
mdcutone merged 27 commits intopsychopy:devfrom
mdcutone:dev-en-movie-readers

Conversation

@mdcutone
Copy link
Copy Markdown
Member

PR which improves movie playback performance when using the MovieStim class and movie component in Builder. Playback will be more responsive and efficient with new frame acquisition and memory management algorithms

Additional features are in development which will permit: high-precision audio-visual synchronization and onset timing, improved memory efficiency for long movies, caching for instant seeking, on-screen playback controls, and visual indicators for seeking and frame drops

The old movie player backend classes have been removed

@codecov
Copy link
Copy Markdown

codecov bot commented May 23, 2025

Codecov Report

Attention: Patch coverage is 0% with 515 lines in your changes missing coverage. Please review.

Project coverage is 12.24%. Comparing base (ce189c5) to head (86994e1).
Report is 8 commits behind head on dev.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #7264      +/-   ##
==========================================
+ Coverage   12.21%   12.24%   +0.03%     
==========================================
  Files         353      350       -3     
  Lines       64680    64505     -175     
==========================================
+ Hits         7899     7900       +1     
+ Misses      56781    56605     -176     
Components Coverage Δ
app ∅ <ø> (∅)
boilerplate ∅ <ø> (∅)
library ∅ <ø> (∅)
vm-safe library ∅ <ø> (∅)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mdcutone mdcutone marked this pull request as ready for review June 11, 2025 13:09
@mdcutone mdcutone merged commit 447725a into psychopy:dev Jun 11, 2025
7 checks passed
@mscheltienne
Copy link
Copy Markdown
Contributor

FYI @mdcutone after a thorough claude-assited debugging of why are our videos not displaying with the MovieStim, it got tracked down to the changes in this PR. psychopy==2025.1.1 works. dev and 2026.1.1 are still broken since that PR. I'll put down Claude's analyis down here.

PsychoPy MovieStim regression

Summary

PsychoPy PR #7264 ("EN: Improved movie playback performance for MovieStim"), merged on 2025-06-11 (commit 447725a15), introduced a regression in MovieStim that causes videos to render as black squares for their entire duration. The bug is present in all releases from 2025.2.0 onwards, including the latest 2026.1.1. The last working release is 2025.1.1.

Affected versions

Version Status Notes
2025.1.1 Working Last version before the breaking change
2025.2.0 Broken First version containing PR #7264
2025.2.1 Broken Version pinned by our experiments
2026.1.1 Broken Latest release as of 2026-02-27

What changed

PR #7264 replaced the old threaded MovieStreamThreadFFPyPlayer architecture with a new synchronous MovieFileReader class. The critical difference is how each handles the ffpyplayer decoder's startup latency after a seek.

Old code (2025.1.1 — working)

The threaded player's seekTo() method retries get_frame() with sleeps between attempts, giving the decoder time to produce frames:

# psychopy/visual/movies/players/ffpyplayer_player.py — seekTo()
n = 0
while n < maxAttempts:  # up to 16 attempts
    frameData_, val_ = player.get_frame(show=True)
    if frameData_ is None:
        time.sleep(0.0025)  # wait 2.5ms and retry
        n += 1
        continue
    # ... check PTS convergence ...

New code (2025.2.0+ — broken)

The new MovieFileReader._openFFPyPlayer() correctly warms up the decoder and gets a first frame, but then seeks back to 0.0 and discards that frame:

# psychopy/visual/movies/__init__.py — _openFFPyPlayer()

# warmup — this works fine
while time.time() - startTime < defaultTimeout:
    frame, _ = self._player.get_frame()
    if frame != None:
        break

# go back to first frame — this is where the problem starts
self._player.set_pause(True)
self._player.seek(0.0, relative=False)

After the seek, the decoder needs ~10ms to produce the first frame. But _getFrameFFPyPlayer() calls get_frame() once, gets None (decoder not ready yet), and immediately gives up:

# psychopy/visual/movies/__init__.py — _getFrameFFPyPlayer()
while 1:
    frame, status = self._player.get_frame()
    # ...
    if frame is None:
        break  # <-- gives up immediately, no retry, no wait

Because no frame is ever stored, updateVideoFrame() returns False on every call, and the texture is never updated. The GPU texture remains black (its initial state) for the entire video duration.

How to reproduce

The bug manifests when using MovieStim with the ffpyplayer backend in a trial-based experiment where videos are loaded with setMovie() between trials. Conditions:

  • movieLib="ffpyplayer"
  • MPEG-4 Part 2 codec (all our stimulus videos; H.264 may behave differently)
  • setMovie() called between trials (creates a new MovieFileReader each time)
  • Non-trivial video resolution and framerate (our stimuli: 512x512, 120fps)

Observed behavior

  • The MovieStim reports status=STARTED and _playbackStatus=PLAYING
  • _movieTime advances correctly (PsychoPy's clock is running)
  • But _pts stays at 0.0 and _recentFrame is always None
  • updateVideoFrame() returns False on every frame
  • The display shows a black square for the entire 2-second video duration
  • The next trial starts, and the same thing happens again (intermittently — some trials
    may work if the decoder happens to be fast enough)

Diagnostic data

With psychopy 2025.2.1, monkey-patching updateVideoFrame() to track its return value shows 100% texture update failure across all trials:

Trial   1: vframes=120 tex_ok=  0 tex_stale=120 has_frame=  0 | mt=[0.00-1.98] pts=[0.00-0.00]
Trial   2: vframes=120 tex_ok=  0 tex_stale=120 has_frame=  0 | mt=[0.00-1.98] pts=[0.00-0.00]
...

Fix suggestion

The _getFrameFFPyPlayer() method needs to either:

  1. Retry with a short sleep when get_frame() returns None, similar to the old seekTo() approach — e.g., sleep 2.5ms and retry up to N times.
  2. Not discard the warm-up frame in _openFFPyPlayer() — store it in _frameStore so it's available immediately (partially attempted in commit 20ce06aae for 2026.1.1 but insufficient since the seek still invalidates the decoder state).
  3. Not seek after warmup — if the warm-up frame is at or near PTS 0.0, there's no need to seek back.

Workaround

Pin psychopy to 2025.1.1:

psychopy==2025.1.1

This version uses the old threaded MovieStreamThreadFFPyPlayer with proper retry logic and does not exhibit the bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants