Help with a simple multitrack player in Python + PySide6 #1032

jonata · 2024-10-03T21:25:21Z

jonata
Oct 3, 2024

Hi! I'm experimenting with MLT for a side project, and I just want to create a multitrack player, that is, I want to basically play a video file in sync with an audio file. I tried the following basic code, with no success. I just want to make sure that's not something on my system and that's really something I am missing:

from PySide6.QtWidgets import QApplication, QWidget, QVBoxLayout, QPushButton
import mlt7 as mlt
import os


class VideoPlayer(QWidget):
    def __init__(self, video_file: str, audio_file: str):
        super().__init__()
        self.setWindowTitle("MLT Video Player")
        self.setGeometry(100, 100, 800, 600)

        mlt.Factory().init()

        profile = mlt.Profile()

        self.playlist = mlt.Playlist(profile)
        self.video_producer = mlt.Producer(profile, video_file)
        self.audio_producer = mlt.Producer(profile, audio_file)
        
        self.playlist.append(self.video_producer)
        self.playlist.append(self.audio_producer)

        self.tractor = mlt.Tractor()
        self.tractor.preview = True
        self.tractor.set_track(self.playlist, 0)

        self.consumer = mlt.Consumer(profile, "sdl2")
        self.consumer.connect(self.tractor)
        
        layout = QVBoxLayout()
        self.main_player = QWidget()
        layout.addWidget(self.main_player, 1)

        self.play_button = QPushButton("Play")
        self.pause_button = QPushButton("Pause")
        self.stop_button = QPushButton("Stop")
        
        self.play_button.clicked.connect(self.play_video)
        self.pause_button.clicked.connect(self.pause_video)
        self.stop_button.clicked.connect(self.stop_video)

        layout.addWidget(self.play_button)
        layout.addWidget(self.pause_button)
        layout.addWidget(self.stop_button)
        
        self.setLayout(layout)
        
        os.putenv('SDL_WINDOWID', str(self.main_player.winId()))

        self.tractor.set_speed(1)
        self.consumer.start()

    def play_video(self):
        self.tractor.set_speed(1)
        
    def pause_video(self):
        if not self.consumer.is_stopped():
            self.tractor.pause()
        
    def stop_video(self):
        if not self.consumer.is_stopped():
            self.consumer.stop()
        
    def closeEvent(self, event):
        self.consumer.stop()
        mlt.Factory().close()
        event.accept()

if __name__ == "__main__":
    import sys

    app = QApplication(sys.argv)

    video_file = "video.mp4"
    audio_file = "audio.mp3"
    
    player = VideoPlayer(video_file, audio_file)
    player.show()
    
    sys.exit(app.exec())

It keep crashing ("Floating point exception"), no matter what I do. If I just add the audio file, the window open, but wont play anything. Otherwise, it just crash.

Versions: Ubuntu 24.04.1, Python 3.10, mlt 7.22. I compiled the latest version 7.28, but still crashing.

Thanks for your attention!

ddennedy · 2024-10-03T21:51:38Z

ddennedy
Oct 3, 2024
Maintainer

SDL_WINDOWID will not embed the sdl2 consumer into the window. That only works with the very old SDL 1.2. I do not know if that is your problem, but it could be. Just because SDL2 might still have something in this regard, but that does not mean it will work with X, Wayland, or Qt. I do not know if this is your only problem. If you want to embed the video, you have a big job ahead of you if you want it to also be efficient.
What if you comment out os.putenv('SDL_WINDOWID', str(self.main_player.winId()))?
If you want to continue I suggest to start with src/swig/python/play.rb example and slowly make changes.

0 replies

jonata · 2024-10-04T01:24:45Z

jonata
Oct 4, 2024
Author

I would eventually try to use GL texture (qglsl?), I already use this technique with mpv.

Commenting the line don't fix it.

play.py returns a Segmentation Fault. 😐

0 replies

ddennedy · 2024-10-05T18:21:53Z

ddennedy
Oct 5, 2024
Maintainer

play.py uses the sdl consumer. Do you have it, or did you change it to sdl2. I do not use python much, but it worked for me after changing it to sdl2 since I do not build sdl.

I just updated the play.py example because it has some bad examples of error handling. It is not sufficient to test if p due to the nature of our C++ wrapper based on our reference counting in C. Rather, you need to use if p.is_valid(). I also added our concept of "auto-profile" to the example.

Then, I extended the play example to use a playlist and multitrack tractor as you did, and it worked

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

# Import required modules
from __future__ import print_function
import mlt7 as mlt
import time
import sys

# Start the mlt system
mlt.Factory().init()

# Establish a default (usually "dv_pal") profile
profile = mlt.Profile()

# Create the producer
p = mlt.Producer(profile, sys.argv[1])

if p.is_valid():
	# Derive a profile based on the producer
	profile.from_producer(p)

	# Reload the producer using the derived profile
	p = mlt.Producer(profile, sys.argv[1])

	# Create the consumer
	c = mlt.Consumer(profile, "sdl2")

	if not c.is_valid():
		print("Failed to open the sdl2 consumer")
		sys.exit(1)

	# Setup a multitrack
	playlist = mlt.Playlist(profile)
	playlist.append(p)
	tractor = mlt.Tractor()
	tractor.set_track(playlist, 0)

	# Connect the producer to the consumer
	c.connect(tractor)
	
	# Start the consumer
	c.start()
	
	# Wait until the user stops the consumer
	while not c.is_stopped():
		time.sleep(1)
else:
	# Diagnostics
	print("Unable to open ", sys.argv[1])
	sys.exit(1)

If you still have trouble you need to validate your build and install of MLT using melt. For example melt -query consumers tells you which consumers you have. Can you play a video with melt? I discourage trying to get new development to work from a custom build if you are new to this project due to numerous dependencies and install-sensitive (libs and data loaded at runtime are location sensitive).

0 replies

jonata · 2024-10-08T05:33:07Z

jonata
Oct 8, 2024
Author

Thank you very much! It worked.

Then I tried to embed the SDL2 into a QWidget with no success, but I made a PR (#1033) that would enable it. We need to set it as in c.set("window_id", WINDOW_ID).

Thanks again!

0 replies

ddennedy · 2024-10-08T05:35:30Z

ddennedy
Oct 8, 2024
Maintainer

Ok, cool. And you confirm that embedding with Qt worked with your patch?

1 reply

jonata Oct 8, 2024
Author

Yes!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Help with a simple multitrack player in Python + PySide6 #1032

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 5 comments 1 reply

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Help with a simple multitrack player in Python + PySide6 #1032

Uh oh!

Uh oh!

jonata Oct 3, 2024

Replies: 5 comments · 1 reply

Uh oh!

ddennedy Oct 3, 2024 Maintainer

Uh oh!

Uh oh!

jonata Oct 4, 2024 Author

Uh oh!

ddennedy Oct 5, 2024 Maintainer

Uh oh!

jonata Oct 8, 2024 Author

Uh oh!

ddennedy Oct 8, 2024 Maintainer

Uh oh!

jonata Oct 8, 2024 Author

jonata
Oct 3, 2024

Replies: 5 comments 1 reply

ddennedy
Oct 3, 2024
Maintainer

jonata
Oct 4, 2024
Author

ddennedy
Oct 5, 2024
Maintainer

jonata
Oct 8, 2024
Author

ddennedy
Oct 8, 2024
Maintainer

jonata Oct 8, 2024
Author