Performance Regression on macOS #21014
Description
I've discovered a performance regression on MacOS 12.0.1 (21A559).
When initialising a slider with a large maximum value, performance is impacted. Getting something to appear on screen can take several seconds.
Using cv2.createTrackbar, and setting the slider to reflect the number of frames in a video (15,000) I experience very poor playback performance. After profiling, it seems that the creation of the trackbar is expensive. Further, it seems that the entire playback is significantly slower when the trackbar with a large max value is present.
opencv/modules/highgui/src/window_cocoa.mm
Line 945 in 17234f8
Looking at the implementation, a few things spring to mind:
- In my experience, NSSlider encodes position as a double and is often mapped from a 0-1 range.
- In the case where there are a lot of tick marks, a lot of tick marks have to be rendered. Disabling tick marks on MacOS could help with the performance of creating a NSSlider.
- OpenCV sets the slider to only allow tick mark values. This could also be interfering.
- NSSlider allows the manual setting of the number of tick marks.
Possible solutions to this issue:
- Force NSSlider to show no tick marks at all. This is in fact the default behaviour and will probably fix the issue.
- Force the NSSlider to only show a maximum of 100 tick marks. setAllowsTickMarkValuesOnly to NO.
- Map everything into the range 0-1.
Showing tick marks is of questionable value when there maximum value is over about 100 anyway. Snapping to tick marks is a convenient way to ensure that we get an int value back but it is not worth the squeeze here.
// Create slider
CVSlider *slider = [[CVSlider alloc] init];
[[slider name] setStringValue:cvname];
[[slider slider] setMaxValue:max];
[[slider slider] setMinValue:0];
[[slider slider] setNumberOfTickMarks:(max+1)];
[[slider slider] setAllowsTickMarkValuesOnly:YES];
I have a repro case attached. The time to first draw on my MacBook Pro was ~30seconds with a video containing 13252 frames.
The results of my line profiler run:
*** KeyboardInterrupt exception caught in code being profiled.
Timer unit: 1e-06 s
Total time: 48.8547 s
File: /var/folders/mg/jckm172j3rx0v3sl_x74jzc00000gp/T/ipykernel_9212/1458464341.py
Function: start at line 24
Line # Hits Time Per Hit % Time Line Contents
==============================================================
24 def start(self):
25 1 7223.0 7223.0 0.0 cv2.namedWindow(self.window_name)
26 1 6703565.0 6703565.0 13.7 cv2.createTrackbar(self.position_trackbar, self.window_name, 0, self.frame_count, self.seek_video)
27 1 3.0 3.0 0.0 self.play = True
28 while True:
29 21 45.0 2.1 0.0 if self.play:
30 21 16112.0 767.2 0.0 self.ret, self.frame = self.video.read()
31 21 233.0 11.1 0.0 self.pos = max(int(self.video.get(cv2.CAP_PROP_POS_FRAMES))-1, 0)
32 21 17190.0 818.6 0.0 cv2.setTrackbarPos(self.position_trackbar, self.window_name, self.pos)
33
34
35 21 56.0 2.7 0.0 if self.frame is not None:
36 21 14361256.0 683869.3 29.4 cv2.imshow(self.window_name, self.frame)
37 21 27748684.0 1321365.9 56.8 k = cv2.waitKey(30)
38
39 # if space is pressed, pause
40 20 144.0 7.2 0.0 if k == ord(' '):
41 self.play = not self.play
42 # if j is pressed, step back
43 20 31.0 1.6 0.0 elif k == ord('j'):
44 self.seek_video(max(self.pos - 1, 0), set_trackbar_pos=True)
45 # if k is pressed, step forward
46 20 21.0 1.1 0.0 elif k == ord('k'):
47 # cv2.setTrackbarPos(self.position_trackbar, self.window_name, min(self.pos + 1, self.frame_count))
48 self.seek_video(min(self.pos + 1, self.frame_count), set_trackbar_pos=True)
49 20 24.0 1.2 0.0 elif k == ord('J'):
50 self.seek_video(max(self.pos - 10, 0), set_trackbar_pos=True)
51 # if k is pressed, step forward
52 20 21.0 1.1 0.0 elif k == ord('K'):
53 self.seek_video(min(self.pos + 10, self.frame_count), set_trackbar_pos=True)
54 # if s is pressed, send the unlabelled frame to labelstudio
55 20 23.0 1.1 0.0 elif k == ord('s'):
56 register_video_frame_for_labelling(self.video_path, self.pos, set_trackbar_pos=True)
57 20 24.0 1.2 0.0 elif k == 27 or k == ord('q'):
58 break
59
60 mark_video_done(self.video_path)
61 cv2.destroyWindow(self.window_name)
62 cv2.waitKey(1)
63 self.video.release()