You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've created a plugin for managing external Android views related to #128198
But it is missing the position synchronization feature. When the external view is moving, its position does not match the frame produced by Flutter.
I was able to implement the synchronization feature but it required exposing some callback from the flutter engine.
This ticket is for merging these changes into Flutter.
Proposal
The mechanism:
At the end of the compositing phase, after updating the view position and before rasterization, send the new position and the timestamp (framework build end time) to the native plugin.
At the start of rasterization, send the engine build end time and the current timestamp to the native plugin. (requires adding a new callback into the engine)
Using FlutterImageView to render the UI. It provides Image from ImageReader.
The Image provides getTimestamp() which is the nano time when the image is created. Use it to find the rasterization start time -> find the engine build end time -> find the framework build end time -> find the corresponding position. Update the corresponding position and draw the Image. (requires adding callback and modifying FlutterImageView)
Add FlutterEngine.addOnFrameTimeListener(), when rasterization starts, it fires the callback that provides buildStartTime, buildEndTime, raster nanoTime for the plugin.
Add FlutterImageView.addOnImageAvailableListener which exposes ImageReader.OnImageAvailableListener callback.
FlutterImageView delays the Image.close() call until the Image is drawn on screen. Because in practice, calling Image.close() too soon will cause miss synchronization event if the Image and its position are correct.
The image is closed at the Choreographer.FrameCallback after the nearest View.onDraw.
Detail: ImageReader.acquireLatestImage() -> first Choreographer.FrameCallback -> View.onDraw() -> second Choreographer.FrameCallback (Image is closed here).
Is there an existing issue for this?
Use case
I've created a plugin for managing external Android views related to #128198
But it is missing the position synchronization feature. When the external view is moving, its position does not match the frame produced by Flutter.
I was able to implement the synchronization feature but it required exposing some callback from the flutter engine.
This ticket is for merging these changes into Flutter.
Proposal
The mechanism:
FlutterImageViewto render the UI. It providesImagefromImageReader.ImageprovidesgetTimestamp()which is the nano time when the image is created. Use it to find the rasterization start time -> find the engine build end time -> find the framework build end time -> find the corresponding position. Update the corresponding position and draw theImage. (requires adding callback and modifyingFlutterImageView)Detail modification:
Engine (PR flutter-team-archive/engine#45430):
FlutterEngine.addOnFrameTimeListener(), when rasterization starts, it fires the callback that provides buildStartTime, buildEndTime, raster nanoTime for the plugin.FlutterImageView.addOnImageAvailableListenerwhich exposesImageReader.OnImageAvailableListenercallback.Image.close()call until the Image is drawn on screen. Because in practice, callingImage.close()too soon will cause miss synchronization event if theImageand its position are correct.The image is closed at the
Choreographer.FrameCallbackafter the nearestView.onDraw.Detail:
ImageReader.acquireLatestImage()-> firstChoreographer.FrameCallback->View.onDraw()-> secondChoreographer.FrameCallback(Image is closed here).FlutterImageView.getPendingImage()Result
After:
5699459854376307995.mp4
Before:
6821630647257156189.mp4