Skip to content

[Android] Add callback for external view synchronization #133990

Description

@XuanTung95

Is there an existing issue for this?

Use case

I've created a plugin for managing external Android views related to #128198
But it is missing the position synchronization feature. When the external view is moving, its position does not match the frame produced by Flutter.

I was able to implement the synchronization feature but it required exposing some callback from the flutter engine.
This ticket is for merging these changes into Flutter.

Proposal

The mechanism:

  1. At the end of the compositing phase, after updating the view position and before rasterization, send the new position and the timestamp (framework build end time) to the native plugin.
  2. At the start of rasterization, send the engine build end time and the current timestamp to the native plugin. (requires adding a new callback into the engine)
  3. Using FlutterImageView to render the UI. It provides Image from ImageReader.
  4. The Image provides getTimestamp() which is the nano time when the image is created. Use it to find the rasterization start time -> find the engine build end time -> find the framework build end time -> find the corresponding position. Update the corresponding position and draw the Image. (requires adding callback and modifying FlutterImageView)

Detail modification:

Engine (PR flutter-team-archive/engine#45430):

  • Add FlutterEngine.addOnFrameTimeListener(), when rasterization starts, it fires the callback that provides buildStartTime, buildEndTime, raster nanoTime for the plugin.
  • Add FlutterImageView.addOnImageAvailableListener which exposes ImageReader.OnImageAvailableListener callback.
  • FlutterImageView delays the Image.close() call until the Image is drawn on screen. Because in practice, calling Image.close() too soon will cause miss synchronization event if the Image and its position are correct.
    The image is closed at the Choreographer.FrameCallback after the nearest View.onDraw.
    Detail: ImageReader.acquireLatestImage() -> first Choreographer.FrameCallback -> View.onDraw() -> second Choreographer.FrameCallback (Image is closed here).
  • Expose the FlutterImageView.getPendingImage()

Result

After:

5699459854376307995.mp4

Before:

6821630647257156189.mp4

Metadata

Metadata

Assignees

Labels

P2Important issues not at the top of the work listc: new featureNothing broken; request for a new capabilityc: proposalA detailed proposal for a change to Flutterengineflutter/engine related. See also e: labels.frameworkflutter/packages/flutter repository. See also f: labels.platform-androidAndroid applications specificallyr: solvedIssue is closed as solvedteam-androidOwned by Android platform team

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions