Skip to content

Conversation

@camsim99
Copy link
Contributor

@camsim99 camsim99 commented Feb 23, 2023

Wraps ImageCapture CameraX UseCase and uses it to implement takePicture().

Fixes flutter/flutter#111140.

Pre-launch Checklist

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • I read the Tree Hygiene wiki page, which explains my responsibilities.
  • I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/packages repo does use dart format.)
  • I signed the CLA.
  • The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
  • I listed at least one issue that this PR fixes in the description above.
  • I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
  • I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is test-exempt.
  • All existing and new tests are passing.

private void handleImageFileError(
@NonNull GeneratedCameraXLibrary.Result<String> result, @NonNull String errorDescription) {
// Send empty path because image was not saved.
result.success("");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking particularly for feedback on using an empty path to signify a file error.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the other options? In general I would expect either a typed result with meta data or thrown exception to handle (in java). If there was going to be a string return value and neither of the others then I would expect null not empty.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you. I think I got caught up in the confusion of resulting with an error versus sending a camera error to the Dart stream. It's my understanding now (based on the current plugin) that we should send a camera error to the stream if an error occurs while an operation is ongoing. Otherwise, we can throw the error directly from the native side. I changed it to that here.

Let me know what you think. Also @bparrishMines will this work well with the wrapping pattern? I want to make sure we throw errors consistently from here on out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I agree that this is the ideal way to handle it and is how we handle it for the webview_flutter implementations as well. The native code can just return the exception in result.error and then the Dart platform implementation can handle converting it to a CameraException.

@camsim99 camsim99 marked this pull request as ready for review February 24, 2023 22:05
return new SystemServicesFlutterApiImpl(binaryMessenger);
}

public ImageCapture.Builder createImageCaptureBuilder() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why make this a method instead of just calling ImageCapture.Builder() were you need it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For injecting a mock for testing

ImageCapture.Builder imageCaptureBuilder = cameraXProxy.createImageCaptureBuilder();
if (flashMode != null) {
// This sets the requested flash mode, but may fail silently.
imageCaptureBuilder.setFlashMode(flashMode.intValue());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be an error or warning if a flashMode is passed that is larger than an int? Is there a more typesafe variable we can use here like an enum?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://developer.android.com/reference/androidx/camera/core/ImageCapture#FLASH_MODE_ON()
The values are not large enough to be a long and in your test you are converting from an int to a long. Why use longs at all?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Longs are used in the code generated by pigeon. That's the only reason I'm using them here

private void handleImageFileError(
@NonNull GeneratedCameraXLibrary.Result<String> result, @NonNull String errorDescription) {
// Send empty path because image was not saved.
result.success("");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the other options? In general I would expect either a typed result with meta data or thrown exception to handle (in java). If there was going to be a string return value and neither of the others then I would expect null not empty.

///
/// [cameraId] is not used.
@override
Future<XFile> takePicture(int cameraId) async {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xizhang this is the Dart implementation for taking a picture. Do you have any feedback on binding the ImageCapture use case to take picture, and then, immediately unbinding it, as seen below? Wondering if this is best performance-wise. cc @gmackall

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I mentioned in the sync meeting. We should leave ImageCapture always bound.

I think something like this would work: Preview, ImageAnalysis and ImageCapture are always bound, since the combination is always supported by the camera hardware. VideoCapture is lazy loaded: it's only bound when the user start recording Video.

The reason to special case VideoCapture is that VideoCapture is not always supported by camera hardware. When the device doesn't support it, CameraX supports it with OpenGL which will introduce an additional cost. We should avoid that cost if necessary.


/// Takes a picture and returns the absolute path of where the capture image
/// was saved.
Future<String> takePicture() async {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this implementation of this method, but I have a few notes to add.

This is returning a String despite the output being a OutputFileResult. I'm assuming this is done because it only has one method, getSavedUri. But when a library returns a class with a single method, there is a pretty good chance they intend to add more at a later date and they don't want to introduce a breaking change.

This is also missing the OutputFileOptions.

But I would guess that the tedious process of wrapping native apis contributed to creating a simpler method. And that's still on me to improve it. So, I would suggest including in the documentation that this method doesn't completely wrap the method from ImageCapture and serves as a convenience method that was created for Dart.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't wrap those since the goal right now is to reach feature parity, and it was simpler to configure taking a picture the same as the current plugin on the Java side than wrap something we aren't likely to change soon.

I think you make a good point though. I'll leave a comment, and I'm definitely open to coming back to wrap those classes later if needed. Should be straightforward to change.

camsim99 and others added 2 commits March 3, 2023 08:59
…o/flutter/plugins/camerax/ImageCaptureHostApiImpl.java

Co-authored-by: Maurice Parrish <10687576+bparrishMines@users.noreply.github.com>
Co-authored-by: Maurice Parrish <10687576+bparrishMines@users.noreply.github.com>
camera = await processCameraProvider!
.bindToLifecycle(cameraSelector!, <UseCase>[preview!, imageCapture!]);
_previewIsPaused = false;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reidbaker @gmackall @bparrishMines To address @xizhang's comment, I did some significant refactoring in this method if any of you are willing to take a look.

The main change is that I configured the Preview and ImageCapture use cases here and bind them to the lifecycle together.

assert(cameraSelector != null);
assert(preview != null);

final bool previewIsBound = await processCameraProvider!.isBound(preview!);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reidbaker @gmackall @bparrishMines The other major change I made is not relying on class variables to know if a UseCase has been bound or not. I wrapped isBound to achieve that.

@camsim99
Copy link
Contributor Author

camsim99 commented Mar 8, 2023

Note to self: This needs to land after #3318 to avoid issues internally.

Copy link
Contributor

@bparrishMines bparrishMines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@camsim99 camsim99 added the autosubmit Merge PR when tree becomes green via auto submit App label Mar 22, 2023
@auto-submit auto-submit bot merged commit 19d19b5 into flutter:main Mar 22, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Mar 23, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Mar 23, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Mar 23, 2023
nploi pushed a commit to nploi/packages that referenced this pull request Jul 16, 2023
[camerax] Implement image capture
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autosubmit Merge PR when tree becomes green via auto submit App p: camera platform-android

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[camera] Add ImageCapture use case to CameraX plugin

5 participants