Skip to content
This repository was archived by the owner on Feb 25, 2025. It is now read-only.

Conversation

@nturgut
Copy link
Contributor

@nturgut nturgut commented Aug 5, 2020

Saves the information on the form. So far TextFields which have AutofillHints were autofilled by the browser, however new information was not saved. This PR adds the part to save the information.

Fixes: flutter/flutter#59378

This PR is manually tested extensively on different browsers. Some other issues found on the way are also fixed:

  • There was an unnecessary append line for Firefox, which added the element to the twice.
  • There was a flickering on the autofill elements for Safari.
  • username/password was not autosuggested in Safari since id was not password. Made all the ids equal to autofill hints. (This didn't negatively effected other tested browsers)

Comment on lines 1266 to 1318
while (html.document.getElementsByTagName('form').length > 0) {
html.document.getElementsByTagName('form').last.remove();
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is too aggressive. It'll remove any form elements on the page, including ones that were created by someone else.

Is there a reason we are not using forms in the map formsOnTheDom?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it. I initially tried to avoid iterating over the map.

When I iterate over the maps, I need to use a cast like:
final html.FormElement form = formsOnTheDom[formsOnTheDom.entries.last.key] as html.FormElement;

Otherwise I get a non-nullability error such as A value of type 'T?' can't be assigned to a variable of type 'T'.

@nturgut nturgut requested a review from mdebbar August 6, 2020 01:06
Copy link
Contributor

@mdebbar mdebbar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for working on this! Is this the last piece to complete the autofill feature?


// Form elements are always sent with the same order from Framework
// therefore no sorting necessary for the ids.
final StringBuffer ids = StringBuffer();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if it's safe to make this assumption. The developer may add/remove items, or reorder the group, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reordering should be easy to address with a sort algorithm. Thanks for the suggestion. I'll address it in the next commit.

However, add/remove cases will always cause problem unless Framework sends some additional signals to the engine. This can only be addressed on the Framework level.

Let me give an example solution algorithm and 3 examples:
(1) A possible solution: we can match 2 forms as equal if there are a shared elements.
Previous form: address1 address2 (saved on the DOM due to a previous user edit)
New form: address1 address2 tel email
At this point, when a new set_client platform message arrives to the engine, we can identify these forms as equal, first remove the one on the DOM, then add a new form to the DOM. So looks like this solution can handle some cases, but not all.

(2) This case will give an error even with the algorithm on 1.
If user fills a formA: address1 address2 tel email , later it's split into 2 parts on the framework such as formB: address1 address2 formC: tel email
If the user clicks on address1 to edit, this will trigger a set_client message. According to the algorithm in (1) the engine will remove FormA and will only add info for FormB to the DOM. Information on FormC (Tel email) is no longer attached to the DOM. But visible on the framework. User hits submit button, but browser only saves some part of the info.

(3) This case will give an error even with the algorithm on 1.
If user fills a formA: address1 address2 tel email , later framework code removes tel email.
If the user never clicks any form elements, no platform messages are send to inform the engine of the change, all four elements are still on the DOM. Only 2 of them are in the framework thus 2 elements are visible to the user. User hits submit button, but browser saves more information than necessary.

There might be multiple ways to address this in the framework:

  • Every-time the autofill group elements change (addition/removal), framework can send a finalize message with save=false, therefore.
  • Autofill finalize call, also sends a list of ids of the elements still on the page. Engine creates a new form with the information visible to the user on the time they triggered a finalize event. Only information still on the page is saved to the browser.

I assume other engine's such as desktop which can show larger number of form elements will experience similar issues in the long run, therefore it is better to open an issue and record all the possible problems that can arise from adding/removing elements to an autofill group dynamically.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the check for reordered element.

@nturgut
Copy link
Contributor Author

nturgut commented Aug 7, 2020

LGTM. Thanks for working on this! Is this the last piece to complete the autofill feature?

I would love to say yes, the answer is more complicated :) This will be the final big change. The more manual tests I do the more bugs I discover though. There are two main areas.
Differences in browsers: since most of the work depends on the browser for autofill, each browsers have different way of handling things. This goes as far as using different autofill hints:

Issues in semantics: I'll start on these next, since I think keeping a11y working is critical:

I'm doing my manual tests in this matrix. I'm filing issues as they surface.

Browser User login info Payment info Address info
Firefox Desktop      
Chrome Desktop      
Chrome Android      
Safari Desktop      
Safari IOS      

Then I test a11y with VoiceOver on Mac.

engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Aug 8, 2020
// https://github.com/flutter/flutter/issues/59378
final bool saveForm = call.arguments as bool;
// Close the text editing connection. Form is finalizing.
implementation.sendTextConnectionClosedToFrameworkIfAny();
Copy link
Member

@xu-baolin xu-baolin Dec 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why close the connection here?

The framework AutofillGroup disposal will send 'TextInput.finishAutofillContext' message, But dispose is an asynchronous process, and the wrong client ID may be closed here.

  @override
  void dispose() {
    super.dispose();

    if (!_isTopmostAutofillGroup || widget.onDisposeAction == null)
      return;
    switch (widget.onDisposeAction) {
      case AutofillContextAction.cancel:
        TextInput.finishAutofillContext(shouldSave: false);
        break;
      case AutofillContextAction.commit:
        TextInput.finishAutofillContext(shouldSave: true);
        break;
    }
  }

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue reported at flutter/flutter#71476

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[web] Save the autofill information in the browser

5 participants