[Modal Layout Picker] Add analytics metrics for the layout picker by antonis · Pull Request #13097 · wordpress-mobile/WordPress-Android

antonis · 2020-10-09T12:21:42Z

Fixes: wordpress-mobile/gutenberg-mobile#2454

Related PRs:
Depends on: #13085

Description

This PR adds analytics metrics to the layout picker's actions

To test:

An easy way to test for these analytics is to debug the app and filter Logcat to look for the string 🔵 Tracked:

Layout Picker should show when creating a new page from My Site or Site Pages the Modal Layout Picker appears.

Start by navigating to the Modal Layout Picker

Template Create Page

Select a layout
Select Create Page
In the console, you'll see an event like:
🔵 Tracked: editor_session_template_apply, Properties: {"has_unsupported_blocks":"0","template":"about-4","editor":"gutenberg","content_type":"new","session_id":"ba5e7e37-a29e-48a8-bbac-7e34625ee5c0","post_type":"page","blog_type":"wpcom","editor_has_hw_disabled":"0"}

Template Preview

Select a layout
Select Preview
In the console, you'll see an event like:
🔵 Tracked: editor_session_template_preview, Properties: {"has_unsupported_blocks":"0","template":"about-4","editor":"gutenberg","content_type":"new","session_id":"af74365c-abbe-42fc-b969-bafc7c3745b1","post_type":"page","blog_type":"wpcom","editor_has_hw_disabled":"0"}

Template Preview, then Create

Select a layout
Select Preview
In the console, you'll see an event like:
🔵 Tracked: editor_session_template_apply, Properties: {"has_unsupported_blocks":"0","template":"about-4","editor":"gutenberg","content_type":"new","session_id":"ba5e7e37-a29e-48a8-bbac-7e34625ee5c0","post_type":"page","blog_type":"wpcom","editor_has_hw_disabled":"0"}
Repeat selecting back and choosing a new layout as many times as you want
Select Create Page
In the console, you'll see an event like:
🔵 Tracked: editor_session_template_preview, Properties: {"has_unsupported_blocks":"0","template":"about-4","editor":"gutenberg","content_type":"new","session_id":"af74365c-abbe-42fc-b969-bafc7c3745b1","post_type":"page","blog_type":"wpcom","editor_has_hw_disabled":"0"}

Validate Editor Session

Create a new page from a selected layout using either of the paths mentioned above.
Perform some actions in the editor session like publishing, saving as a draft, or switching to the classic editor.
Expect to see editor events like the below containing the template field:
🔵 Tracked: editor_session_start, Properties: {"has_unsupported_blocks":"0","template":"about-4","editor":"gutenberg","content_type":"new","startup_time_ms":7461,"session_id":"ba5e7e37-a29e-48a8-bbac-7e34625ee5c0","post_type":"page","blog_type":"wpcom","editor_has_hw_disabled":"0","unsupported_blocks":[]}

Create a Blank Page

Create a Blank Page
Expect The events mentioned in the last step to not have the template field populated.

PR submission checklist:

I have considered adding unit tests where possible.
I have considered adding accessibility improvements for my changes.
I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

# Conflicts: # WordPress/src/main/java/org/wordpress/android/ui/main/WPMainActivity.java # WordPress/src/main/java/org/wordpress/android/ui/pages/PagesFragment.kt # WordPress/src/main/java/org/wordpress/android/viewmodel/mlp/ModalLayoutPickerViewModel.kt # build.gradle

# Conflicts: # libs/gutenberg-mobile

peril-wordpress-mobile · 2020-10-09T12:21:47Z

You can trigger optional UI/connected tests for these changes by visiting CircleCI here.

peril-wordpress-mobile · 2020-10-09T12:26:40Z

You can test the changes on this Pull Request by downloading the APK here.

# Conflicts: # WordPress/src/test/java/org/wordpress/android/viewmodel/mlp/ModalLayoutPickerViewModelTest.kt

# Conflicts: # WordPress/src/main/java/org/wordpress/android/ui/mlp/ModalLayoutPickerFragment.kt # WordPress/src/main/java/org/wordpress/android/viewmodel/mlp/ModalLayoutPickerViewModel.kt # build.gradle

chipsnyder · 2020-10-14T17:35:21Z

Hey @antonis, This is looking good for the events we're tracking. One thing I noticed that might be worth considering. Since we're reusing the same editor with the preview We're also firing the events editor_opened, editor_session_start, editor_session_end, and editor_closed. We aren't firing these on iOS so I just wanted to raise the question, should these be fired for the preview? I don't know if this would affect any funnels we have that might be looking at how often the editor is opened.

antonis · 2020-10-15T06:49:46Z

Hello @chipsnyder 👋 ,
thank you for reviewing this 🙇

One thing I noticed that might be worth considering. Since we're reusing the same editor with the preview We're also firing the events editor_opened, editor_session_start, editor_session_end, and editor_closed. We aren't firing these on iOS so I just wanted to raise the question, should these be fired for the preview? I don't know if this would affect any funnels we have that might be looking at how often the editor is opened.

I'm not sure either what is the correct approach on this. Implementation wise I can mute the extra logging by passing the mIsPreview flag to the PostEditorAnalyticsSession.

On the other hand leaving this as is resembles the behaviour when the MODAL_LAYOUT_PICKER build flag is off.

MLP ON

2020-10-15 09:33:08.411 17591-17591/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_session_template_preview, Properties: {"has_unsupported_blocks":"0","template":"about-4","editor":"gutenberg","content_type":"new","session_id":"409b1f45-1070-4b23-9036-faf5064eba2b","post_type":"page","blog_type":"wpcom","editor_has_hw_disabled":"0"}
2020-10-15 09:33:08.417 17591-17591/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_post_created, Properties: {"blog_id":174576032,"created_post_source_detail":"no-detail","post_type":"page","created_post_source":"post-list","is_jetpack":false}
2020-10-15 09:33:08.429 17591-17591/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_opened, Properties: {"blog_id":174576032,"has_gutenberg_blocks":false,"post_type":"page","is_jetpack":false,"editor_has_hw_disabled":"0"}
2020-10-15 09:33:15.230 17591-28990/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_session_start, Properties: {"has_unsupported_blocks":"0","editor":"gutenberg","content_type":"new","startup_time_ms":6816,"session_id":"409b1f45-1070-4b23-9036-faf5064eba2b","post_type":"page","blog_type":"wpcom","editor_has_hw_disabled":"0","unsupported_blocks":[]}
2020-10-15 09:33:19.671 17591-17591/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_closed
2020-10-15 09:33:20.238 17591-17591/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_session_end, Properties: {"has_unsupported_blocks":"0","editor":"gutenberg","content_type":"new","session_id":"409b1f45-1070-4b23-9036-faf5064eba2b","post_type":"page","blog_type":"wpcom","editor_has_hw_disabled":"0","outcome":"cancel"}

MLP OFF

2020-10-15 09:36:50.122 29904-29904/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_post_created, Properties: {"blog_id":174576032,"created_post_source_detail":"page-from-my-site","post_type":"page","created_post_source":"post-list","is_jetpack":false}
2020-10-15 09:36:50.137 29904-29904/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_opened, Properties: {"blog_id":174576032,"has_gutenberg_blocks":false,"post_type":"page","is_jetpack":false,"editor_has_hw_disabled":"0"}
2020-10-15 09:36:56.229 29904-30504/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_session_start, Properties: {"has_unsupported_blocks":"0","editor":"gutenberg","content_type":"new","startup_time_ms":6107,"session_id":"4edba602-ce81-4e67-b147-3187ed508422","post_type":"page","blog_type":"wpcom","editor_has_hw_disabled":"0","unsupported_blocks":[]}
2020-10-15 09:37:02.262 29904-30504/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_session_template_preview, Properties: {"has_unsupported_blocks":"0","template":"about","editor":"gutenberg","content_type":"new","session_id":"4edba602-ce81-4e67-b147-3187ed508422","post_type":"page","blog_type":"wpcom","editor_has_hw_disabled":"0"}
2020-10-15 09:37:35.734 29904-29904/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_closed
2020-10-15 09:37:36.257 29904-29904/org.wordpress.android.beta I/WordPress-STATS: 🔵 Tracked: editor_session_end, Properties: {"has_unsupported_blocks":"0","editor":"gutenberg","content_type":"new","session_id":"4edba602-ce81-4e67-b147-3187ed508422","post_type":"page","blog_type":"wpcom","editor_has_hw_disabled":"0","outcome":"cancel"}

cc @Tug

Tug · 2020-10-15T15:54:17Z

On the other hand leaving this as is resembles the behaviour when the MODAL_LAYOUT_PICKER build flag is off.

Seems like a good opportunity to fix event parity between iOS and Android.

chipsnyder · 2020-10-15T16:03:38Z

On the other hand leaving this as is resembles the behaviour when the MODAL_LAYOUT_PICKER build flag is off.

This is the flow on iOS as well in the way it is today. I think this case doesn't fully apply though because the selection of the template technically occurs outside of the editor session.

The big difference here is Android loads the same editor fragment for previews as it does for editing. The difference in analytics though really comes down to describing the different implementations so to me the different analytics wouldn't be a big deal unless we use these for important funnels.

I think what might help us decide would be, do we use the events editor_opened, editor_session_start, editor_session_end, and editor_closed to track engagement at all? Ie If I preview all of the layouts will that affect any funnels? @Tug that might be something you can help us determine.

antonis · 2020-10-16T08:48:13Z

@chipsnyder , @Tug Thinking of this again the best approach is to align with iOS on this. The behaviour when MODAL_LAYOUT_PICKER build flag is off is not relevant anymore.

I fixed this with 0812184

# Conflicts: # WordPress/src/main/java/org/wordpress/android/ui/posts/EditPostActivity.java # WordPress/src/test/java/org/wordpress/android/viewmodel/mlp/ModalLayoutPickerViewModelTest.kt

chipsnyder

All is working well for me. Thanks for making those adjustments!

antonis added 11 commits October 7, 2020 19:49

Requests scaled thumbnail images

c201d76

Merge branch 'issue/2452-Preview' into issue/2445-ScaledThumb

01b7f8a

Shows message after creating a new page

3cfbbd9

Merge branch 'issue/2445-ScaledThumb' into issue/2453-NewPageToast

d0b0245

# Conflicts: # libs/gutenberg-mobile

Removed deprecated tests related to title visibility in landscape mode

85289e2

Merge branch 'issue/2445-ScaledThumb' into issue/2453-NewPageToast

7a1c8b3

Updates tests broken after merge

61e1620

Fixes landscape xml issue

1acae0f

Merge branch 'issue/2445-ScaledThumb' into issue/2453-NewPageToast

88bffa7

Adds analytics for template preview and apply

99427fe

antonis added this to the 16.0 milestone Oct 9, 2020

antonis self-assigned this Oct 9, 2020

Moved analytics to the editor

c24a506

antonis marked this pull request as ready for review October 9, 2020 14:12

antonis requested a review from chipsnyder October 9, 2020 14:12

antonis added 4 commits October 9, 2020 17:14

Removes unused import

cf4b09b

Prevents VM on rotation

6c85c8f

Merge branch 'issue/2445-ScaledThumb' into issue/2453-NewPageToast

94f60e7

Merge branch 'issue/2453-NewPageToast' into issue/2454-MLP_Analytics

4af9c97

# Conflicts: # WordPress/src/test/java/org/wordpress/android/viewmodel/mlp/ModalLayoutPickerViewModelTest.kt

antonis mentioned this pull request Oct 12, 2020

[Modal Layout Picker] Cache the results of the fetched the layouts call #13105

Merged

3 tasks

Sets page title according to the selected template

1c3f941

antonis mentioned this pull request Oct 12, 2020

[Modal Layout Picker] "Preview" button functionality #13022

Merged

3 tasks

antonis added [Type] Enhancement Part of a WIP Feature labels Oct 13, 2020

antonis added 3 commits October 13, 2020 10:22

Merge branch 'issue/2452-Preview' into issue/2445-ScaledThumb

5b75ddd

# Conflicts: # WordPress/src/main/java/org/wordpress/android/ui/mlp/ModalLayoutPickerFragment.kt # WordPress/src/main/java/org/wordpress/android/viewmodel/mlp/ModalLayoutPickerViewModel.kt # build.gradle

Merge branch 'issue/2445-ScaledThumb' into issue/2453-NewPageToast

bc53c64

Merge branch 'issue/2453-NewPageToast' into issue/2454-MLP_Analytics

40528bf

antonis requested review from develric, malinajirka and planarvoid October 13, 2020 10:08

antonis added 5 commits October 16, 2020 10:24

Merge branch 'develop' into issue/2445-ScaledThumb

b7c5a92

Merge branch 'issue/2445-ScaledThumb' into issue/2453-NewPageToast

f2e3a23

Prevents new page message from showing when reopening local drafts

80ba44f

Merge branch 'issue/2453-NewPageToast' into issue/2454-MLP_Analytics

ece4b82

Mutes unneeded analytics in editor preview mode

0812184

antonis requested a review from Tug October 16, 2020 08:50

Base automatically changed from issue/2453-NewPageToast to develop October 16, 2020 11:11

Merge branch 'develop' into issue/2454-MLP_Analytics

8916f73

# Conflicts: # WordPress/src/main/java/org/wordpress/android/ui/posts/EditPostActivity.java # WordPress/src/test/java/org/wordpress/android/viewmodel/mlp/ModalLayoutPickerViewModelTest.kt

chipsnyder approved these changes Oct 16, 2020

View reviewed changes

antonis merged commit 80bdfe9 into develop Oct 16, 2020

antonis deleted the issue/2454-MLP_Analytics branch October 16, 2020 14:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Modal Layout Picker] Add analytics metrics for the layout picker#13097

[Modal Layout Picker] Add analytics metrics for the layout picker#13097
antonis merged 26 commits intodevelopfrom
issue/2454-MLP_Analytics

antonis commented Oct 9, 2020 •

edited

Loading

Uh oh!

peril-wordpress-mobile bot commented Oct 9, 2020 •

edited

Loading

Uh oh!

peril-wordpress-mobile bot commented Oct 9, 2020 •

edited

Loading

Uh oh!

chipsnyder commented Oct 14, 2020

Uh oh!

antonis commented Oct 15, 2020 •

edited

Loading

Uh oh!

Tug commented Oct 15, 2020

Uh oh!

chipsnyder commented Oct 15, 2020

Uh oh!

antonis commented Oct 16, 2020 •

edited

Loading

Uh oh!

chipsnyder left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

antonis commented Oct 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

To test:

Template Create Page

Template Preview

Template Preview, then Create

Validate Editor Session

Create a Blank Page

Uh oh!

peril-wordpress-mobile bot commented Oct 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

peril-wordpress-mobile bot commented Oct 9, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chipsnyder commented Oct 14, 2020

Uh oh!

antonis commented Oct 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Tug commented Oct 15, 2020

Uh oh!

chipsnyder commented Oct 15, 2020

Uh oh!

antonis commented Oct 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chipsnyder left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

antonis commented Oct 9, 2020 •

edited

Loading

peril-wordpress-mobile bot commented Oct 9, 2020 •

edited

Loading

peril-wordpress-mobile bot commented Oct 9, 2020 •

edited

Loading

antonis commented Oct 15, 2020 •

edited

Loading

antonis commented Oct 16, 2020 •

edited

Loading