Next step
Per this week's offline discussion, assigning this task over to @MNeisler who is going to:

• 1. Take first pass at drafting the measurement plan for evaluating the impact of the work we are doing in T387175
• 2. Once a first draft of the measurement plan is in a good place, learn - through a conversation with the Experiment Platform Team – which (if any) aspects of the measurement approach we're proposing is not compatible with what Test Kitchen currently offers

In T409112#11355415, @ppelberg wrote:

Next step
Per this week's offline discussion, assigning this task over to @MNeisler who is going to:

• 1. Take first pass at drafting the measurement plan for evaluating the impact of the work we are doing in T387175
• 2. Once a first draft of the measurement plan is in a good place, learn - through a conversation with the Experiment Platform Team – which (if any) aspects of the measurement approach we're proposing is not compatible with what Test Kitchen currently offers

"1." is done (see: measurement plan) and "2" is in progress

Next steps

• @MNeisler to continue working on "2."
• @ppelberg to update this task description with what Megan proposed, and we converged on, in the measurement plan.

Tech News draft (WIP):

Later this week, some [Wikipedia? all?] editors using the mobile visualeditor who are editing a section in Visual editor will be shown a button that can expand their editing session to the whole page. This A/B test should enable editors to more easily access the rest of the article mid-edit and lower the edit-abandonment rate. [1]

In T409112#11410589, @Quiddity wrote...

Thank you, Nick. Some edits reflected in the draft below...

v2

Later this week, a controlled experiment affecting editors [100 Wikipedias] who are editing a section in the mobile Visual editor will begin. The experiment will last ~4 weeks. 50% of the editors in the experiment will not notice any difference in their experience and 50% of editors will notice a new button that will enable them to expand their editing session to the whole page. This interventions is meant to make it easier for people on mobile web to edit any article section, regardless of which edit icon they first tap. Please visit T409112 to see more details about the metrics we will be using to evaluate the impact of this intervention. [1]

Next step
@ppelberg: update task description to include contents of measurement plan

Tech News draft slightly adjusted/simplified, and added to Description. It will be ready for announcement in the 8th December edition.
We still need to get the correct link for the "100 Wikipedias", and potentially find a place onwiki to document the screenshot and a sentence explaining it. [I'll work on those aspects]

In T409112#11411611, @Quiddity wrote:

Tech News draft slightly adjusted/simplified, and added to Description. It will be ready for announcement in the 8th December edition.
We still need to get the correct link for the "100 Wikipedias", and potentially find a place onwiki to document the screenshot and a sentence explaining it. [I'll work on those aspects]

Wonderful. All that you described sounds great. Thank you, Nick.

Per offline discussions with @MNeisler, we've settled on targeting a 1% decrease in edit abandonment rate. I've updated the task description in 409112#11449802 to reflect this.

Note: in parallel, we're going to investigate the feasibility of further limiting the population of people we examine through this experiment to people who we assume to be arriving with an edit in mind they are intent on making. This investigation will happen in T412318.

Findings (high-level)

• While the feature seems to create some friction for registered newcomers, it appears to be effective for both more experienced Junior Contributors (≤100 cumulative edits) and logged-out users completing constructive edits.
• The most concerning metric is the Section Edit Completion rate of logged-in users, which saw a statistically significant 5.8% decrease. Additionally, the time to first change increased by 13.1% for logged-in users and decreased -14.5% for logged-out users, suggesting the UI addition is slowing down the initial editing momentum for some subset of registered users and accelerating it for folks who are not logged in.

• • Further investigation into section edit completion rate (T415336) indicates that the button is very helpful to users that directly engage with it. Logged-in users who click on the Edit Full Page Button are 55.6% more likely to complete their edit compared to those in the treatment group who don't click the new button.
  • The new edit full page button also proved highly effective for users who started typing, increasing edit completion rates by 8.5% for those who started typing and achieving a 96.8% edit completion rate for those who engaged with the feature.
  • The overall 5.8% decrease in edit completion rate is mostly due to logged-in users in the treatment group who abandon their edits before starting to type. This indicates that the new Edit Full Page button might be:
    • Slightly distracting or
    • Confusing to some newcomers or
    • Deter low-intent editors before they start editing.
• We also confirmed statistically significant improvements in section edit constructive edits rate for logged-out users.

Findings (detailed)

The findings below are drawn from Test Kitchen Experiment Analytics.

* = not statistically significant

Path forward

Based on these findings, we understand this experiment to fall in Scenario 5:

The new {nav Edit] button UX is effective at increasing the likelihood that people publish constructive edits and/or decreasing the rate at which they abandon the edits they start without a meaningful change in any guardrail metrics

In line with the above, we recommend moving forward with deployment to additional Wikipedias.

While we observed a slight decrease in overall completion rate – primarily driven by users abandoning before they begin typing – the feature proved highly effective for high-intent editors, significantly increasing their success rate.

We will continue to monitor the impact on edit completion and explore design iterations, such as adjusting button prominence or placement, to reduce initial distraction for newcomers while preserving the tool's utility for registered newcomers.