Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

SG-61976 - Run a test to remove steps from the new user signup flow#62413

Merged
gitstart-sourcegraph merged 2 commits into
mainfrom
contractors/SG-61976
May 21, 2024
Merged

SG-61976 - Run a test to remove steps from the new user signup flow#62413
gitstart-sourcegraph merged 2 commits into
mainfrom
contractors/SG-61976

Conversation

@gitstart-app

@gitstart-app gitstart-app Bot commented May 3, 2024 •
edited by gitstart-sourcegraph
Loading

Copy link
Copy Markdown
Contributor

Description

PR for running a test to remove steps from the new user signup flow

Ticket

SG Issue
GitStart Ticket

Related issues

What has been done

  • Remove AB testing implementation
  • Implement the use of ab-shortened-install-first-signup-flow-cody-2024-04 feature flag

Demo

https://www.loom.com/share/eaada5fdd93b46e7901e5e7beb1d6056?sid=b43cc80f-42a0-47d5-a411-9a3551b79b84

Test plan

  • Pull the test branch contractors/SG-61976-test
  • Run export SOURCEGRAPHDOTCOM_MODE=true
  • Sign in

Sign up from the web

  • Visit the route /cody/manage, you should also see the "hi" and the "Hotspot form"

Sign up from an IDE

  • Create the feature flag ab-shortened-install-first-signup-flow-cody-2024-04.
  • Navigate to the /site-admin/feature-flags page
  • Create a new flag: name: ab-shortened-install-first-signup-flow-cody-2024-04, type: Rollout, then save
  • Configure the feature flag(Mimic enabled and 50% chance for being in the treatment group)
  • Navigate to site-admin/feature-flags/configuration/ab-shortened-install-first-signup-flow-cody-2024-04 page
  • Set the value to the maximum
  • Visit the routes: /cody/manage?requestFrom=CODY, you should not access the Hotspot form
  • Configure the feature flag(Mimic enabled and 50% chance of being in the control group)
  • Navigate to site-admin/feature-flags/configuration/ab-shortened-install-first-signup-flow-cody-2024-04 page
  • Set the value to the minimum(0)
  • Visit the routes: /cody/manage?requestFrom=CODY, you should also see the "hi" and the "Hotspot form"

@cla-bot cla-bot Bot added the cla-signed label May 3, 2024
@gitstart-sourcegraph gitstart-sourcegraph self-assigned this May 3, 2024
@gitstart-sourcegraph gitstart-sourcegraph linked an issue May 3, 2024 that may be closed by this pull request
Run a test to remove steps from the new user signup flow #61976
Closed
@gitstart-sourcegraph gitstart-sourcegraph marked this pull request as ready for review May 3, 2024 07:16
@gitstart-sourcegraph gitstart-sourcegraph mentioned this pull request May 3, 2024
Closed
6 tasks
@gitstart-sourcegraph

gitstart-sourcegraph commented May 3, 2024

Copy link
Copy Markdown
Contributor

Hi. @vdavid @chenkc805
I am unable to access the failing CI details as it leads to a 404 page.
Kindly help look into this.
Thank you.

Video

Screen.Recording.2024-05-03.at.10.29.20.mov

@chenkc805

chenkc805 commented May 3, 2024

Copy link
Copy Markdown
Contributor

Thanks for the Loom @gitstart-sourcegraph.

Clarifying question to make sure the flow is working right, if the user:

  • has requestFrom=CODY
  • AND is coming from the install-first flow
  • AND
    • if they're in the treatment flow, after they successfully signup
    • OR if they're in the control flow, after the HubSpot form

THEN:

  • They should not see this page
    Screenshot 2024-05-03 at 10 16 45 AM
  • They should instead see this page
    Screenshot 2024-05-03 at 10 17 02 AM

Is that correct?

@gitstart-sourcegraph

gitstart-sourcegraph commented May 6, 2024

Copy link
Copy Markdown
Contributor

Thanks for the Loom @gitstart-sourcegraph.

Clarifying question to make sure the flow is working right, if the user:

  • has requestFrom=CODY

  • AND is coming from the install-first flow

  • AND

    • if they're in the treatment flow, after they successfully signup
    • OR if they're in the control flow, after the HubSpot form

THEN:

  • They should not see this page
    Screenshot 2024-05-03 at 10 16 45 AM
  • They should instead see this page
    Screenshot 2024-05-03 at 10 17 02 AM

Is that correct?

Hi @chenkc805,
Just to clarify, the current behavior on production matches what you've described.
The loom video demonstrates the IDE signup process by simulating it on the web, thus showcasing a list of IDEs since we're using the web. However, the list of IDEs shouldn't showup for the actual experience if tested directly within the IDE.

Currently, we lack documentation on how to test the IDE signup process within a local development environment.
Thank you
Cc: @vdavid

@chenkc805

chenkc805 commented May 6, 2024

Copy link
Copy Markdown
Contributor

Ok sounds good to me. Let's ship once @vdavid approves!

@chenkc805

chenkc805 commented May 20, 2024

Copy link
Copy Markdown
Contributor

@vdavid can I get a quick stamp here so we can ship this?

CC: @gitstart-sourcegraph

vdavid
vdavid approved these changes May 20, 2024
View reviewed changes

@vdavid vdavid left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code LGTM, and I assume @chenkc805 checked the functionality, so this should be good to merge.

@gitstart-sourcegraph gitstart-sourcegraph merged commit e2a0fd6 into main May 21, 2024
@gitstart-sourcegraph gitstart-sourcegraph deleted the contractors/SG-61976 branch May 21, 2024 03:28
@gitstart-sourcegraph gitstart-sourcegraph mentioned this pull request May 27, 2024
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

2 more reviewers

@vdavid vdavid vdavid approved these changes

@chenkc805 chenkc805 chenkc805 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@gitstart-sourcegraph gitstart-sourcegraph

Labels

cla-signed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Run a test to remove steps from the new user signup flow

3 participants

@gitstart-sourcegraph @chenkc805 @vdavid