feat: onboarding wizard #6#2768
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughThis PR adds comprehensive onboarding event instrumentation across CLI and Studio components. The CLI now captures telemetry for demo command setup steps with categorized error types (support files, docker readiness, router, resource), while Studio components emit PostHog events for onboarding step completion, failures, and skips, with referrer parameter propagation through the flow. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Comment |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## ondrej/eng-9192-onboarding-wizard-from-signup-to-live-metrics-topic #2768 +/- ##
========================================================================================================
- Coverage 40.63% 10.36% -30.28%
========================================================================================================
Files 1021 451 -570
Lines 127725 57081 -70644
Branches 5889 1102 -4787
========================================================================================================
- Hits 51906 5916 -45990
+ Misses 74087 50859 -23228
+ Partials 1732 306 -1426
🚀 New features to boost your workflow:
|
f92bb55 to
9d25dac
Compare
3f5ce69 to
2512d57
Compare
9d25dac to
34cbcfb
Compare
2512d57 to
355af5b
Compare
54b956c to
089708c
Compare
089708c to
24f1ee3
Compare
355af5b to
c687548
Compare
24f1ee3 to
5a7a197
Compare
c687548 to
1792cb0
Compare
5a7a197 to
2d0042e
Compare
1792cb0 to
05e08dc
Compare
2d0042e to
23a86ed
Compare
05e08dc to
dcd81d8
Compare
be76b54 to
bd2278c
Compare
This comment has been minimized.
This comment has been minimized.
bd2278c to
a6ce6a4
Compare
This comment has been minimized.
This comment has been minimized.
dcd81d8 to
59b6c78
Compare
a6ce6a4 to
ac30ef1
Compare
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.
Tip: disable this comment in your organization's Code Review settings.
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
studio/src/components/onboarding/onboarding-provider.tsx (1)
43-49:⚠️ Potential issue | 🟡 MinorNormalize persisted
currentStepafter reducing max step to 3.If a user already has
cosmo-onboarding-v1-step=4in session storage, the app can still carry that invalid step beforesetStep()runs again.🛠️ Minimal guard
+ const normalizedCurrentStep = + currentStep === undefined ? undefined : Math.max(Math.min(currentStep, ONBOARDING_V1_LAST_STEP), 1); + const value = useMemo( () => ({ onboarding, enabled: Boolean(onboardingFlag.enabled && featureFlagStatus === 'success' && onboardingFlag), setOnboarding, - currentStep, + currentStep: normalizedCurrentStep, setStep, setSkipped, resetSkipped, skipped, }), - [onboarding, onboardingFlag, featureFlagStatus, currentStep, setStep, setSkipped, resetSkipped, skipped], + [onboarding, onboardingFlag, featureFlagStatus, normalizedCurrentStep, setStep, setSkipped, resetSkipped, skipped], );Also applies to: 73-85
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@studio/src/components/onboarding/onboarding-provider.tsx` around lines 43 - 49, Persisted currentStep may exceed the new max (ONBOARDING_V1_LAST_STEP), so add a normalization guard: after reading currentStep (the useSessionStorage value) and in any place that updates it (e.g., where setCurrentStep or setStep is called, including the effect handling onboarding/featureFlag changes around the existing logic at lines ~73-85), check if currentStep !== undefined and currentStep > ONBOARDING_V1_LAST_STEP and if so call setCurrentStep(ONBOARDING_V1_LAST_STEP) (or set to undefined if that fits UX) to clamp the stored value; apply the same clamp any time you restore or initialize the step so an old value like 4 is normalized to the valid max.cli/src/commands/demo/util.ts (1)
136-168:⚠️ Potential issue | 🟠 MajorWrap GitHub tree fetch/parse in
try/catchto avoid untracked hard failures.Network failures or JSON parse errors skip the current non-OK/safeParse branches, so the command can abort without spinner failure messaging and without onboarding failure telemetry.
Suggested fix
- const treeResponse = await fetch( - `https://api.github.com/repos/${config.demoOnboardingRepositoryName}/git/trees/${config.demoOnboardingRepositoryBranch}?recursive=1`, - ); - if (!treeResponse.ok) { - spinner.fail('Failed to fetch repository tree.'); - const errorText = `GitHub API error: ${treeResponse.statusText}`; - captureOnboardingEvent({ - name: 'onboarding_step_failed', - properties: { - step_name: 'init', - entry_source: 'wgc', - error_category: 'support_files', - error_message: errorText, - }, - }); - program.error(errorText); - } - - const parsed = GitHubTreeSchema.safeParse(await treeResponse.json()); - if (!parsed.success) { - spinner.fail('Failed to parse repository tree.'); - const errorText = 'Unexpected response format from GitHub API. The repository structure may have changed.'; - captureOnboardingEvent({ - name: 'onboarding_step_failed', - properties: { - step_name: 'init', - entry_source: 'wgc', - error_category: 'support_files', - error_message: errorText, - }, - }); - program.error(errorText); - } + let parsedTree: z.infer<typeof GitHubTreeSchema>; + try { + const treeResponse = await fetch( + `https://api.github.com/repos/${config.demoOnboardingRepositoryName}/git/trees/${config.demoOnboardingRepositoryBranch}?recursive=1`, + ); + if (!treeResponse.ok) { + const errorText = `GitHub API error: ${treeResponse.status} ${treeResponse.statusText}`; + spinner.fail('Failed to fetch repository tree.'); + captureOnboardingEvent({ + name: 'onboarding_step_failed', + properties: { + step_name: 'init', + entry_source: 'wgc', + error_category: 'support_files', + error_message: errorText, + }, + }); + program.error(errorText); + } + + const parsed = GitHubTreeSchema.safeParse(await treeResponse.json()); + if (!parsed.success) { + const errorText = 'Unexpected response format from GitHub API. The repository structure may have changed.'; + spinner.fail('Failed to parse repository tree.'); + captureOnboardingEvent({ + name: 'onboarding_step_failed', + properties: { + step_name: 'init', + entry_source: 'wgc', + error_category: 'support_files', + error_message: errorText, + }, + }); + program.error(errorText); + } + parsedTree = parsed.data; + } catch (err) { + const errorText = err instanceof Error ? err.message : String(err); + spinner.fail('Failed to fetch repository tree.'); + captureOnboardingEvent({ + name: 'onboarding_step_failed', + properties: { + step_name: 'init', + entry_source: 'wgc', + error_category: 'support_files', + error_message: errorText, + }, + }); + program.error(errorText); + } - const files = parsed.data.tree.filter((entry) => entry.type === 'blob' && entry.path.startsWith('plugins/')); + const files = parsedTree.tree.filter((entry) => entry.type === 'blob' && entry.path.startsWith('plugins/'));As per coding guidelines:
Add proper error handling with try-catch blocks.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cli/src/commands/demo/util.ts` around lines 136 - 168, Wrap the GitHub tree fetch and JSON parse in a try/catch around the await fetch(...) and await treeResponse.json() + GitHubTreeSchema.safeParse logic so network failures and JSON parse errors are caught; in the catch block call spinner.fail with an explanatory message, build an errorText from the caught error (e.g. error.message), call captureOnboardingEvent with the same properties used for other failures (step_name: 'init', entry_source: 'wgc', error_category: 'support_files', error_message: errorText), and finally call program.error(errorText); keep the existing branches for non-ok responses and safeParse failures but ensure any thrown exceptions are handled by this new try/catch around the treeResponse fetch/parse.
🧹 Nitpick comments (3)
studio/src/components/onboarding/step-1.tsx (1)
120-123: Trim unused dependencies from thesetStepeffect.
posthogandreferrerare not used in this effect body, so they can trigger unnecessarysetStep(1)re-runs.♻️ Optional cleanup
useEffect(() => { setStep(1); - }, [setStep, posthog, referrer]); + }, [setStep]);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@studio/src/components/onboarding/step-1.tsx` around lines 120 - 123, The useEffect that calls setStep(1) currently lists posthog and referrer in its dependency array even though the effect body only references setStep; remove posthog and referrer from the dependency array so the effect becomes dependent only on setStep (i.e., update the effect that contains useEffect(() => { setStep(1); }, [setStep, posthog, referrer]) to only include setStep) to prevent unnecessary re-runs.studio/src/lib/track.ts (1)
6-10: Remove stalewindow.kotyping from globalWindowdeclaration.
window.kois dead tracking surface and keeping it typed here adds confusion for future changes.♻️ Suggested cleanup
declare global { interface Window { - ko: any; Reo: any; } }Based on learnings: "In studio/src/lib/track.ts, remove all references to window.ko (Koala tracking) as it is no longer used."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@studio/src/lib/track.ts` around lines 6 - 10, Remove the stale Koala tracking typing by deleting the window.ko entry from the global Window interface declaration in track.ts and any direct references to window.ko; update the declaration so it only exposes Reo (i.e., remove "ko: any" from the interface) and search for and eliminate any usages of window.ko or ko in functions like the tracking exporters to avoid unused/undefined references, then run type checks to ensure no remaining references break compilation.cli/test/demo/command.test.ts (1)
271-477: Add coverage forrun_router_send_metricscompletion on first successful sample query.The production code now emits this completion event only on the first successful query; this branch/dedup behavior is not asserted yet.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cli/test/demo/command.test.ts` around lines 271 - 477, Add a new test that asserts the onboarding_step_completed event for run_router_send_metrics is emitted on the first successful sample query: call runDemo (via runDemo), simulate the user choosing the router flow (keys.enter(); keys.press('r')), and mock demoUtil.runRouterContainer (and if needed createFederatedGraphToken/deleteRouterToken) so the first sample query returns success while subsequent queries may also succeed; then assert demoUtil.captureOnboardingEvent was calledWith { name: 'onboarding_step_completed', properties: { step_name: 'run_router_send_metrics', entry_source: 'wgc' } } and that this completion event for run_router_send_metrics is only emitted once (e.g., toHaveBeenCalledTimes / filter calls by step_name). Ensure you reference runDemo, demoUtil.runRouterContainer, demoUtil.captureOnboardingEvent, createFederatedGraphToken, and deleteRouterToken when locating test scaffolding.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@cli/src/commands/demo/command.ts`:
- Around line 467-468: The code awaits waitForKeyPress({ Enter: retryFn }, ...)
but then executes break, which discards the retry's return value and lets
handleStep1 resolve undefined; replace the break with a return (or return the
awaited call) so the recursive retry result from waitForKeyPress / retryFn is
propagated back to handleStep1 (look for waitForKeyPress and handleStep1 to
apply this change).
- Around line 308-323: The deleteRouterToken and createRouterToken calls
currently only inspect returned { error } objects and will crash on
transport-level ConnectError exceptions; wrap both
deleteRouterToken(tokenParams) and createRouterToken(tokenParams) calls in
try/catch blocks, catch transport errors (e.g., ConnectError), build the same
errorText and call captureOnboardingEvent with the same properties, log the
error, and call await waitForKeyPress({ r: retryFn, R: retryFn }, 'Hit [r] to
retry. CTRL+C to quit.') before returning; ensure you still handle the existing
deleteResult.error/createResult.error path inside the try and only rethrow
unexpected non-ConnectError exceptions if necessary.
In `@cli/src/commands/demo/util.ts`:
- Around line 198-207: The telemetry payload is leaking local filesystem details
by including cosmoDir and raw per-file error strings in error_message; update
the captureOnboardingEvent call in util.ts to omit cosmoDir and to sanitize
entries from failed (use safe identifiers like path.basename(f.path) or just the
filename and/or an error code/category instead of full error text) so
error_message contains only non-sensitive summary info (e.g., "Failed to fetch N
support files" plus a sanitized list like " filename.ext: NETWORK_ERROR").
Ensure you update the variables failText/errorText and the error_message
property accordingly and keep captureOnboardingEvent usage intact.
In `@studio/src/components/onboarding/step-1.tsx`:
- Around line 51-53: The current assignment to referrer uses document.referrer
during render which breaks SSR; update the derivation so document is only
accessed on the client (e.g., compute the source using router.query.referrer ||
(typeof window !== 'undefined' ? document.referrer : '') or move the logic into
a useEffect/useMemo that runs client-side) and then pass that value into
normalizeReferrer; locate the referrer variable assignment and the
normalizeReferrer call in step-1.tsx to apply the guard.
In `@studio/src/components/onboarding/step-3.tsx`:
- Around line 201-215: The effect currently only fires when both timeouts are
true and mislabels the category; change the guard to run when either timeout is
true (use || instead of &&) and compute error_category/error_message from the
individual flags so that if polling.routerTimedOut is true it sets category to
'router' (and message to 'Router not detected within 5 minutes'), otherwise if
polling.metricsTimedOut is true it sets category to 'metrics' (and message to
'Metrics not detected within 5 minutes'); adjust the effect surrounding
captureOnboardingEvent(posthog, ...) and its dependency array if needed to
ensure it fires correctly for single-timeout cases.
In `@studio/src/hooks/use-onboarding-navigation.ts`:
- Around line 62-67: The code constructs pathWithParams by concatenating
?referrer=${referrer}, which can break on reserved characters; update the logic
in use-onboarding-navigation (where params, referrer, path, pathWithParams and
Router.replace are used) to URL-encode the referrer before appending (e.g., use
encodeURIComponent(referrer) or build the query via URLSearchParams) so
Router.replace receives a safe, correctly encoded URL.
---
Outside diff comments:
In `@cli/src/commands/demo/util.ts`:
- Around line 136-168: Wrap the GitHub tree fetch and JSON parse in a try/catch
around the await fetch(...) and await treeResponse.json() +
GitHubTreeSchema.safeParse logic so network failures and JSON parse errors are
caught; in the catch block call spinner.fail with an explanatory message, build
an errorText from the caught error (e.g. error.message), call
captureOnboardingEvent with the same properties used for other failures
(step_name: 'init', entry_source: 'wgc', error_category: 'support_files',
error_message: errorText), and finally call program.error(errorText); keep the
existing branches for non-ok responses and safeParse failures but ensure any
thrown exceptions are handled by this new try/catch around the treeResponse
fetch/parse.
In `@studio/src/components/onboarding/onboarding-provider.tsx`:
- Around line 43-49: Persisted currentStep may exceed the new max
(ONBOARDING_V1_LAST_STEP), so add a normalization guard: after reading
currentStep (the useSessionStorage value) and in any place that updates it
(e.g., where setCurrentStep or setStep is called, including the effect handling
onboarding/featureFlag changes around the existing logic at lines ~73-85), check
if currentStep !== undefined and currentStep > ONBOARDING_V1_LAST_STEP and if so
call setCurrentStep(ONBOARDING_V1_LAST_STEP) (or set to undefined if that fits
UX) to clamp the stored value; apply the same clamp any time you restore or
initialize the step so an old value like 4 is normalized to the valid max.
---
Nitpick comments:
In `@cli/test/demo/command.test.ts`:
- Around line 271-477: Add a new test that asserts the onboarding_step_completed
event for run_router_send_metrics is emitted on the first successful sample
query: call runDemo (via runDemo), simulate the user choosing the router flow
(keys.enter(); keys.press('r')), and mock demoUtil.runRouterContainer (and if
needed createFederatedGraphToken/deleteRouterToken) so the first sample query
returns success while subsequent queries may also succeed; then assert
demoUtil.captureOnboardingEvent was calledWith { name:
'onboarding_step_completed', properties: { step_name: 'run_router_send_metrics',
entry_source: 'wgc' } } and that this completion event for
run_router_send_metrics is only emitted once (e.g., toHaveBeenCalledTimes /
filter calls by step_name). Ensure you reference runDemo,
demoUtil.runRouterContainer, demoUtil.captureOnboardingEvent,
createFederatedGraphToken, and deleteRouterToken when locating test scaffolding.
In `@studio/src/components/onboarding/step-1.tsx`:
- Around line 120-123: The useEffect that calls setStep(1) currently lists
posthog and referrer in its dependency array even though the effect body only
references setStep; remove posthog and referrer from the dependency array so the
effect becomes dependent only on setStep (i.e., update the effect that contains
useEffect(() => { setStep(1); }, [setStep, posthog, referrer]) to only include
setStep) to prevent unnecessary re-runs.
In `@studio/src/lib/track.ts`:
- Around line 6-10: Remove the stale Koala tracking typing by deleting the
window.ko entry from the global Window interface declaration in track.ts and any
direct references to window.ko; update the declaration so it only exposes Reo
(i.e., remove "ko: any" from the interface) and search for and eliminate any
usages of window.ko or ko in functions like the tracking exporters to avoid
unused/undefined references, then run type checks to ensure no remaining
references break compilation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 2e4d14eb-8a64-47ee-b274-408610e866c4
📒 Files selected for processing (10)
cli/src/commands/demo/command.tscli/src/commands/demo/util.tscli/test/demo/command.test.tsstudio/src/components/onboarding/onboarding-provider.tsxstudio/src/components/onboarding/step-1.tsxstudio/src/components/onboarding/step-2.tsxstudio/src/components/onboarding/step-3.tsxstudio/src/components/onboarding/step-finished.tsxstudio/src/hooks/use-onboarding-navigation.tsstudio/src/lib/track.ts
440ca36 to
f775983
Compare
f3cd3b8 to
9e00102
Compare
9e00102 to
b606160
Compare
…etrics-topic' into ondrej/eng-9192-onboarding-wizard-from-signup-to-live-metrics-06
dcb2b18
into
ondrej/eng-9192-onboarding-wizard-from-signup-to-live-metrics-topic
Summary by CodeRabbit
Bug Fixes
Improvements
Checklist
Open Source AI Manifesto
This project follows the principles of the Open Source AI Manifesto. Please ensure your contribution aligns with its principles.
PR (6): for onboarding wizard - event tracking
Parent: #2750