Skip to content

feat(cli): add marketplace funnel telemetry to integration add command#15065

Merged
tonypan2 merged 1 commit intomainfrom
tonypan/cli-marketplace-telemetry
Feb 23, 2026
Merged

feat(cli): add marketplace funnel telemetry to integration add command#15065
tonypan2 merged 1 commit intomainfrom
tonypan/cli-marketplace-telemetry

Conversation

@tonypan2
Copy link
Copy Markdown
Contributor

@tonypan2 tonypan2 commented Feb 15, 2026

Summary

Adds marketplace funnel telemetry events to the auto-provision path of vercel integration add (FF_AUTO_PROVISION_INSTALL=1). The legacy path is not instrumented — it will be removed.

Events tracked (9):

  • marketplace_install_flow_started
  • marketplace_checkout_plan_selected — includes plan_selection_method (cli_flag / server_default)
  • marketplace_checkout_provisioning_started
  • marketplace_checkout_provisioning_completed — includes resource_id
  • marketplace_checkout_provisioning_failed — includes error_message
  • marketplace_project_connected — includes project_id, resource_id
  • marketplace_project_connect_failed — includes project_id, resource_id, error_message
  • marketplace_install_flow_web_fallback — reason passed through from server response
  • marketplace_install_flow_dropped — input validation failures after funnel entry (metadata_parse_error, metadata_validation_failed, resource_name_invalid)

All events carry base props: integration_id, integration_slug, integration_name, product_id, product_slug, team_id, source: "cli", is_auto_provision: true

Web fallback events include pass-through fields from the auto-provision API response: auto_provision_result_kind, auto_provision_result_reason, auto_provision_error_message. No client-side interpretation — server values forwarded as-is.

Implementation

  • One generic telemetry method: trackMarketplaceEvent(event, props) — no per-event interfaces or typed wrappers
  • Base props built once, spread into each event call
  • AutoProvisionFallback.kind and .reason typed as string for forward-compatibility with new API responses
  • onProjectConnected callback added to PostProvisionOptions to decouple telemetry from post-provision setup
  • Fallback reason: fallback.reason ?? fallback.kind — pure pass-through from server
  • Every post-flow_started exit has a terminal event — no silent funnel drops

Fallback guard fix

isAutoProvisionFallback changed from allowlist (['metadata', 'unknown'].includes(kind)) to denylist (kind !== 'provisioned'). This is a behavioral change beyond telemetry:

  • Bug fix: The old allowlist missed kind: 'install' (returned when policies not accepted) — would have thrown an unhandled error instead of falling back to the browser
  • Forward-compatible: New API kind values (e.g. multiple_installations from https://github.com/vercel/api/pull/61258) are automatically handled as browser fallbacks
  • Safe: The structural checks (url, integration, product fields present + HTTP 422) are the real guard, ensuring only actual step responses are caught

The API currently returns 4 kind values: provisioned (201 success), install, metadata, unknown (all 422 fallbacks). The denylist ensures the CLI doesn't need updating each time the API adds a new fallback reason.

Event Lifecycle

Happy Path

 ① flow_started
       │
 ② plan_selected (cli_flag | server_default)
       │
 ③ provisioning_started
       │
 ④ provisioning_completed
       │
 ⑤ project_connected (if project linked)

Unhappy Paths

PATH 1: Policy Declined
───────────────────────
 ① flow_started
 ✖ flow_dropped (reason: policy_declined)
 → EXIT

PATH 2: Input Validation Failure
─────────────────────────────────
 ① flow_started
 ✖ flow_dropped (reason: metadata_parse_error
                       | metadata_validation_failed
                       | resource_name_invalid)
 → EXIT

PATH 3: Server Fallback
───────────────────────
 ① flow_started
 ② plan_selected
 ③ provisioning_started
 ✖ web_fallback (reason: server kind/reason)
 → Open browser fallback → EXIT

PATH 4: Provisioning API Error
──────────────────────────────
 ① flow_started
 ② plan_selected
 ③ provisioning_started
 ✖ provisioning_failed
 → EXIT

Terminal Events by Exit Type

Exit Point Event Reason
Terms declined flow_dropped policy_declined
Metadata parse error flow_dropped metadata_parse_error
Metadata validation failed flow_dropped metadata_validation_failed
Resource name invalid flow_dropped resource_name_invalid
Server non-provisioned web_fallback server kind/reason
Provisioning API error provisioning_failed

Dependency

  • Depends on: https://github.com/vercel/api/pull/60858 (adds reason field to auto-provision responses) — merged and deployed
  • Without the API PR: CLI uses fallback kind value as the telemetry reason
  • With the API PR: CLI reads the specific reason for full funnel specificity

Test plan

  • 73/73 auto-provision tests passing
  • 75/75 legacy path tests passing (no changes to legacy path)
  • Telemetry events asserted for happy path
  • TypeScript compiles cleanly
  • CI passes

Fixes: MKT-2615

🤖 Generated with Claude Code

Note

Low Risk Change

This PR adds telemetry instrumentation to a CLI command with one behavioral change: the fallback guard logic changed from an allowlist to a denylist, which is a bug fix that handles additional API response kinds as browser fallbacks rather than throwing errors.

  • Telemetry: adds 9 marketplace funnel events throughout auto-provision flow
  • Behavioral change: isAutoProvisionFallback now uses denylist (kind !== 'provisioned') instead of allowlist
  • Adds callbacks onProjectConnected and onProjectConnectFailed to PostProvisionOptions

Risk assessment for commit 87e2258.

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Feb 15, 2026

🦋 Changeset detected

Latest commit: 87e2258

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
vercel Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 15, 2026

📦 CLI Tarball Ready

The Vercel CLI tarball for this PR is now available!

Quick Test

You can test this PR's CLI directly by running:

npx https://vercel-pwrsgeeud.vercel.sh/tarballs/vercel.tgz --help

Use in vercel.json

To use this CLI version in your project builds, add to your vercel.json:

{
  "build": {
    "env": {
      "VERCEL_CLI_VERSION": "vercel@https://vercel-pwrsgeeud.vercel.sh/tarballs/vercel.tgz"
    }
  }
}

Python Runtime Wheel

A Python runtime wheel was also built for this PR.
To use in your Python project builds, also set this environment variable:

VERCEL_RUNTIME_PYTHON="vercel-runtime @ https://vercel-pwrsgeeud.vercel.sh/tarballs/vercel_runtime-0.6.0.dev1771879991+87e2258-py3-none-any.whl"

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 15, 2026

🧪 Unit Test Strategy

Comparing: 8244a9687e2258 (view diff)

Strategy: Affected packages only

✅ Only testing packages that have been modified or depend on modified packages.

Affected packages - 1 (3%)
  1. vercel
Unaffected packages - 39 (98%)
  1. @vercel-internals/get-package-json
  2. @vercel/backends
  3. @vercel/build-utils
  4. @vercel/cervel
  5. @vercel/cli-auth
  6. @vercel/client
  7. @vercel/config
  8. @vercel/detect-agent
  9. @vercel/edge
  10. @vercel/elysia
  11. @vercel/error-utils
  12. @vercel/express
  13. @vercel/fastify
  14. @vercel/firewall
  15. @vercel/frameworks
  16. @vercel/fs-detectors
  17. @vercel/functions
  18. @vercel/gatsby-plugin-vercel-builder
  19. @vercel/go
  20. @vercel/h3
  21. @vercel/hono
  22. @vercel/hydrogen
  23. @vercel/koa
  24. @vercel/nestjs
  25. @vercel/next
  26. @vercel/node
  27. @vercel/oidc
  28. @vercel/oidc-aws-credentials-provider
  29. @vercel/python
  30. @vercel/python-analysis
  31. @vercel/redwood
  32. @vercel/related-projects
  33. @vercel/remix-builder
  34. @vercel/routing-utils
  35. @vercel/ruby
  36. @vercel/rust
  37. @vercel/static-build
  38. @vercel/static-config
  39. examples

Results

  • Unit tests: Only affected packages will run unit tests
  • E2E tests: Running in parallel via E2E Tests workflow
  • Type checks: Only affected packages will run type checks

This comment is automatically generated based on the affected testing strategy

@tonypan2 tonypan2 requested a review from zbalda February 20, 2026 19:37
@tonypan2 tonypan2 marked this pull request as ready for review February 20, 2026 19:38
@tonypan2 tonypan2 requested review from a team as code owners February 20, 2026 19:38
@tonypan2 tonypan2 force-pushed the tonypan/cli-marketplace-telemetry branch from 710965c to b71d69c Compare February 23, 2026 20:08
@vercel
Copy link
Copy Markdown
Contributor

vercel bot commented Feb 23, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
zero-config-go Ready Ready Preview, Comment Feb 23, 2026 8:08pm

@tonypan2 tonypan2 force-pushed the tonypan/cli-marketplace-telemetry branch from b71d69c to 87e2258 Compare February 23, 2026 20:53
@tonypan2 tonypan2 merged commit cfc59ca into main Feb 23, 2026
264 of 275 checks passed
@tonypan2 tonypan2 deleted the tonypan/cli-marketplace-telemetry branch February 23, 2026 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants