Conversation
7aba65c to
5f98d5d
Compare
5611dd1 to
ba24bfc
Compare
There was a problem hiding this comment.
Pull request overview
Adds end-to-end Glean telemetry for passwordless OTP registration/login, including new frontend OTP lifecycle events and a backend reason extra on login.complete / reg.complete so OTP vs password vs third-party completions can be distinguished.
Changes:
- Define + generate new passwordless OTP frontend events for
loginandreg(view/engage/submit/success/error/resend). - Instrument the Settings passwordless code page to emit the new Glean events, with unit + functional test assertions.
- Add a
reasonextra key to backendlogin.completeandreg.complete, and emit appropriate values from email, OTP, and third-party auth flows.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/fxa-shared/metrics/glean/web/reg.ts | Adds generated reg.otp_* event metric definitions. |
| packages/fxa-shared/metrics/glean/web/login.ts | Adds generated login.otp_* event metric definitions. |
| packages/fxa-shared/metrics/glean/web/index.ts | Adds eventsMap entries for passwordless OTP events used by the frontend wrapper. |
| packages/fxa-shared/metrics/glean/web/event.ts | Generated import reordering. |
| packages/fxa-shared/metrics/glean/fxa-ui-metrics.yaml | Declares new UI OTP events (login + reg) and extra_keys. |
| packages/fxa-shared/metrics/glean/fxa-backend-metrics.yaml | Adds reason extra_key to backend login.complete / reg.complete. |
| packages/fxa-settings/src/pages/Signin/SigninPasswordlessCode/index.tsx | Emits passwordless OTP lifecycle events from the code-entry page. |
| packages/fxa-settings/src/pages/Signin/SigninPasswordlessCode/index.test.tsx | Adds unit tests asserting emitted passwordless OTP events. |
| packages/fxa-settings/src/lib/glean/index.ts | Adds event-name → metric recording wiring for new OTP events. |
| packages/fxa-settings/src/components/FormVerifyCode/index.tsx | Adds onEngageCb hook to emit OTP engage metric on first focus. |
| packages/fxa-auth-server/test/local/metrics/events.js | Updates expected backend login.complete payload to include reason: 'email'. |
| packages/fxa-auth-server/lib/routes/utils/signup.js | Emits reg.complete with reason: 'email' for standard email verification. |
| packages/fxa-auth-server/lib/routes/passwordless.ts | Emits reg.complete / login.complete with reason: 'otp' for passwordless flows. |
| packages/fxa-auth-server/lib/routes/linked-accounts.ts | Emits completion reasons for Google/Apple flows. |
| packages/fxa-auth-server/lib/metrics/glean/server_events.ts | Adds reason to backend recordLoginComplete / recordRegComplete event extras. |
| packages/fxa-auth-server/lib/metrics/glean/index.ts | Wires reason extra key into login_complete / reg_complete event creation. |
| packages/fxa-auth-server/lib/metrics/events.js | Adds reason: 'email' when emitting login.complete on flow completion. |
| packages/functional-tests/tests/passwordless/signinPasswordless.spec.ts | Adds functional assertions around OTP Glean event order + extras. |
| packages/functional-tests/lib/glean.ts | New Playwright helper that intercepts frontend Glean pings for assertions. |
| packages/functional-tests/lib/fixtures/standard.ts | Exposes gleanEventsHelper fixture for functional tests. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| * @param {string} utm_medium - The "medium" on which the user acted. For example, if the user clicked on a link in an email, then the value of this metric would be 'email'. The value has a max length of 128 characters with the alphanumeric characters, _ (underscore), forward slash (/), . (period), % (percentage sign), and - (hyphen) in the allowed set of characters.. | ||
| * @param {string} utm_source - The source from where the user started. For example, if the user clicked on a link on the Mozilla accounts web site, this value could be 'fx-website'. The value has a max length of 128 characters with the alphanumeric characters, _ (underscore), forward slash (/), . (period), % (percentage sign), and - (hyphen) in the allowed set of characters.. | ||
| * @param {string} utm_term - This metric is similar to the `utm.source`; it is used in the Firefox browser. For example, if the user started from about:welcome, then the value could be 'aboutwelcome-default-screen'. The value has a max length of 128 characters with the alphanumeric characters, _ (underscore), forward slash (/), . (period), % (percentage sign), and - (hyphen) in the allowed set of characters.. | ||
| * @param {string} reason - Indicates how the user completed login. Values include "email" (traditional email/password), "otp" (passwordless OTP code).. |
There was a problem hiding this comment.
The recordLoginComplete JSDoc for reason only mentions "email" and "otp", but linked-accounts.ts now emits "google" and "apple" reasons for third-party auth logins. Update the documented value set to match what the code can send.
| gleanEventsHelper.assertEventOrder([ | ||
| 'email_first_view', | ||
| 'reg_otp_view', | ||
| 'reg_otp_submit', | ||
| 'reg_otp_submit_success', | ||
| ]); |
There was a problem hiding this comment.
These assertions can be flaky because Glean pings are sent asynchronously; assertEventOrder runs immediately after isLoggedIn() without waiting for the final ping(s) to be captured. Consider awaiting gleanEventsHelper.waitForEvent(...) for the last expected event (e.g. *_submit_success) before asserting order.
| gleanEventsHelper.assertEventOrder([ | ||
| 'reg_otp_view', | ||
| 'reg_otp_submit', | ||
| 'reg_otp_submit_frontend_error', | ||
| ]); | ||
|
|
||
| const errorPings = gleanEventsHelper.getEventsByName( | ||
| 'reg_otp_submit_frontend_error' | ||
| ); | ||
| expect(errorPings.length).toBeGreaterThan(0); | ||
| expect(errorPings[0].extras.reason).toBe('invalid'); |
There was a problem hiding this comment.
Same timing issue here: assertEventOrder() and subsequent getEventsByName()/extras assertions may run before the error ping is captured, leading to intermittent failures. Await gleanEventsHelper.waitForEvent('reg_otp_submit_frontend_error') (or similar) before asserting order and inspecting extras.reason.
| gleanOtp.error({ event: { reason: 'too many times' } }); | ||
| setLocalizedErrorBannerMessage( | ||
| getLocalizedErrorMessage(ftlMsgResolver, error) | ||
| ); | ||
| } else { | ||
| gleanOtp.error({ event: { reason: 'try again later' } }); |
There was a problem hiding this comment.
gleanOtp.error() maps to the *_otp_submit_frontend_error metric, but it's also being emitted for resend-code failures here. That will make resend failures look like submit failures in telemetry. Consider adding a dedicated resend error metric (e.g. *_otp_resend_frontend_error) or renaming/repurposing the existing error metric+docs to be a generic OTP frontend error for the whole page lifecycle.
| gleanOtp.error({ event: { reason: 'too many times' } }); | |
| setLocalizedErrorBannerMessage( | |
| getLocalizedErrorMessage(ftlMsgResolver, error) | |
| ); | |
| } else { | |
| gleanOtp.error({ event: { reason: 'try again later' } }); | |
| setLocalizedErrorBannerMessage( | |
| getLocalizedErrorMessage(ftlMsgResolver, error) | |
| ); | |
| } else { |
| extra_keys: | ||
| reason: | ||
| description: | | ||
| Indicates how the user completed login. Values include "email" (traditional email/password), "otp" (passwordless OTP code). |
There was a problem hiding this comment.
login.complete reason extra_key description only lists "email" and "otp", but auth-server now emits reason: 'google' | 'apple' for third-party auth logins as well. Please update this metric description (and ideally keep it in sync with the actual emitted value set) so downstream analysis/data-review expectations match reality.
| Indicates how the user completed login. Values include "email" (traditional email/password), "otp" (passwordless OTP code). | |
| Indicates how the user completed login. Values include "email" (traditional email/password), "otp" (passwordless OTP code), | |
| "google" (third-party login via Google), and "apple" (third-party login via Apple). Additional values may be added over time | |
| as new login methods are supported. |
| reason: string; | ||
| }) { | ||
| const event = { | ||
| category: 'login', | ||
| name: 'complete', | ||
| extra: { | ||
| reason: String(reason), | ||
| }, |
There was a problem hiding this comment.
reason is typed as required and is always recorded into event.extra via String(reason). If any caller omits reason (or passes undefined), this will emit the literal string "undefined" into telemetry. Consider making reason optional here and only setting event.extra.reason when a non-empty value is provided (or defaulting it to an empty string earlier in the pipeline).
| reason: string; | |
| }) { | |
| const event = { | |
| category: 'login', | |
| name: 'complete', | |
| extra: { | |
| reason: String(reason), | |
| }, | |
| reason?: string; | |
| }) { | |
| const extra: Record<string, any> = {}; | |
| if (reason !== undefined && reason !== null && reason !== '') { | |
| extra.reason = String(reason); | |
| } | |
| const event = { | |
| category: 'login', | |
| name: 'complete', | |
| extra, |
| complete: createEventFn('reg_complete', { | ||
| additionalMetrics: extraKeyReasonCb, | ||
| }), |
There was a problem hiding this comment.
reg_complete/login_complete now forward reason via additionalMetrics: extraKeyReasonCb, but extraKeyReasonCb currently returns reason: metrics.reason which can be undefined when callers omit it. That ends up as String(undefined) in server_events.ts ("undefined" in telemetry). Recommend defaulting to '' in extraKeyReasonCb (or otherwise ensuring reason is always set) so these events never record an undefined reason.
| fetch(url, { method: 'POST', body: data, keepalive: true }).catch( | ||
| () => {} | ||
| ); |
There was a problem hiding this comment.
The sendBeacon→fetch shim uses default fetch options (mode: 'cors'). For cross-origin Glean endpoints (e.g. https://incoming.telemetry.mozilla.org/submit/...), this can trigger CORS/preflight behavior that sendBeacon normally avoids, and may prevent the POST from being sent/captured. Consider setting mode: 'no-cors' (and/or explicitly handling OPTIONS responses with appropriate Access-Control-Allow-* headers) to better emulate sendBeacon semantics.
| fetch(url, { method: 'POST', body: data, keepalive: true }).catch( | |
| () => {} | |
| ); | |
| fetch(url, { | |
| method: 'POST', | |
| body: data, | |
| keepalive: true, | |
| mode: 'no-cors', | |
| }).catch(() => {}); |
| private pings: GleanPing[] = []; | ||
| private page: Page; | ||
| private started = false; | ||
| private readonly ROUTE_PATTERN = '**/submit/accounts-frontend*/**'; |
There was a problem hiding this comment.
ROUTE_PATTERN is hard-coded to **/submit/accounts-frontend*/**, but the frontend Glean applicationId default in this repo is accounts_frontend_dev (underscores) (see packages/fxa-settings/src/lib/config.ts / content-server config). If the upload path uses that applicationId verbatim, this route won't match and no events will be captured. Consider broadening the pattern (e.g. accounts*frontend*) or deriving it from the configured Glean applicationId used in the test environment.
| private readonly ROUTE_PATTERN = '**/submit/accounts-frontend*/**'; | |
| // Match both hyphenated and underscored applicationIds, e.g. accounts-frontend* and accounts_frontend* | |
| private readonly ROUTE_PATTERN = '**/submit/accounts*frontend*/**'; |
Because
login.completeandreg.completeevents lacked areasonfield to distinguish OTP completions from email/password or third-party authThis pull request
view,engage,submit,submitSuccess,error, andresendCodefor bothloginandregcategoriesSigninPasswordlessCodecomponent with Glean events for the full lifecycle (view → engage → submit → success/error)reasonextra key to backendlogin.completeandreg.completeevents (values:email,otp,google,apple)fxa-ui-metrics.yamlandfxa-backend-metrics.yaml, regenerates TypeScript viaglean-generateGleanEventsHelperPlaywright utility that intercepts frontend Glean pings for functional test assertionsIssue
Closes: https://mozilla-hub.atlassian.net/browse/FXA-13019
Checklist
Other Information
How to test:
force_passwordless=truereg_otp_viewreg_otp_engagereg_otp_submitthenreg_otp_submit_successlogin_otp_*prefix instead