Correct product analytics cohorts and reporting
This commit is contained in:
@@ -4,6 +4,10 @@ This document is the canonical definition of product metrics. All dates and
|
||||
cohorts use UTC calendar boundaries. Counts are based on distinct accounts when
|
||||
an installation is linked, otherwise on the pseudonymous installation.
|
||||
|
||||
Admin presets cover exactly 7, 30, or 90 UTC calendar dates, starting at 00:00
|
||||
on the first date and ending at the current instant. The current UTC date is
|
||||
therefore explicitly partial.
|
||||
|
||||
## North-star metric
|
||||
|
||||
### Weekly AI active users (WAIU)
|
||||
@@ -39,9 +43,10 @@ Accounts whose `accounts.created_at` falls in the selected period.
|
||||
|
||||
### 24-hour AI activation rate
|
||||
|
||||
The percentage of new installations that successfully complete any AI feature
|
||||
within 24 hours of their first open. The numerator uses the same value-event
|
||||
rules as WAIU.
|
||||
The percentage of new installations that have completed their full 24-hour
|
||||
observation window and successfully complete any AI feature within 24 hours of
|
||||
their first open. Unmatured installations are excluded from both numerator and
|
||||
denominator. Managed usage before that installation's first open is ignored.
|
||||
|
||||
### Time to first value
|
||||
|
||||
@@ -49,6 +54,14 @@ Elapsed time from `FIRST_OPEN` to the first successful AI feature. The dashboard
|
||||
reports the median in minutes. Users without a successful AI feature are not
|
||||
included in the median and remain visible in the activation denominator.
|
||||
|
||||
### 24-hour growth funnel
|
||||
|
||||
A strict cohort of installations with a completed 24-hour observation window:
|
||||
first open, account registration after first open, first AI value event after
|
||||
registration, and first server-verified purchase after that value event. Every
|
||||
downstream step must occur within 24 hours of first open. D7 belongs only to the
|
||||
retention report and is not mixed into this funnel.
|
||||
|
||||
## Activity
|
||||
|
||||
### AI DAU, WAU and MAU
|
||||
@@ -69,6 +82,13 @@ Managed client success events are excluded from this total.
|
||||
|
||||
`successful AI requests / distinct value-active users` for the selected period.
|
||||
|
||||
### Registered product-active users
|
||||
|
||||
The operations overview counts distinct registered accounts with either a
|
||||
successful AI value event (managed, local, or BYOK) or a finalized manual
|
||||
keyboard-input summary in the selected period. The displayed rate divides this
|
||||
population by all registered accounts.
|
||||
|
||||
## Keyboard input usage
|
||||
|
||||
Keyboard input metrics use finalized UTC-day summaries produced on-device.
|
||||
@@ -175,8 +195,11 @@ server credits and are excluded.
|
||||
### 7-day and 30-day free-to-paid conversion
|
||||
|
||||
The percentage of newly registered accounts with a first credited StoreKit
|
||||
purchase no later than 7 or 30 days after registration. Cohorts whose conversion
|
||||
window has not elapsed are reported separately from mature cohorts.
|
||||
purchase no later than 7 or 30 days after registration. The selected report
|
||||
period filters when each observation window matures: a 7-day report cohort uses
|
||||
registrations shifted exactly 7 days earlier, and the 30-day cohort is shifted
|
||||
30 days earlier. This keeps every denominator fully observed and makes the rate
|
||||
available even when the selected preset is no longer than the conversion window.
|
||||
|
||||
### Paying users
|
||||
|
||||
@@ -187,21 +210,29 @@ Distinct accounts with at least one credited StoreKit purchase in the period.
|
||||
The percentage of paying accounts with at least two credited StoreKit purchases
|
||||
across their lifetime.
|
||||
|
||||
### Purchase intent funnel
|
||||
|
||||
A strict installation cohort: `PURCHASE_VIEWED`, followed by
|
||||
`PURCHASE_STARTED`, followed by a StoreKit purchase verified by the server for
|
||||
the linked account. Each event must occur after the previous step and before the
|
||||
report's `until`. `PURCHASE_CANCELLED` is a separate signal, not a funnel step.
|
||||
|
||||
StoreKit transaction count and granted credits are operational proxies. Net
|
||||
revenue, App Store commission and refunds require App Store financial data and
|
||||
are outside this service's first version.
|
||||
|
||||
## Referral funnel
|
||||
|
||||
The ordered growth funnel is:
|
||||
The ordered cohort contains bindings created in the selected period:
|
||||
|
||||
1. `REFERRAL_SHARED` distinct sharing installations.
|
||||
2. Invitation opens: accepted `INVITE_OPENED` client events plus anonymous
|
||||
first-party invitation page views. Page views are aggregate requests rather
|
||||
than distinct people and must be interpreted as a directional funnel signal.
|
||||
3. Referral-bound accounts.
|
||||
4. Referral-bound accounts that reach their first value event.
|
||||
5. Rewarded referral bindings.
|
||||
1. Referral binding created.
|
||||
2. The same invitee reaches an AI value event after binding.
|
||||
3. The same binding is rewarded before the report's `until`.
|
||||
|
||||
`REFERRAL_SHARED` distinct installations and invitation opens are independent
|
||||
directional signals. Invitation opens combine accepted `INVITE_OPENED` events
|
||||
with anonymous first-party page-view counters, so they are not people and must
|
||||
never be placed in the ordered conversion funnel.
|
||||
|
||||
Pending and ineligible bindings are parallel status counts, not sequential
|
||||
funnel steps.
|
||||
@@ -212,8 +243,8 @@ funnel steps.
|
||||
terminal success or failure event.
|
||||
- Managed request failure rate: terminal non-settled `provider_requests` divided
|
||||
by terminal managed requests.
|
||||
- P50/P95 latency: client duration bucket distribution for all modes; exact
|
||||
server duration percentiles may be added later.
|
||||
- Client latency: successful and failed terminal events grouped by declared
|
||||
duration bucket. Exact P50/P95 values are not inferred from buckets.
|
||||
- Credit-blocked users: distinct installations reporting
|
||||
`INSUFFICIENT_CREDITS` during the period.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user