Analytics Trial: Design an Analytics Funnel for a Free-Trial Conversion Flow
Interview Experience
Problem
Your product has a free trial that converts to paid. Define the metrics and instrumentation for tracking the trial conversion funnel, diagnosing drop-offs, and running an A/B test on the trial length (7 days vs. 14 days).
Funnel stages:
Landing -> Sign-up -> Trial Start -> Feature Activation -> Trial End -> Paid Conversion
Your task: Define (1) the key metrics at each stage, (2) the events you would log with required properties, (3) a SQL query to compute stage-by-stage conversion rates, and (4) the A/B test design for trial length.
sql
-- Events table
events(user_id, event_name, properties JSONB, created_at)
Follow-ups
- What is your randomization unit for the A/B test — user, session, or device? Why?
- How do you detect and handle users who sign up under multiple accounts to extend their trial?
- If feature activation rate drops after a UI change, how do you distinguish a causal effect from a seasonal trend?
- What is the minimum detectable effect size you would target, and how does that determine sample size?
Full Details
Problem
Your product has a free trial that converts to paid. Define the metrics and instrumentation for tracking the trial conversion funnel, diagnosing drop-offs, and running an A/B test on the trial length (7 days vs. 14 days).
Funnel stages:
Landing -> Sign-up -> Trial Start -> Feature Activation -> Trial End -> Paid Conversion
Your task: Define (1) the key metrics at each stage, (2) the events you would log with required properties, (3) a SQL query to compute stage-by-stage conversion rates, and (4) the A/B test design for trial length.
sql
-- Events table
events(user_id, event_name, properties JSONB, created_at)
Follow-ups
- What is your randomization unit for the A/B test — user, session, or device? Why?
- How do you detect and handle users who sign up under multiple accounts to extend their trial?
- If feature activation rate drops after a UI change, how do you distinguish a causal effect from a seasonal trend?
- What is the minimum detectable effect size you would target, and how does that determine sample size?
About This Question
This is a candidate experience report from a mercor interview.