Interview Questions › Metrics

Define success for Netflix "Downloads For You"

Netflix automatically downloads recommended shows to your phone when on Wi-Fi so you can watch offline. How would you measure whether it works?

Context

Popular in markets with patchy mobile data. It uses device storage, which some users dislike.

Clarifying questions to ask first

• Which markets?
• Is retention the business goal?

How to approach it

1. Goal: more viewing in low-connectivity moments
2. Usage of auto-downloaded titles
3. Costs: storage, bandwidth, user annoyance
4. Business link: retention

Framework: Goal of the product -> Users and value exchange -> North Star -> Supporting metrics by funnel stage -> Guardrails and counter-metrics -> Decision or target

Sample answer

• Share of auto-downloaded titles started within 7 days
• Offline hours per member per week
• Turn-off rate of the feature

1. Only 22% of auto-downloaded titles get started
2. Members using it watch 35 more offline minutes a week
3. Turn-off rate is 15%, mostly on phones with under 32 GB

Primary: share of auto-downloads watched, because it shows recommendation quality and justifies storage use. Outcome: offline hours and retention in target markets. Guardrails: turn-off rate and storage complaints.

What interviewers look for

• Hours watched offline
• Share of auto-downloads watched
• Retention