Interview Questions › Metrics

Ship or not: Shorts in the subscriptions feed

An experiment adds Shorts from channels you subscribe to into the Subscriptions feed. Total watch time is up 1.5% but long-form watch time from Subscriptions is down 4%. Do you ship?

Context

Long-form videos earn several times more ad revenue per minute than Shorts. Creators care about subscribers watching their long videos.

Clarifying questions to ask first

• How long did the test run and is there novelty effect?
• What is the revenue impact, not just watch time?
• Which creators lose views?

How to approach it

1. Clarify the goal of the Subscriptions feed
2. Look past the top-line: revenue per minute, creator impact
3. Segment: which users and creators drive the change
4. Make a decision with conditions

Framework: Goal of the product -> Users and value exchange -> North Star -> Supporting metrics by funnel stage -> Guardrails and counter-metrics -> Decision or target

Sample answer

• Ad revenue per user per day
• Long-form views per subscribed creator
• Return visits to Subscriptions tab

1. Ad revenue per user is down 0.8% in treatment
2. The long-form drop is concentrated in creators with fewer than 100K subscribers
3. Daily active users of Subscriptions are flat; return visits are flat

Do not ship as is: revenue per user falls and small creators lose the long-form views that sustain them. Test a capped version (for example, one Shorts row) and judge on revenue per user and small-creator long-form views as guardrails.

What interviewers look for

• Watch time
• Revenue per user
• Creator long-form views