Interview Questions › Metrics

Raise ad load in Reels? Make the call

An experiment increases ads in Reels from 1 in 8 to 1 in 6 items. Revenue per user is up 9%, time spent in Reels is down 2%, and daily sessions are flat. Do you ship, and how do you decide?

Context

Reels competes with TikTok and YouTube Shorts for time. Short-term revenue matters this year.

Clarifying questions to ask first

• How long has it run?
• Are heavy or light users losing time?
• Do we have a long-term holdout?

How to approach it

1. Short-term vs long-term effects
2. Which user segments lose time
3. Long-term holdout to detect slow decay
4. Decision rule and guardrails

Framework: Goal of the product -> Users and value exchange -> North Star -> Supporting metrics by funnel stage -> Guardrails and counter-metrics -> Decision or target

Sample answer

• Revenue per user over 8+ weeks
• Time spent by light users
• Week-over-week retention of new Reels users

1. The time-spent drop doubles to 4% among users with under 10 minutes a day
2. A 6-month holdout from a similar past change showed time spent decaying 0.5% a month
3. Ad click-through falls 6% per ad as load rises

Do not ship to everyone. Ship to heavy users where the time cost is small, keep light and new users at current load, and run a long-term holdout. Decide on 6-month revenue per user net of the projected time decay, with new-user retention as a hard guardrail.

What interviewers look for

• Revenue per user
• Time spent
• Long-term retention