Interview Questions › Metrics

Guest reviews submitted fell after a redesign

After Airbnb shortened the review form, the share of stays that get a guest review went from 72% to 75%, but average review length fell by half. Is this a good change? How would you decide?

Context

Reviews build trust for future guests and give hosts feedback.

Clarifying questions to ask first

• What do future guests use reviews for?
• Do we know how review content affects booking?

How to approach it

1. Why reviews matter: booking conversion for future guests
2. Quantity vs quality
3. Measure downstream impact
4. Decision

Framework: Goal of the product -> Users and value exchange -> North Star -> Supporting metrics by funnel stage -> Guardrails and counter-metrics -> Decision or target

Sample answer

• Booking conversion on listings by review count and detail
• Share of reviews mentioning specifics (cleanliness, location)
• Host response to feedback

1. Short reviews mention a specific detail 40% less often
2. Booking conversion is unchanged so far on affected listings
3. New listings gain their first 3 reviews 5 days faster

Keep the shorter form because faster first reviews help new listings convert, and conversion is unchanged. Add an optional prompt for one specific detail to restore useful content. Judge by listing conversion and review helpfulness, not review length.

What interviewers look for

• Review submission rate
• Review helpfulness
• Listing conversion