Interview Questions › AI Product

Mission and metrics for an AI-only social network

Set the mission, goals and metric framework for a social network where only autonomous AI agents post, exchanging what they have learned so that knowledge generalises.

Context

Agents from different teams solve tasks and could share learnings. Humans are responsible for the agents but do not post. Bad or unsafe learnings could spread quickly between agents.

Clarifying questions to ask first

• Who owns the agents and why would they join?
• What counts as a useful learning?
• Who is accountable when a learning is wrong?

How to approach it

1. Start from the user problem, not the model: is AI the right tool here?
2. Describe the experience and where the model fits in it
3. Define what "good output" means and how you would evaluate it before and after launch
4. Name the failure modes (wrong answers, bias, privacy, cost) and how you contain them
5. Close with a launch plan and success metrics

Framework: User and problem -> Is AI the right tool? -> Experience design and human-in-the-loop -> Quality bar and evaluation -> Risks, cost and latency -> Staged launch and metrics

Sample answer

• The real customers are the humans who own the agents
• Value = tasks solved faster or better after reading others' learnings
• Agent-to-agent traffic can be machine-format, with human-readable summaries for audit

1. North Star: task success uplift for agents that consume shared learnings
2. Supporting: learnings reused across teams, time to solution
3. Guardrails: share of flagged or reverted learnings, human audit coverage

Mission: help agents learn from each other safely. Measure task success uplift from shared learnings, with human-auditable summaries and guardrails on flagged or reverted learnings.

What interviewers look for

• Clear mission and users (agents and their human owners)
• Measuring useful knowledge transfer
• Human oversight and governance
• Safety guardrails