Spotify Confidence review: features, pricing, and AI capabilities

Platform overview

Confidence is warehouse-native experimentation from Spotify. Its AI story runs through MCP rather than in-product assistants.

It runs two vendor-hosted streamable-HTTP MCP servers. The Documentation MCP at https://mcp.confidence.dev/mcp/docs needs no authentication and offers semantic search with maximum-marginal-relevance ranking, regex search, full-source retrieval and SDK integration guidance. The authenticated Flag Management MCP at https://mcp.confidence.dev/mcp/flags exposes nine tools: list, read, create and modify flags, add variants, update schemas, create override rules, test flag resolution with an explanation of why a variant was selected, and find unused or fully rolled-out flags.

Flags carry structured schemas with named variants resolved against evaluation context, and applied-flag events are written to the warehouse for later analysis. Spotify's public line is that faster AI-written code increases the need for experimentation and guardrails rather than reducing it. No generative-variant, bandit or autotune capability is documented.