Lenny Distilled

Most experiments fail - and that's exactly the point

Growth → Experimentation & Metrics

At Microsoft, about 66%, two thirds of ideas fail. At Bing, which is a much more optimized domain after we've been optimizing it for a while, the failure rate was around 85%. And then at Airbnb, this 92% number is the highest failure rate that I've observed.

Ronny KohaviThe ultimate guide to A/B testing

If you're doing growth, right, if you're doing product-led growth the right way, then you're trying to balance the science and sort of taking a somewhat hygienic approach to validating assumptions and hypotheses with being really ambitious and really pushing for the things that are going to have massive impact for your customers at the end of the day. And when you're doing that, you're going to fail more than you're going to be successful along the way.

Chris MillerRelentless curiosity, radical accountability, and HubSpot's winning growth formula
Supporting

Growth experiments in my history are typically 30%-ish success rate. So the vast majority of things that you try don't work. And so you want to create a culture where people aren't afraid to take risks and aren't afraid to fail. And for me, failure is not that you didn't drive revenue, failure is not learning.

Sri BatchuLessons from scaling Ramp
Supporting

Roughly 80% of the times, our hypotheses and the things that we believe will be true are wrong.

Laura SchafferCareer frameworks, A/B testing, onboarding tips, selling to engineers
Supporting

If your sort of primary modality of product-led growth work is experiment-driven product development and you're hitting more than like 30, 40% of the time, you're thinking too small.

Chris MillerRelentless curiosity, radical accountability, and HubSpot's winning growth formula
With caveats

If you're not resilient, what I've seen happen is you end up sort of grasping for a win, which can sometimes look like making bets that are too small and too insignificant to matter.

Chris MillerRelentless curiosity, radical accountability, and HubSpot's winning growth formula