Experiments Vs. Commitments

Distinguish between experiments and commitments with customers. Share prototypes and ask for feedback but keep your promises. The easiest way to keep a promise is not to make one, so take great care in what you sign up for.

Experiments vs. Commitments

Seth Godin’s post today on “A Hierarchy of Failure Worth Having”  crystallized one of my concerns with some recent startup practices. Godin outlines the following desirable hierarchy of failures:

FAIL OFTEN: Ideas that challenge the status quo. Proposals. Brainstorms. Concepts that open doors.

FAIL FREQUENTLY: Prototypes. Spreadsheets. Sample ads and copy.

FAIL OCCASIONALLY: Working mockups. Playtesting sessions. Board meetings.

FAIL RARELY: Interactions with small groups of actual users and customers.

FAIL NEVER: Keeping promises to your constituents.

I think he gets it exactly correct. And I think a number of entrepreneurs, in particular in the early market, get it almost exactly backward by

• Putting up a landing page that promises a capability or extra feature that doesn’t exist.
• Focusing more on a potential solution when talking with prospects–using them to prototype– instead of ensuring that they really understand the prospect’s perspective on the problem.
• Looking for funding before they look for customers, using investor interest as validation for their business concept.

Godin concludes:

Most organizations do precisely the opposite…They rarely take the pro-active steps necessary to fail quietly, and often, in private, in advance, when there’s still time to make things better.

Better to have a difficult conversation now than a failed customer interaction later.

The foundation of a successful business is the ability to make and meet commitments to customers, partners, employees, suppliers, and other stakeholders.  If you inform them in advance, “we are going to try the following experiment” they may or may not take part, but  they can offer informed consent. I see too many instances more recently where founders undervalue a relationship with a customer based on mutual trust and commitment.

Related Blog Posts

• Honor Customer Commitments To Avoid Poisoning the Well
• Expiration Dates: Deadlines and Commitments
• We Help You Design Experiments That Move Your Business Forward
• If We Killed Our Product Would You Miss It?

Follow up Questions from Comments

Mariya Genzel: Sean, love reading your stuff on LS list, but not sure I agree with this. It seems to me that the only way to ensure that failure is infrequent for the kinds of failures it’s recommended above is to put in the kind of non-minimal development time that would go against the grain of the entire LS movement.

Sean: Put a mockup of your feature on a slide or a piece of paper and have a conversation.

Mike A. Smith: So am I correct in thinking you disagree with Eric Ries’ suggestion that the Minimum Viable Product can be a landing page to gauge interest? I also recall reading/hearing an interview with him where he described a similar practice at IMVU. They would intentionally include links/access to features that didn’t exist, yet, within the application. When a user tried to access the new feature, they’d receive a message saying it was temporarily unavailable due to a randomly generated reason. They used this technique to prioritize their internal backlog of features. Items that lots of users clicked on went to the top of the queue in development.

I recognize that this was a trade-off in customer disappointment vs. scarce development time, and it was really a gamble on their part that they wouldn’t upset that many customers in the time it would take them to develop and deploy the new feature. Are you saying that gamble is never worth it?

Sean: In B2B markets I am saying that the downside far exceeds any gain to the point that the gamble is never worth it.