A Conversation With A Bootstrapping EdTech Startup On Customer Interviews

What follows is a sequence of E-mails with an entrepreneur bootstrapping an EdTech startup around the challenges of doing customer interviews that have been recast as a conversation, with the original content edited for length and clarity.

Entrepreneur (E): I am working with a couple of friends–we all have day jobs–on an idea for helping students improve their completion of on-line courses, for example from coursera.org or udacity.com. We have a persona for our target student, whom we call Hazel. Hazel is in college and very motivated to learn programming techniques that she aims to integrate into her current degree and later employment. She has taken several on-line courses but has had poor completion.

Sean Murphy (SKM): For Hazel to be willing to talk to you–much less try out your offering–she will need to recognize that she has a problem and believe you have a potential solution. What is her perspective on how well she is doing in her courses?

E: Hazel has enrolled in about half a dozen courses in the last nine months (since Spring Quarter of last year, mainly over the summer) but has completed only one. She feel badly about her lack of progress and has tried a few things to improve her consistency but so far nothing has worked.

SKM: Does she believe he has a problem or need to improve? Why?

E: She is painfully aware of her drop out rate, when she talks to her friends she complains about having to study alone compared to study groups she is able to take part in with her regular college courses.

SKM: What key capability is she looking for to improve her performance?

E: She wants the accountability and peer support she enjoys in her regular college classes.

SKM: I understand what you mean by “accountability and peer support” but are there some specific symptoms of poor performance that she would recognize and acknowledge that you can probe for to get an interview?

E: Have you registered for more than three on-line courses and failed to complete them? Do you take part in informal study groups for your college classes?

SKM: Those sound like good probes using the same language that a student would. I worry that since you are bootstrapping you might want to target folks who have more money or motivation than college students. You want your customers to get more value out of on-line courses; I would be tempted to target people who are already employed who are adding to skills to get a promotion or a new job.

E: We have another persona, Edward, who looks like that but we have decided to start with Hazel. We will ultimately interview people who fit both personas.

SKM: What are your challenges trying to find students who match Hazel?

E: Because we are all working it’s hard to schedule interviews with college students; do you think they need to be face to face?

SKM: I think you will learn a lot more in a synchronous conversation: face to face will tell you the most but a phone call or a Skype call also works, even a chat session is often more useful than an exchange of emails. My last choice would be a survey.  How have you been reaching out to folks to interview?

E: We have been posting in the on-line course forums when we see someone post something that looks like they are having trouble completing the course:

“I read your message <Title> on <Forum> at <URL> and I’m very interested your take on <Course> in particular and your experience with on-line learning in general. Do you have 15 minutes for a Skype call to share your opinion on online courses? A few friends and I, after having some problems with a course we took together, started working on something we believe will improve online learning and we’d love to hear your take on it. If you don’t have the time for a call, would you be willing to share your insight over email? We also have this survey <URL> that only takes 3 minutes to complete if you’d prefer.”

SKM: 15 minutes is a big ask for a stranger and you don’t offer any specifics in your e-mail as to how you may be able to help or enable them to do some self-diagnosis to see if your solution may be something that they are looking for. It’s not clear how they will benefit from the conversation.

E: People can be pretty eager to tell you what they think about something they care about so I thought that in itself might be enough of a driver. But we have had a low response rate when we asked just for a Skype call. We have more people either giving us a short reply over e-mail or taking the survey. Would it be better to drop the other options and just suggest a Skype call?

SKM: I would offer Skype and E-mail. An exchange of E-mail can lead to a Skype conversation; I think it’s harder to move from a survey to a conversation.  You might consider blogging about ways to improve your performance/results/learning from an on-line course: outlining methods or techniques that would complement your solution. This would offer some credibility that you can help students and give you something to point to in the offer letter.

E: Blogging sounds like it would take a lot of time, and time is a pretty big constraint for us. How high would you rank blogging compared to other activities related to customer interviews? How about offering a gift card? $20 for a gift card is totally worth an hour of my time to author a decent blog post. I understand a blog post might have less perishable value in the long run.

SKM: You can share the same “decent blog post” with many people who may each value it and who can each share it with friends they think it may help. A gift card is proof that you value their time but does not substantiate any expertise or offer direct assistance in addressing the problem or need your solution is targeting.

Let me suggest another way to look at your question. Would you rather interview?

1. Ten college students for 5 minutes who read your blog post “10 tips for finishing an on-line course” and are interested in talking to you about their challenges in completing on-line courses
2. Ten college students for 15 minutes who heard that you were giving away $20 gift cards as a reward for an interview.

I think many in the first group may talk to you for more than five minutes if they don’t feel that the first five was a waste of their time. It may be hard to talk to few in the second group without promising another gift card.  Because I work primarily in B2B niche markets and have to cherish any prospects I come across I don’t look at informational interviews as transactions but the start of a potential business relationship.

There is a trap that some entrepreneurs fall into: they look at prospects they are interviewing as a consumable–somewhat akin to the way a scientist might budget for lab rats–instead of potential customers who will require several conversations of escalating mutual disclosure to establish a business relationship with.

E: I understand your point, but it seems that it may be worth it in some circumstances to shortcut that process. I wouldn’t think that one blog post with some useful tips would be enough to repay the person for the time I’m asking for. Yes, it’s certainly better than nothing, but does it really help to move the needle? I can see that providing great content and supporting Hazel could help me to understand her needs better and led to us doing business. But is trying to close her and get her to become a customer the goal here?

SKM: I work in niche markets and cannot afford to look at any potential relationship as disposable. Try both, pay some people and see if you can write something that offers enough insight that folks feel like giving you five minutes of their time. What you do in that five minutes may encourage them to give you another five minutes. That’s how relationships are built. Not everyone you talk to turns out to be a fit and many of those that are don’t care to continue the conversation. But I like to engage in a way that does not preclude further conversation.

E: I know I don’t have a lot of experience yet doing interviews so I’m looking for guidance. But on paper it would seem worthwhile to try and get some feedback as quickly as possible on the potential validity of the idea and then start to build some more “content” and stronger relationships. Where is the risk in that approach?

SKM: If you are in a hurry it can work against empathy and appreciative inquiry, both of which are critical to forming a deep understanding of a prospect’s situation and needs.

E: One of our values as a business is to be grateful to anyone who tries to help us and to treat them with respect. And even though it can hurt when people decide not to have a conversation, it’s still a useful signal so we respect that decision as well. We don’t treat anyone as a lab rat and try to help him or her as best we can, but this doesn’t seem to preclude the possibility of an initial shallow conversation. Now whether that lack of depth is an issue in building my business or not that I’m not really sure.

SKM: I had two additional thoughts for you:

1. I don’t think Hazel views her problem as an inability to finish on-line courses. She is signing up for those courses to meet some need or to enable an opportunity. I would dig a little deeper into what she will do once she finishes the course.
2. An on-line class provider may view their “dropout rate” as a problem and they may be the real customer.

[We adjourned for two weeks and then had a second shorter exchange.]

E: We decided to opt for the “blog post” option. We also managed to go to some evening Meetups and tried to add value for people right there while talking to them. That worked surprisingly well. Hopefully it’ll continue that way. I still feel like there’s a place for the paid interviews, maybe even as an intermediary step between face to face interviews and flat out surveys.

SKM: When you can find a way for your discovery conversations to offer value to a prospect, then they are more likely to share information and more willing to agree to continue the conversation or agree to a second conversation. One test for interest is that it becomes less of an interview (or worse an interrogation where you are asking all of the questions) and more of a dialog where you are both asking questions of each other.

E: As far as your additional thoughts, we now believe that you were correct on both accounts and based on the last few rounds of interviews we have a better idea of Hazel’s reasons for not finishing the course. We are now trying to determine how that intersects with the course provider’s perception of that problem: the course provider is interested in drop out rate but Hazel is not really focused on dropping out.

We are now trying to get a better understanding of Hazel’s goals in taking the course so that we have a shot at helping her succeed. In parallel we are trying to figure out how to have some conversations with the course providers. If we decided that selling to the course providers was the way to monetize this, would you talk to them first? Even if we are not sure we can find a way to help Hazel improve her odds of finishing the course?

SKM: If you plan to sell to course providers I would try and gain a better appreciation for their perspective and their business model: e.g. what are cost and revenue drivers; what do they view as the key risks in launching a new course or making a profit on a course.

One possibility to consider is that they may be less interested in helping Hazel and more interested in how to predict who will complete the course, in particular at or before signup, and how to attract more of those students. In other words: if the course provider is your customer I would figure out if they would pay me to help Hazel complete the course before I invested effort figuring out how to help Hazel complete the course.

If Hazel is the paying customer then you have to determine if she would pay you something to help her complete a “free” course: again I would focus on this question before trying to dig too deeply into how to help her complete the course. Prospects are often more interested in outcomes, costs, and timeframes than the details of your solution. One good book on this is “Great Demo” by Peter Cohan, who suggests that you open very quickly with the “ta da” or final result and see if the prospect wants it before diving into the details.