What’s the point of trying a beta product?

Beta Customers are the Best Test Customers You Have

In Steve Blank’s excellent blog post on Startup First Principles, he describes a startup as “an organization built to search for a repeatable and scalable business model.” A key driver in building a repeatable and scalable business model is to find the kind of (friendly and unfriendly) customers who are not only willing and able to try partially completed software but also capable of providing relevant, actionable feedback. These beta customers might not be your ultimate customers – or they might be exactly the customers you’re looking for – and finding them and getting them to try beta products is a key requirement to give you the information you need to make business decisions.

When thinking about beta customers – especially early in the cycle of creating ideas and getting feedback on those ideas – it might be good to source your customers both from among people who are experienced “triers of beta product” and also those who are more natural matches for customers in your potential target markets. The former can give you hard-nosed feedback about what a product in that general industry area and type should be able to do in addition to content and usability feedback, and the latter will be more effective at helping you understand whether you have product-market fit. But it’s often hard to find these characteristics in the same person – both the ability to provide effective, actionable feedback, and the persona to match your target market.

Why don’t more people try new things? (Or, why isn’t it easy to find people to try your stuff?)

Nifty, right? It’s cool to try new things. It can also be frustrating to experience the (very visible and tangible) gap between the MVP (Minimum Viable Product) you are being asked to try as a beta customer and the fully realized vision in your head – or in the descriptions of the product shared by the people who are asking you to test a beta (or alpha product). Yet don’t despair – the kind of feedback you’re providing is exactly what product builders and designers need to hear to – in Steve Blank’s words – “get out of the building” so that they can reconcile the ideas they have about a product and market and combine those with the attitudes and behaviors of potential real customers who have a real personal or business problem to hire.

It turns out that it’s hard for the average software or product user to tell you what they like and don’t like, what they were doing when that happened, and to suggest ways that might make them more likely to use your product in the future. In addition, your more seasoned product user may be able to give you solid feedback on how to use the service (and what feels “off”) but may not know very much about the actual business problems faced by a user in a target market.

What can you do as an interested product builder or designer to make this process better?

Customers (and testers alike) usually have a limited amount of time to send on your orocess or idea. so start by giving them a summary (“We’re trying to build x for y; we think that you should be able to problem solution statement here, and we think that the whole process should take xx minutes”). Be concrete enough to allow them to know whether are done, and to open the opportunity for the customer to use your software or product more if they want.

Here are a few techniques for getting more (and more actionable) feedback:

• Ask customers to do something – and then ask them to tell you how they felt when they were doing it.
• Ask those same customers to show something – you how they think they should be using the product
• Ask for just one more thing … If there is only one thing that you can change, you’ll be more economical about asking for it. So ask the customer to spend their virtual money (or ideas) on that one thing.

And don’t forget, thank them for the feedback, and offer to provide some follow up on the results of what they helped you learn. It doesn’t have to be huge and detailed feedback – but it’s always nice to know that your effort helped from the perspective of the tester.

What can you do as a beta tester to make this process easier?

As a tester, if you focus on the biggest small thing that can be fixed, and also compare what you think you should be doing to what the software or customer flow is asking you to do, you’ll provide great value for a product team. It may be obvious, but just saying “it doesn’t feel right”, or “I don’t want to use it” is less useful than “I am trying to replace an existing process that does x and y and the way you want me to think about that process is missing some things I’m used to seeing, like a and b and c.”

Your goal should be to give them concrete observations to which they can respond and decide whether or not to address in the product. If you’re thinking with the target user in mind, you might also avoid the interesting but not terribly useful feedback that pops out of your head (An aside here: your “outlier” feedback might point the team in a new and useful direction or confirm assumptions that they have, so don’t censor yourself.)

How can you bridge the gap between customers and skilled testers?

So therein lies the problem for a startup – bridging the gap between skilled testers who can provide great feedback but might not have the business problem, and potential customers who can better inform you of how close you are to true product-market fit.

Here are a few ways that you can get information from pilot customers and give them less to do:

• Use in-app surveys (Testflight, Uservoice, Desk.com, and Intercom.io) are some of the tools you might consider here
• Instrument the product into tasks that define the stages of user activity, and measure them
• Ask customers to rank the things they like best (top 5, 3, or 1) and focus more on those things

It also helps to remember that when you test a product, you are trying an uncertain product pointed at an uncertain customer occupying an uncertain market with an uncertain likelihood of success. There might be a lot of variance between that and your ideal product or service, or even the same product 6 months from now. So be flexible. and if you don’t like it, go on and provide advice to the next beta product or service that catches your eye.