“You can’t just ask customers what they want and then try to give that to them. By the time you get it built, they’ll want something new.” – Steve Jobs
These quotes are both worth considering. When you get customer feedback, you don’t know if it’s the next thing that is going to make your business successful. Perhaps the feedback is just a random thought from a person who might not have the same concern the next time they open your product and use it. The feedback might also be the key to understanding what makes your customer tick. So what’s an intelligent, thoughtful person to do with user feedback. What should you do to balance your long-term vision against the thing your customer said today?
I believe you should start with empathy when you’re considering the customer’s feedback. When someone takes the time to give you feedback, it’s often because you failed them or because there is a mismatch of expectations between what you told them and what they experienced. Trying to understand what they are feeling and repeating that effort back to them is a very important way that you can learn more about your customer’s experience today.
Having the presence of mind to be in the moment with that customer and focus on what they are thinking and saying right now is another key way that you can address their feedback. As Field suggested so many years ago, the Customer is Right not because your business needs to change drastically but because your business is the sum of small relationships that continue over time. Your customer’s feedback is a sign that the relationship can be nurtured and that it’s your job to figure out how to make it better for them. If you can’t make it better for the customer, you owe it to them to explain what you can do and to make the steps of the process as transparent as possible.
Does this mean you are going to be able to make things right for every customer? Is every customer’s feedback relevant to your long-term vision? Maybe. It’s really hard to know at the moment of receiving the feedback, listening to it, acting upon it, and finishing whether the customer’s feedback is an issue for the long term. And you do know it is an issue for today: so start today by responding to that customer in the best way possible.
And a note on vision. After a while, you’ll start to have a “sixth sense” or internal compass that will tell you whether the customer’s feedback should and does fit in your long term vision. You and your organization get bonus points if you have a living document that explains the tenets you believe in, and if you actively debate those tenets to arrive at a vision matching some of what your customer tells you every day. The customer will not always be right, and they will always be more right than you in letting you know what they want today. It’s your job to figure out how that fits into the longer term vision.
This post is part of the Startup Edition writing collaborative. Read more about this here.
When the folks at LittleBits asked me to test their latest kit of snap-together DIY electronics, I was really excited. As a gadget-loving person, I love the idea of creating a Rube Goldbergian perpetual motion machine. I even the idea of putting together models and understanding how electronic circuits work. Yet I’ve never gotten around to learning how to solder, putting electronics boards together by hand, and building electronics from the ground up.
LittleBits prides itself as being an educational, open company, and you can even dream up new bits in their GitHub repo. The company produces sets of “bits” – individual units that perform different functions and snap together with magnets – that you can use to make projects. It’s dead simple. Bits come for Power (e.g. a battery or USB), Input (a dimmer or motion switch), Wire (connect all of that stuff together and branch it if necessary), and Output (a buzzer, motor, light, or fan).
LittleBits is great because it gives you the feeling of building electronic gizmos from scratch while providing a color-coded, magnetized set of pieces that you just can’t screw up. Buck Rogers ingenuity with no-brainer simplicity and instant benefit: I love it. The sets are deceptively simple because you start playing with something and how it goes together, and then almost instantly rearrange it into a new configuration because there is almost no switching cost to putting it back together again. Need a diagram? Take a quick photo of the pieces snapped together.
LittleBits allow you to play with electronics in the same way you might sketch a quick drawing, play a piece on an instrument or just goof around. They are a great way to learn about circuits and motors visually – there are lights, pressure switches, motion sensors, and lots of fun switches to make your “stay out of my room motion sensing alarm with buzzer” device, or just to make the simple morse code machine my assistant and I built tonight. It’s time to build more what-is-that-thing-calleds.
(disclaimer: LittleBits provided me with the Extended Kit free of charge in exchange for help testing the product. My young assistant and I are still doing the testing and are enjoying ourselves profusely.)
Let’s say you’re starting a new company, feature, or product. You have an idea that you want to test with some beta customers. So what would you do today to “get out of the building” (in best Steve Blank Lean Startup style) once you’ve done some initial customer development to determine your Minimum Viable Product and have some ideas and data about the kind of customer who might use or provide feedback on your idea? One way to do this to set up a beta program, where you combine your ideas, your preliminary feedback (and some actual people) to see what will happen. (The feedback will likely be both sweet, and a bit tart.)
What are the goals of a beta program?
At face value, the goals of a beta program seem simple: find some potential customers, ask them to use the product, identify bugs, and get feedback on what’s working and what’s not working in a defined period of time. “Customers” probably look the same as the initial persona you identified during your customer development phase, and since you’re looking for a directional indication at this point, you don’t have to get them completely right (the beta program is an extension of your customer development efforts.) “Using the product” and “defining bugs” will work best if you define a few tight scripts at first to help people understand your vision of what they should be doing, and not just what it looks like in your prototype. And “getting feedback” means finding out the most important thing to improve or fix at that point of your development process.
The Perils of Talking to Customers You Know
Finding potential customers is the first step, and also not the easiest. When you start, you’re likely to ask people you know – which is great because they will be more receptive and accommodating of problems, and not so great because they’re biased to give you good feedback – so you need a mix of people you know and don’t know. One way to solve this problem is to ask the people you know to recommend 2-5 people that they know who will provide practical feedback and who don’t know you all that well. Once your group reaches 30 people, you’ll be able to know more confidently that you have at least directional statistical information.
Practical tip: manage the list of customers in a Google Spreadsheet, identifying their name, email, Twitter handle, external ID in your system, “customer type” (this could be ‘experienced, noob, mainstream’ or another taxonomy), the date they joined the beta or the identifier for their cohort group, a comment field, and a “last contacted” date. This will allow you to model the list of beta customers by cohort, give you a way to communicate with them, and provide you with a data structure you can use to pivot their feedback by customer type, externalID, beta cohort, and date range.
Using the Product is Not Following The Instructions
It’s tempting to think that customers (any customers) will “read the manual” and follow your instructions to the “T”. Well, when was the last time you read the manual? It’s important when considering your beta group to include both those people who are likely to follow your instructions, those who are not very imaginative and who just want to “hire your product” to do a job for them, and those who want to break your product or will think of unusual ways to interact. And if you want really great bug descriptions, you need to make it really easy for customers to provide inline feedback and to give them a prompt every time to identify the key items you need to understand what’s going on.
Practical tip: provide instructions for your scenario using multiple learning styles, including listing items in an ordered list (“Step 1, Step 2, Step 3”), asking an open-ended question (“What’s the 1 thing you’d like us to improve”), and communicating in other media, e.g. asking them to record a Skype conversation, a screencast, or setting up a group Google+ Hangout to discuss the “how do you do it” aspect of your product.
The “90-9-1” rule for participation inequality suggests that some of your beta participants are going to give you a LOT of information, and the feedback they provide will be overweighted toward the one or five or ten people who feel really passionate in your first group of 100 or so beta participants. So how can you take what they’re telling you and make it more meaningful? First, you should identify the functional bugs they point out, and fix those: these are potential blockers for any new customers. If you’re not going to fix an issue, log it and let the reporter know you’re placing it in a lower priority queue (and then once every month or so, prune the queue aggressively to remove the “not-urgent not-important” items.) Second, you can use a Kanban or other scrum technique to organize the volume and priority of the work (Asana, Trello, Do, Jira, Pivotal, and others are all good for this task). And third, keep asking your beta testers short surveys frequently (1-3 minutes) to see whether what you think you’re doing is actually working for them.
Practical tip: use some bug tracking software to manage this process. Don’t reinvent the wheel: if you don’t like Jira, Bugzilla, or another solution you can always find a bug tracking template in another tool. And make sure you identify the type of customer for which you’re solving the problem; the severity of the issue (does it stop them from using the product; is it a major deficiency; or is it just nice to have) and the priority (get it done now, get it done soon, log it to see if it becomes more important later).
What’s realistic to expect from your results?
It’s reasonable to think you’ll get feedback, and that some of your customers will like (or even love) what you’re working on for them. And it’s also likely that some will dislike, or even hate, the thing you think is cool. To get the most mileage out of this feedback, use a dedicated email alias or distribution list (e.g. beta-feedback@) to share this information with as wide a group as possible within your company. And view the feedback in the context of the type of customer who’s providing that feedback. When you get positive feedback clustered among multiple people in the same persona in the same area of your product, that’s a good sign you’re heading in the right direction. And then use that feedback as a data input to make decisions: what kind of company do you want to be? Which customers are you serving, and will you serve them better by improving this feature or fixing this bug? If the answer is yes, it’s time to invest time and money in making that change.
One of the best ways to learn how real people view your product is to ask them to complete a set of tasks that you think all customers “should” be able to do. Think of this as a directional usability test, where you can get some feedback on the way “normal” folks use your product without sitting right next to them and telling them how to complete the task. Yet you can also learn a lot by sitting in the same room as someone who has tried your product and just having a conversation. Even if these people are not perfect examples of your persona definitions, setting up “Friends of the Company” sessions are a great way to make a tremendous leap in usability in a short period of time.
“Friends of the Company” sessions might look like this: every two weeks, line up two or three people to visit your office and ask them to complete a common customer task (set up an account, use the product the way they “normally”, and talk through the progress as they do it.) You should have someone from your design team, your engineering team, and your executive team in attendance, and make sure to give the person some homework before they arrive so that you can capture their feedback.
When your F.O.C. session is running, you should use this focused time to listen, learn, and suggest. You can listen by hearing what a “typical” customer does when you’re not around and hear more about the features that people outside of your building think are pain-killers, not vitamins. You can learn by identifying “cringe” moments that show up during the session, and plan which of these items to address and which to log for later effort. And you can suggest by using this time with a customer to bring up ideas that need additional feedback.
It’s important to note that the feedback you receive in these sessions is just that: feedback. It’s not usually enough to make major changes in usability, and it is an amazing way, however, to note little items that trip customers up when you think they should be able to complete (what you consider to be) routine tasks. Friends of the Company sessions give you a temperature reading of customers and let you know what those people are thinking and whether your message matches their experience with the product.
And matching that message to the product is an important task that’s very easy to practice during the F.O.C. Session. Remember, some of the people who are coming to see you are very talented and want to help, and some are just there to see what you’re up to in building your product and culture. All of this feedback can be really useful if you use it as a opportunity to refine your pitch, your usability, and the real-world functionality of your product.
In Steve Blank’s excellent blog post on Startup First Principles, he describes a startup as “an organization built to search for a repeatable and scalable business model.” A key driver in building a repeatable and scalable business model is to find the kind of (friendly and unfriendly) customers who are not only willing and able to try partially completed software but also capable of providing relevant, actionable feedback. These beta customers might not be your ultimate customers – or they might be exactly the customers you’re looking for – and finding them and getting them to try beta products is a key requirement to give you the information you need to make business decisions.
Information Maven 