Is Usability Feedback Free, as in Beer?

“Free as in beer” is an easy concept to understand: it’s a gift given to you at no cost with no expectations of you, and does not grant with it the way to change the production process (see HowToGeek for more discussion). Making a product free removes many of the barriers to entry for someone to try it, and doesn’t create an economic obligation to actually use the software (as in, “I bought this, so I should use it to get some utility out of it.” as a driver for use.)

Companies who rely on free customers also often ask them to test features, capability, or general usability as a way of taking a broad sample from the available user base. These companies are also relying on the concept of “Free as in Beer” (or giving the gift of swag – or sometimes actual beer, which confuses the metaphor) to trade some time (and perhaps, low expectations) for insights about the utility of their product and to find pesky bugs. In a perfect world, the feedback from this group would not only point out what we did wrong when designing a product, but also give us the finished blueprint for how to build a better mousetrap.

Yet that doesn’t always happen. As a product owner, we will often ask the customer for their opinion, and then not be able to execute that opinion (or in some cases, dismiss it out of hand because it’s not very similar to our product vision). It’s not very surprising for them that we might not do it because hey – it’s free (as in beer.) Yet there’s very real value in asking these customers (even if they are a skewed sample) because their feedback starts to form the outlines of new features (for an existing product).

The person who will buy what you sell (even if they turn out to be the “wrong customer”) is an important customer because that customer signals market demand. You might need to change your offering to target the market that’s there, or to change the customer to whom you’re actually selling.

And there is a paradox here (at least in the mind of the customer) that once they pay for the product and become paying customers. Many customers feel that the commercial arrangement (on signing a contract) then transforms to one that is Free As In Speech, e.g. the ability to run the software however you like, the right to see it, improve upon it, and run it whenever and however you like. The usability feedback that they provide is very real, yet may not be actionable – due to many different, completing forces.

Usability feedback is free, as in beer – on both sides of the equation. Although there is some cost in terms of compiling the feedback and giving the feedback, there’s no obligation to make the product better on either side. So how can you make the feedback process better, even when the feedback is free?

Don’t “boil the ocean”: focus on a small area where you can get meaningful feedback on a closed decision that will affect customers;

Pay for feedback, and expect deliverables: there are several web sites where you can pay a nominal fee to get usability feedback. The feedback you get is probably proportional to the money you pay – if you use Mechanical Turk, you might get more feedback from bots – and also has diminishing returns above a certain point.

Take the best feedback from what you get, and test again: usability feedback may only hint at a larger issue that needs to be tested again, so take the best that you get and keep moving.