Which users do we include in our user testing, and why?

When rating something new and unknown to us, we look to familiar anchor points for reference. This can result in evaluations that don’t do justice to the new and unfamiliar. The best-known quote in this respect stems from Henry Ford: “If I’d asked people what they wanted, they would have said faster horses.”

So, is it worth doing interviews on innovations? Yes, it is – but to really understand the responses, it’s important to consider from whom you’re likely to obtain meaningful information, what questions you should ask and which topics you should include. You also need to think about how to ask your questions to avoid ‘selling’ the innovation or overwhelming the user. In this post we discuss how to select the right people to participate in user testing:

Defining the ‘right’ users

In the early stages of developing an innovation, it makes sense to invite lead users, heavy users or users with specific know-how who will complement the expertise of the innovation team. These are often the early adopters, i.e. the first to try out a new product or an innovative service. These people tend to be very familiar with a specific item, product field, or sector and are able to reflect on their views, attitudes and evaluations, thereby giving well-founded information. The innovation team can then use this to acquire a good understanding of user habits and how users handle something new.

So never use ‘average’ users?

To successfully establish an innovative product with appeal that goes beyond the early adopter niche, one must weigh up the potential within the market as a whole. Hence there comes a time when ‘average’ users need to be included in the innovation development process.

Based on our experience with this version of user testing, we advise paving the way for these users to approach the topic.  When dealing with future developments, this means attuning the participants to scenarios that could become reality further down the line to help them assess the test object in a (likely) future setting rather than in the here and now. It’s vital here to understand existing habits and allow for the necessary learning and relearning processes. The interviewing technique must also be adjusted accordingly to obtain reliable statements. Only then can the risk of deciding on future products or services based on artefacts be minimised.

What general considerations are there when it comes to conception?

JU-KNOW considers three aspects important:

I
High-quality, reliable respondent recruitment based on careful definition of the target persons. To this end, we deploy a proven and efficient screening procedure for our projects. We don’t think it expedient to interview randomly selected individuals. After all, the innovations are being developed for the subsequent user and not for the innovation team.

II
A sound interview concept that covers the main topics, key questions and sub-items while leaving enough scope for the respective respondents to come up with their own analysis. This must be coupled with interviewers skilled in the established interviewing techniques that allow them to uncover even hidden motives and unconscious reactions.

III
Allow for the interview to broach topics that throw up information in support of the innovation. Margaret Mead, the ethnologist, realised that “What people say, what people do and what people say they do are entirely different things.”. This expresses in a nutshell a vital aspect of user testing. Our consulting projects reveal time and time again how companies launched an innovation based on an enthusiastic reception during user testing, only to see it ultimately flop. We therefore warn against using basic questions such as “Would you purchase this product?”, or “How much would you pay for it?” – and instead recommend valid interviewing techniques that are more likely to deliver sustainable insights for decision making.

We support innovation teams with ‘How to’ Learning & Coaching or with our ‘User Testing’ and ”Validating the Winner’ modules.