The whole point of qualitative research (such as focus groups or in-person interviews) is to help developers move their products in the right direction. These practices leverage user feedback to squish bugs, fix any friction in the UI, add useful new features and optimise a mobile app for improved user experience and user flow. But can they be misleading?
Can User Feedback Bring More Harm Than Good?
Definitely. Especially if you are not careful.
User feedback can indeed be harmful. This is because people know what they want. They are just not good at voicing it. There is a quote that was reportedly said by Henry Ford (although this is debatable) that usually gets thrown around in discussions like these: “If I had asked people what they wanted, they would have said faster horses.” Yes, people know they want to go faster, they just do not know how.
If you thought it all comes down to the imagination, you would be partially right. Another reason is bias. Pure, scientifically-proven, bias. Our brains are wired in a way that we sometimes tend to ignore facts that do not fit our pre-conceived conclusions. We are quick to jump on bandwagons and stick to old conclusions, regardless of how irrelevant they might have become in the meantime.
That thing with horses? It is called ‘anchoring bias‘.
“Anchoring or focalism is a term used in psychology to describe the common human tendency to rely too heavily, or “anchor,” on one trait or piece of information when making decisions”, says Science Daily. In this example, the anchor is the horse, which humans of the time saw as the only means of transportation. All other conclusions are drawn from that fact.
But why is this relevant to mobile app professionals? It is relevant because they rely on their users’ feedback to shape and steer their apps. If their users’ opinions are biased (and they most likely are), and if app developers are not aware of that fact, they might end up trying to build faster horses, instead of building a car.
So the challenge for mobile app developers here is to differentiate between useful input and bias, and then optimise their products.
There is an app for that.
Well, sort of. Let us take a look at some of the primary forms of bias in user research, before moving onto possible solutions.
Main Biases in User Research
Usually, research bias can be split into two groups – bias from the respondents’ side, and bias from the researchers’ side.
As a user researcher, you want to make sure you are asking the right questions, the data you are getting is honest, and that you are not interpreting answers the wrong way. For this piece we will not go deeper into response interpretation. Instead, we will show you how to use technology to make sure you are getting honest, unbiased feedback from your respondents. In order to be able to use this technology properly, you first need to understand some of the more common types of biases that occur in research.
So let us take a look:
- Friendliness bias: The manifestation of this kind of bias is that respondents seem to agree and support anything the researcher claims. They like every idea, and would positively act on every stimulus. It happens for several reasons, including seeing the researcher as a professional and thus valuing their opinion too much, or being too tired of answering questions and just going with the principle of least effort.
- Social desirability bias: Ever since our ancestors dwelled in caves, being accepted into a group meant the difference between life and death. Being such an important issue, we have developed a subconscious drive to do whatever necessary to be accepted into a group. As a result, we sometimes tend to answer questions in a way we believe the team would consider best, regardless of what we thought of the question.
- Familiarity bias: People tend to give similar answers to questions with similar words. Seriously. Mostly, familiarity changes people’s perceptions. We are wired to choose familiar over the unknown when forming beliefs or making important decisions. If you give a person a choice of three answers, one of which is familiar to him/her regarding the words used, they might choose that answer. Familiarity bias is widespread – we see it hurting business investment decisions, programming decisions, even life-or-death decisions. It is, thus, only logical that we will see it in app research, as well.
- Funding bias: If respondents know what is being researched and who is trying to prove what, it will most likely influence the way they answer questions. When companies selling Cloud services do a market research, they often come to the conclusion that Cloud is the most important IT service there is, and that the sky would fall if everyone were not using it. Similar things happen with companies selling backup solutions, unified communications or collaboration tools. Also, companies outside the IT sector sometimes end up doing the same – their reports somehow usually support their interests. According to, well, everyone, this can sometimes be due to funding bias, or sponsorship bias. If the respondents in the survey know who is asking the question (for example, a Cloud company), and if they are aware that the Cloud company is trying to prove that Cloud is the most important IT service in the world today, their answers might be biased.
Conducting more in-depth research, you will probably find many other types of biases, but these four are generally considered the most influential. As mobile app professionals, you should keep both eyes open on these biases when analysing your app. If you ignore them, you might come to the wrong conclusions from your users’ feedback and steer your app in the wrong direction.
Eliminating Bias Through Qualitative Analytics
We can draw two conclusions from the four types of biases mentioned above:
- Respondents should not know who is doing the research
- Questions should not be worded similarly, and should not lead to an answer
If respondents do not know who is asking the questions, they cannot:
- Be (too) friendly
- Try to fit into a group
- Know what the goal of the research is
And if there are no questions asked, then the wording can never be similar.
So if you want feedback on your app, that feedback needs to be anonymous. Users should not even be aware that they are being asked questions, and those questions should not have words. Freaky, right? Indeed it is, but it is also entirely possible with qualitative analytics tools.
These tools come with features like user session recordings and touch heatmaps. As the name itself says, user session recordings are real-time, unfiltered, unbiased recordings of app sessions. With a tool that supports this, app pros can see just how their users are navigating through the app, without having to ask that question and risk a biased answer. They get to see which app features are used most, and which are being ignored. They can see if the app has bugs or crashes frequently. They can also better understand if certain UI elements frustrate the users and deduce why.
Touch heatmaps, on the other hand, is a tool which aggregates all interactions users have with an app, like swipes, taps or pinches. Again, anonymously and completely unfiltered, this tool combines all this data and creates a visual touch map, showing app pros exactly how users interact with the app.
With such a tool, app pros can get an honest, unbiased answer on questions like is their navigation intuitive, user-friendly, or even worse – does the navigation work properly at all? Unresponsive gestures occur when users try to interact with a non-intractable element, like a picture or a logo. For mobile apps, unresponsive gestures are a huge issue, which app pros look to remedy as fast as possible.
Elude Bias
User feedback is fundamental to building any solid product, mobile apps included. However, you need to make sure your respondents are giving you ‘clean’ information. Otherwise, you might end up steering your app down a cliff.
In this article, we showed you how new technology solutions could be used to supplement user research and negate biases. These solutions can help you recognise biases and get first-hand, unfiltered and unbiased answers regarding your mobile apps. Which conclusions you will draw out of the information pulled from qualitative analytics is something we will leave for another time.
Source: Usability Geek
Author: Hannah Levenson
