Measure twice and cut once is a universal recommendation that works both for tailors and product designers. The only way to know if your product is doing well is to start measuring design success. Yet, measuring success might not that easy, especially if you’ve just recently started to do it. You can struggle with one of the following five misconceptions about measuring UX:

1. UX can be measured with a single metric

“How are we doing?” Many businesses believe that it is possible to answer this question with a single metric. NPS is notoriously known as a metric that businesses rely on to evaluate the current state of things. They believe that a high NPS score means great UX.

In reality, UX is multidimensional, and it requires multiple qualitative and quantitative metrics like task success rate, user satisfaction score, and summary of user reviews. NPS alone doesn’t cover these areas.

Also, when you rely on one metric, the risk of data manipulation increases. In other words, NPS might be gamed.

Campbell’s law states that the more important a metric is in decision-making, the more likely it is to be manipulated

Because executives like this metric, the product team might be forced to introduce practices that ‘guarantee’ high scores. As a result, the team will use tricks to pressure customers to give high scores — for example, by sending an NPS survey only to customers who will likely give a rating (i.e., customers who had a smooth experience with a product or service).

By using a holistic set of metrics, you can achieve a much better understanding of your current state of UX. One good framework to try is Google HEART.

2. More data always leads to better insights

Data is the new gold. Companies that are able to effectively analyze and extract valuable insights from raw data have a strong advantage over their competitors. Yet, it doesn’t mean that the more data you collect, the better your understanding of the current state of UX will be.

Be aware that there are things you don’t know you don’t know

With data-driven design, you prioritize design decisions in areas where you have data. If you don’t have data in some area, you won’t even know that a certain problem exists. As a result, you will refine design in areas that won’t have a tremendous impact on overall UX.

3. Self-reported data is always reliable

Many organizations believe that the best way to learn about users is to ask them directly. Such companies rely on user surveys and feedback to evaluate user experience.

In reality, what people say and what people do can be quite opposite things.

There are many reasons why people may not articulate their true experiences — biases, memory issues, or even question misunderstanding. It doesn’t mean you cannot rely on the results of surveys and feedback in your design process. Yet, you need to validate the hypothesis you have about user experience via testing and observational methods like heatmaps and session recordings. These will provide more objective insights.

4. Usability testing equals UX measurement

Test early, test often is the right method for building products. Yet, the results of usability tests alone cannot replace UX measurement because usability testing is only one part of UX measurement.

Usability testing is only one part of the UX measurement

With usability testing, you measure the usability (ability to use) of a product. However, you don’t consider other important factors like emotional response during real interactions with a product and engagement during the actual journey. These things play a key role in retention and conversion rates.

5. Negative feedback means bad UX

If you see a lot of negative feedback your first gut reaction will likely be — our UX is failing. It’s especially painful to receive negative feedback right after a product redesign. You invest so much time and energy in redesigning a product just to receive a pile of negative comments from your existing users about it. But in reality, every redesign, no matter how brilliant it is, will likely cause negative feedback from your users.

Why?

Because people don’t like changes

People see design changes as disruption to their workflow. They might complain simply because they have to invest their time to learn how to deal with a new design. And since this investment is completely unplanned, this causes frustration. The more time & effort users have to invest in re-learning how to use the new system, the higher the level of frustration will be.

Want to learn more? Check my course “How to measure design success!”

You will learn how to choose the right metrics for your product, how to measure product’s success in meeting business goals and communicate findings to business owners.

Measure & Communicate Product Design Impact by Nick Babich on Maven

Top 5 Misconceptions About Measuring UX was originally published in UX Planet on Medium, where people are continuing the conversation by highlighting and responding to this story.