Usability Metrics

Usability Metrics

The Nielsen Norman Group, in one of its articles, pointed that these type of metrics can help you with the following: track the progress of your product, compare your product with the competitors, and make relevant decisions about the product.

Metrics

There are different metrics we can use to evaluate our products. We'll mention some of the basic ones here: the time that a task requires, whether the users can perform all the tasks or not, the error rate, the navigation path. We also need to consider that we can have new and experienced users, this article by the Nielsen Norman Group talks more about it. Most of the time new users can give better feedback as they are not used to the flow of the product. This allows them to come up with new observations and ideas.

Testing 20 users

The Nielsen and Norman Group recommend that for quantitative testing we should test twenty users. But if we are not able to test this amount of users we can do some qualitative testing, such as small tests of the product that can be run between three to five users.

Measuring Usability

We can measure the effectiveness, the efficiency and the satisfaction of the product. There are different parameters that we need to consider when we are measuring these type of factors. Tutplus has a good article which goes into more detail about how to measure usability. The Nielsen Norman Group recommends the use of the geometric mean method, which can help gather better information.

UX Metrics

There are different metrics we can use to evaluate our product. There is an article by UX for the Masses where they share some ideas and examples about these models.

Creating a usability metrics table in Notion

For this exercise, we will create an evaluation table for a user persona. I'm choosing Notion because it is a versatile and collaborative tool. But feel free to work on any other tool you prefer. The first thing we will do is create a new document. Set its title to: John Doe, which is the name of our persona.

Next, we will upload a picture to our template, you can use the unsplash library for that.

Underneath the title, we will create a table. Name the first, second and third column as follows: "Question", "Score" and "Counter".

Then we will change the Multiselect option on Score for Select.

Next, we will create some new tags like 👍 Success, 👊 Partial Success and 👎 Fail. Also, name the whole table User 1.

Now, we will change the Counter column from Text to Formula so we can add a script that will help us with our table.

Select the first row in the Counter column and copy the following script, that I wrote in advance. The main goal of the script is to give a numeric value to our tags. So 👍 Success will be equal to 1, 👊 Partial Success to 0.5 and 👎 Fail will be equal to 0.

if(equal(prop("Score"), "👍 Success"), 1, if(equal(prop("Score"), "👊 Partial Success"), 0.5, 0))

Then, in the Questions column we can start adding the different questions that we think will be relevant for our metrics. These questions can vary depending on the context of our product and the market. Feel free to add any type of questions, you can also use the questions that we will add in this template or get inspired via some of the questions in this great article on UXM.

Now that we have the questions added, we can start with the evaluation process. Next to each question on the Score column, you can start classifying our questions between three different options regarding John Doe’s persona. You will notice that the Counter column will be filled automatically with the equivalent value.

Next, we will create a new table which will help us measure (in percentage) the quality of our app in relation to the qualification we added to each one of the questions. In other words, we will put a value to our Score column. Add a new table underneath our main table, and name it Summary User 1. Name the first row as User 1.

Next we will name the second column of the new table Relation, and we will change Multi-select to Relation. ****

A new window will pop-up and it will ask us to choose a data base, let's select the User 1 table.

Next, we will click the first row of the Relation column and add all the questions by clicking blue plus signs.

Now, let's name the third column as Total Questions and the fourth column as Total Answers. Then, change the type of both the columns from Text to Rollup. Rollup will allow us to sum up the values from our columns.

Then, click the first row in the Total Question column. Here, set it up in the following order: RELATION = Relation, PROPERTY = Score, CALCULATE = Count all.

For our fourth column, we will click on the first row and set it up in the following way: RELATION = Relation, PROPERTY = Counter, CALCULATE = Sum.

Then we will create a new column called Percentage and we will make sure that the type of the column is Formula.

Finally, we will click on the first row of our new column and add this script:

divide(multiply(prop("Total Answers"), 100), prop("Total Questions"))

This formula will help us translate the number of the total questions and answers to the percentage over 100%, where 100% is equal to good usability. For this example, we got 71.4%.

Conclusion

Usability is measured relative to users' performance on a given set of test tasks. It's up to us, as designers, to define the parameters we want to evaluate.