So, which user experience metrics should you use to assess your EdTech product? The answer is, it depends.
There’s no such thing as a one-size-fits-all UX metric. Rather, there are a number of key metrics you can pull from depending on the situation. Your prototype’s fidelity, the scope of what you are testing, your test objectives, and your internal stakeholders’ preferences all play a role in determining which metrics make the most sense at any given time. Here’s what you need to know to select the right metrics for your EdTech product.
Following are five of the most common UX metrics for EdTech products. UX researchers frequently leverage these standardized assessments to supplement their qualitative research.
The extremely popular System Usability Scale is a subjective assessment of a product’s usability. This questionnaire-based assessment is often used as a way to compare systems by measuring effectiveness, efficiency, and user satisfaction. To get a SUS score, researchers ask users to rank the following 10 statements on a 5-step scale ranging from Strongly Disagree (1) to Strongly Agree (5).
Users’ responses are scored on a scale of 0-100. The average SUS score is a 68. Scores above 80.3 are considered to be in the top 10% of scores, while any score below 51 is in the bottom 15% of scores. If you compared SUS scores to letter grades on an exam, anything above an 80.3 would be considered an A, while anything below 51 would be considered an F.
SUS is best used to judge a product’s overall usability. However, it should be used only after a user has interacted with your product deeply enough to have a holistic understanding of how it (or a significant portion of it) works. Only then can they provide meaningful answers to SUS’s questions.
A Net Promoter Score measures how willing a user would be to recommend your product to others. This measurement of loyalty can be helpful in promoting user-centric design solutions across your team. To determine a NPS, you only need to ask one question: “On a scale of 0 to 10, how likely are you to recommend this product to a friend or a colleague?” (where 0 equals not likely to recommend and 10 equals extremely likely to recommend.)
Based on their answer to this single question, users are divided into three categories:
In order to calculate the final score, researchers subtract the percentage of users who are detractors from the percentage of users who are promoters. The number you are left with is your NPS score. A score of 35 or above is considered good. However, looking at the percentage of detractors, passives, and promoters can also be illuminating.
In order to use the NPS, you have to at least be able to demo your product and explain its features to your users. They have to understand what they would be recommending before they could give you an honest answer. However, NPS scores shouldn’t be used in a vacuum. They should always be accompanied by qualitative data that sheds light on the score. You also need to strongly consider your audience—how frequently would a student recommend an EdTech product to a friend?
The affect grid is a tool that measures the way your product impacts your users’ mood or affect.
The grid uses two dimensions: Depressed versus excited and stressed versus relaxed. Users rate the way your product makes them feel using the following Likert scales (a scale used to measure respondents’ attitudes):
Next, their responses are plotted on an Affect Grid that gives a quick visual reference of your users’ quality of experience, ranging from poor to exceptional. Plotting data points from all users on the same graph gives a holistic picture of the overall experience.
Exceptional scores fall high into the top right quadrant of the grid, meaning that users have high excitement but also feel relaxed. Poor experiences fall into the bottom left quadrant of the chart, indicating an experience that is highly unpleasant and stressful.
You can use the Affect Grid to quantify the user experience of one or more tasks. For example, you could use it to individually measure each of the various tasks a user might take over the course of an average session with your product. From there, you can determine which tasks cause the most negative reactions and prioritize improvements accordingly.
The UEQ is a standardized method to measure and quantify users’ subjective product experiences. It comes in a short and long version. The long version evaluates a product’s attractiveness, clarity, efficiency, dependability, excitement, and novelty. The short version combines the seven data points listed below into three: Pragmatic Qualities, Hedonistic Qualities, and Attractiveness. Users rank the product according to a series of adjective pairs (such as annoying versus enjoyable) that are arranged on likert scales.
Similar to SUS, the UEQ is best used after you have given your user one or more tasks to complete in your EdTech product. However, the UEQ yields more nuanced scores that can give further direction for exploration or next steps. The UEQ provides information from each individual scale, along with defined scores for different aspects of the product. By identifying the adjective scales that contribute to each category, you can also tailor your own series of questions to look specifically at the categories that address the goals of testing.
The Single Ease of Use Question is derived from SUS. It isolates and evolves the single most impactful question in the SUS assessment (I thought the system was easy to use) to a 7-point scale question that can stand on its own and be administered directly after a user attempts a task. The question is:
The average SEQ score is around a 5.5. The SEQ is most appropriate for robust prototypes and user assessments in which the user controls the mouse.
Consider the following factors when selecting and administering UX metrics.
UX metrics are a key piece of the user experience research puzzle. Use them wisely, and you can quickly measure your product’s problems and progress in ways that validate qualitative findings, track progress, and bring stakeholders into alignment.