Sometimes, every UX designer is faced with questions: “Will it work properly for my user?” “Is it clear to understand?” “Which option is better to develop?”. There is no point in guessing or tossing a coin — just ask your user in the early stages and improve your product based on facts.

You definitely can use some other tools to analyze and make improvements in your design such as analytics, logs, search requests, or other sources of data, but they tell you “what” when you often need to know “why”.

Usability testing is a process when you (or another person as a moderator) asks participants to complete tasks, observing their behavior and listening to feedback.

Usability testing allows to:

• Identify problems in the design, which may not have even been thought about;
• Find opportunities to improve your product;
• Learn about users’ behavior;

We prepared the ultimate guide that will help you navigate through all the stages of usability testing.

1. Pretest Setup

Defining Test Objectives

The most important step in design process is to figure out the objective, i.e. what we want to find out. The type of testing directly depends on the task. It could be:

• Checking usability flow;
• Comparing one design version with another;
• Determining, how clearly the user sees what is shown on the page;
• Navigation test, etc.

When facing a problems that a user has already encountered, we need to:

Choosing a Usability Test Type

Chose test type depending on the goal:

Each usability test is suitable for different situations and purposes. Remote usability testing appears to be more popular in COVID times.

Concerning types, for example, five-second testing is only appropriate when you need to measure users’ very first impressions of some of the layouts or even a separate section.
The 1-click test allows you to measure the usability of your product (site/app/platform) and to check how easily the user can complete a given task. The navigation tests help analyze users` navigating process through your product (site/app/platform), solving a specific task. With the help of this test, you can identify critical user flows, track navigation difficulties and improve your information architecture.

Depending on the goals you can choose moderated or unmoderated usability testing. Moderated usability testing will give you more control over the testing process. You can read more about unmoderated vs moderated usability testing in NNG article.

Choosing Target Audience for Usability Testing

The first thing you need to determine is whether you will be provided with respondents for the test or whether you will need to choose a group of users yourself. Access to direct users would be ideal, but if that’s not possible, you can determine the target audience based on parameters such as demographics, behaviors, interests, needs, and goals. However, the parameters should be determined based on the goals and priorities of the test.

• Age;
• Gender;
• Household income;
• Education;
• Employment type (full-time/part-time);
• Job sector;
• Company size;
• Duty and level (middle/head etc);
• Location (place of address);
• Web expertise (advanced/beginner PC user).

While conducting remote usability testing, it is worth specifying, if necessary, the operating system (ios/windows/android) / type of device (desktop/mobile/tablet)

To define your target audience more accurately, you should form screener questions. That types of questions cover different areas and provide detailed information. Depending on the answer, you either “approve” or “screen out” the user.

Choosing a Remote Usability Testing Tool

The platform is chosen mainly for remote testing. If you’ve chosen lab testing, it would be best done in pairs, where one person would be a “guide” giving tasks and interacting, and the other put down all the actions and/or questions.

Figma or Sketch-Craft-Invision link is used for linking layouts

Loop11 is suitable for testing your own respondents (if you don’t have your own participants database). Allows conducting moderated and unmoderated testing. With the help of audio and video, you can also view the participants` actions from any device in live mode (or recordings) (for mobile devices, participants need to download the application). Free Trial 14 days. Here you can upload a link to a website or a linked prototype, or upload an image.

Usertesting.com — Suitable if you need a third-party database of testers. Here you can set up your target audience in detail and quickly get test results (1 test — $ 50). Working with your own respondents (by paid subscription) is also possible. Allows conducting unmoderated testing and interviews. Screen recording helps respondents articulate all their thoughts and actions. You can test both a template website and a prototype or application. Site comparison is also possible.

UsabilityHub — Allows you to conduct different types of tests (5 seconds, navigation, preference test, survey, 1 click). You can use it either for your audience ($ 200 / month) or for the platform audience (depending on the type of test, about $ 1 per test). A heatmap or a list of answers should be provided depending on the test type etc.

Defining the Scenario/Objectives

Defining the Scenario and Objectives for Usability Testing

YYou need to come up with a realistic scenario for the user and clearly articulate any necessary circumstances in which they should act (such as urgency or location). It is best to use open test scenarios where you simply provide the task, such as “Buy a T-shirt in your size online and make a payment with the specified data.” For any type of test, be sure to provide the participant with information they can refer to and reread the task.

The task should not include specific actions the user should perform or sites they should take action from. It is also not recommended to provide hints or any information not directly related to the task. To articulate the task correctly, it is necessary to set the goals you want to achieve when performing a specific task.

For each task, it is also helpful to establish metrics, such as asking the respondent what was the most difficult aspect of the task or rating the task on a 5-point scale.

Test Parameters

After determining all the test parameters, all the information should be collected in a single document/presentation and all the details should be approved by the customer, preferably in written form. Several types of testing can be conducted simultaneously, to get a more reliable response.

The document should indicate all of the above points (test goals, type, audience, platforms, scenarios, and final data acquired for each task).

Preparing to Usability testing. Checklist.

1. Layout Check-List

• If you have the variability of the task performance (you can perform the same action from different screens, e.g.: assign a patient to a doctor from the doctor’s or patient’s profile; select a diagnosis first, and then add examinations to it, or do these actions in turn), you need to take this into account when linking layouts, thus giving the user the opportunity to go different ways.
• Check that everything is consistent in the layouts, all the identical elements (buttons, headers, cards, icons) are in the same style and in the same place
• Create layouts update after completing the task. When a user, for example, has added a new patient, the last step should be the screen he started from (for example, a list of patients) with the new patient added.
• Draw the success scenario if needed (tooltip that the action is performed)
• Check that there is no LOREM IPSUM in the layouts and that the information is consistent from screen to screen. For example, if you create insurance for a patient named Patric Cruz, then he must also be indicated both in the insurance field and in the lists of insurance, etc.
• Make sure the clickable areas do not overlap and are large enough for a click (especially on a mobile device)
• Make sure there are no dead-end screens and the user can return to flow
• Make sure navigation works everywhere. Do not link unnecessary screens, irrelevant to the test task. Even if they are part of a well-developed flow
• Make sure that you can exit all popups, drovers, face sheets, etc.
• Make sure the navigation (Tabbar /menu/controls/main actions) is fixed on the screen. This is more applicable to mobile devices, but also to laptops to some extent. Different devices have different heights, so you need to see the navigation when scrolling.

If the controls are inactive, use the disabled state. Don’t focus on non-clickable elements.

2. Test Planning Check-List

• Check the linking on the templates (figma/invision) by yourself.
• Verify the platforms you will conduct testing from, as well as the devices you`ll be testing. If it will be remote testing, on a mobile device, you should 100% simulate the situation of how you will carry out communication.
• Bundled applications, for example, a zoom call/ hangsout + invision; if you use Loop11, don’t make a call to the participant because you will hear him during the test (otherwise, the access to the microphone won`t allow testing, since it will be busy calling another application).
  In order to share testing in live mode with other observers, you should call each of them separately and share the screen
• Make 3–5 trial tests in a team
• Make the test on different devices (OS) and browsers
• Send invitations to interviewees in advance.
  There are should be at least 3 messages:

3. Conducting Remote Usability Testing

Basic Rules to follow:

1. Send the respondents instructions in advance. State what and how they will test, what and for what they will need.

2. Check the mic and camera.

3. Switch on the camera; do small talk to make the participant feel comfortable.

4. Ask permission for recording the test.

5. Make it clear that we are not testing the participant`s insightfulness or IQ level, all we are aiming at is to test the application which he will help to improve greatly.

6. Articulate tasks one by one, and point out where they can be found.

7. When you see that the participant is experiencing some difficulties, you may ask him leading questions, but without telling him what to do or where to click. Even if the participant asks you, ask him instead, “What do you think”, or use other techniques.

8. Ask the participant to comment on their thoughts\actions

9. After completing the task, ask what caused the most difficulties. Rate the task on a 5-point scale.

10. For the unmoderated remote usability testing, after the completion of the task, you can use the SUS to rate the interface.

11. Don’t forget to thank the participant and tell them they made a great contribution to the development of the product :)

Technical Checklist

1. Check your equipment in advance, including your camera, microphone, and screen recorder application.
2. If necessary, book a meeting and turn off the sound on your phone to avoid distractions.
3. Check the participants’ time zones to ensure you schedule the test at the correct time.
4. Ensure that all required applications are downloaded and logged in before the test.
5. Keep the task script and prototype references in a separate document, so you can quickly navigate and test in a different way if something goes wrong.
6. It is more convenient to conduct tests in pairs so that while one person is “guiding” the participant, the other can make notes. However, if you are conducting the test alone, that is also fine as all the data can be taken from the screen recording.

Data Collection

Basic positive and negative points should be noted in a worksheet right away. Additionally, both quantitative and qualitative indicators should be noted along with the methods used for problem-solving (if there were any variations).

3. Data Processing and Usability Testing Report

Data Systematization

All the data obtained should be summarized in a table (as described above) during or immediately after testing, so as not to miss any details. It is also recommended to create a presentation that includes a summary of the data, conclusions, and further suggestions.

Usability Testing Report

The usability testing report should contain the data summary and proposals on improving usability based on the experience. The report outline could be like this:

1. Key Findings
2. Test parameters and audience (test type, services, how many participants for each device, etc)
3. Test Scenario
4. Information on every task:

• Description of the task scenario
• Average execution time
• Average rating
• Progress (8/10)
• Basic issues
• Positive points

5. General conclusions (positive and negative points)

6. Recommendations

7. Links to videos, summary worksheet

4. Usability Testing Check-List

✔️ Develop a scenario and tasks.

✔️ Prepare and link layouts.

✔️ Choose a platform.

✔️ Choose the target audience.

✔️ Plan a test schedule.

✔️ Do pilot testing.

✔️ Check everything in the layout again.

✔️ Check if everything is ok with the platforms.

✔️ Prepare a post-test questionnaire.

✔️ Rehearse testing.

✔️ Enable session recording.

A very popular question is ‘How many tests need to be done?’ It depends. Usually, it is good to test until you receive repeatable test results, or you may test as long as you have the resources and time.

Hopefully, this article will help you prepare for usability testing properly, remember to iterate on your design, apply enhancements, and use the best practices for usability testing.

If usability test doesn`t show you what has to be improved, one thing you can be sure of: the test wasn`t carried out right. — Jakob Nielsen.

‍

‍

Check useful details about our Product design process.

‍