A guide to choosing the right user testing method

CRO | 7 MIN READ

No matter how much of a UX expert you are, there really is no telling how 'real' users will interact with your site until you see them doing it first-hand. It's common for seemingly obvious blockers that have remained unnoticed and even unconsidered to only be revealed once real users have tested a website or new functionality. Use our simple guide to determine what type of user testing your organisation should undertake.

What is user testing?


User testing involves observing people attempting to complete key tasks on your website. For example:

  • Purchasing a product
  • Researching a holiday
  • Signing up for an account
  • Finding a suitable course to study
  • Getting in contact with a company

Watching users undertake tasks allows businesses to find out where their audiences may be experiencing confusion and any difficulties they are having. Findings from user testing give clear direction for design changes or A/B tests.

Companies and organisations can fix the problems that may be affecting your revenue or making users leave a website without engaging or converting. Observing real users interacting with your website is the most effective way to identify any problem areas and uncover opportunities for improvement. 

Headshot of Adam Stafford

User testing is a vital ingredient for creating and optimising your website – it tests how users react to your website, whether they can complete key tasks or goals and how they find the overall experience.

Adam Stafford, Co-founder

Why run user testing?


The goals of user testing are likely to vary from company to company. A lot will depend on what is being tested (for example a prototype or completed website) and what insight is required.

Broadly speaking, though the goals of user testing fall into three categories:

  • Identifying usability problem areas
  • Finding potential areas for improvement 
  • Gaining a deeper understanding of the user’s behaviour and needs

User testing provides qualitative (and some quantitative) data around how the look, feel and structure of websites and platforms can be improved to better meet the requirements of the user. Essentially, it gives you the insights that you need to shape or validate your design ideas.

Benefits of running user testing


There are many benefits of running user testing on your website, including:

  • Shows how users actually perform tasks, rather than how you assume they perform them
  • Reveals unexpected differences in behaviour between user groups
  • Identifies barriers that may prevent users from converting or interacting with a brand
  • Allows a greater understanding of users’ mental models and their expectations from your website
  • Uncovers which features or areas of a website should be prioritised when it comes to redesign
  • Can be used as a way to solve design disputes as it provides more robust feedback than opinion-based design
  • Brings the client closer to their customer

A better understanding of your users and how they interact with your website will help you to build a website that meets their needs.

User testing at Fresh Egg


User testing can take different forms, which we discuss in more detail in the next section. The steps we take to organise, run and analyse tests are as follows:

  • Planning. We kick off the project by discussing what our client is trying to achieve, and then create a project plan around that to ensure that the research meets the requirements
  • Recruiting. We find users who share the same needs, motivations and demographics as those of ‘real’ users
  • Building the test. We create tasks or questions for those users based on the likely actions of real users; ensuring that they help us achieve the overall goal
  • Testing. We run the research in whatever form is most appropriate – at all times making sure we don’t lead or mislead the users
  • Analysis. We analyse the research in detail and report back with evidence-based recommendations for next steps
  • Follow-up. We can run other services, including other forms of research, based on the findings of the project

It’s crucial that each step of the process is completed in order to ensure accurate, useful results at the end of the process.

Conducting user research with older people

Read our guide on conducting user research with older people

Globally, the population aged 65 and over is growing faster than all other age groups; discover why your organisation could be missing simple tricks in engaging with older people to increase ROI

What types of user testing are there?


User testing takes three main forms, all of which have different strengths and can be used to uncover a range of findings. They are:

All three involve participants attempting to complete key tasks on your website in order to help you identify areas of improvement.

What types of user testing are there?

Moderated testing, both in-person and remotely, can be more thorough and in-depth than their unmoderated test counterpart. The key difference is that moderated testing can give you a better idea of why users are doing what they’re doing. It gives you the chance to ask follow-up questions during the test that remove the guesswork from the process.

Read on for an overview of each type of testing, what they entail, when they should be used and how they are carried out.

Remote unmoderated testing

Remote unmoderated testing involves users attempting to complete key tasks on your website from their own homes.

During remote unmoderated testing, users are given a list of tasks to complete in their own home in their own time with no moderator present. The output from these sessions are videos of the user attempting to complete the tasks while thinking aloud.

The main advantage of remote unmoderated testing is that it is normally much quicker to recruit participants and produce findings. Online testing companies have huge databases of users broken down by demographics who are ready to take part in tests almost immediately.

As you can select participants who match your target audience, remote testing will normally give you a more diverse range of users than if you are recruiting locally. 

With a large number of users waiting to start, you can run tests concurrently and outside of office hours. You can also include participants from around the world, without having to leave your office.

Remote unmoderated testing can be very cost-effective. Companies run bespoke online user tests for around £50 per participant, which is a small price to pay for the insights you get.

Remote unmoderated testing is best for you if… you’re looking for a quick way of getting some top-level insight into the usability of your website.

Remote moderated testing

Remote moderated testing allows you to run user testing sessions with users no matter where in the world they are.

Remote unmoderated user testing

During remote moderated user testing, participants are taken through a list of tasks by an experienced facilitator. The sessions are run using audio and screen sharing software like Skype to allow two-way communication between people who are based in different locations. The output from these sessions are videos of the user attempting to complete the tasks while discussing their thoughts with the moderator.

The main advantage of remote moderated testing is that it allows you to run tests with users based anywhere without needing to spend precious time travelling to different locations. The only things users need to participate is a reliable internet connection and a computer or other device.

The moderated aspect of this form of testing allows for interaction with users. It allows the facilitator to engage with participants to find out more about their needs, giving you a richer source of insight and understanding.

The facilitator can really observe and interact with each participant, making them feel at ease and encourage them to act naturally. A good facilitator will avoid leading users in any way but react to different situations to get the most insight from each test. 

Remote moderated testing is best for you if… you’re looking to get deeper insights from your testing while engaging with users in different locations.

In-person moderated testing

In-person moderated testing allows you to meet users in person and get deep and meaningful insights from sessions.

In-person moderated user testing is live and takes place in a test ‘lab’. A good lab is often a relaxed lounge-style environment where users feel at ease. As with remote moderated testing, participants are taken through a list of tasks by an experienced facilitator. The output from these sessions are videos of the user attempting to complete the tasks while discussing their thoughts with the moderator.

The main advantage of in-person moderated testing is that it allows the facilitator, and any observers, to pick up on non-verbal communication, which may not be as visible with other forms of testing. 

As the facilitator is in the same room as the test participant, they can build more rapport and put the user at ease. This helps to put the user in a more relaxed frame of mind allowing for a more ‘natural’ test to take place. The ability to engage more organically with users allows the facilitator to get the most value from each test and go beyond the basic usability of a website to find out more about the needs and motivations of each participant. 

Stakeholders should be encouraged to attend these sessions so they can observe the tests live.

In-person moderated testing is best for you if… you’re looking to get rich insights into your users’ needs and motivations as well as seeing how they interact with your website.

User research being carried out

How to choose the right user research method

This guide will help you map your user research goals and questions to the right methods.

When to use which type of user testing


There are benefits and drawbacks to each of the different types of user testing. To decide which is right for you, it is important to consider what you’re trying to discover and what level of detail you need from your findings and insights. The following table gives a quick overview of the features of each user testing method:

Remote unmoderatedRemote moderatedIn-person moderated
CostLowMediumHigh
Run remotelyYesYesNo
Turnaround timeFastSlowSlow

Two-way discussion

NoYesYes

Non-verbal
forms

NoSomeYes
Rapport buildingNoneSomeYes
Live observationNoYesYes
Level of detailLowMediumHigh

Understanding what each type of testing is, and how it works, will help you to choose the right method for you.

What does user testing offer your business?


The purpose of user testing is to gain insight into how people interact with and understand your website. User testing will help you highlight usability performance issues and can validate or debunk assumptions and ideas around website improvements.

Running user testing and acting on the findings from your sessions, will result in a better user experience and a better performing site.

Chat with us about your user testing and research needs