At the moment of this writing the Corona virus is holding a firm grip on the economy in Europe and the rest of the world. It is clear now that we have entered a new world, with strict rules on personal physical engagements and terrifying economic turmoil.
For many companies investing in user experience (UX) may not seem a top priority right now (which would be a strategic mistake), and that due to the lockdown measures you can’t meet your users anyway (which would be a tactical mistake).
The strategic risk is that your company downplays the importance of user experience, while for users the critical argument to choose the competition over you is the experience that you offer.
What is true however is that you need to adapt and move online. One domain for instance where you can easily move online is user testing. Preferably you do user testing live; you pick up on so many visual and other non-verbal cues. But how to do this when both the user and the researcher are locked down in remote locations?
As it turns out, online testing is a very viable alternative that wields very good qualitative results. What’s more, there are quite a few advantages to online testing.
Advantages of moderated online user testing
With live user testing the user and you need to be in the same physical location. With remote user testing all it takes is a high-speed internet connection. This offers some important benefits.
Lower costs
Remote user testing is less expensive. There is no travel budget needed to get one (preferably two) researchers on site. You also do not need to compensate your user for travel (normally this is part of the incentive that users receive for their participation). Finally, you don’t have to rent a separate location. To have several additional people observing the test you need a labo environment with a one-way mirror and that is quite expensive. With remote testing you can have as many people call in a non-obtrusive way and at no additional cost.
Easier to plan
Remote user testing is also easier to plan. In a traditional live setup, you organize all the tests on one or two fixed days. Most of the time the tests take place during daytime. It’s possible to organize sessions in the evening, but the time there is limited. With remote testing it is much easier for testers to do the test somewhere in between their work hours when it suits them.
In addition, the act itself of organizing the schedules and making appointments for fixed days is quite an effort and no-shows are always a terrible waste of time. With remote testing you have much more flexibility in scheduling in the tests and considerably reduce the cost and effort.
Better test profiles
We noticed that one of the side-effects of going online is that the quality of the test users has increased. In a live setting, you typically only attract test users that are willing to sacrifice some hours off work to participate in your test. Due to the flexibility remote testing offers you get access to a much larger group of possible candidates and these candidates are much more inclined to participate. The result is that your participants are much closer to the profiles you want to research and less skewing of results.
Extended geographical reach
More and more companies are active globally and distribute products and solutions all over the world. This means they have a very diverse user base and regional and cultural differences play a huge role in how users use and experience products. It is very expensive to have researchers travel all over the world to cover this diverse user base. What happens is that by necessity companies limit their testing to places where the user base is easily reachable. Mostly this is the home country or some neighboring countries. The results of these tests are then extrapolated towards the rest of the world. This may of course lead to biased results and is a very dangerous strategy to follow. Remote user testing eliminates all these boundaries, and you can reach anyone, anywhere (as long as you take into account the time difference of course and accept that you’ll find yourself conducting user research in the middle of the night or early morning 😊)
More iterative testing
Because it is less expensive and easier to organize, the threshold to structurally include testing in your overall design process is lower. Instead of focusing all efforts on one specific occasion –often more towards the end of the design process- you can now include more pinpointed tests from the start. By working more iteratively, you are true to the agile design process. By getting constant user feedback you can adjust as you go and ensure you come up with the most useful and valuable version of your product.
User testing remains a craft
While we are experiencing these many advantages of remote testing, we are also very aware that the way you conduct the tests has a great impact on the results you will get.
- In general people’s attention span has decreased dramatically, especially online. This means you need to adapt the level of complexity of the tasks you are asking the user to execute. Also test one task at a time. Online, users get easily confused if you combine different tasks or if you are probing too many things at the same time.
- You still need to master the right interview techniques. Asking the right questions is key of course, but also things like keeping strategic silences are more important than ever. The right dose of strategic silence will make your user talk more and express more useful feedback.
- Most online audio and screensharing software look self-evident and easy to use. But we still experience a lot of glitches. There are of course technical problems, such as bad audio or broken connections over which you don’t always have power. But other issues such as users that are not able to share their screen, work with outdated browsers etc. Are things you can avoid by good preparation. Clear instructions and sufficient effort to ‘prepare the participant’ up front is a must to ensure a smooth process and good results.
- Recruiting the right users remains key. You can have a well-structured test but if the participant does not have the right profile you are missing the mark. If this is a problem for you, we advise to work with a recruitment agency that can help you get the right people with the right profiles. They are also very strong in making sure all participants show up and time-wasting things like no-shows are minimal.
- If you really want the best and most objective results, it’s a good idea to have a third party perform the tests. In our agency for instance we also abide to this principle: you don’t test your own designs, we always have another consultant doing the tests than the consultant who made the designs. Consider working with an outside party with experience in user testing. Keep in mind that being able to identify a problem with your designs is one thing, coming up with design solutions is another. Here also, design agencies can help you out.
In conclusion, remote user testing has many benefits but also some caveats you need to consider making sure you are getting the results you need. In any case, now is not the time to scale back your efforts. Digital is here to stay. Make use of this time that you have and organize your online testing today. As a UX design agency with a lot of experience we are here to help to make sure you get the most out of your efforts.
