In a world before Covid we would conduct user research in person. We would hire somewhere or use one of our meeting rooms, grab a load of coffee, tea and biscuits from the office kitchen and crack on. But what happens when your office kitchen is now your kitchen at home and the meeting room is your living room? How are you going to conduct user testing from your sofa?
During the summer we conducted some user testing for one of our clients. It was a pretty straightforward requirement and something we love doing. We were tasked with testing a redesigned part of the website. However, the biggest hurdle for us was conducting it remotely as it wasn’t something we had done during the Covid lockdown.
How did we do it?
The process for conducting remote user testing remotely does not differ from what we do in person apart from putting more focus on the technical aspects. In addition to the usual writing of test plans, discussion guides and other documentation, we also undertake the recruitment of our participants. At this moment in the project our process hasn’t changed.
At this junction we normally liaise with a company or office managers to hire a location to conduct the sessions. However with Covid and working from home we can’t do that. To get around this we have to find an online tool that facilitates video calls and the ability to screen share and something affordable. After using many user testing tools and video conferencing applications, we’re leaning towards Zoom. It is something that we all seem to be familiar with during the year at one point or another whether it’s for a catch up with family or a quiz night with friends.
Once we have completed the recruitment we start to book participants in and this step involves a lot of Zoom links, meeting invites and managing people’s diaries. A calendar invite goes out with a time, date and location, something quite generic. One thing we have to do here is include a unique Zoom URL for each participant, this process takes some organisation.
Once we’ve laid the groundwork, we always do a dummy run. Even if this had been an in-person user testing session we would do a practice run to check the process works and what improvements can be made in time for the user testing to begin. The dummy run is a perfect opportunity to make sure the script is clear, and that it captures the goals we have set. We check the screen sharing works, that we can see each other, the chat works without much set up needed by the participant and foremost that the prototype works.
What did I find?
One thing I immediately picked up on during user testing was that participants need a minimum level of knowledge and experience using Zoom. I had cases where it has taken the participant a few minutes to log in, turn their camera on and share their screen. Whilst this can be frustrating it can also reduce the time you have with the user. I had one experience where the participant became so irate at Zoom that it practically derailed our session and we gained little or no insights from them. It is important to send an email in advance to take them through the technical details and to include some extra time at the start of the session.
“Email in advance to take [users] through the technical details and to include some extra time at the start of the session”
It is important to build a rapport with the participant before you begin. I always make sure at the beginning to ask the participant a few questions. It is often very trivial but it helps build that rapport and puts them at ease. Rapport doesn’t always extend to verbal communication. One thing I miss during remote research is lack of physically being in the room with someone. It’s hard to pick up on those body language cues as they are replaced by bad camera angles and lighting. So you might not understand or see their frustrations as easily.
One thing that is always in the back of my mind when I am conducting remote sessions, is praying to the Internet Gods that Zoom does not kick me out of the session. It happens and there is nothing you can do about it. If you’re organised you can quickly remedy it. If the interviewer is booted, the note taker should be able to take over whilst they get back online and everything should be fine, that’s why you practice beforehand. Unfortunately, it’s happened to me a few times to myself, but luckily the note taker has my back and runs things till I get back online – thanks guys!
One of the biggest benefits was the ability to successfully recruit participants from across the country.
Typically we get participants from across London and whilst this is great we don’t get a lot of variety. Over the summer it was refreshing to speak to participants from Scotland, Northern Ireland and the south west all in the same day. The diversity was complimentary to the product we were testing, because it was aimed at a national audience, not a London centric user base. It was also to hear how people were going about their lives during lockdown compared to somebody complaining about the tardiness of the Northern Line.
My final thoughts
I’m writing this as we embark on Lockdown 2.0 (sequels are the worst, I’m looking at you Speed 2: Cruise Control) and so it’s hard to see anything post Covid. The sessions we conducted during the summer were successful. We gained invaluable insights that led to improvements to the product.
In summary, I recommend making the most of remote user testing. It won’t replace conventional face to face sessions, but it’s something we will continue to do, as we’ve seen that even during a pandemic it is still possible to arrange and host research without comprising the results.