Monthly Archives: October 2009

Two Days of Remote Usability Testing

I’ve been looking forward to remote usability testing a major manufacturer’s website for some weeks now.  While some people may have doubts about the value of remote usability testing compared to face-to-face testing, I’m here to attest that the value is comparable, and in some respects, perhaps even higher.

In our arrangement, we observed the testing in a conference room on a large projection screen, while a facilitator (in another state) moderated the test sessions, and the participants sat at home at their own computers.

While you don’t typically see a user’s facial expressions in this remote testing set-up, here’s what you do get:

  • You hear what the user is saying (and perhaps listen better since you’re not distracted by what he/she looks like).
  • You see the user’s on-screen activity including what browser they use and how it’s set up (e.g., number of toolbars).
  • You get a better understanding of the user’s natural environment (whether their speakers are hooked up, whether there’s a dog in the house, etc.).
  • You may get more honest feedback.  I’m not sure what the research says about this, but my impression was that people not meeting face-to-face with a facilitator may be more comfortable saying what they really think. 
  • No need to sit in a dark room behind one-way glass, nor keep our voices down when we wanted to discuss users’ comments during a test.  (Our conference line was on mute, of course.)
  • No travel costs!

A few things that I thought made our remote testing experience even better included:

  • A two-way video conference set up with the remote facilitator so that we could see and talk to her after each session — as if she were in the room with us.
  • A research director who sat with our team and could chat online with the facilitator when we wanted to interject a question in the middle of a test.
  • Because the usability vendor was local, it was feasible for more project team members and the client to attend the test sessions.  This should help ensure that the findings are seriously considered, since nothing beats seeing users interact with your site first-hand. 

A few things I would keep in mind for next time include:

  • Add a question in the screener to ensure the participant has a LAN line which can be called for the test session.  Cell phone connnections can be unreliable.
  • As in face-to-face testing, consider hiring floaters who can be available to replace a participant who doesn’t answer the telephone for the test, has technical problems, or turns out not to be a great match to the target audience.

I look forward to more remote usability testing experiences!