March 17, 2009

Let Your Users Bail: four usability testing ideas

At SXSW 09, I listened to a panel* of UX game designers talk about the unique challenges around building games. Much of their discussion, however, applied to usability testing in other software. Four big takeaways that I heard:

Let testers bail

Usability tests–especially structured task tests–often put the participant in an artificial situation. While the task may be representative, the user may spend more time on the task than normal. In fact they might keep banging away at the task even if, in reality, they would have bailed a long time ago. At the onset of a usability test, give the participant a “stop word” so that they can clue you in to where they would have abandoned your site.

Let go of the formality.

Some clients are forever looking for formalized tests. These include a “magic” number of users, tested during a specific set of time, and using very structured and repeatable tasks. While there are merits to this approach, it’s important to recognize that conducting tests is not the end game. Making the product better is.

So grab your neighbor, your friends and family, and evaluate the design without a whole lot of hullabaloo. Its better to get some data often than a lot late. Remember that, despite usability testing having roots in science–it is most often not scientific. That’s a good thing because it gives you more flexibility in test execution.

Test in person

There are definite kinesthetic and verbal reactions that users have to products, which can’t be observed remotely. Erica Firment noted one that she sees often is the “eye flash,” or the point at which the user’s eyes widen in enjoyment. Jason Schklar noted hearing the repeated smack of the joystick against a controller as evidence that the gamers were playing in a way less subtle than expected.

Go into the wild

Usability is as much about analyzing reactions to your products in the wild. So, set up Google Alerts, Twitter searches, etc. to automatically scan and report on problem phrases you anticipate your users saying. This can help you get a leg up on those people who aren’t submitting trouble tickets, but are being vocal about problems your product has.

* Funologists Live & In Person: Guerrilla Game Research