Last Wednesday Vimcar product team (well, my pal Krille and I) attended 23rd Usability Fix. The event was expertly run by Holger Dieterich and hosted by You is Now incubator.
We took along some of the early prototypes for a browser-based, desktop-focused version of SPOTS. A week later, we are still acting on the insight that we were able to gather at the event.
Here are some of the tips on how to get even more out of Usability Fix or similar "speed dating meets usability testing" events.
1. Keep it lo-fi
Go there with a plan. Know what you are testing, what do you need to get out of it and how to get it.
Forget about fancy stuff, though. You'll be in a middle of a buzzing room, people are milling around and it all happens fairly quickly.
Keep it simple - all you need is a keen pair of eyes, a pen and a notebook.
Work in tandem. You should be at least two people, one of you driving the test and the other taking observations. Should you have a third,give her a roaming role and a remit to pin down whoever looks the best fit with your target demographic.
2. Give testers a choice
We were testing a browser-based app optimised for desktop. Doesn't come more platform-independent than that, I thought. So I brought along my MBA with Chrome on it.
Turns out this stumped at least a few of our testers. In a slightly alien context of a meetup even an unfamiliar keyboard can become a serious distraction.
While you can't make this perfect, bringing along another machine with an alternative OS or setting up several different browsers would go a long way.
If you are testing a mobile service, remember that not all Android flavours are equal, bring as many devices as you can and let the users pick whatever they are most comfortable with.
3. Segment testers
What about selection biases ask you? Yeah, what about those? Beyond attempting to shanghai someone who vaguely looks like your audience, you don't have much control over your sample.
That's OK, though. Remember, it's free. Besides, you can still ask a few questions from your users, before or after you test - just to know how best to interpret the observations.
From my experience there are three group of people you should be on the lookout for:
QA folk - they are here because they like to test. Or rather, to break your app by systematically attacking it.
QA folk will be a bit disappointed if they see some rough edges of the app and might struggle with providing constructive input on rough prototypes. Not to worry though, as long as you don't take their requests for non-essential features too seriously.
Millennial Engineers - these kids just know what is going on and will breeze though pretty much any task. If your product is not specifically targeted at the demographic of millennials already working in IT, as it is the case with Spots, take their dazzling performance with a pinch of salt.
It's not your product that is brilliant, it's them.
Startup Smartarses - the reason why people do all sorts of surveys is so that they can feel their opinions are heard. Nonetheless, most of the testers in the attendance will understand that they are here to test, not to tell you what to do. Some won't.
These people will tell you what you're doing wrong and what should you do instead. Nod politely if you must and then look at what they did, not what they said.
4. Know when to fold them
If you have all pieces in place, your tests can be pretty quick. Providing there are many testers in the attendance this means you can really get a lot of observations done.
Don't rush it though. This is not a number's game. Besides,the chances are you will be seeing the same thing over and over again after 5th or 6th tester.
Instead, go for quality. Make sure you have plenty of time to observe each tester, ask plenty of relevant background and follow-up questions and know when to stop.
5. See what other teams are up to
Don't forget to join in the fun too. Maybe wrap it up one round of testing earlier than you thought and see if you can get to the other side of the table.
It's fun to be a tester, that's why people keep coming back. I know I'll make sure to attend the next one as a tester. See you across the table!
Cover photo by Aaron Fulkerson - CC BY-SA 2.0.