Guerrilla usability testing workshops

Yesterday I had the pleasure of running a workshop in Norwich for NorfolkDev on getting started with guerrilla usability testing. It was a half day session, covering:

  • How to design a guerrilla usability study
  • How to approach participants
  • How to write a script with scenarios for your sessions
  • How to record and analyse your results

Everyone split into teams of 3 to test real websites and services, and when each group had prepared their study plan they had a chance to try it out on 3 different people from other groups.

We had some really insightful questions come up during the session, so I thought I’d answer some of them on my blog.

The first one was asked by a few teams:

“I have a task scenario that requires someone to be logged in, what should I do?”

This is a difficult question with guerrilla usability testing, because you’ve got no guarantee that your participant has an account, or if they do, that they’ll be willing to log into it in front of you.

So, what should you do? Well, it depends on exactly what you’re trying to find out. If you want to evaluate how easy it is to log in, for example, then you’ll obviously need your participants to give it a try. But if you just need some speedy turnaround on a new feature, then as long you’ve tested the sign up or log in process in another study, it’s probably ok to log them into a fake account.

On a practical level, there are three options:

  1. Have the participant pre-logged in to a fake account

    If you are going to do this, it’s worth considering what content the participant should see on the fake account. If it’s something like a social media platform, then you might want to create a fake account with the kind of content you know your target audience is interested in. For example, if you had a persona which represents your users you could create that persona an account. However, you’ll need avoid things like profile photos with real people in them; that will make it harder for your participant to imagine that it is their account or content.

  2. Provide fake credentials for the participant to sign up with

    Such as username, password, mobile phone number and email address. If you’re dealing with payment, you might even want to provide a fake credit card (this was a very clever idea by @rosebotanic for a project we worked on together). Just remember, if your registration process requires a confirmation step, like receiving a text message or email, you’ll need to provide your participants access to the appropriate medium so that they can process that confirmation.

  3. Ask your participant to sign in and create and account

    You should think very carefully before asking for this in a guerrilla setting, particularly as the participant probably won’t be logging in on their own device. If you do ask for it, you need to be prepared for participants to say no and have a back-up plan (like a pre-logged in option, or a set of interview questions instead). If they are comfortable doing it (and I stress the “comfortable” in this sentence) then make sure you look away while they type in their password, and let them watch you log out and clear any stored data afterwards.

Next up

In my next few blog posts I’ll be covering some of the other questions that came up during the workshop, including:

  • What happens if I discover my participant isn’t part of my target audience?
  • What happens if my participant isn’t used to using the device I’m testing with?
  • How do I deal screening participants for difficult topics, like disability or income?

If your team or organisation is interested in learning more about guerrilla usability testing, then I’d love to come and run a workshop with you. For my full-day workshops, I’ll be stepping you through how to prepare a study in the morning, and taking you out to try the newly designed study sessions on real participants in the afternoon. Drop me an email if you’d like to hear more.