Level up your moderation game

5 tips to get more out of usability testing.


Dan Kossoy

3 years ago | 5 min read

UX design is the practice of making technology more usable and enjoyable for users.

Sure, this sounds easy enough, but as Kate Moran of the Nielsen Norman Group could tell you, “even the best UX designers can’t design a perfect — or even good enough — user experience” on their first try.

This is exactly where usability testing, and the idea of iterative design, comes in. The popular research method involves learning from participants (aka potential users of your product) by observing them perform a set of tasks.

This is one of my favorite parts of the entire design process because it really digs into the essence of what we as designers do: we use our expertise to create a product (ideally rapidly and inexpensively), then rely on our users to show us how we did, learn from their feedback, iterate, and improve.

Now, what you came for.

Before you read out that first task, your moderating duties have already begun. Your first goal is to put the user at ease through a thoughtful and calculated introduction. Doing so will create a more confident participant who provides more honest feedback and gets you more substantial data. Now who doesn’t like more? To do this:

1. Detach yourself from the product. This is done very subtly and is why a calculated approach is so important. You want your user to know that their feedback won’t hurt your feelings by minimizing your (perceived) personal connection to the product. Just swap out “my/our” for “the/this.”

Note that you can, and should, also express this idea directly. As Steve Krug (usability specialist) does, just tell the participant, “please don’t worry that you’re going to hurt our feelings. We’re doing this to improve the site, so we need to hear your honest reactions.”

This phrase is one I always include in my test script, but paying attention to that extra bit of subtlety takes things to the next level.

2. Make it clear that they are the expert. As mentioned above, a confident, empowered user is one that will give you direct feedback, and thus, great insights. Go above and beyond in letting them know that you value any and all feedback they provide.

Note that saying all of this may feel strange at first, and that’s ok. You’ll find that doing so 1) boosts the user’s confidence for the remainder of the test, and/or 2) at the very least, solicits a flattered and reassured smile that further eases tensions moving forward. Either way, you win.

During the test, your goal is to do your best therapist impression. Be attentive to your participant, inquisitive of their actions, and inviting of feedback.

Your ability to pick up on cues and guide the user is key to uncovering great insights. At the root of it all, the effort you put into this will influence what you get out of it.

Here’s a quick guide for what this may look like in practice. Breaking some of these situations down:

3. Pry out feedback. Thinking aloud is a vital aspect of usability testing, but doesn’t come naturally for most. You may need to prompt this sort of feedback, especially with more introverted users. The benefits of doing so are twofold. At the surface, this very simply allows you to better understand what the user is thinking. Deeper, there is the added benefit of empowering your user to own their actions and thoughts without reservation.

Note that the user is still expressing some sort of confused sentiment on the right-hand side, but it’s the kind you want. The difference is that on the left, the user internalizes this confusion, and as is the case here, begins to put the blame on themselves for what they believe to be an inaptitude,

but is actually revealing of poor design. Worse yet, without prompting the feedback as is done on the right, the moderator may never uncover this key insight.

4. Don’t bail users out. Users will inherently want to please you, telling you what they think you want to hear and completing tasks how they think you want them completed (see social desirability bias). So, yes, your job is to make the participant feel comfortable. Equally as important, though, is remaining neutral so as to keep results honest. At times, this can mean forgoing social standards of conversation in favor of testing best practice.

Note that the recommended response in this case may feel a bit rude or dodgy in a normal context. Very true. But here, it works to draw out insight into real usage behavior, not behavior based on how you want the product used.

After the test, you still have an opportunity to squeeze out some last bits of insight.

Best of all, with “test mode” off, you are likely to get some of the participant’s most genuine thoughts yet (see Hawthorne effect, where people act differently when they know they are being watched — just another reason why putting users at ease is so important). The idea here is to give them an open mic and full control. Here’s how:

5. Reset and reverse roles. It’s simple. Bring the product back to its initial state and have the user click through at their own pace. Keep in mind that the floor is theirs, your only job is to let the conversation flow. Below, I contrast this method to the more traditionally used one that you may be familiar with. This is because I have found the standard to come off as rhetorical for most users. They think it’s a nice way of saying “bye,” which only means that you miss out on an opportunity for more data.

Note that this may add a few minutes to your participant’s time commitment (generally about 5–10). That said, I’ve gotten so many great insights from doing this that it has quickly become a necessary part of my testing.

It comes down to participants running through the product virtually as expert users (all users are “experts,” but an “expert user” cruises through flows and is familiar with the process start to finish). With this holistic knowledge, users are able to ask deeper, more big-picture questions that often yield amazing insights for you.


Created by

Dan Kossoy







Related Articles