[DG: Teaching & Learning] Fwd: UIEtips: 3 Questions You Shouldn't Ask During User Research

[Daphne Ogle]

UIE talks about 3 common pitfalls in the kinds of questions we ask  
when doing user research.  I know I've fallen into these traps as it  
seems intuitive to ask users what they might do or how they would  
design something or putting words in their mouths by deducing from  
what we watched them do.   Sometimes the hardest part of good research  
is not doing what might feel natural :).

Enjoy!

-Daphne


Begin forwarded message:

> From: "Jared M. Spool" <jared.m.spool at uie.com>
> Date: August 18, 2010 1:17:46 PM PDT
> To: daphne at media.berkeley.edu
> Subject: UIEtips: 3 Questions You Shouldn't Ask During User Research
> Reply-To: jared.m.spool at uie.com
>
>
> 	UIEtips: 3 Questions You Shouldn't Ask During User Research
> August 18, 2010
>
>
>
>
> Three Questions You Shouldn't Ask During User Research
>
> 	
> Jared Spool, User Interface Engineering
>
> The participant was struggling. While he was a high-volume customer  
> who had bought tons from this site in the past, today he wasn't  
> getting along with the checkout process. Confusion happened in both  
> directions: the shopper didn't understand what the site was trying  
> to tell him and the site definitely didn't understand what he wanted.
>
> The team spent several painful minutes watching as this poor  
> customer made slow progress. His determination made the purchase  
> happen, but it wasn't smooth.
>
> "It's usually not this bad," the customer shared. "I've had problems  
> before, but this feels especially difficult today." It was just a  
> perfect storm of things-not-going-well.
>
> Sitting next to me was a team member who was becoming increasingly  
> anxious. As the e-commerce site's VP of Customer Service, he could  
> feel this customer's pain. This wasn't the experience he wanted  
> their customers to have, especially a loyal, regular customer like  
> this one.
>
> At the appropriate moment, the session's moderator turned to the  
> observers and asked, "Does anyone have any questions?"
>
> "Yes, I do," exploded the VP. "Do you think... I mean, if you were  
> at home and you went through all this and had all this trouble...  
> and there wasn't a room full of people watching you... do you think  
> you would've called the 800 number that was on every screen? Would  
> you have called our support department?"
>
> The VP wanted to believe that, when customers have a traumatic  
> experience like this, they don't just let it go—they call his people  
> for help. After all, he's made sure they're trained to handle  
> exactly this type of problem and shower the customer with great  
> attention and results. The guy has to want to call.
>
> The customer thought about it carefully for a few moments. You could  
> almost see into his mind as he recreated the situation in the  
> comfort of his own home. Finally, he said, in a very calm voice, "I  
> think so. Yes. I would've called the 800 number."
>
> The VP sat back in his chair, happy with the thought his team would  
> be there for the rescue.
>
> Then I asked, "Have you ever called the support number when you've  
> tried to purchase things in the past?"
>
> "No, I haven't," the customer answered immediately.
>
> That's the puzzle: The customer said he would call support, yet also  
> said he had never done so in the past, even though he's had similar  
> problems. Which should we believe?
>
> 1) Don't Do: Asking About the Future
>
> Unknowingly, the VP broke one of the rules of good participant  
> questions: Don't ask about the future.
>
> When asked about a future scenario, many people will try to answer  
> honestly. They'll answer about what they'd like as their ideal  
> behavior. They want to think they'll behave in the most logical,  
> effective manner. In short, they want to think they'll do the right  
> thing.
>
> Yet, in the thick of the situation, people aren't always so logical.  
> They don't do what's ideal.
>
> We want to predict future behaviors, so we can make designs that  
> service them. Yet just asking participants may not get the actual  
> answer.
>
> I've found the best way to predict future behavior is to look at  
> past behavior. Asking a participant about what they've done in the  
> past is a better way to get those answer.
>
> Instead of asking, "do you think you might need to store messages in  
> different folders?", I've found it better to ask, "when you're done  
> with messages, what do you do with them?" I focus on what they've  
> done in the past, looking for that behavior to tell me what makes  
> sense.
>
> Asking about the future is just one type of question I've learned we  
> shouldn't ask our research participants.
>
> 2) Don't Do: Asking How They'd Design a Feature
>
> "If we gave you a way to annotate your files, how would that work?"  
> "What would be the right way to organize the menu options?" "What  
> other fields should we include on this form?" "How would you want to  
> see this data displayed?"
>
> I've heard these questions many times. (I've even asked them on  
> occasion.) After all, the designers want to create an effective  
> design for the user. The best way is to just ask the participant  
> what that design should be, right? Wrong.
>
> Users aren't designers. (If they were, they wouldn't need us.) They  
> don't know how to deal with constraints. They haven't really thought  
> the problem through.
>
> Any answer they give us is unlikely to be a great design. It'll be  
> the first thing they think of and, unfortunately, they'll take a  
> long time to get there.
>
> I saw this happen just the other day. A smart team was talking with  
> a long-time customer of their product. "How would you..." the team  
> leader asked.
>
> The customer's posture changed completely. Whereas before they'd  
> been leaning towards the team members with solid eye contact for  
> every interaction, they now relaxed in their chair and looked into  
> space. "Hmmm. I guess I'd..." as they tried to imagine their  
> brilliant idea.
>
> Except, it wasn't a brilliant idea. It was simplistic and couldn't  
> handle the myriad of edge conditions that were patently obvious.
>
> The team could see the original problem and asking how to design it  
> wasn't adding anything. They knew what to do.
>
> 3) Don't Do: Asking by Providing Their Reason
>
> "Is the reason you don't click this button because it's really hard  
> to see?"
>
> This is a tough one for many observers. We're watching the  
> participant and they aren't doing what we know to be the optimal  
> method. Our brain races with possible reasons. Need to find out.  
> Must ask.
>
> "Why didn't you click this button?" is only a hair better. It  
> doesn't telegraph the reasons we're hypothesizing, but it implies  
> the person has done something wrong. They'll tell you a reason, but  
> it may be more because you've pointed it out, not because it's what  
> they were actually thinking.
>
> Be creative about exploring the participant's understanding of the  
> interface. For example, when I'm curious about why they didn't click  
> a particular button, I ask them to talk about what each button does.  
> That way, they won't associate the specific button with the action  
> they just did (and I learn their overall understanding of the  
> controls).
>
> One Bad Question Won't End The World
>
> Asking these questions isn't evil. We're not violating some Geneva- 
> Convention-type of international agreement. We won't be hauled away  
> by the User Research Police.
>
> However, there is a price to asking the wrong questions. When  
> conducting user research, the most valuable moments are limited  
> times that the team spends with each participant. It's important to  
> make every second count.
>
> Asking one of these questions wastes those precious moments. They  
> take time without returning value. The participant tries (and,  
> usually tries really hard), but that trying doesn't pay off and eats  
> down the clock.
>
> Learning to focus on the right questions can help you get the most  
> out of each session.
>
> • • •
>
> Jared's article is also available online.
>
> • • •
>
> Once you've learned what you need from your research, you'll want to  
> put it in a form that helps you speed through your design process.  
> That's exactly what scenarios help you do and, coincidentally, why  
> we've asked Kim Goodwin to come to UI15. She'll teach us how to make  
> scenarios work. You can find out about Kim and the other great UI15  
> experts at UICONF.com.
>
> • • •
>
> Have you compiled your own questions that you shouldn't ask? Share  
> your list at the UIE Brain Sparks blog.
>
>
> UI15: Where Good Designers Go to Become Great Designers
>
> Response to the User Interface 15 Conference has been amazing.  
> During the sneak preview alone, we've had over 100 people save their  
> spot and take advantage of the lowest rate for all 3 days.
>
> If you've been thinking about attending UI15 in Boston on November  
> 8-10, now is the time to register. This is the lowest rate you'll  
> find, and you'll be guaranteed to get the workshops of your choice.  
> Get the details at UICONF.com.
>
> Why UI15 is Different
>
> The User Interface Conference is different from many UX conferences.  
> Here's why:
>
> You attend 2 full-day workshops. No skimming a topic for 45 minutes  
> or 60 minutes of "everything you need to know" presentations. We're  
> talking about deep dive, intense workshops on specific topics like  
> scenarios, content strategy, and visual thinking.
>
> You don't just sit in these workshops—you work. The workshops are  
> made-up of lectures, hands-on exercises, and group discussions. If  
> you're expecting to sit back, relax, and listen to someone talk for  
> 6 plus hours, this conference may not be what you're looking for.
>
> We know it's difficult to choose just two speakers to hear so we put  
> together a day between the workshops where you can sample other  
> presenters. Each presenter highlights 90 minutes of their material  
> so you can get a taste of what they have to offer.
>
> There is no other conference that offers full-day workshops on  
> building personas, visual thinking, content strategy, visual design,  
> new techniques to communicate ideas, web form design, and designing  
> scenarios, components and libraries.
>
> Register now and pay the lowest rate of $1295. After August 26, the  
> price goes up $400.
>
>
> Web App Design Advice from 11 Masters All in One Place
>
> Missed the Web App Masters Tour? No problem, we'll bring it to you.
>
> Here's your chance to hear 11 of the Tour Masters present on complex  
> navigation, integrating social components, using design patterns,  
> plus a range of topics that designers continually struggle with.
>
> Hear first hand about the design process that Facebook, 37signals,  
> and Marriott incorporate. Share with your team methods to keep users  
> engaged, steps needed to ensure a team's success, and the importance  
> of creating an experience vision.
>
> When you purchase the Web App Masters Tour proceedings you'll get:
>
> In-depth presentation slides for 12 sessions from our world-renowned  
> speakers
> Audio recordings of all the presentations plus the Q&A from the  
> audience
> Podcasts of interviews with many of the Masters
> Discounts towards the books from our Masters
> Order Your Proceedings by August 31 and Save
>
> Save $50 when you order your proceedings by August 31. You'll get  
> immediate access to the proceedings - no waiting. Down load the  
> files and share them with everyone on your team.
>
> Learn more at UIETour.com
>
>
> Get regular updates about UIE when you follow us on Twitter or  
> become a fan on Facebook.
>
>
> Do you have feedback or comments on our article? Send us your  
> thoughts on our blog.
>
>
> Forward email
>
>
> This email was sent to daphne at media.berkeley.edu by jared.m.spool at uie.com 
> .
> Update Profile/Email Address | Instant removal with SafeUnsubscribe™  
> | Privacy Policy.
> Email Marketing by
>
>
> User Interface Engineering | 510 Turnpike Street | Suite 102 | North  
> Andover | MA | 01845
>
>

Daphne Ogle
Senior Interaction Designer
University of California, Berkeley
Educational Technology Services
daphne at media.berkeley.edu
cell (925)348-4372




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://collab.sakaiproject.org/pipermail/pedagogy/attachments/20100818/a2795ac7/attachment-0001.html