Pat’s Point of View

…choose to literally cut and paste (well, ‘stick’ at least) your document together.

Lately I’ve been wrangling a research report into shape and I’m finding it so much easier to just print stuff out, cut it up, organise and rearrange it, then stick it together with sticky tape. Then I’ll go back and edit the document electronically.

Alternative titles for this blog could have been “You know you’re getting old when you…” or “You know you’ve been playing craft and colouring-in with your kids when you…”. (Actually that last one is very apt since I have been loving colouring in with my daughters lately!)

Almost three years ago I wrote stop calling it usability testing, essentially making the argument that the term “usability testing” has a lot of baggage and gets mistaken for other things.

I still don’t like using the term in most cases, and I’ll explain why. But in the intervening years I have come up with an alternative, which I’d like to share with you. Within the UX team here at NDM, I’ve been referring to user sessions as CIDeR (Collaborative Iterative Design Refinement) sessions. I’ve had some success in convincing my team-mates and the term is starting to permeate out into the business.

Why CIDeR?

Typically our users are involved in our design process by way of a series of one-on-one sessions where users are shown stimuli of some kind, to elicit feedback. The purpose is to guide the design process and allow decisions to be made (usually) regarding the user interface. Successive rounds are used to allow the design to evolve based on user feedback, in effect making users collaborators in the design process.

Hence the name:

• Collaborative – The user is an integral part of the process, as are our colleagues from other disciplines. This word also helps break down the ‘UX guy is expert’ and ‘participant is lab rat’ dynamic that can accumulate.
• Iterative – The approach works best if it’s a process of constantly evolving the design or the idea. This word helps convey to the business that this isn’t a one shot deal, there will be several rounds of user involvement, with some thinking and designing in between.
• Design – Typically these sessions are for the purpose of producing something tangible, whether it’s designing a website or a concept. This word grounds the name/description.
• Refinement – We are working towards producing something. In conjunction with ‘iterative’ this word impresses upon people the fact this is a process, and in conjunction with ‘design’ it gives a sense of progress.

Oh, and of course there’s the added benefit of being able to say “let’s have some CIDeR and think it through” when the team reaches an impasse or isn’t sure how to proceed.

We involve users in our process in many other ways, from up-front ethnographic research through to large quantitative market research, and lots of things in between, but the bread and butter would be the CIDeR sessions. Hence it’s important for us to be clear what this work is and what it delivers—to our team but also to our business.

Why not “usability testing”?

There are four problems with the term usability testing as a label for the type of work done in a CIDeR session, some of which are refinements of the point I made last time:

• Promises conclusive, definitive results – The term sounds too absolute. As you’d expect from “testing”, after all other types of testing deliver conclusiveness or they’re considered a failure.
• Implies a focus on just the UI and usability – Much of what we do is more than usability of the user interface. We’re digging deeper, talking through preferences, perceptions. Part of this is due to the fact that for news products, the content is as much a part of the interface as the buttons, links, labels and code.
• Suggests summative application – To many people, when you say “usability testing” they think that’s something to be done at the end, a validation exercise to make sure we can go live. This isn’t at all the case for most of the work our team does, which is more about exploration over time; a fluid process rather than check-list item.
• Coloured by past experience – Any term that has been around for a while, and widely misunderstood or misused, will be horribly tainted by the experience stakeholders have had with things labelled with that term. This is certainly the case with “usability testing”. I often see this as a tendency towards quant; people expect task failure rates, ‘time on task’ and other rigid measurements and won’t give up on those kinds of outputs from our work. Again, these are rarely the things we are looking to obtain.

Don’t get me wrong, if you practice a method that does live up to all of these things, and you call it usability testing, good on you. Our team rarely does, so I don’t want to set an expectation in the minds of my stakeholders that that is what they’re going to get. We needed a new name.

How does CIDeR work with other techniques?

The CIDeR approach is qualitative and indicative, rather than conclusive. Which means that some findings (ie opinions, perceptions, propensity to buy/use) may not be representative of the larger population, and as such it is necessary to:

1. exercise care in taking these findings on board, using them in the right way, and
2. make use of quantitative methods, either before or after CIDeR, to determine the implications for the broader audience.

Sometimes a more formal method for involving users in the design process is used, which we do call “usability testing”. A more rigorous approach is taken to assessing how easily users are able use a given design, typically later in the design process. Because this technique is dealing strictly with usability, it is acknowledged that relatively small sample sizes (~5) can be used to draw conclusions about the usability of the design for the entire audience.

Questioning regarding opinions or propensity to buy/use, however, do require larger sample sizes. So, alongside both the CIDeR and “usability testing” methods, quantitative research may also be employed, typically to gauge reactions to a product proposition or design. This focuses more on supporting decision making at a product level as opposed to the design or user interface level.

“Pour me a glass!” or “Ewww that’s left a bitter taste”?

What do you think of the name CIDeR? Would you use it in place of the term usability testing? Why or why not? All feedback greatly appreciated.

(Originally posted to the USiT blog, reproduced here with some minor alterations)

The ominous black shape featured in last week’s “Guess that object” post is in fact my take on a mobile device usability testing jig, inspired by the work of Kirk Henry of Lokion Interactive (via Harry Brignull). I’ve been working on this device to help with testing site and app designs on mobile phones and tablets. Quite often these contraptions are called a sled but I’ve been calling this one “The Claw”, for hopefully obvious reasons.

Its purpose is to allow you to get a good view of the screen of a mobile device—handset or tablet—as well as the user’s face, during usability testing (or any other activity that you’d like to see what’s happening while someone uses a mobile device. Using software such as TechSmith Morae 3.0, you can easily record from both cameras.
(more…)

What is this a photograph of?

The answer will be revealed shortly, along with a full explanation.

(Note: you NDM folks are disqualified from entering!)

I’m quite pleased to be able to tell you about UX Storytellers, an eBook containing stories from UX practitioners from around the world, including yours truly.

It’s quite an honour to have my contribution included along side people such as Andrew Hinton, Dave Malouf and Mark Hurst (and many others!). Each author has shared a story, either a story from their career in the UX field, a story about using a certain technique, or like mine, a story based on their experiences with users.
(more…)

Yesterday I presented a short session at Oz-IA 2010, entitled Five user research methods you’ve probably never seen…

I departed from the norm and rather than talk about the five methods I listed in my presentation outline, I went for something a bit more cheeky and light-hearted; the dangers of field research! The idea came to me because so many people I know who have done user or market research have told me stories about the weird and scary situations they’ve found themselves in out in the field. (In particular Stephen Cox and Raymond Van Der Zalm gave me some great anecdotes!)

I got some decent laughs so I was pretty pleased—and relieved—about that. I ended with a practical demonstration of the tongue in cheek self-defense techniques I had talked about, for which I must say a huge thank you to Gary Barber and Oliver Weidlich who volunteered to take part!

A few people have asked me if I’m going to talk about the five methods I originally said I would, and yes I will as there is obviously interest in those topics! Stay tuned.

(This post was originally published on the USiT team blog, republished here June 24th 2011)

A user research method I’ve used many times, and talked and written about several times too, is the cultural probe (also known as a ‘diary study’ or simply ‘user research diary’).

Briefly, the purpose of a cultural probe is to conduct user research from a distance. So rather than having to literally follow the user around for two weeks, they contribute to the probe, either explicitly by writing ‘diary’ entries, or implicitly by leaving ‘digital footprints’ of their online activity. One might label the former as a reflective diary probe and the latter as a ‘lifestream’ log probe. Both types are useful, the lifestream log as evidence akin to analytics of what they actually do, and the reflective diaries in terms of exploring the motivations behind what they think and do.

(more…)

I’m delighted to announce that I’ll be speaking at this year’s Oz-IA conference, October 6-9 in Sydney. I’m running a pre-conference workshop User Research Methods for Information Architecture and I’ll be giving a short presentation during the conference on Five user research methods you have probably never seen.

The workshop is a new version of one I’ve run a few times before, updated with new examples and activities. The presentation will give a short intro to a few research methods that are not so common in the IA field.

Hope to see you there!

I’m slowly and not-so-surely making my way through Good Thinking that I talked about in a recent post. It’s full of really great tips for researhers, not just with regards to methods and methodology but also in terms of mindset.

In a chapter discussing the psychology of small groups, the author talks about the effects of deception and economy with the truth, on the part of the researcher. In her opinion, and I agree, a good researcher must adopt a more respectful attitude with regard to the people they are researching:

It means a shift in perception from treating respondents as laboratory experiment ‘fodder’ or human guinea pigs to consenting adults who have points of view to contribute. It means letting go of the idea that the information the client is looking for can be extracted from respondents whether or not they give permission. It means challenging the view that both client and researcher have the right to be manipulative and controlling because they have paid respondents to attend the session, or have paid the researcher (or company) a large amount of money to retrieve information to solve a marketing problem.

This passage is followed by a ‘quote’ from David Ogilvy:

The consumer may or may not be your wife, husband, child or parent, but you can be sure that he or she is a real living, breathing, thinking and doing human being, who has as much of a right to their way of life as you have to yours.

I like this, and in fact I have used the sentiment embodied in the above quotes as the basis for a ‘principles of user research’ blurb in my research proposals.

(more…)

I originally started writing this post when I was at UPA 2007, but for one reason or another I never published it. On several occasions, I played with the idea of combining the conference notes with some later half-written posts on generally the same topic. But alas it never made it live.

Seeing as I firmly believe that for every unpublished blog post there is one less bit of momentum keeping the interwebs spinning, I’d better put this up. And it’s interesting to look back at what I wrote two and a half years ago…

Day 1 started with a very inspiring talk by Bill Buxton. I think this was just the thing the industry needs, a bit of a reality check and a wake-up call. Firstly usability evaluation is not design and for that reason most people here don’t actually practice User-Centred Design. It’s all about data, rules, strict methodologies, large companies. They’ve even turned agile into something overly defined and bogged down (I have no strong belief either way when it comes to agile methodologies by the way). Bill’s talk about sketching as an important tool for the design process flies in the face of the artefact centric practice many Usability Professionals follow. No there’s no template for it, no there’s no software tool to do it, you have to use your brain! I mean the theme of the conference (“patterns”) says it all really.

This sounds really negative, but I don’t want to be. There are some smart and talented people here, but overall the industry is weighed down by strictness and illusions. Strictness in the sense that many people want some methodology to tell them what to do. I can understand that, but as Bill said, if you find yourself thinking that all the time (being scared of wining it) then maybe this isn’t the job for you. Illusions in terms of the discrepancy between literature and practice. A lot of the things published are not followed in practice (eg rapid, flexible approaches by clever people are replaced by limited, templated projects) and good practice is not published (eg using multiple design alternatives in usability testing). Then there’s the illusions of grandeur, like the way many practitioners think of what they do as some kind of scientific crusade and admitting there is some I-don’t-know-ness to it is an act of heresy.

For me, the best thing I saw at the conference was this talk. It’s a pity someone from outside the field (perhaps technically but really as far as I am concerned he’s slap bang in the centre of what we should strive for) had to be the one to say it. You can’t truly be doing UCD if you’re just evaluating, testing and documenting. This shouldn’t be about statistical analysis techniques.

I remember thinking that my approach to my work seemed at odds with how other attendees appeared to be working, and from the above it seems this annoyed me! Too many practitioners being more worried about following the ‘proper’ process, rather than actually thinking. And the post I did publish at the time, contains similar thoughts.