Pat’s Point of View

The other day I saw what is probably going to be an increasingly common sight. On the way to lunch, I passed an internet cafe that had icons decorating its glass shopfront. The icons were of consistent design, squares with rounded corners, very Apple-esque but they weren’t icons I recognised from existing products or services. That is, I guess the shop owners made them up to look ‘high tech’.

Anyway, as I walked past, a little boy–probably about 4 or 5 years old–was walking along the shopfront pressing the icons with his hand. As he pressed each one, and got no response, he got more frustrated. He’s obviously used a touch screen device such as an iPad, and so just assumed that these icons on the shop windows could also be ‘touched to activate’. When a simple touch failed to work, he progressed to running along the row of icons, touching all of them in quick succession. Still nothing happened.

Will all future generations be so disappointed by our non-interactive built environment?

Almost three years ago I wrote stop calling it usability testing, essentially making the argument that the term “usability testing” has a lot of baggage and gets mistaken for other things.

I still don’t like using the term in most cases, and I’ll explain why. But in the intervening years I have come up with an alternative, which I’d like to share with you. Within the UX team here at NDM, I’ve been referring to user sessions as CIDeR (Collaborative Iterative Design Refinement) sessions. I’ve had some success in convincing my team-mates and the term is starting to permeate out into the business.

Why CIDeR?

Typically our users are involved in our design process by way of a series of one-on-one sessions where users are shown stimuli of some kind, to elicit feedback. The purpose is to guide the design process and allow decisions to be made (usually) regarding the user interface. Successive rounds are used to allow the design to evolve based on user feedback, in effect making users collaborators in the design process.

Hence the name:

• Collaborative – The user is an integral part of the process, as are our colleagues from other disciplines. This word also helps break down the ‘UX guy is expert’ and ‘participant is lab rat’ dynamic that can accumulate.
• Iterative – The approach works best if it’s a process of constantly evolving the design or the idea. This word helps convey to the business that this isn’t a one shot deal, there will be several rounds of user involvement, with some thinking and designing in between.
• Design – Typically these sessions are for the purpose of producing something tangible, whether it’s designing a website or a concept. This word grounds the name/description.
• Refinement – We are working towards producing something. In conjunction with ‘iterative’ this word impresses upon people the fact this is a process, and in conjunction with ‘design’ it gives a sense of progress.

Oh, and of course there’s the added benefit of being able to say “let’s have some CIDeR and think it through” when the team reaches an impasse or isn’t sure how to proceed.

We involve users in our process in many other ways, from up-front ethnographic research through to large quantitative market research, and lots of things in between, but the bread and butter would be the CIDeR sessions. Hence it’s important for us to be clear what this work is and what it delivers—to our team but also to our business.

Why not “usability testing”?

There are four problems with the term usability testing as a label for the type of work done in a CIDeR session, some of which are refinements of the point I made last time:

• Promises conclusive, definitive results – The term sounds too absolute. As you’d expect from “testing”, after all other types of testing deliver conclusiveness or they’re considered a failure.
• Implies a focus on just the UI and usability – Much of what we do is more than usability of the user interface. We’re digging deeper, talking through preferences, perceptions. Part of this is due to the fact that for news products, the content is as much a part of the interface as the buttons, links, labels and code.
• Suggests summative application – To many people, when you say “usability testing” they think that’s something to be done at the end, a validation exercise to make sure we can go live. This isn’t at all the case for most of the work our team does, which is more about exploration over time; a fluid process rather than check-list item.
• Coloured by past experience – Any term that has been around for a while, and widely misunderstood or misused, will be horribly tainted by the experience stakeholders have had with things labelled with that term. This is certainly the case with “usability testing”. I often see this as a tendency towards quant; people expect task failure rates, ‘time on task’ and other rigid measurements and won’t give up on those kinds of outputs from our work. Again, these are rarely the things we are looking to obtain.

Don’t get me wrong, if you practice a method that does live up to all of these things, and you call it usability testing, good on you. Our team rarely does, so I don’t want to set an expectation in the minds of my stakeholders that that is what they’re going to get. We needed a new name.

How does CIDeR work with other techniques?

The CIDeR approach is qualitative and indicative, rather than conclusive. Which means that some findings (ie opinions, perceptions, propensity to buy/use) may not be representative of the larger population, and as such it is necessary to:

1. exercise care in taking these findings on board, using them in the right way, and
2. make use of quantitative methods, either before or after CIDeR, to determine the implications for the broader audience.

Sometimes a more formal method for involving users in the design process is used, which we do call “usability testing”. A more rigorous approach is taken to assessing how easily users are able use a given design, typically later in the design process. Because this technique is dealing strictly with usability, it is acknowledged that relatively small sample sizes (~5) can be used to draw conclusions about the usability of the design for the entire audience.

Questioning regarding opinions or propensity to buy/use, however, do require larger sample sizes. So, alongside both the CIDeR and “usability testing” methods, quantitative research may also be employed, typically to gauge reactions to a product proposition or design. This focuses more on supporting decision making at a product level as opposed to the design or user interface level.

“Pour me a glass!” or “Ewww that’s left a bitter taste”?

What do you think of the name CIDeR? Would you use it in place of the term usability testing? Why or why not? All feedback greatly appreciated.

(Originally posted to the USiT blog, reproduced here with some minor alterations)

My colleague Angus Fraser recently gave The Claw a good workout on some mobile site testing in Brisbane. The screenshot below shows the output of the two webcams side-by-side, viewed in a neat piece of software called AMCap. The two AMCap windows, and the audio from one of the webcams, was recorded and broadcast to observers using GotoMeeting.

There’s nothing like some real world feedback to improve a product, and Angus was most helpful in this regard. Besides the introduction of AMCap, Angus pointed out that the webcam mounting solution wasn’t the best. The original screws from the webcam base were too short to make it all the way through the perspex and securely hold the webcam, even with the countersinking. So I’ve enhanced the design by using meatier screws (wood screws actually) that are longer than the original and with wider thread to really bite into the plastic of the webcam. Note this is a destructive enhancement, the new screws will wreck the hole for the original screws and you’ll no longer be able to attach the webcam’s circular desk stand.

The new screws also have bigger heads that a normal screwdriver will drive (not requiring a jeweller’s screwdriver like the original screws). So not only are the webcams held nice and securely, but it’s easier to undo and move them.

The other enhancement Angus suggested was to use ‘velcro’ on the handset, so it can be taken off the Claw for setup changes, but then securely re-attached. The ‘velcro’ strips are Command Picture Hanging Strips (annoyingly, they don’t refer to them as ‘velcro’).

I also tied the two USB cables for the webcams together with cable ties, making them less messy when using the Claw.

The Claw is evolving!

(Originally posted to the USiT blog, reproduced here with some minor alterations)

What is this a photograph of?

The answer will be revealed shortly, along with a full explanation.

(Note: you NDM folks are disqualified from entering!)

At least it is when it comes to customer experience.

Today I purchased two gifts online, from a small proprietor. Their website doesn’t calculate the shipping costs when you place the order, instead stating that they will work out the best method of shipping the goods (many of which are bulky or odd shaped) and then contact you with a final price.

An hour or two later I got a phone call from one of their staff who informed me of the shipping costs, which were quite reasonable, and confirmed that I still wished to go through with the order. The person who called was a pleasure to deal with; friendly, efficient and helpful.

Now, in this day and age we’d label this as backward, inefficient…a bad customer experience. But I put it to you that this is preferable to some of the awfully complex or unusable interfaces for online ordering I’ve seen.

This particular organisation has accepted the limitation of their online ordering system—the fact that it can’t accurately calculate the cost of shipping for their variety of item shapes and sizes—and turned it into an opportunity for a positive customer interaction. I got personal service, solid confirmation that all was well, and I got the best possible shipping price (calculated by a human!).

Sure, it would be nice if everyone who does business online had a highly sophisticated system that would handle complex shipping calculations whilst providing a good user/customer experience, but that’s just not realistic. In the absence of that, it’s much better to take part of the process offline and optimise both the online and offline parts for the best possible experience.

What examples have seen where a good offline customer experience beats a bad online one?

[image credit: Yanko Design]

I originally started writing this post when I was at UPA 2007, but for one reason or another I never published it. On several occasions, I played with the idea of combining the conference notes with some later half-written posts on generally the same topic. But alas it never made it live.

Seeing as I firmly believe that for every unpublished blog post there is one less bit of momentum keeping the interwebs spinning, I’d better put this up. And it’s interesting to look back at what I wrote two and a half years ago…

Day 1 started with a very inspiring talk by Bill Buxton. I think this was just the thing the industry needs, a bit of a reality check and a wake-up call. Firstly usability evaluation is not design and for that reason most people here don’t actually practice User-Centred Design. It’s all about data, rules, strict methodologies, large companies. They’ve even turned agile into something overly defined and bogged down (I have no strong belief either way when it comes to agile methodologies by the way). Bill’s talk about sketching as an important tool for the design process flies in the face of the artefact centric practice many Usability Professionals follow. No there’s no template for it, no there’s no software tool to do it, you have to use your brain! I mean the theme of the conference (“patterns”) says it all really.

This sounds really negative, but I don’t want to be. There are some smart and talented people here, but overall the industry is weighed down by strictness and illusions. Strictness in the sense that many people want some methodology to tell them what to do. I can understand that, but as Bill said, if you find yourself thinking that all the time (being scared of wining it) then maybe this isn’t the job for you. Illusions in terms of the discrepancy between literature and practice. A lot of the things published are not followed in practice (eg rapid, flexible approaches by clever people are replaced by limited, templated projects) and good practice is not published (eg using multiple design alternatives in usability testing). Then there’s the illusions of grandeur, like the way many practitioners think of what they do as some kind of scientific crusade and admitting there is some I-don’t-know-ness to it is an act of heresy.

For me, the best thing I saw at the conference was this talk. It’s a pity someone from outside the field (perhaps technically but really as far as I am concerned he’s slap bang in the centre of what we should strive for) had to be the one to say it. You can’t truly be doing UCD if you’re just evaluating, testing and documenting. This shouldn’t be about statistical analysis techniques.

I remember thinking that my approach to my work seemed at odds with how other attendees appeared to be working, and from the above it seems this annoyed me! Too many practitioners being more worried about following the ‘proper’ process, rather than actually thinking. And the post I did publish at the time, contains similar thoughts.

I recently received an email from someone asking me for some advice on starting off in the field of user research field.

Hi Patrick,

I’m Hans, a technical writer from Zurich and wanted to say a quick hello.

I happened to visit your website and liked the article The 7 Stages of User Frustration :)

I have a interest towards user research and have been reading books.

I’m looking out for projects (websites or applications) where I could help with user research and data analysis. I was wondering how to get started in terms of skills required. I know it is not easy for people to entrust such an important activity to me. Could you help me with your suggestions?

Thanks,
Hans

I thought about how I would respond, but since this isn’t the first time I’ve been asked this question, I figured it might be helpful to respond in the form of a blog post for everyone to see.

(more…)

There are many user research methods one can use, and there are even more variations and names for them. But regardless of what name they’re given, methods should be chosen that are suitable for the situation at hand.

In this article I give a quick overview of the methods I commonly use, broken down in to main categories:

• Direct user contact—where the researcher does very much interact with users, or members of the audience as I prefer to call them
• Indirect user contact—where the researcher does not actually interact with members of the audience

The list is by no means exhaustive but it gives a good indication of the breadth and depth of methods that can be employed in user research.

(more…)

The best experiences customers have with a product or service are when that product or service really nails a particular problem.

But how do you come up with that “killer app”? How do you “innovate”? (I say this with tongue firmly in cheek because simply meeting people’s needs is not innovation, even though that’s the word many people use to describe it).

To come up with a great solution you need to single out a specific problem. Not a whole bunch of issues that affect the experience, just one.
(more…)

On Friday I presented with my colleague Alun Machin at the first day of Oz-IA 2009, the 4th Australian Information Architecture conference. It was a great day, with good company to keep us talking and some interesting stuff to keep us thinking. The venue was quite good too, my first time at Star City Convention Centre as it happens.

Unfortunately I couldn’t make it for the second day of the conference due to family commitments, which is a shame since the programme looked even better for the Saturday!

Our presentation on our new SuperRacing site (not yet live at the time of writing thissite is now live) went down well considering we had to cram everything we wanted to talk about into 25 minutes. The slides are shown below.

After two conference presentations in one week, it’ll be a while before I make another appearance, not to mention that I’ll be pretty brain dead for a while after the new baby arrives.