Introducing “CIDeR” (or why I don’t like the term “usability testing”)
Almost three years ago I wrote stop calling it usability testing, essentially making the argument that the term “usability testing” has a lot of baggage and gets mistaken for other things.
I still don’t like using the term in most cases, and I’ll explain why. But in the intervening years I have come up with an alternative, which I’d like to share with you. Within the UX team here at NDM, I’ve been referring to user sessions as CIDeR (Collaborative Iterative Design Refinement) sessions. I’ve had some success in convincing my team-mates and the term is starting to permeate out into the business.
Why CIDeR?
Typically our users are involved in our design process by way of a series of one-on-one sessions where users are shown stimuli of some kind, to elicit feedback. The purpose is to guide the design process and allow decisions to be made (usually) regarding the user interface. Successive rounds are used to allow the design to evolve based on user feedback, in effect making users collaborators in the design process.
Hence the name:
- Collaborative – The user is an integral part of the process, as are our colleagues from other disciplines. This word also helps break down the ‘UX guy is expert’ and ‘participant is lab rat’ dynamic that can accumulate.
- Iterative – The approach works best if it’s a process of constantly evolving the design or the idea. This word helps convey to the business that this isn’t a one shot deal, there will be several rounds of user involvement, with some thinking and designing in between.
- Design – Typically these sessions are for the purpose of producing something tangible, whether it’s designing a website or a concept. This word grounds the name/description.
- Refinement – We are working towards producing something. In conjunction with ‘iterative’ this word impresses upon people the fact this is a process, and in conjunction with ‘design’ it gives a sense of progress.
Oh, and of course there’s the added benefit of being able to say “let’s have some CIDeR and think it through” when the team reaches an impasse or isn’t sure how to proceed.
We involve users in our process in many other ways, from up-front ethnographic research through to large quantitative market research, and lots of things in between, but the bread and butter would be the CIDeR sessions. Hence it’s important for us to be clear what this work is and what it delivers—to our team but also to our business.
Why not “usability testing”?
There are four problems with the term usability testing as a label for the type of work done in a CIDeR session, some of which are refinements of the point I made last time:
- Promises conclusive, definitive results – The term sounds too absolute. As you’d expect from “testing”, after all other types of testing deliver conclusiveness or they’re considered a failure.
- Implies a focus on just the UI and usability – Much of what we do is more than usability of the user interface. We’re digging deeper, talking through preferences, perceptions. Part of this is due to the fact that for news products, the content is as much a part of the interface as the buttons, links, labels and code.
- Suggests summative application – To many people, when you say “usability testing” they think that’s something to be done at the end, a validation exercise to make sure we can go live. This isn’t at all the case for most of the work our team does, which is more about exploration over time; a fluid process rather than check-list item.
- Coloured by past experience – Any term that has been around for a while, and widely misunderstood or misused, will be horribly tainted by the experience stakeholders have had with things labelled with that term. This is certainly the case with “usability testing”. I often see this as a tendency towards quant; people expect task failure rates, ‘time on task’ and other rigid measurements and won’t give up on those kinds of outputs from our work. Again, these are rarely the things we are looking to obtain.
Don’t get me wrong, if you practice a method that does live up to all of these things, and you call it usability testing, good on you. Our team rarely does, so I don’t want to set an expectation in the minds of my stakeholders that that is what they’re going to get. We needed a new name.
How does CIDeR work with other techniques?
The CIDeR approach is qualitative and indicative, rather than conclusive. Which means that some findings (ie opinions, perceptions, propensity to buy/use) may not be representative of the larger population, and as such it is necessary to:
- exercise care in taking these findings on board, using them in the right way, and
- make use of quantitative methods, either before or after CIDeR, to determine the implications for the broader audience.
Sometimes a more formal method for involving users in the design process is used, which we do call “usability testing”. A more rigorous approach is taken to assessing how easily users are able use a given design, typically later in the design process. Because this technique is dealing strictly with usability, it is acknowledged that relatively small sample sizes (~5) can be used to draw conclusions about the usability of the design for the entire audience.
Questioning regarding opinions or propensity to buy/use, however, do require larger sample sizes. So, alongside both the CIDeR and “usability testing” methods, quantitative research may also be employed, typically to gauge reactions to a product proposition or design. This focuses more on supporting decision making at a product level as opposed to the design or user interface level.
“Pour me a glass!” or “Ewww that’s left a bitter taste”?
What do you think of the name CIDeR? Would you use it in place of the term usability testing? Why or why not? All feedback greatly appreciated.
(Originally posted to the USiT blog, reproduced here with some minor alterations)
About the author
Patrick Kennedy is a user experience strategist and design researcher based in Sydney Australia. He leads research activities that improve the user experience of cross-channel products and services; helping both designers and business decision makers in bringing those products and services to fruition. Read more.
Great article – which I’d like to see expanded further. Although I would be hesitant to introduce another acronym to an industry which has already far too many… I have to agree with many of the points made here.
However, what is here called ‘usability testing’ is perhaps better known as ‘user testing’? However, both terms currently cover far too wide a scope of design research – from those early “insights” gained from exposing potential customers to concepts, right through to the critical pre-release “usability testing” described above.
And as UX teams increasingly rub shoulders with Market Research teams, there does need to be greater transparency in describing the benefits – and in particular, the limitations – of the tools and activities UX practitioners currently embrace.
Thanks Jim. I guess I didn’t intend for CIDeR to become another industry term, only that we use it to better describe the work our team is doing. If anything, I’d like some discussion about how others in the industry have named or described their activities in order to ensure clients/stakeholders have suitable expectations, or to distinguish their work from that of similar fields such as marketing research–which as you rightly point out we are rubbing shoulders with much more often.
And on that last point, we’re increasingly seeing the need for collaboration. A CIDeR session is very much a qualitative method, and even with several rounds of application, may require complimentary quantitative research in order to properly decide on certain aspects of design and product development (what proportion of content topics is most appealing etc). This is where you need to be ‘besties’ with your marketing team to be able to leverage their skill at obtaining information that will help with that. Or your business intelligence team. Or your sales data team.
Actually, one thing I will expand on with regard to CIDeR is the fact that it’s not the answer to everything. Not by a long shot.
When you focus purely on usability, you can use a small number of tests. Ask Jackob. But when you do something like CIDeR, where you’re not just looking at functional usability but also perception, preferences, appeal etc, you can easily run into troubles with a small sample size.
Potentially massive problem…and one I’m very concerned about. Ask my colleagues.
Thus, as I touch on briefly in this post, you need to be very careful how you use the outputs of CIDeR. You can’t make conclusive statements about what your audience likes or how a product should be structured. Experience architects can use what they learn to guide their early design process. And they gain empathy. But they have to accept that they may make design decisions based on an anomalous comment.
That’s why you need quant to compliment the qual.
Hey Pat – would be interested to know how you get around your issues with sample size. We have been working on some pretty nifty mixed methods approaches here – but I dunno about you but I reckon you should be able to nail a lot of the “fluffy” stuff (percpetion, need, appeal) with some well thought out pre design research.
Good to see you are back writing (must be the improved eyesight?!)
I wouldn’t quite call it “nifty”, but we’re also trying to tackle sample size issues with a mixed method approach :)
This includes quite a bit more quant than when you were here, Stephen. Working closely with marketing and business intelligence teams has proven very useful, we now have a lot of data on brand appeal, brand emotional connection, better competitor landscapes, better readership measures, cross-channel audience sizing, feature appeal, propensity to pay, pricing models etc etc. Along with web analytics, this data allows us to have a solid understanding of the basics, on top of which we can research specifics–including running CIDeR sessions.
In essence we’re trying harder to answer the right questions with the right methods.
We should catch-up, you can show me your nifty stuff.
Sounds awesome – I’d love to catch up and chat, sounds like there’s heaps we can learn from the work you guys are doing!