I still don’t like using the term in most cases, and I’ll explain why. But in the intervening years I have come up with an alternative, which I’d like to share with you. Within the UX team here at NDM, I’ve been referring to user sessions as CIDeR (Collaborative Iterative Design Refinement) sessions. I’ve had some success in convincing my team-mates and the term is starting to permeate out into the business.

Why CIDeR?

Typically our users are involved in our design process by way of a series of one-on-one sessions where users are shown stimuli of some kind, to elicit feedback. The purpose is to guide the design process and allow decisions to be made (usually) regarding the user interface. Successive rounds are used to allow the design to evolve based on user feedback, in effect making users collaborators in the design process.

Hence the name:

• Collaborative – The user is an integral part of the process, as are our colleagues from other disciplines. This word also helps break down the ‘UX guy is expert’ and ‘participant is lab rat’ dynamic that can accumulate.
• Iterative – The approach works best if it’s a process of constantly evolving the design or the idea. This word helps convey to the business that this isn’t a one shot deal, there will be several rounds of user involvement, with some thinking and designing in between.
• Design – Typically these sessions are for the purpose of producing something tangible, whether it’s designing a website or a concept. This word grounds the name/description.
• Refinement – We are working towards producing something. In conjunction with ‘iterative’ this word impresses upon people the fact this is a process, and in conjunction with ‘design’ it gives a sense of progress.

Oh, and of course there’s the added benefit of being able to say “let’s have some CIDeR and think it through” when the team reaches an impasse or isn’t sure how to proceed.

We involve users in our process in many other ways, from up-front ethnographic research through to large quantitative market research, and lots of things in between, but the bread and butter would be the CIDeR sessions. Hence it’s important for us to be clear what this work is and what it delivers—to our team but also to our business.

Why not “usability testing”?

There are four problems with the term usability testing as a label for the type of work done in a CIDeR session, some of which are refinements of the point I made last time:

• Promises conclusive, definitive results – The term sounds too absolute. As you’d expect from “testing”, after all other types of testing deliver conclusiveness or they’re considered a failure.
• Implies a focus on just the UI and usability – Much of what we do is more than usability of the user interface. We’re digging deeper, talking through preferences, perceptions. Part of this is due to the fact that for news products, the content is as much a part of the interface as the buttons, links, labels and code.
• Suggests summative application – To many people, when you say “usability testing” they think that’s something to be done at the end, a validation exercise to make sure we can go live. This isn’t at all the case for most of the work our team does, which is more about exploration over time; a fluid process rather than check-list item.
• Coloured by past experience – Any term that has been around for a while, and widely misunderstood or misused, will be horribly tainted by the experience stakeholders have had with things labelled with that term. This is certainly the case with “usability testing”. I often see this as a tendency towards quant; people expect task failure rates, ‘time on task’ and other rigid measurements and won’t give up on those kinds of outputs from our work. Again, these are rarely the things we are looking to obtain.

Don’t get me wrong, if you practice a method that does live up to all of these things, and you call it usability testing, good on you. Our team rarely does, so I don’t want to set an expectation in the minds of my stakeholders that that is what they’re going to get. We needed a new name.

How does CIDeR work with other techniques?

The CIDeR approach is qualitative and indicative, rather than conclusive. Which means that some findings (ie opinions, perceptions, propensity to buy/use) may not be representative of the larger population, and as such it is necessary to:

1. exercise care in taking these findings on board, using them in the right way, and
2. make use of quantitative methods, either before or after CIDeR, to determine the implications for the broader audience.

Sometimes a more formal method for involving users in the design process is used, which we do call “usability testing”. A more rigorous approach is taken to assessing how easily users are able use a given design, typically later in the design process. Because this technique is dealing strictly with usability, it is acknowledged that relatively small sample sizes (~5) can be used to draw conclusions about the usability of the design for the entire audience.

Questioning regarding opinions or propensity to buy/use, however, do require larger sample sizes. So, alongside both the CIDeR and “usability testing” methods, quantitative research may also be employed, typically to gauge reactions to a product proposition or design. This focuses more on supporting decision making at a product level as opposed to the design or user interface level.

“Pour me a glass!” or “Ewww that’s left a bitter taste”?

What do you think of the name CIDeR? Would you use it in place of the term usability testing? Why or why not? All feedback greatly appreciated.

(Originally posted to the USiT blog, reproduced here with some minor alterations)

Related posts:

1. Stop calling it usability testing
2. Testing a Web site’s first impression
3. The Claw – mobile device usability testing jig

There’s nothing like some real world feedback to improve a product, and Angus was most helpful in this regard. Besides the introduction of AMCap, Angus pointed out that the webcam mounting solution wasn’t the best. The original screws from the webcam base were too short to make it all the way through the perspex and securely hold the webcam, even with the countersinking. So I’ve enhanced the design by using meatier screws (wood screws actually) that are longer than the original and with wider thread to really bite into the plastic of the webcam. Note this is a destructive enhancement, the new screws will wreck the hole for the original screws and you’ll no longer be able to attach the webcam’s circular desk stand.

The new screws also have bigger heads that a normal screwdriver will drive (not requiring a jeweller’s screwdriver like the original screws). So not only are the webcams held nice and securely, but it’s easier to undo and move them.

The other enhancement Angus suggested was to use ‘velcro’ on the handset, so it can be taken off the Claw for setup changes, but then securely re-attached. The ‘velcro’ strips are Command Picture Hanging Strips (annoyingly, they don’t refer to them as ‘velcro’).

I also tied the two USB cables for the webcams together with cable ties, making them less messy when using the Claw.

The Claw is evolving!

(Originally posted to the USiT blog, reproduced here with some minor alterations)

Related posts:

1. The Claw – mobile device usability testing jig

The ominous black shape featured in last week’s “Guess that object” post is in fact my take on a mobile device usability testing jig, inspired by the work of Kirk Henry of Lokion Interactive (via Harry Brignull). I’ve been working on this device to help with testing site and app designs on mobile phones and tablets. Quite often these contraptions are called a sled but I’ve been calling this one “The Claw”, for hopefully obvious reasons.

Its purpose is to allow you to get a good view of the screen of a mobile device—handset or tablet—as well as the user’s face, during usability testing (or any other activity that you’d like to see what’s happening while someone uses a mobile device. Using software such as TechSmith Morae 3.0, you can easily record from both cameras.

A key feature of this particular design is its flexibility, it’s attached to the device and moves with it, and it can be used for two different sizes of device: smaller phones, handsets, smartphones (eg iPhone, Android, Blackberry), as well as larger tablets (eg iPad, Samsung Galaxy Tab 10.1). This second feature is achieved by having two sets of attachment locations for the cameras: one lower down for handsets and one higher up for tablets.

Design and development

After a much earlier prototype built out of a desk lamp (left side of photo below) a major design decision was made, that the rig needed to move with the mobile device, rather than sticking the mobile to the jig and asking the user to use it fixed in place.

While the claw is tethered to a PC via USB cables, it still allows quite a lot of freedom of movement and lets the user hold the device more or less naturally.

A second prototype was built using a wire coathanger, to get the dimensions and angles right before committing to a building material that wasn’t as pliable (right side of photo above). Two things became obvious and resulted in two distinctive features of the final design.

The downward-facing camera that records the screen of the device needs to be at the right height and angle to get a good view of the screen. If angle A is too much more than 90°, you either can’t see the whole screen, or the view you get is at a steep angle meaning the perspective distortion is high (the end of the device closest to the camera appears much smaller than the end that is furthest away from the camera). Additionally the end of the mobile device furthest away from the camera may be out of focus compared to the closer end.

Also, you’re balancing getting a good view of the screen with obstructing the user’s view of the screen. This is especially an issue when the camera is this close to the device.

Another factor to take into account is the ‘focal length’ of the camera being used. The Microsoft VX-6000 webcams I used couldn’t be placed any closer to the device than I’ve placed them, otherwise the picture was blurry and the camera also gets in the way of the user viewing the screen.

I also discovered that in order to provide a good view of a larger device like a tablet (without too much of any angle) the downward-facing camera needed to be more directly overhead. Hence the second bend in the claw that brings the jig back past vertical (ie angle B). The height of the camera was also a factor, as having it too close to the device means that you can’t see the whole screen of a 9-10” device. As before, a balance needs to be struck between getting a good view of the screen and obstructing the user’s view.

After some experimenting, the angles and heights were worked out and it was time to build the next prototype…The Claw.

Manufacture

With very little budget, and because it was just a prototype, I decided not to use something like Ponoko but instead to make it myself. The Perspex is 10mm thick, to give it strength and stop it bending when in use. You can get it at any plastics fabricator, I bought this black specimen from Australian Plastic Fabricators for about $20.

There are also places that can cut, bend and drill any design you want (including Australian Plastic Fabricators) but it can be expensive if you’re ordering a low number of units, like one. So I again decided to do it myself.

Note: 10mm Perspex is hard to bend! :) If you have a heat gun (used for heat-shrinking, stripping paint, welding copper plumbing pipes etc) it should be fairly easy, especially if you have a nozzle that focuses the heat to a narrow strip. I didn’t have one. So I used my toaster. I don’t exactly recommend it, but it does work.

1. Print out a scale diagram of your design and place it on a heat resistant surface right next to the toaster (you’ll use it as a guide to bend the perspex)
2. Mark the Perspex where you want to bend it
3. Place the Perspex over the top of the taster, with the mark positioned above one of the slots (don’t stick the Perspex into the toaster!)
4. If you have a multi-slot toaster you might need to cover some of the slots so that your Perspex only gets heated in one spot. I used slices of bread to do this, since they were right there next to the toaster and I got a bonus snack! Don’t completely cover the slots of your toaster might have a meltdown
5. Adjust the toaster setting to a fairly high/long setting, I had to heat the Perspex for about 4 minutes, but experiment a little and see what it takes for your toaster to make it pliable enough. You don’t want to melt it, firstly because it will give off fumes but also because it’ll likely stretch as you bend it, you want it about as pliable as a paper clip
6. Once you think it’s ready to bend, use oven mitts or some heat resistant gloves to pick up the Perspex and place it on the scale diagram and bend to the desired angle and hold it in place
7. Do this quickly as the plastic will cool and stop being flexible within 10 seconds or so

[A toaster oven might also work, but it wouldn’t be very precise in terms of where it heats the plastic, and you’ll probably burn your hands. Just buy/rent a heat gun :) ]

After bending is when you drill and cut the holes. If you drill and cut before bending, the lower structural rigidity will probably mean it won’t keep its shape. The purpose of the drill holes is to mount the cameras. The cameras I used are both Microsoft LifeCam VX-6000 models, fairly cheap but also fairly good quality, but probably their best attribute of this webcam is it’s tilt and swivel bracket that attaches to a round base. If you unscrew the base and removed it, you can mount the tilt and swivel bracket to the perspex, using the original screw. The hole for the screw will need to be countersunk because the screw is not long enough to make it all the way through 10mm of Perspex.

The purpose of the large cavities is to save weight. That’s why I used 10mm Perspex, because that thickness would mean there would be enough strength after the weight-saving holes were cut. Also these large holes are a neat way to thread the cameras’ USB cables out the back of the claw.

To make the cavities I drilled holes around the perimeter, very close together and then bashed out the centre and filed down the edges. It’s an old trick I picked up somewhere, probably making stuff in the garage when I was a kid, but you could drill one hole and use a jigsaw to cut around the perimeter. The weight-saving cavities remove about 32% of the weight, but leave structural integrity.

Cameras

The cameras I chose are good enough for the job, but to further advance the design, smaller and higher resolution cameras might be used. But is likely to mean a move away from USB webcams, making for a more expensive and complicated exercise.

To attach the mobile device to the rig, I use 3M Command Strips (normally used for hanging pictures on walls etc). The advantage of these strips over double-sided tape is that they can be easily removed and they don’t damage the mobile device or leave any sticky stuff on it.

Morae setup

Morae 3 allows you to record from two webcams, and it’s pretty easy to do so. Start with a hardware or mobile recording configuration and set the “Main” source as the bottom webcam (the one pointing down at the device). I mount the webcams upside down to allow the tilt mechanism to work, giving an easy way to adjust the angle of the camera in relation to the rig. So in Morae you need to flip the camera using the camera settings.

Set the “PiP” source as the top webcam (the one pointing at the user’s face). Again I mount it upside down so you need to flip the image in the settings.

For a reason I can’t currently figure out, Morae won’t use the microphone in either of the webcams as the audio source for recording, so I use a Logitech USB desk microphone placed nearby, it picks up audio well enough.

That’s it, the only things you need to do each time you use the claw is adjust the camera tilt and angle, and maybe adjust the focus ring (in case it has been bumped since last use). Then you plug in the USB cables, stick on your mobile device, launch Morae and away you go!

Future development

The claw is currently being is used for several different projects, on both handset and tablet devices. Next steps are to make some refinements to the angles and dimensions and then produce a better quality version using a service such as Ponoko.

(Originally posted to the USiT blog, reproduced here with some minor alterations)

UPDATE October 31st 2011: I’ve just written a new post about some enhancements I’ve made to The Claw

Related posts:

1. The Claw 2.1
2. My laptop thinks it’s a mobile
3. Stop calling it usability testing

The answer will be revealed shortly, along with a full explanation.

(Note: you NDM folks are disqualified from entering!)

No related posts.

Seeing as I firmly believe that for every unpublished blog post there is one less bit of momentum keeping the interwebs spinning, I’d better put this up. And it’s interesting to look back at what I wrote two and a half years ago…

Day 1 started with a very inspiring talk by Bill Buxton. I think this was just the thing the industry needs, a bit of a reality check and a wake-up call. Firstly usability evaluation is not design and for that reason most people here don’t actually practice User-Centred Design. It’s all about data, rules, strict methodologies, large companies. They’ve even turned agile into something overly defined and bogged down (I have no strong belief either way when it comes to agile methodologies by the way). Bill’s talk about sketching as an important tool for the design process flies in the face of the artefact centric practice many Usability Professionals follow. No there’s no template for it, no there’s no software tool to do it, you have to use your brain! I mean the theme of the conference (“patterns”) says it all really.

This sounds really negative, but I don’t want to be. There are some smart and talented people here, but overall the industry is weighed down by strictness and illusions. Strictness in the sense that many people want some methodology to tell them what to do. I can understand that, but as Bill said, if you find yourself thinking that all the time (being scared of wining it) then maybe this isn’t the job for you. Illusions in terms of the discrepancy between literature and practice. A lot of the things published are not followed in practice (eg rapid, flexible approaches by clever people are replaced by limited, templated projects) and good practice is not published (eg using multiple design alternatives in usability testing). Then there’s the illusions of grandeur, like the way many practitioners think of what they do as some kind of scientific crusade and admitting there is some I-don’t-know-ness to it is an act of heresy.

For me, the best thing I saw at the conference was this talk. It’s a pity someone from outside the field (perhaps technically but really as far as I am concerned he’s slap bang in the centre of what we should strive for) had to be the one to say it. You can’t truly be doing UCD if you’re just evaluating, testing and documenting. This shouldn’t be about statistical analysis techniques.

I remember thinking that my approach to my work seemed at odds with how other attendees appeared to be working, and from the above it seems this annoyed me! Too many practitioners being more worried about following the ‘proper’ process, rather than actually thinking. And the post I did publish at the time, contains similar thoughts.

No related posts.

There were some good things being said, and by real people who have done the hard yards. They’re not “industry luminaries”, but people out there in the trenches working out how to create effective intranets. Grounded and real are two words I would apply to the conference.

Then there was my presentation, a tad more abstract, but I felt it went well. Slides below.

I felt compelled to steer my presentation towards audience participation, if only because of the collective knowledge in the room; about half the room were presenting at the conference so I was learning as much, if not more, than I was dishing out. That’s the downside of being a consultant, you rarely get that rich experience that in-house staff have. Some great examples were offered by the audience, complementing my own examples.

There were one or two people twittering, you can follow the conversation on #smid.

Happy to hear your comments on my slides, either here or on slidehsare.

Related posts:

1. Strategically managing intranet developments
2. Using cultural probes for intranet user research
3. Design research workshop at OZ-IA

More than likely your content management system (CMS) will have many usability problems if you just use it “out of the box”. Having been involved in a number of projects tasked with implementing a these types of systems—including content management systems for websites, intranets and wikis for knowledge management—I’ve noticed that there are a number of key areas of the user interface that frequently need fixing from a usability point of view.

All the usability tips you see here link back to general usability principles, and they apply to any software package or web application, it just seems that they are an issue in most CMS implementations.

Use these tips to improve your current CMS or to help you when implementing a new one.

1. If in doubt, leave it out

The user interface should be devoid of everything that is not necessary in terms of users completing their tasks. Most CMS products will have capabilities in excess of what is being used, but don’t show it if they don’t use it. And many products will have optional extras and upgrade possibilities, so your version might not have all the bells and whistles. For better or worse, some vendors will leave a stub to these missing features (possibly to help encourage up-sell). Don’t show it if they can’t use it.

Use CSS to hide stuff if you have to, but clean up that interface. We’re talking about main navigation, links, and irrelevant details spat out by the system. This also applies to words; as Steve Krug said “Krug’s third law of usability: get rid of half the words on each page, then get rid of half what’s left”. Each page title, sub heading, button label, navigation label, form field label, icon and graphic should be useful and meaningful, clearly communicating what it should.

2. Shield users from the complexities of the system

Your CMS might be complex and all powerful, but users don’t care. The user interface should provide a level of abstraction away from the inner workings of the system, performing a translation between what users want to achieve and the technical functions that make it happen. Don’t expose the user to the inner workings of the system by referring to things such as “asset models”, data structures and other things found under the bonnet. These things relate to how the system performs what the user needs it to. Users do not need to know about these things and will be confused by them. They are a dead giveaway that an interface has been built “by developers for developers”.

Frequently you’ll find this kind of thing popping up in “error” messages and terminology used. But also in the convoluted processes users have to go through and the assumed knowledge they need to interact with the system in even the most basic ways. For example, a common problem in many CMS products is ‘pogo-sticking’ or ‘hub and spoke’ where users are forced to go back and forth between different parts of the system to complete their task, usually with some degree of ‘double handling’ or duplication.

3. Speak the users’ language

On that note, the user interface is fundamentally about communication. You have to communicate clearly, using terminology the user will understand and in a way that allows them to take the necessary action and continue with what they were doing. This effects navigation, notifications and messages, button text, form labels. Of course, it goes without saying that jargon and internal system techno-talk should be nowhere in sight (although you might want to use business jargon that is specific to your particular organisation, see tip 10).

User research can help uncover the language that should be used, and Defensive Design for the Web by 37signals gives excellent advice on error messages, notifications etc.

4. Know your real users

What the heck, let’s go all the way and find out who it is we’re creating the system for. Who are the end-users? Chances are they are not techies; they’re not developers and they’re not systems administrators. Which is a shame because they won’t get how perfect the database schema is or how scalable the architecture is. In fact they don’t care about how it’s built at all!

Sure, developers and sysadmins may be users of the system, but they unlikely to be the majority of the users. The end users are going to be writers, producers, product managers, sales people, editors…human beings (ok maybe not the sales people :).

Do some research, find out what, how, when and why these users are using the system. Talk to them, watch them, crunch some server logs, do a task analysis. It’ll all inform the design and help you work out how to apply the tips you see here.

5. Don’t forget the real-world purpose

The acronym “CMS” is made up of three words, one of which is much less important than the other two: the S. Don’t get caught up in the magnificence of the system and forget its true purpose, which is to manage (and create) content. That’s why users are using it!

Everything in the design and construction of the system should keep this in mind. The purpose of the system is not scalability, data synchronisation, referential integrity or network security. These are simply a means to an end, and should stay behind the scenes. Techies will focus on these things, which is fair enough, it’s their responsibility and their reason for being. Which is why there needs to be someone calling the shots in the project who understands that the job of the user is not these technical aspects, it is about creating and managing content.

6. Do a few things really well

Getting to know your users should include some kind of task analysis, find out what the most important tasks are that your users perform. Prioritise and focus on these key tasks, do them well. Customise the interface, automate some tasks to simplify processes, whatever it takes but make doing those tasks as easy as possible for users. Allow them to initiate those tasks from the home page.

For a busy, content heavy website, a key task might be writing content. This is “on the critical path”, to use project management parlance, so ensure you get that right if nothing else. Similarly, for a very large site with tens of thousands of content pieces, a key activity might be searching for content (either to re-use it or attach it as a related item). In this case make search work really well, perhaps structure the system around searching.

7. Use natural mappings where possible

In the case of a CMS, a natural mapping is where the user interface mimics the action being performed. A great example is when editing the placement of items on a web page. Typically in a CMS this is achieved using a form to capture input (unless you’re using “in-place editing”). But this isn’t natural, since the form doesn’t bear any resemblance to the finished product. It seems trivial but re-arranging the form fields on the edit page to be similar to the finished page being created, will make it much easier to understand which bit is which and where they should type what. Or provide a visual map that illustrates that each form field relates to in the real world. Or your could just label each field so it makes sense :)

A good user interface mirrors the mental model of the user as opposed to the internal system model.

Another example is if dealing with images, show them. This could be thumbnails when searching or listing, or bigger versions when viewing properties. It’s much easier to identify content by visual attributes that will appear in the end product, such as what an image looks like or title text or name of a page. Compare this to an arbitrary system generated ID, which means nothing to a user.

8. Be consistent

The user interface should be consistent across screens, pages and components. This includes navigation, buttons, form controls, text styling, link styling, form layout, terminology and feedback mechanisms (ie alert boxes or ‘yellow fade’).

This is especially the case if certain parts have been customised, which normally stand out as being substantially different in look and feel. Delivering a “Frankenstein” is a sure fire way of making the system look like a hodge podge which won’t help user’s perception or acceptance of a new system. They must feel it’s a quality product.

9. Stick to established conventions

Not only should the CMS be consistent within itself, but it should also be consistent with what users are likely to expect from other similar systems. For example, if the CMS is a web application, standard website conventions apply, such as the placement of the search box, use of form controls such as dropdowns, and single click links. As Rich Internet Applications become more prevalent, additional conventions will emerge, but this is not a licence to do whatever you please. Functionality hidden behind unconventional methods of interaction (such as right-click or double-click for web) will not be discoverable because users will not expect they can interact with the interface in that way.

10. Tailor to suit your specific environment

Consistency also extends to making the CMS look and feel like other tools and applications provided to your users in their Standard Operating Environment (SOE). Integration with these tools will be hugely beneficial, be it ensuring content from the SOE word processor can be easily copied and pasted into the CMS, or ensuring the web-based CMS works in the SOE browser, or perhaps integrating the CMS into the organisation’s single-sign-on so that users don’t have to login separately.

Tailoring the CMS might also mean using consistent visual design and branding, making users feel like the CMS is “theirs” can go a long way towards them accepting the new system and creating emotional investment.

This tailoring should also include removing unnecessary items (tip 1) and abstracting a simpler interface (tip 2) especially for key tasks (tip 6) and speak their language (tip 3). For example, if the specific purpose of the CMS is for an intranet, then the user interface should be tailored to make reference to that, rather than a “website” which might be the out-of-the-box terminology.

11. Create an effective “home page”

A home page or start page or dashboard for the CMS is a great idea if well crafted, containing useful items. Don’t load it up with out-of-the-box clutter (typically “workflow” related items) find out what your users need to know when they first login. Use this page to give easy access to key tasks, by prioritising all the things that the system can do, and giving prominence to the core tasks users need to perform (displayed bigger, higher on the page and perhaps with icons). Infrequently used or less important items can be smaller and further down the page, if they’re shown at all.

Hand in hand with this is streamlining the login process itself. If the CMS allows editing of multiple sites, but a user only ever edits one of those sites, then don’t make them have to choose the site every time they login; an unnecessary step standing between the user and them getting on with their work.

Furthermore, consider dispensing with a dashboard or home page altogether and take users straight to the single most common task (if there is one) and let them navigate to the home page when they need to.

12. Get the forms right

The primary method of data entry in most CMS products is a form. And typically they’re not very usable, so get this part right. The forms should be well designed, with logical structure, meaningful labels, proper validation and a clear call to action. As mentioned in tip 7, making the form more natural in terms of its relation to the end-product will make it much easier to use and understand. Web Form Design by Luke Wroblewski is an excellent reference for creating usable forms.

13. Get a designer in on the action

Don’t leave the design of the user interface up to the developers who are building/installing the system. Even if they do have the required skills for interface design, they necessarily lack the perspective because they have been involved in the nitty gritty of the inner workings of the CMS. Even experienced UX practitioners find it hard to maintain an end-user perspective if they get wrapped up in the technical complexities of building the system.

Most organisations would not think to use their graphic designers or UX people on the tool as opposed to the end-product. But this underestimates the importance of what is often a mission critical tool, particularly if you’re in the business of publishing information on a large website.

14. Don’t rely on training

Training users will not completely resolve usability issues. Long after training sessions and user manuals have been disregarded, an unintuitive interface or overly complex system will still be causing problems. In the worst case, even the trainers themselves will have difficulty explaining how the system works and how it should be used, as they try to bridge the gap between the technical system and the users.

While CMS training programmes make everyone feel better, they are rarely effective in my opinion. Why not just spend that time and money making the system easier to use? Then anyone should be able to use it without needed repeat or refresher training sessions.

15. Provide user assistance

Training won’t save a poorly designed system, but you should provide help. This might involve some instructions that explain what users are supposed to do. These can be shown during a “training wheel” stage or in a “n00b” mode, such that they turn off once users are comfortable with using the CMS.

Other than instructions, there should be help content in-situ, when and where it’s needed to help users perform a task. As with notifications and “error” messages, the help should be helpful. It shouldn’t simply re-state what users can already see for themselves on the user interface (eg “this screen contains the XYZ buttons”) but should tell them how to use a function or feature, what’s expected from them and how to recover if something goes wrong.

Of course, there are ways to reduce the need for instructions and help content, simply through the design of the interface making it more obvious how users should use it: good labelling for buttons and navigation, clear visual hierarchy for pages and forms, and prominence given to the most important items on a screen or in a list. The sequence a user should follow, and what they should be focussing on at each step, should also be clearly indicated.

16. Consider an expert interface

There will be some (if not a lot) of push-back from developers when it comes to these tips: “but it is useful…for developers”. And, there is certainly an argument for this if developers and other technical roles will be using the system alongside normal users (although perhaps fewer in number). You want to give them access to use the power of the system and the information they need to administer the CMS and the end-product site, without affecting what normal users see. The view a user sees could be based on their login.

17. Don’t release something half cooked

There is also usually a really strong push to do the bare minimum and then fix it later. That is, launch the CMS out-of-the-box and then “take care of usability later”. This is a bad idea. Firstly, launching something that isn’t working well will not engratiate you to your users nor give them much faith in the new CMS. Secondly, fixing usability problems later on will require users to re-learn the system all over again. Thirdly, “later” rarely ever comes, especially if the business has just invested a lot of money in buying/building a CMS, they will naturally expect something that is finished and the best it can be.

This doesn’t rule out continual improvements to the system—including the user interface—over time but it means the CMS should be usable and useful from the get-go. The user experience need to be taken into account when planning the implementation of the CMS and shouldn’t be tacked on to the end, otherwise there will never be enough time or resources to include them before launch.

Conclusion

Is this important? As long as the tool works everything else is just icing, right? Well no. Lack of usability will make the system difficult for end-users to learn and use, resulting in lower up-take of the CMS, or the rise of unauthorised workarounds.

And these tips are just the start, the obvious things. They can help you improve the usability of the admin end of your website but perhaps it’d be better to choose better in the first place. Concentrate on the key things that need to be done and don’t get caught up in “featuritis” or go over the top with future-proofing (ie including requirements for things you might use one day). Make sure the CMS you’re considering does the key things well, possibly using scenarios to evaluate products. If you need to customise—particularly the user interface—then make sure this can be easily done and doesn’t interfere with upgrades in the future. There should be independence between the user interface and the system.

[Photo courtesy of Brymo]

Related posts:

1. The IM family

In my recent IA tutorial for OZCHI08, I told my students about performing low-fi usability testing on a draft information architecture. I introduced them to a technique which Donna Spencer called Card Based Classification Evaluation (CBCE) and is known to other people as tree testing or task-based information architecture testing (which really doesn’t have the same ring to it).

(Incidentally, CBCE is an evolution of closed card sorting but crucially different. I quite like Donna’s explanation for the need for such a technique from about the time she coined the term: Categorising information and finding it are two entirely different tasks, with entirely different cognitive processes. The only way to test whether a classification will allow people to find information, is to ask them to find information…You don’t learn it by asking them to place information in the classification. Hence just using closed card sorting won’t do.)

As usual, some people quickly tired of my affection for fiddly index cards and asked if you can do this electronically or even online. It reminds me of that line in Star Wars where the dude says to Vader “your sad devotion to that ancient religion has not helped you conjure up the stolen data tapes”…and you know what happened to that guy. Rather than crush their throats with my mastery of the Force, I explain that there are some advantages to good old cards:

• Cards are quick and easy to create and modify
• Cards remove the distractions associated with ‘using a computer’
• Cards allow a greater affinity between participant and facilitator
• Cards function consistently for all participants (whereas software may not)
• Cards can be made by anyone (eg don’t require special skills or software)
• Cards are cheap

However, I must admit there are situations when something more computery might be more useful, such as:

• You need to include remote participants (eg widely dispersed target audience)
• You intend to involve a large number of users in the testing (eg more than 20)
• You want to spend less time recording results into an electronic format (ok I’m reaching now)

You can use one of the many card sorting tools—either desktop software or online—but there are also a few dedicated tools. For example, Treejack which Sam Ng from Optimal Workshop showed at OZ-IA this year. It’s quite a nice tool and would do nicely for getting that all important user feedback on your design decisions early (and possibly later) in the IA design process.

Of course, one can use something like PowerPoint or Dreamweaver if one likes, it all depends on the resources at your disposal and the capabilities of your test participants—you might be a FileMaker whizz but there’s no point sending out a test app built in it if your users can’t open it.

Feel free to leave a comment below if you have any other suggestions.

No related posts.

Now that I’ve got your attention, let me clarify what I mean. When we refer to this activity called “usability testing” there are often a lot of misunderstandings. It’s really not very applicable for the thing we should be doing. Here are some reasons why…

• It gets mistaken for UAT.
• It gets mistaken for technical testing.
• It makes it sound more ‘scientific’ than it (usually) is.

Usability testing is not UAT

UAT, or User Acceptance Testing, is a term used in software engineering to describe when the client would give final approval for the built system to be delivered. This becomes really confusing when the term is taken out of that context and used in web design, for example. When you don’t understand the “user” it refers to and what is being “accepted”, you might very well think UAT is the same thing as “usability testing”.

Conceptually, the intention of UAT might be to make sure those who will ultimately be using the system are happy with it. But in practice, this is rarely the case. I could go on, but this is not a discussion of why I don’t like UAT (which I don’t) but rather my point is that we don’t want stakeholders thinking that usability testing is UAT, and thus something that can be dispensed with because “we always go through UAT”.

Usability testing is not done by Mr Test Manager

The term “usability testing” often gets misconstrued by technical types, project managers and business analysts. It gets turned into a stale, rigid, bureaucratic affair. The old “unit, integration, system” mantra. It’s done as a matter of course, at the end of the gantt chart, to tick a box. That’s pointless.

Again, in theory, test driven design is not a bad thing. Software, websites and anything technically complex should be checked to make sure it has been built as was required. That assumes a lot though, for example are the requirements valid? do users actually want or need what is being built? But let’s leave that one alone.

What I’m trying to say, is that usability testing shouldn’t be mistaken for the technical testing done by a Test Manager according to a test plan, using a test script. (At least not the kind of usability testing I want to be doing, which is possibly a qualification I should have stated up-front). People do run highly structured usability tests—typically summative in nature—which are very similar to technical testing. In my experience this is the minority of cases and the least valuable. On to my next point.

Usability testing sounds really scientific

Following on from the tail end of that last point, the term “usability testing” makes the activity sound more definitive, more scientific. Let’s be honest, even when we try, it usually isn’t. But it doesn’t have to be!

There is a place for large scale, highly structured, task-timing testing, but often what is most useful in terms of formative (but also summative) user input is something more simple. Many use the term “guerilla usability testing”, but that’s really just cowering in the face of academics and purists who scoff at our “lax methods” and “dismal sample size”.

If we don’t put that connotation of science on it, we won’t have to battle questions over statistical significance, or waste time defending something that gave useful results and improved the design process.

Closing the loop with user feedback

So let’s stop calling it usability testing. Let’s call it what it is: feedback, confirmation, validation. Showing people who will be using the thing we’re designing, and getting their feedback. It should be a natural part of the design process, closing the loop to ensure that what we’re designing it usable and useful for the intended audience.

Don’t show your designs to your boss, project manager or stakeholders for “approval” (ok not just to them). Show them to the only people who can truly sign off on them, your users.

I’m talking informal sessions. Collaborative or participatory design, if you will, but not testing.

UX practitioners often call it that to make it sound more than it is, give it more persuasive weight and the importance—or should that be respect?—that it deserves. I did it today in fact (well the day I started writing this post).

But no more. Let’s call it what it is and act like it’s part of the process. We can start to educate our colleagues and get them to the point they assume it’s part of the project too. “No, Miss Project Manager, we don’t need to wait until UAT to see if all the money we spent has paid off”.

What do you think? All feedback appreciated.

[Photo credit: Closing the loop by jspad]

Related posts:

1. Testing a Web site’s first impression
2. The Claw – mobile device usability testing jig

I was reading the Qantas in-flight magazine yesterday, and came across a review for the Olympus Tough Smart 1050SW which was claimed to be “astonishingly resilient to holiday rough and tumble”. And as a keen snowsports fan, it would seem to be a perfect camera for skiing. Not only is it sealed, waterproof and able to withstand freezing temperatures (which I can attest most electronic devices are not able to do) but the “entire camera body is touch-sensitive, making it easier to use [whilst wearing] gloves”.

All you need to do is touch the top of the camera to take a photo or you can select “play” mode by tapping the LCD screen. To browse to the next photo you just tilt the camera to the left or right. If only all products were this straightforward to use, and contextually respectful. I’ve not used one, so I don’t know how sophisticated it is at detecting these gestures, but it sounds perfect for snowsports. Apart from not wanting to take off your gloves because it’s cold, doing so takes precious time that might mean you miss that spectacular stack that you really want to capture forever.

This is excellent design for the context of use. There may be other cameras like this, but this is a great example of seeing the problems people have using technology, and then solving those problems in a suitable way. Well done.

No related posts.