Pat’s Point of View

I spent today at Ark Group’s Strategically Managing Intranet Developments conference, which I blogged about before.

There were some good things being said, and by real people who have done the hard yards. They’re not “industry luminaries”, but people out there in the trenches working out how to create effective intranets. Grounded and real are two words I would apply to the conference.

Then there was my presentation, a tad more abstract, but I felt it went well. Slides below.

I felt compelled to steer my presentation towards audience participation, if only because of the collective knowledge in the room; about half the room were presenting at the conference so I was learning as much, if not more, than I was dishing out. That’s the downside of being a consultant, you rarely get that rich experience that in-house staff have. Some great examples were offered by the audience, complementing my own examples.

There were one or two people twittering, you can follow the conversation on #smid.

Happy to hear your comments on my slides, either here or on slidehsare.

More than likely your content management system (CMS) will have many usability problems if you just use it “out of the box”. Having been involved in a number of projects tasked with implementing a these types of systems—including content management systems for websites, intranets and wikis for knowledge management—I’ve noticed that there are a number of key areas of the user interface that frequently need fixing from a usability point of view.

All the usability tips you see here link back to general usability principles, and they apply to any software package or web application, it just seems that they are an issue in most CMS implementations.

Use these tips to improve your current CMS or to help you when implementing a new one.

1. If in doubt, leave it out

The user interface should be devoid of everything that is not necessary in terms of users completing their tasks. Most CMS products will have capabilities in excess of what is being used, but don’t show it if they don’t use it. And many products will have optional extras and upgrade possibilities, so your version might not have all the bells and whistles. For better or worse, some vendors will leave a stub to these missing features (possibly to help encourage up-sell). Don’t show it if they can’t use it.

Use CSS to hide stuff if you have to, but clean up that interface. We’re talking about main navigation, links, and irrelevant details spat out by the system. This also applies to words; as Steve Krug said “Krug’s third law of usability: get rid of half the words on each page, then get rid of half what’s left”. Each page title, sub heading, button label, navigation label, form field label, icon and graphic should be useful and meaningful, clearly communicating what it should.

(more…)

In my recent IA tutorial for OZCHI08, I told my students about performing low-fi usability testing on a draft information architecture. I introduced them to a technique which Donna Spencer called Card Based Classification Evaluation (CBCE) and is known to other people as tree testing or task-based information architecture testing (which really doesn’t have the same ring to it).

(Incidentally, CBCE is an evolution of closed card sorting but crucially different. I quite like Donna’s explanation for the need for such a technique from about the time she coined the term: Categorising information and finding it are two entirely different tasks, with entirely different cognitive processes. The only way to test whether a classification will allow people to find information, is to ask them to find information…You don’t learn it by asking them to place information in the classification. Hence just using closed card sorting won’t do.)

As usual, some people quickly tired of my affection for fiddly index cards and asked if you can do this electronically or even online. It reminds me of that line in Star Wars where the dude says to Vader “your sad devotion to that ancient religion has not helped you conjure up the stolen data tapes”…and you know what happened to that guy. Rather than crush their throats with my mastery of the Force, I explain that there are some advantages to good old cards:

• Cards are quick and easy to create and modify
• Cards remove the distractions associated with ‘using a computer’
• Cards allow a greater affinity between participant and facilitator
• Cards function consistently for all participants (whereas software may not)
• Cards can be made by anyone (eg don’t require special skills or software)
• Cards are cheap

However, I must admit there are situations when something more computery might be more useful, such as:

• You need to include remote participants (eg widely dispersed target audience)
• You intend to involve a large number of users in the testing (eg more than 20)
• You want to spend less time recording results into an electronic format (ok I’m reaching now)

You can use one of the many card sorting tools—either desktop software or online—but there are also a few dedicated tools. For example, Treejack which Sam Ng from Optimal Workshop showed at OZ-IA this year. It’s quite a nice tool and would do nicely for getting that all important user feedback on your design decisions early (and possibly later) in the IA design process.

Of course, one can use something like PowerPoint or Dreamweaver if one likes, it all depends on the resources at your disposal and the capabilities of your test participants—you might be a FileMaker whizz but there’s no point sending out a test app built in it if your users can’t open it.

Feel free to leave a comment below if you have any other suggestions.

Now that I’ve got your attention, let me clarify what I mean. When we refer to this activity called “usability testing” there are often a lot of misunderstandings. It’s really not very applicable for the thing we should be doing. Here are some reasons why…

• It gets mistaken for UAT.
• It gets mistaken for technical testing.
• It makes it sound more ’scientific’ than it (usually) is.

Usability testing is not UAT

UAT, or User Acceptance Testing, is a term used in software engineering to describe when the client would give final approval for the built system to be delivered. This becomes really confusing when the term is taken out of that context and used in web design, for example. When you don’t understand the “user” it refers to and what is being “accepted”, you might very well think UAT is the same thing as “usability testing”.

Conceptually, the intention of UAT might be to make sure those who will ultimately be using the system are happy with it. But in practice, this is rarely the case. I could go on, but this is not a discussion of why I don’t like UAT (which I don’t) but rather my point is that we don’t want stakeholders thinking that usability testing is UAT, and thus something that can be dispensed with because “we always go through UAT”.

Usability testing is not done by Mr Test Manager

The term “usability testing” often gets misconstrued by technical types, project managers and business analysts. It gets turned into a stale, rigid, bureaucratic affair. The old “unit, integration, system” mantra. It’s done as a matter of course, at the end of the gantt chart, to tick a box. That’s pointless.

Again, in theory, test driven design is not a bad thing. Software, websites and anything technically complex should be checked to make sure it has been built as was required. That assumes a lot though, for example are the requirements valid? do users actually want or need what is being built? But let’s leave that one alone.

What I’m trying to say, is that usability testing shouldn’t be mistaken for the technical testing done by a Test Manager according to a test plan, using a test script. (At least not the kind of usability testing I want to be doing, which is possibly a qualification I should have stated up-front). People do run highly structured usability tests—typically summative in nature—which are very similar to technical testing. In my experience this is the minority of cases and the least valuable. On to my next point.

Usability testing sounds really scientific

Following on from the tail end of that last point, the term “usability testing” makes the activity sound more definitive, more scientific. Let’s be honest, even when we try, it usually isn’t. But it doesn’t have to be!

There is a place for large scale, highly structured, task-timing testing, but often what is most useful in terms of formative (but also summative) user input is something more simple. Many use the term “guerilla usability testing”, but that’s really just cowering in the face of academics and purists who scoff at our “lax methods” and “dismal sample size”.

If we don’t put that connotation of science on it, we won’t have to battle questions over statistical significance, or waste time defending something that gave useful results and improved the design process.

Closing the loop with user feedback

So let’s stop calling it usability testing. Let’s call it what it is: feedback, confirmation, validation. Showing people who will be using the thing we’re designing, and getting their feedback. It should be a natural part of the design process, closing the loop to ensure that what we’re designing it usable and useful for the intended audience.

Don’t show your designs to your boss, project manager or stakeholders for “approval” (ok not just to them). Show them to the only people who can truly sign off on them, your users.

I’m talking informal sessions. Collaborative or participatory design, if you will, but not testing.

UX practitioners often call it that to make it sound more than it is, give it more persuasive weight and the importance—or should that be respect?—that it deserves. I did it today in fact (well the day I started writing this post).

But no more. Let’s call it what it is and act like it’s part of the process. We can start to educate our colleagues and get them to the point they assume it’s part of the project too. “No, Miss Project Manager, we don’t need to wait until UAT to see if all the money we spent has paid off”.

What do you think? All feedback appreciated.

[Photo credit: Closing the loop by jspad]

I was reading the Qantas in-flight magazine yesterday, and came across a review for the Olympus Tough Smart 1050SW which was claimed to be “astonishingly resilient to holiday rough and tumble”. And as a keen snowsports fan, it would seem to be a perfect camera for skiing. Not only is it sealed, waterproof and able to withstand freezing temperatures (which I can attest most electronic devices are not able to do) but the “entire camera body is touch-sensitive, making it easier to use [whilst wearing] gloves”.

All you need to do is touch the top of the camera to take a photo or you can select “play” mode by tapping the LCD screen. To browse to the next photo you just tilt the camera to the left or right. If only all products were this straightforward to use, and contextually respectful. I’ve not used one, so I don’t know how sophisticated it is at detecting these gestures, but it sounds perfect for snowsports. Apart from not wanting to take off your gloves because it’s cold, doing so takes precious time that might mean you miss that spectacular stack that you really want to capture forever.

This is excellent design for the context of use. There may be other cameras like this, but this is a great example of seeing the problems people have using technology, and then solving those problems in a suitable way. Well done.

Last week I went along to the Australian International Motor Show (formerly the Sydney International Motor Show) since I had a bit of spare time and a free ticket. Despite being an “automotive enthusiast” I usually find these shows dead boring (but that’s a whole other topic) so I tried to think of a way of making it more interesting.

I combined my career as a usability geek with my new role of daddy, and went in search of what might be our next family car. We’re not looking to buy one yet, but our ever-expanding family will necessitate an upgrade before too long. I visited every stand and examined what they had on offer for this market segment (wagons, SUVs and maybe some large sedans).

After a bit of snooping around, two things became obvious. Firstly the “booth bimbos”, female and male, had very little product knowledge. What’s the point? OK I know the point is to have a bit of totty that will attract people (mainly guys) to the stand. But if you’re seriously shopping for a car, you need to bypass the bimbos and find the actual sales staff. I can’t believe it, I’m stating a preference for car yard dealers!

(In this regard the highlight of the show would have to be the Subaru stand, which was staffed by knowledgeable sales people dedicated to each product line. It was one such lady who told me about the stand-out practical features of the Forrester and in essence started this little usability study.)

The other thing I found was that there are several key measures of usability for a vehicle intended for use by a young family. By that I mean two parents and two kids, one in a child’s car seat and the other in a baby capsule, who go on the occasional driving trip and need to take some luggage in addition to a pram and a stroller. And the mum is rather short :)

The emergent metrics were:

A. Size of boot. A fairly obvious metric. When you need to fit prams, strollers, bags of shopping and maybe some luggage, you need a decent sized boot. Most small SUVs fall down in this area, particularly if their child seat anchors are not well placed (see next point).
Particularly good: Subaru Liberty/Outback, Ford Territory
Particularly bad: Peugeot 308 Touring, Volvo V50, Holden Commodore Sportswagon (space is severely hampered by sloping rear hatch)

B. Location of child seat anchor points. Often overlooked, indeed most staff on the stands didn’t have a clue where they were located in their own vehicles. Quite often they are located on the back of the rear seats (which doesn’t seem like a good design to me) or in a spot in the boot where the car seat straps would decimate the luggage space. Also, in many cars it’s too easy to mistake the luggage net hooks or shopping bag hooks for the anchor points (yes there’s a huge difference, you idiot on one of the stands who suggested I just attach my baby to a plastic hook that wouldn’t take the weight of a half a flea’s butt!).
Particularly good: Subaru Forrester (mounted on roof)
Particularly bad: Skoda Octavia Scout, VW Passat Wagon (on both of which they were surprisingly difficult to find)

C. Opening of side doors. Trying to get a car seat or baby (or both) in through the door and onto the back seat is pretty difficult when the doors don’t open wide enough. Ideally, the doors open out to 90 degrees perpendicular to the vehicle.
Particularly good: Subaru Forrester

D. Height of seating position. With a car that is quite low to the ground, trying to get babies or bags into the back seat can be really difficult; you need to crouch and lean in. Not good for your back. Full size 4WDs have the opposite problem, especially for those who are “vertically challenged”. Most small SUVs are at about the right height, whilst most sedans and wagons are too low. Also related to this metric is the ease of entry and exit for the driver.

E. Height of boot “lip”. For similar reasons to above, the height of the lowest part of the rear door is crucial when it comes to lifting luggage or prams into the boot. If you have a flat lip and a low boot floor level, it’s much easier.
Particularly good: Mitsubishi Outlander (had a high lip but it folds down like a tailgate, very handy)

F. Opening of back door. The space needed to open the rear door of the vehicle to access the boot. Holden talk this up in their advertising for the Sportswagon, the fact that the hatch opens within the length of the vehicle. Some other models do a similar thing with their ‘lift up’ hatch, such that you require less space behind the vehicle and they open almost vertically so you get good access to the boot space.
Particularly bad: Suzuki Grand Vitara, Toyota RAV (both of which have a big ’swing open’ door instead of a ‘lift up’ hatch…very disappointing!)

Note: this list ignores the more common facets normally associated with vehicles—at least by me—such as price, engine size/power, performance, fuel economy. These would of course be important factors in choosing a vehicle, but in terms of usability they are not relevant.

Based on all these metrics, the front runners in my mind are the Subaru Forrester, Mitsubishi Outlander and Hyundai Santa Fe. Should we wish to go a bit bigger then we’d be looking at a Toyota Kluger, Ford Territory or Mazda CX-9 (though the Mazda doesn’t score that well in terms of practicality in my opinion).

Interestingly, there was no stand for Land Rover, Audi, BMW or Mercedes. All three of these manufacturers have vehicles that fit into my category, if not my price range, but I guess a motorshow in Sydney is not high on their priorities. Can you blame them when the majority of the local marketplace can’t see past Commodores and Falcons?

[Diagrams based on: http://www.brian894x4.com/LC62drawing.gif]

My IA for Agencies survey is now closed. Thanks to everyone who took the time to fill in the survey, all 206 of you from across the globe.

I’ve also selected the winner of the prize, a copy of Don’t Make Me Think: A Common Sense Approach to Web Usability, 2nd Edition by Steve Krug. Congratulations!

Now comes the task of analysing the submissions and compiling the results into a useful format. I’m also compiling a list of lessons learned from the process, as there have been a few. I’ll make all of this available in due course (read as when I get some time to spend on it!).

Thanks again to everyone who filled in the survey, those who helped me with it, those who offered encouraging words and constructive feedback, and thank you even to those who felt the need to leave negative comments.

While not everyone agreed with the manner in which I conducted the survey, it’s all been very educational.

Here’s a Friday afternoon thought regarding a classic mistake in trying to get people to participate in a survey. When you tell people about your survey, make sure they can complete it!

Sounds simple, right? Well that’s because it is.

Some background perhaps. A common sight within organisations is the satisfaction survey, your HR department might call it something else but essentially they’re all the same. And like all satisfaction surveys—think of a motel or car rental company—it’s really a poor attempt at communicating with your audience, or giving the perception that you are communicating with them (but I digress). People know this, that you’re not really listening to them (the vague, clichéd questions are a dead giveaway) so you’re already off to a bad start if you choose to run such a survey. But you’re really going to struggle when you ask staff to complete a survey that doesn’t exist…yet.

I saw an example of this today, where an email—and a rather long winded email full of ‘management speak’—was sent to all staff in a large organisation, telling them about the flagship staff engagement activity for the year, an all-encompassing survey (!!). At the end there was a link to the online survey. At this point I was simply surprised it was an online survey and out of sheer excitment decided to click on the link. Error. Not a nice “survey not open please come back later” message, an HTTP 404 error.

Apparently I was meant to wait until the survey opened in a few days time and then go back to the email, click the link and complete the survey. Hmmm. Delete!

Survey completion rates are typically quite low. Without a tangible incentive they’re even lower. If it has anything to do with work, waaaay low. So you’ve got very little chance of getting a decent set of responses, but when the link doesn’t work at the point when people are the most likely to click on it, you’ve lost before you’re even begun.

Let’s not even discuss whether a survey is a good method for engaging with staff, the moral of the story is if you create a call to action make sure the people who you want to act are able to take action. This goes for surveys but also competitions, sales promotions and website launches.

(You may be thinking this post is rather ironic given my own recent attempts at surveying, but this is one mistake I definitely didn’t make :)

I’m running a survey and I’d like your help.

Best practice design of websites, and other digital media, involves a set of skills known broadly as Information Architecture (IA) which generally means making designs user friendly. IA is also known to people doing this work, by such terms as User Experience (UX), User Centred Design (UCD), Interaction Design (IxD) or simply “usability”.

A significant amount of this sort of work is performed by agencies—whether they be advertising agencies, digital agencies or communication agencies. As a practitioner and educator in the field of IA, I am interested in learning how people go about practicing it, in particular how agencies “do IA”. This is to both confirm and challenge my own understanding of the way agencies work and how IA fits into their processes, who it gets done by and how it might be possible to give agencies the skills they need to perform better in this regard.

To this end, I’ve launched an online survey to get some answers straight from the people who work in agencies (or used to). The survey will take approximately 5-10 minutes to complete and I’ll give away, to one lucky person who completes the survey, a copy of the acclaimed best-selling book by Steve Krug Don’t Make Me Think: A Common Sense Approach to Web Usability, 2nd Edition. I recommend this book for anyone considering doing anything to do with IA or usability, so it’s only fitting to offer it as an incentive.

You’ll find the survey at: http://www.gurtle.com/survey/index.php?sid=61824

If you don’t work in an agency, you can still help me out by forwarding this to your clients, peers and friends who do work in agencies. I may run a more general survey in the future, but for now I’m focused on agency folks.

The survey will run until the end of September, so there should be plenty of time for word to get around.

I’ll share the results of the survey (aggregated not raw data), either here on this blog or through conferences in and around the IA community. Stay tuned.

Yesterday I attended the half-day WIPA Usability and Eyetracking Seminar, and found it fairly good use of a few hours of my time. Largely because it helped confirm some things in my own mind.

First up was Challenges for Usability in Agile Development presented by John Eklund of UX Research. There has been much talk about agile development methodologies in recent, and probably as much talk about how user experience practitioners can remain valuable in such an environment.

To paraphrase John, my summary of the discussion is as such:

• Agile is about “bringing design forward” (I like this definition)
• It’s about less documentation and specification up-front
• Acknowledge that requirements will not be fully correct, complete or fixed in stone; learn to live with it rather than boxing requirements gathering into one neat discrete step that must be finished before anything else can begin
• Agile is also iterative or incremental development
• Partial prototypes help elicit requirements and specifications from the client
• Clients rarely read spec documentation and often can’t articulate what they want until they see it (you know it’s true!)
• Creativity is less bounded by specification when the specification is yet set (this is not just in terms of visual creativity but also the overall design directions)
• For UX to fit into this methodology it needs to be embedded, flexible, fast and practiced by an experienced practitioner
• For best results in agile environments, UX expertise should be independent of designer (and client)
• Additionally, UX practitioners must play the “expert advocacy” role (providing ad-hoc advice on simple issues without the need for a costly ‘engagement’ or bulky reports)
• Generally faster turn-around is needed for activities like usability testing

I enjoyed John’s perspective on this topic, and a few of his points in particular are closely aligned with my own views on UX practice. I’ve used mentoring in the same way as John’s “expert advocacy” where UX or IA expertise is bought by the hour, allowing for much greater freedom to add value to the design team without having to get approval for a project each time they want to ask a question.

Next up was Eyetracking – Applications in Digital and Media by Peter Brawn of Eyetracker. I was impressed by Peter’s presentation of eyetracking as part of UX practice, as opposed to how I have seen it pitched in the past (that is as the answer to all your problems). This makes sense, since there is a lot that gaze paths and fixation data can not tell you about the usability of a website, and vice versa there are some things you can’t really get out of traditional usability testing and ethnographic research techniques.

For example, a certain section of a website is not receiving much traffic. Usability testing might tell you that users are interested in the content in that section, but perhaps not why they aren’t getting to it. Eyetracking can tell you that users simply don’t look at the obvious, big, fat link that goes to that section, but not why they don’t look at it. Combined you are getting a more complete picture.

That said, I still believe that, dollar for dollar, other methods are better value than eyetracking. For the cost of the hardware and software (or consultants to do it for you) quite a lot of low-tech testing and research could be done. Ideally, you’d do both, but going back to John’s topic, if you want to ensure UX keeps its foot in the door in agile environments—or any other—the approach needs to be lean, mean and cost effective.

I do understand that many clients want the snazzy visualisations you get from eyetracking, not to mention the snob value, but this is only going to be realistic for big corporate clients. Smaller clients should save the cash and use other methods.