Pat’s Point of View

I’m a big fan of Wordpress, the content management system I use for this blog, and I like the changes that have been introduced since version 2.0 including the admin interface. There are clearly many people who disagree, but I’m liking the direction the admin interface is taking.

The Wordpress team are currently in the process of redesigning this interface for the next version (2.7) and as is becoming quite popular these days, they are opening some of the process to the public and even enlisting their huge user-base in the design process. The latest output of this process has been a set of wireframes. Wireframes are a tricky thing, and everyone does them differently based on the needs of the project. I would have done some things differently to the Wordpress 2.7 wireframes, but overall they look reasonable.

However, I was quite surprised that a survey was used to “give WordPress users the ability to play a part in deciding how the navigation options should be grouped and labeled”. This doesn’t sound like a job for a survey. Unfortunately I caught onto this only after that survey had closed and so I don’t know what sort of questions were asked. Was it collecting feedback on the current navigation? Was it asking for design ideas? Or was it something more like online card sorting?

The first survey was followed by another survey intended to “take your opinions regarding: Where to put the search box, Where to put the Add New Post button/favorites menu, How to label the Future Publish/Edit Timestamp function”. I missed this survey as well, but again it sounds like a survey wasn’t an appropriate method to use. I fear this was a glorious example of design by committee.

It seems they got 5000 responses to the second survey, in a matter of days, so it’s likely there at least as many for the first survey too. That’s a pretty good sample size and would be great for learning more about the audience. What troubles me is the team’s apparent reliance on the surveys as a major input into the design. Were there other methods used, such as usability testing, interviews, analytics, workflow analysis? I hope so.

Getting input from users is commendable, but a better approach to a similar challenge is Leisa Reichelt’s work on the Drupal.org redesign. Lisa also had massive online participation from the user-base, but used a variety of methods to research requirements, needs and preferences. The key difference is the appropriateness of each technique to the information intended to be gathered. A survey was used “to get in touch with people who are willing to help us out with some more structured feedback once we start getting into the design phase” rather than to elicit design preferences. Participatory design or collaborative design shouldn’t be done via survey.

But, of course this criticism could be completely unfounded, as I haven’t seen the survey questions. And perhaps I’m totally jaded by the big surveys I’ve seen being used inappropriately in the past. So, if you completed either of the Wordpress surveys, let me know what kind of questions were asked by leaving a comment below.

Popularity: 100% [?]

Short stories are an excellent way of communicating user needs; not just use cases but needs, desires, joys and frustrations. While it was written as a creative short story, “To do” is a very realistic depiction of how someone might use personal organisation tools, and more importantly how they might feel about their experiences. Here’s a taste:

Every time I cross one item off the list, another seems to appear. Making a dental appointment should be a cause for celebration, a satisfying tick; instead, it leads to more appointments, white waiting rooms, condescending child-receptionists and x-rays. And while I now know exactly what I need to do, it’s getting harder to keep track of the when.

Read more

This story has feeling, it is human. Compare this to the way in which user experience requirements that have been captured for one of your projects. I think there’s a huge benefit in presenting the needs of our audiences in this way. It goes beyond a mere scenario, in that it uses the power of creative writing to put you inside the head of the person using—or should that be surviving?—the things we design.

I know the author of this story probably wasn’t describing a real series of events, but the story would have been inspired by real situations, pieced together to create a whole story. This story represents his experiences using to-do lists and other productivity tools. You can easily do the same thing based on the findings of design research.

Popularity: 10% [?]

They say you learn more from your mistakes than you do from your successes. Amen to that. They also say that you should be man (or woman) enough to admit your mistakes. So here I am, bearing my soul to the internet, saying I screwed up and have learnt valuable lessons.

Yesterday I ran my Research Methods for IA workshop for OZ-IA, and whilst I was actually quite happy with the content, the timing was off big-time. Luckily my gracious guests for the afternoon agreed with this summary, the feedback forms were overwhelmingly positive for the content, materials and myself…but the time allocated was totally inadequate. You see, I tried to cram a whole day into half a day.

Why did I do such an idiotic thing? Why did I break one of the fundamental rules of workshops (and lots of other things) and not keep it simple? Well there are lots of excuses I could give, like when I first planned the workshop it was for a full day, or that I’m so used to doing full-day workshops that my ‘internal rhythm’ was set to that, or that this is my first workshop I’ve designed entirely on my own from the ground up, or that I’ve been burning the candle from both ends lately and didn’t give this as much thought as it deserved. But they’re not the real reasons.

The real reason is that I try to pack too much in because I want to tell people everything. I was trying to be everything to everyone. And that’s not achievable.

Moving forward I know I need to do this workshop over a whole day. Given more time it would have gone brilliantly, and I need to make sure I do it that way from now on. But the reverse is also true, if I only have half a day then I need to tailor the content to half a day. I shall be ruthless for my next workshop, which is on a different topic, paring it right down and allowing plenty of time for activities and Q&A.

Another lesson I’ve learnt through doing this workshop is how much harder it is to do all this on your own. My previous workshops with Step Two were much easier, in that the groundwork had already been done and there were more hands to make light work of tasks such as printing, creating card sorting cards, shipping materials to the venue etc.

And I also learnt that I should take help from those around me, even if it means putting aside time when in the thick of it, to stop and talk it through and get advice. Particularly my long-suffering wife, who is an organisational master. I, being hopeless at organising things, should have consulted her when preparing for the workshop.

For my first time out of the gates on this particular horse, it could have gone better. Whilst this jockey didn’t exactly fall right off his mount, he did get dragged along a bit. But you know what they say, you need to get back in the saddle, and I’m certain that if I keep refining this workshop it will serve me well.

And I truly hope, so because I really like running workshops. I get to help others by passing on my knowledge and experience; it causes lovely warm fuzzy feelings.

Popularity: 8% [?]

I recently conducted a quick survey among my colleagues and peers in the UX field, regarding how to determine which research methods to use for a particular project. My objective was to try and work out the four qualities of any research method that need to be considered when putting together a plan for design research.

For example, interviews don’t allow as many people as focus groups but they can dig deeper into the thoughts of the participant, thus ‘breadth’ and ‘depth’ might be two such differentiating qualities.

I’ll be using this information for an ‘edutainment’ game I’m including in my upcoming OZ-IA workshop, whereby participants will select a set of research methods for a given scenario and limited by a ‘budget’ for each quality.

I got lots of useful replies from a small sample size. Fortunately—I got more than what I had literally asked for (my bad…a poorly worded email). Almost everyone gave me great insight into how they go about planning research, as well as some gentle rhetoric regarding the subtle complexities of the subject and how perhaps I was oversimplifying :) So, I stand duly chastised, planning a research programme is definitely more complex than choosing four qualities, but that’s what I needed for my workshop game. Lesson learned :)

The survey question was framed in terms of user research for web design, but I expected that I would get responses from a broader perspective than this, and my recipients did not disappoint (eg BA’s gave their views from the point of view of business requirements gathering).

My findings were numerous, starting with the fact that it’s hard to analyse responses when they are all wildly different; some people ranked, some gave low/med/high values and some simply discussed the relative importance of the qualities I had listed in my question. So I had to interpret some of the responses in order to collate them. I should have used a proper survey form, but I was under the influence of spontaneity. Another lesson learned :)

Without further delay, here are the results of the survey. These are the top ranked qualities, taking into account all the information I received from respondents:

I’m not so concerned with the order, but it was clear the top factors were the breadth, depth and amount of analysis required.

The cost directly associated with a method were not seen as a useful differentiator since they are all much the same in this regard. Similarly the facilitation time, or face-to-face contact time, was not seen as a crucial factor since it can be varied as needed on a case by case basis, regardless of the method employed. Difficultly, or the effort required, was seen as a crucial factor only by a few people, the rest didn’t see it as a sufficient differentiator (perhaps reflecting their own level of experience in the field).

Total cost (ie time and materials) was seen by many as a crucial factor in that a research undertaking might “live or die” on the basis of this alone, but it’s an aggregate of facilitation time, analysis time and material costs, which are already in the list. So I removed it from the final list.

Two other factors are worth a mention. Firstly, rigour was nominated by one respondent as a reasonably important factor. I took this to mean how easily the research method can be repeated with consistency across sessions. However, I grouped this with difficulty, or effort required, because I see it as a quality of the practitioner rather than the techniques themselves.

Secondly, the difference between qualitative and quantitative analysis was suggested as a key consideration. That is, some techniques will require more discourse analysis as opposed to data analysis. Excellent point, but probably too fine-grained for my purposes.

My conclusion is that I should use reach, coverage, analysis time and effort required as the differential qualities for my workshop game. Feel free to comment on this below.

Thanks to everyone who responded to my email, and for taking the time to make sense of it. Thanks also to James Breeze for reminding me to blog about this…I have been neglecting my blogging lately!

See what people are blogging about: design research, survey, user experience, user research, research

Popularity: 6% [?]

I’m happy to announce I’ll be running a half-day workshop entitled Research methods for information architecture at this year’s OZ-IA conference (actually the workshop is on Thursday the 18th of September).

This is an important part of successful IA, and I intend to give practical insight into some key techniques that will allow IA practitioners, or other disciplines doing some IA, to gain valuable insight into their audience. Here’s my blurb:

Information architecture (IA) is a form of user-centred design (UCD) which requires an understanding of users and their needs and designing with those needs in mind – balanced by factors such as business objectives and available or necessary content. Without a solid understanding of these inputs into the process, design is blind.

Design research is the process of uncovering and understanding those needs, whether it be direct user research or other means of gathering requirements.

Yet, many experienced web designers, developers and IA practitioners don’t feel well-equipped to take on design research in their projects.

In this half-day workshop, Patrick Kennedy will present the fundamentals of design research from the perspective of IA. Specifically, the workshop will be conducted in the context of organising and designing information systems such as websites, intranets and software applications.

This workshop will introduce design research, explain the fundamental principles and teach some simple techniques. The aim is to give the audience a heads-up on the subject and point them in the right direction so they can integrate research into their own work or just better collaborate with design researchers.

You can register for the conference at www.oz-ia.org/2008/register.shtml and don’t forget to quote the discount code PK0265!

Popularity: 10% [?]

What do you do when your client seems to be missing the target?

Take, for instance, the situation I recently experienced regarding a fashion-related website whose owners have—how shall I say—an aspirational view of their audience. That is, they want to target an older, more sophisticated, style conscious women. I’ll call them the ‘fashionistas’. This happens to be the audience for their parent brand, an offline publication, and also happens to be the people with whom the brand is most associated.

However, the reality is that those who use the website are younger women who primarily want to chat about gossip, fashion bargains, celebrities and the like. I’ll call them fashion ‘fans’.

I suppose there is nothing wrong with having an aspirational target audience; those people whom you would like to be your users. But research shows that the fashionistas are simply not interested in using a website; it’s too low rent, not part of their world. These are people who sip lattes in exclusive fashion boutiques. They live high fashion, they’re not chatting about it with hotThang267. Lol. Wtf?

Research was conducted into a broad cross-section of fashion-minded people and it showed this segmentation; the fans, who are less capable of actually partaking in designer fashion, were at one end and the fashionistas who read the magazine, were at the other. The latter were unlikely to use the website. Unfortunately, the client said we recruited the wrong people, because the participants didn’t match their audience. Rudimentary web analytics showed that the most used part of the website is the discussion forum, and the demographic of users of the forum matches the ‘fans’ segment. More weight to the argument that fashionistas don’t use the website.

I think there are two main reasons for this. The first is something that many designers and experience architects have to deal with: clients who design for themselves instead of who will actually be using their product. My clients are, or want to be, the fashionista. They are style conscious and do participate in the world of high fashion. Through the website design process, they’re trying to cater for themselves, either consciously or subconsciously.

The second reason, closely related to the first, is that the client won’t accept who the real audience is. It reminds me of the “serious” rock band who refuses to accept that their fan base is predominantly 14 year old girls from middle class suburbs—that just ain’t cool. I believe this issue is part and parcel with the field I work in; as people creating websites, magazines and advertising get older, wiser and wealthier, it is increasingly difficult to relate to audiences that are not in the same life-stage. Is there a point at which you can’t, or don’t want to?

Aiming for a certain target audience is one thing—stretching yourself and your product—but it would appear in this case that it’s unrealistic since the people they’re aiming for won’t use their product. Gone are the days when online properties were simple, “me too” attachments to offline brands, with typically little more to offer than a placeholder for the domain name. Now days, these websites need to stand on their own and be successful, even if that means catering for a markedly different audience.

In the future, the ‘fans’ might very well grow up to become the ‘fashionistas’ that my client wants to target, but how long is that going to take? And would they continue to use the web if this did happen? Perhaps I’m missing the point; perhaps the plan is to leverage the fact that the ‘fans’ aspire to be ‘fashionista’, and that’s part of the attraction of the brand. If we targeted the ‘fans’ in a way that reflects their actual age and socio-economic status, maybe they would no longer be attracted because that wouldn’t be “on brand”. If this is the case, it should be openly stated, which hasn’t been.

Having said all this, the website is being developed with functionality for the ‘fans’, so the issue I have is really the perception my client has of who their audience is. Yet this is still troubling, since if you don’t have a clear picture of who you’re designing for, or if the team is not in total agreement regarding this, it’s difficult to design and deliver successful solutions.

I’m not sure how this will play out, my clients are good people and excel in their respective fields, but perhaps are “too close to it”. I’m also open to suggestions, have you found an effective way of addressing a situation such as this?

Photo credit: Missed the target… by malavoda.

See what people are blogging about: fashion website, target audience

Popularity: 10% [?]

So many people have extracted lessons from Gordon Ramsay’s style, from management to consulting (for instance there’s me, Ruth, Donna, Craig and even The Australian) that it’s becoming boring. He’s obviously good value, but I’m now looking elsewhere for analogies.

Sticking to the reality TV theme, though unintentional, bears much fruit and I can’t think of any better example than Trinny and Susannah. I think they are brilliant at what they do, without having to drop the F-bomb every few seconds (don’t get me wrong I don’t mind Gordon’s profanity but it does make him less accessible).

I first happened across Trinny and Susannah (T&S) in their UK show and series of books “What Not to Wear”, where they gave ordinary people fashion tips tailored to their body-shape, lifestyle and budget. For those of you not familiar with the duo, this isn’t your typical fashionistas spouting on about “What’s hot this season, darling”, it’s honest and useful advice for real people. And it goes much deeper than vanity, most often the cause of the problems they solve are the emotional hang-ups we all have inside. They make people feel good about themselves and accept who they are.

Since then they have continued to help people across several more TV series, as well as across the globe. Theirs is an excellent model for consultants, with core traits such as:

• Working as a pair, they balance each other out
• Breaking it down into simple rules we can understand
• By exposing themselves, literally and figuratively, they foster trust and empathy
• A true desire to help others
• Ruthlessly forward, there’s no room for shyness, excuses, self-denial or apathy
• A simple process: research, understand, extract, boil down, try, validate, rollout

The process I speak of in the last point, was well documented in the more recent series “Trinny and Susannah Undress…”, where the girls showed that their approach can scale too. Doing some very ethnographic-like formative research around a particular problem, they extract insight and develop their rules, followed by application of the results to thousands of people. A series of huge publicity stunts for the BBC, no doubt, but this is consulting on a level rarely seen (and with genuine results).

I think possibly the most interesting aspect of their approach is the development of rules. This is what makes their work so accessible; anyone can get simple and straight-talking advice that suits them. What colour combinations go together? What choice of clothing hides this, or accentuates that? What colours go well with my complexion?

I’ve even done this myself, making use of the rules for guys-without-six-pack-abs and it works. In fact T&S’s work with men is probably the best of all, because if there is anyone who needs simple rules to supplement bugger-all fashion sense it’s the male population.

See what people are blogging about: Trinny and Susannah, consulting, Gordon Ramsay

Popularity: 11% [?]

My latest article has just been released, in which I interview Gerry Gaffney on the use of cultural probes in the context of intranet development. This was actually to return the favour extended by Gerry to interview me for his UX podcast. Here’s a taste:

Often it’s difficult to tell exactly who is using an intranet, and how they are using it. Obviously, some research is required to help answer these questions, but what technique can capture the tacit knowledge without shadowing users in an uncomfortable and expensive way?

A relatively recent research technique that can be very useful in this situation is known as a ‘cultural probe’. In essence, the technique involves getting users to give you information without you actually being there. Often this means giving them a diary to write things down in, but the technique can make use of all manner of objects.

I think there are many benefits to this technique for those designing, or redesigning, an intranet. Firstly, it’s often the case that intranet teams are working with very little budget, and thus including all user groups—say interstate or overseas—in research activities can be next to impossible. Sending out a probe which can collect data for you at low cost is a good solution.

It’s also a great way to reduce the ‘Hawthorne effect’, that is the effect you will have on those you are researching simply by being there. This effect is especially difficult for intranet teams to avoid, since they are often researching their own colleagues. It’s hard to blend into the background when you know everyone in the room! External consultants have it easy when it comes to quietly observing or doing contextual inquiries.

I really like the cultural probe technique, especially the sorts of whacky and creative things done by design researchers such as Bill Gaver. These might be a bit ‘out there’ for the corporate environment, but certainly get your attention.

(I’m interested in case studies that describe the use of probes, particularly ‘inside the firewall’. Have you probed your enterprise? What did you find?)

See what people are blogging about: cultural probes, user research, intranet, design research

Popularity: 24% [?]

My latest article has just been published, despite me being on a bit of baby leave. This latest one, Audience personas for the Macquarie University Library website, explores one of the key outcomes of a recent user research project I undertook, that is, a set of personas. Here’s a snippet of one persona:

Nicole the novice scholar

Nicole is an undergraduate student whose course requires a fair bit of research in the library. Students like Nicole, studying humanities and similar subjects, need to use a lot of academic material: books, reserve books and journals as well as media and the internet. However, Nicole is not well equipped for this kind of work when she arrives at university and is unfamiliar with information systems or research strategies.

It was interesting that the existing view of the library audience was that held by many academic institutions—I have had some exposure to other libraries and universities—and while it is fairly logical, it doesn’t really help in terms of designing a website. So the segmentation and prioritisation I recommended was quite different; rather than lumping all undergraduates together I followed the natural ‘fault lines’ in library usage to split these students into groups that make it easier to design and deliver online library services.

In contrast, I found that many ‘higher degree’ students and academic staff were very similar in terms of their library skills and information needs. Thus there is no need to distinguish between them (even though there might be 20 years of experience and respect between a senior lecturer and a masters’ student). It was all about context, we were segmenting and profiling the audience for the purpose of designing a website, and other online services.

As always, all feedback welcome.

See what people are blogging about: personas, user research, online library, audience segmentation

Popularity: 31% [?]

Today I attended a pre-conference tutorial at OzCHI. The topic was Cultural Probes and it was run by Gerry Gaffney and John Murphy. It was very interesting and I can’t wait to use the technique on a real project.

I’ve blogged about the technique before, but in summary it’s a form of longitudinal user research for situations where you can’t normally go. For example, you might get your participants to keep a diary for a few weeks whilst they are renovating their house. Analysing what they give you back will give you very rich information about who they are, what they go through and what they think. (You’ll find a very good intro on Gerry’s Information & Design website)

I can see lots of situations where I could use this, along side other research techniques, to gain a much better understanding of an audience. Whilst the costs of producing the ‘kit’, and the participant incentives, might be higher than methods such as contextual enquiry, the amount of effort and person-hours required on the part of the design team are less (ie compared to direct observation). And of course the level of detail you get is much higher than techniques such as interviews or focus groups.

Anyway, off to finish my presentation for Friday.

See what people are blogging about: ozchi2007, cultural probes, UX

Popularity: 7% [?]