Pat’s Point of View

Comments

1. Lisa | January 23rd, 2009 | 11:41 am

   Good question Pat.

   I refer to them as surrogates. But I won’t use them. I think there’s almost always a way of recruiting the actual users, you might just have to be patient or creative in getting them.

   For example, when I was recruiting participants to test the in fight entertainment for the A380 a few years ago, we had to recruit passengers who flew first class so that they could test the prototype, seating, hand controls etc. As you can imagine there’s a huge difference between 1st class and economy :)

   But Qantas has a policy of not contacting their 1st class customers. So the solution was to identify Business class passengers who travelled frequently and who had been upgraded to first class a number of times.

   So I really don’t think using surrogate users is a good idea. In fact I’d even go so far as to say that lazy recruiting practices can equate to interviewing surrogate/ intermediary users.

   This is just my 2cents worth ;)

   It’s a great question. I’m really interested to hear what others think, and if you have any example of when you think it’s perfectly fine.

2. Patrick Kennedy | January 23rd, 2009 | 11:54 am

   Thanks Lisa, I definitely appreciate your views on this. I wouldn’t do what you have described, using one type of user in place of another. It may be pure semantics, but what I have done in the past is use people within the client organisation as “expert witnesses”, as a partner in the project. I guess my justification (hope?) is that they are speaking from their experiences with the third party and not confabulating.

   By the way “surrogates” is a good label for this. Doc Baty suggested “proxies”, I like both words.

3. Stephen | January 23rd, 2009 | 12:55 pm

   In Anthropology they call them key informants. Sounds a little bit cloak and dagger, but an accurate term.

   http://web1.msue.msu.edu/msue/imp/modii/iii00004.html

   “The key informant method is based on obtaining information, over time, from a community resident who is in a position to know the community well. The person or persons selected to be key informants must therefore have a broad knowledge of the community, its services, and its people. It is an excellent way to recover information about past events or ways of life that are no longer observable.”

   Cheers,
   Stephen

4. Patrick Kennedy | January 23rd, 2009 | 7:27 pm

   It’s a long weekend, so I doubt my esteemed USiT colleagues Stephen Cox(Stephen did leave a comment but it Akismet caught it) and Doctor Chris Khalil will be leaving a comment, so I’ll have to paraphrase the conversation we had on this topic :)

   Stephen told me about the anthropological concept of “key informants”, those people you include in your research that have a perspective you can not otherwise tap into. The example he used was if the subject you were researching exists only in the past, you would speak to someone who remembers it.

   Very similar to this was Chris’ view of the intermediary as a Subject Matter Expert (SME), who can give you an excellent primer in the behaviour of your target audience; the language they use, the issues that concern them etc.

   Both of these are exactly what I was thinking of in terms of including people in your research who will speak about another group of people. In addition to the call center example I used above, an example of what I’m talking about would be going to a liaison librarian to discuss what university academics commonly request from their library.

   Yet the question still remains, what are the rules for this? Off the top of my head I would say that you want these intermediaries/surrogates/proxies/key informants/SMEs to talk about the target audience, not on their behalf. In a similar way to using analytics, this source of information can only tell us facts: what users did, what they ask for the most, when they put in requests. They relay evidence, they can’t tell us why.

   Thoughts, comments?

5. Steve Baty | January 24th, 2009 | 5:35 pm

   Pat,

   Here’s the article I was referring to on Twitter: http://www.uxmatters.com/mt/archives/2008/03/closing-the-communication-loop.php – it’s one of mine, and basically talks about proxy users as a source of user insight.

   Steve

6. Harry Brignull | January 25th, 2009 | 6:50 pm

   Hi Pat,

   I have had great success using intermediaries – specifically, advisors from the Citizen’s Advice Beareau who talked to me about young people with severe financial problems. These intermediaries worked with and advised my target user group on a daily basis, giving the exact assistance in person that our online service was attempting to provide. I also worked with the target users, but they were by poor interviewees (kids who had been failed by the education system), who had very poor abilities to reflect on their situation and understand what kind of assistive materials might help them take control of their finances.

   The advisers, on the other hand, had tried many different ways of providing assistance and support. They knew what didn’t work from bitter experience.

   I see this kind of intermediary is dfferent from the “second grade” type where you recruit a similar user-type because you can’t get your hands on the real thing. I remember once being on an international research project looking at family scheduling in the home. The US client wanted to use Germany as a proxy for the whole of Western Europe! This is a perfect example of why you have to be very careful…

7. Patrick Kennedy | January 27th, 2009 | 8:15 am

   @Harry those are good examples, one that illustrates a situation where you would definitely want to use intermediaries (young people with financial problems) and another that illustrates all that is bad about the mis-use of intermediaries (Germany as proxy for all of western Europe). Any user research requires careful selection of participants, but the use of intermediaries requires even more attention in terms of ensuring the intermediary is truly representative of the target audience.

8. Patrick Kennedy | January 28th, 2009 | 9:29 am

   Thanks for the link @Steve.

   One of the comments on your article had a link to the Boxes & Arrows article entitled Extreme User Research which also discussed this topic:

   The main idea behind extreme user research is that instead of going for the real users, we go for surrogate users. Those are the ones within a company who talk directly to the customers. We want to talk to the people who talk to the people.

   Unfortunately the author doesn’t go into a lot of detail, in particular when not to use surrogates, which is what I’m interested in right now. However, trawling through the comments reveals a number of useful tips (emphasis added by me):

   By talking to the people who talk to users, we introduce another layer of bias (political, cognitive, whatever) and thus could end up a bit further from facts than if we’d gathered them ourselves.

   …in my view this is also second hand information. Because you are asking people what they think they would do on your site. You never know what they would really do.

   (I think the point here is that you shouldn’t be asking surrogates what users would do on the website, because that is second hand information. You ask them only what they know users have done/said to them, the surrogate)

   …surrogate users can be problematic…the feedback we get from our internal surrogate users is often clouded by their knowledge of business priorities, and whatever ethnographic research we’ve managed to do externally has often revealed dramatically different things.

   While I think that user research by proxy isn’t a bad idea per se, it seems to me that the article isn’t quite egalitarian enough. When designing a web site, it would seem a bit odd not to account for the obvious possibility that people who call the call center are different from people who prefer to use web. The approach would seem to demand some corresponding research of other channels before making hard design decisions…

   (Good point)

   I’ve also found that working with surrogates can sometimes help engage people throughout your company, turning them into allies that you can approach in the future for assistance.

   One additional thing to keep in mind when using surrogates: it’s helpful to match surrogates to the users they serve…in your Quebec city example you interviewed call center reps to get info on consumers but you probably also spoke to people in the permits area to get data on the needs of building contractors.

   …in many cases I think surrogate users may have other interests besides the clear truth. Extreme example: Not in every case a call center employee would want a user to find parking information on the website. Where would her job go if 90% of the calls is about that information?

   …to elicit the functionality a new or redesigned site should provide…If you ask the citizens you will elicit invididual needs but not get the overall picture – except if you ask a very large group of citizens.

   (I think the point here was that surrogates can help aggregate information without having to speak to a massive number of users)

   …it would be an awesome idea to ask that tech support people keep track of their calls in one week. They don’t need to do anything but have a sheet of paper and for every call write down the main words associated with that call. Then when you ask these surrogates for what are the top problems, they can refer back to this sheet…

   So the consensus seems to be that intermediaries/surrogates/proxies can be useful, particularly if you have no other option, but shouldn’t replace research with actual members of the target audience. In fact do both. And one must proceed with caution, ensuring the surrogates can talk with credibility about the audience group you’re studying.

   I added my own comment to the list, which I’ll repeat here:

   From the comments here, I think some people are confused about the purpose of surrogates. We’re not talking to surrogates as members of the target audience, nor are we asking them to pretend they are users. We’re talking to them about the target audience. So using surrogates is not the same as “grabbing someone in the corridor” and asking their opinion (such an approach has it’s place, but you need to be very careful in terms of what you take away from your corridor encounter because that person is unlikely to be representative of your target audience). Likewise, surrogates are not “power users” either.

   Surrogates (or proxies or intermediaries) are used to learn more about the target audience, indirectly, but based on their direct experience with the target audience.

   I wrote a blog post on this topic: Intermediaries in user research. Happy to hear more feedback.

   One more thought, I think the idea of reflexivity is important here; not only should you be aware of your own perspectives and biases that might affect your research and analysis, but you should be mindful of the perspectives and biases of your surrogates too. This goes a long way to helping avoid some of the problems with using surrogates.

9. A summary of user research methods | Pat's Point of View | December 6th, 2009 | 3:41 pm

   [...] potentially useful variation is known as the proxy interview (also known as ‘key informant’ or ‘intermediary’ [...]

Post a comment