Choosing design research methods
I recently conducted a quick survey among my colleagues and peers in the UX field, regarding how to determine which research methods to use for a particular project. My objective was to try and work out the four qualities of any research method that need to be considered when putting together a plan for design research.
For example, interviews don’t allow as many people as focus groups but they can dig deeper into the thoughts of the participant, thus ‘breadth’ and ‘depth’ might be two such differentiating qualities.
I’ll be using this information for an ‘edutainment’ game I’m including in my upcoming OZ-IA workshop, whereby participants will select a set of research methods for a given scenario and limited by a ‘budget’ for each quality.
I got lots of useful replies from a small sample size. Fortunately—I got more than what I had literally asked for (my bad…a poorly worded email). Almost everyone gave me great insight into how they go about planning research, as well as some gentle rhetoric regarding the subtle complexities of the subject and how perhaps I was oversimplifying :) So, I stand duly chastised, planning a research programme is definitely more complex than choosing four qualities, but that’s what I needed for my workshop game. Lesson learned :)
The survey question was framed in terms of user research for web design, but I expected that I would get responses from a broader perspective than this, and my recipients did not disappoint (eg BA’s gave their views from the point of view of business requirements gathering).
My findings were numerous, starting with the fact that it’s hard to analyse responses when they are all wildly different; some people ranked, some gave low/med/high values and some simply discussed the relative importance of the qualities I had listed in my question. So I had to interpret some of the responses in order to collate them. I should have used a proper survey form, but I was under the influence of spontaneity. Another lesson learned :)
Without further delay, here are the results of the survey. These are the top ranked qualities, taking into account all the information I received from respondents:
| Ranking | Method | Description / Synonyms |
|---|---|---|
| 1 | Depth | Coverage, level of detail |
| 2 | Breadth | Reach, number of participants |
| 3 | Analysis | Time required for analysis, turnaround time |
| 4 | Difficulty | Effort required |
| 5 | Facilitation | Face-to-face contact time |
| 6 | Cost | Material or capital expenses |
I’m not so concerned with the order, but it was clear the top factors were the breadth, depth and amount of analysis required.
The cost directly associated with a method were not seen as a useful differentiator since they are all much the same in this regard. Similarly the facilitation time, or face-to-face contact time, was not seen as a crucial factor since it can be varied as needed on a case by case basis, regardless of the method employed. Difficultly, or the effort required, was seen as a crucial factor only by a few people, the rest didn’t see it as a sufficient differentiator (perhaps reflecting their own level of experience in the field).
Total cost (ie time and materials) was seen by many as a crucial factor in that a research undertaking might “live or die” on the basis of this alone, but it’s an aggregate of facilitation time, analysis time and material costs, which are already in the list. So I removed it from the final list.
Two other factors are worth a mention. Firstly, rigour was nominated by one respondent as a reasonably important factor. I took this to mean how easily the research method can be repeated with consistency across sessions. However, I grouped this with difficulty, or effort required, because I see it as a quality of the practitioner rather than the techniques themselves.
Secondly, the difference between qualitative and quantitative analysis was suggested as a key consideration. That is, some techniques will require more discourse analysis as opposed to data analysis. Excellent point, but probably too fine-grained for my purposes.
My conclusion is that I should use reach, coverage, analysis time and effort required as the differential qualities for my workshop game. Feel free to comment on this below.
Thanks to everyone who responded to my email, and for taking the time to make sense of it. Thanks also to James Breeze for reminding me to blog about this…I have been neglecting my blogging lately!
Comments
Post a comment