Location: Home » UXblog » User Research Triangulation: A Practical Approach
07.02.2011: User Research Triangulation: A Practical Approach
Over the past couple of years, the concept of triangulation related to end-user research has gained a lot of traction on the interwebs (here's Patrick Kennedy's excellent Johnny Holland article on this exact theme). As it turns out, most of the end-user research I've done on enterprise scale projects over the past 7+ years does fall quite elegantly under this definition, even though I never thought of labeling it that way until recently. My goal for approaching user research in this manner has always been to eliminate bias as much as possible, and use different datasets to justify research findings.
While this post covers my own approach, I won't waste any virtual ink on the theory of user research triangulation, Patrick Kennedy's article referenced above does it more justice than I ever would. Also, my approach is in no way prescriptive, it is simply a three-step research methodology that I personally find particularly useful when it comes to working with samples of large user populations, more often than not scattered across multiple geographical locations. I've settled on this pattern based on a number of trial-and-error attempts to combine various user research methods, in some of the larger projects I've been involved with. In most cases, I was either working in tandem with another UX professional or I was leading an entire UX team.
Step 1: QuestionnairesThe first thing I typically do is I create an online questionnaire which I send out electronically to a large number of users, normally over 100. The purpose of this questionnaire is to assess the user demographics (this will later allow me to hone in the most relevant personas) as well as identify what works and what doesn't work well in the current environment. The value of sending out such questionnaire is especially valuable when dealing with a redesign or a consolidation/upgrade of existing systems.
The questionnaires also capture optional comments for each question allowing questionnaire participants to justify their answers and even make suggestions about how a future system could better address their needs. As a result, the subsequent questionnaire report will be able to quantitatively show the perceived issues of the current environment and will pinpoint some areas that are prime for improvement.
My preference is also to avoid anonymity, for two reasons. First, because when a survey is anonymous a lot of times people don't take it very seriously, and second, because the quality of the comments allows me to identify individual participants in the second phase of my end-user research triangulation methodologyy, the interviews.
Moreover, once the questionnaires are completed, I am ready to create a draft of demographical data that will be included in the personas document (no behavioural data is available at this point).
Step 2: InterviewsBefore we look at this in detail, let me point out that the activity that makes up the second phase is not fixed. While interviews have typically been the most prevalent technique used in my projects, other methods (such as contextual inquiries) have been equally effective for this activity. In the event that the questionnaires were not anonymous, I would choose to interview up to 20 users carefully selected from the list of those who answered the questionnaire, ensuring that they fall across all of the demographics identified in the draft of the personas document. If the questionnaires were anonymous, than I would simply make sure that every demographic is covered, without asking for specific individuals.
The prime candidates for these interviews would be those who have identified system flaws and/or features that are working smoothly in the comments section of the initial questionnaire. The questions prepared for them would explore in more depth the areas that have been quantitatively flagged as potential trouble spots. Other potential question material is related to task flows, information grouping and other time-saving opportunities from a user interface perspective.
The interviews also allow me to articulate the behavioural aspect of my personas document. As a result, at the end of this second step there are already a number of deliverables ready for the client: a questionnaire report, individual and overall interview reports and the personas document.
Step 3: Focus GroupsThe problem with questionnaires, interviews (or contextual inquiries) and personas is that they are very prescriptive and do not include a lot of feature-based, research-based cross-functional problem solving activities. This is addressed in the third and last step of my user research triangulation methodology: the focus groups.
The setup for my focus groups is simple. Maximum 8 people including myself as the facilitator and a dedicated scribe. The rest of the invitees should include a minimum of two end-users, a representative from the business and a representative from the technical team (sometimes an architect and a hands-on developer is even better). Unless the purpose of the focus group is getting feedback from novice users, the participants should already be somewhat familiar with the functionality that will be evaluated based on a previously established agenda.
Everyone in the focus has the opportunity to state their perceptions, opinions, beliefs and attitudes towards the features discussed, and then, if necessary, I would facilitate a (re)design exercise of the user interface and/or the user flow associated with those features. During the discussions, I typically juggle between two roles: UX professional and focus group facilitator.
The object of the focus groups is to solve problems. As a result I try to focus on critical tasks rather than minor or aesthetic improvements, as there will typically be ample time for addressing smaller details later in the prototyping process. This is also a visual step so sketching and the use a wide range of visual aids is highly recommended. As I previously mentioned, these focus groups should have a design exercise flavour built into them.
Once the focus groups reports are done, written reports are created and my user research triangulation stage is done. At this point, I am ready to combine the triangulation deliverables with other datapoints such as existing analytics and essentially wrap up the end-user research stage of my project by articulating the findings in a user requirements document.
What do you guys think? Are there other user research triangulation combinations that have proven useful in your projects?
Tags: research, thoughts, UX tools
I wonder why this approach doesn't get a lot of airtime on the web. It sounds like a great way of covering your bases when it comes to research. I've combined a few methods myself, but i haven't quite figured out a logical structure, so thank you for this.
accessibility branding business canUX community conference design GoC CLF marketplace ottawa privacy project management public sector research security standards TEDx thoughts usability user experience user interface UX tools UXcamp wireframes
- Thoughts on CLF 3.0 From Outside the Firewall... 22527 views » 32
- Best and Worst of CLF 2.0 Public Web Design 13798 views » 20
- Why the Ottawa UX scene is a little slow... 6681 views » 2
- CLF 3.0 Crowdsourcing: A Public Traction Pill for OpenGov Initiatives 5069 views » 7
- One Step Forward, Two Steps Back: 'His Clerkiness' is Online 4755 views » 7
- UX+Agile: Like Dr.Jekyll and Mr.Hyde? 3889 views » 2
- UXWG: The Dawn of a New UX Era in the Canadian Government? 3847 views » 9