Andrew Harder (@thevagrant) is a researcher who likes to make things. He specialises in aligning emerging market user insights with shipping software using ethnography, usability testing, product sprint workshops and elbow grease.
Hannah Scurfield (@theduchess) is a design research manager working for Intel in London. She works with technologists and designers to drive software innovation and strives to institutionalise user empathy.
Editors note: This blog post is from Andrew Harder (@thevagrant) and Hannah Scurfield (@theduchess) who ran the workshop What we buy when we buy design research at EPIC 2013. I invited Andrew and Hannah to guest contribute to the January EPIC 2013 theme because their workshop speaks to a much needed and missing conversation on what exactly clients are buying and what agencies are delivering in design research. Their articles allows us to peek into some of the important discussions that emerged from the workshop. They share with us several strategies that should be considered in the execution of design research processes.
Both Andrew and Hannah a very unique background that enables them to speak from the perspective of agencies and clients. Having moved from agency research to in-house research, they understand the affordances and challenges that boutique firms and large corporations experience. All views expressed are the authors own not those of their employers.
For more posts from this January EPIC edition curated by contributing editor Tricia Wang, follow this link.
Like most good ideas to come out of England, the inspiration for our workshop at EPIC 2013 came from conversations in the pub. In this case, we were talking about “the great divide” between client and agency research teams.
Within a few years of each other, we had both left user experience agencies to work as design research managers inside big companies. Despite having worked in-house previously, this marked a transition in both our careers.
In agencies we both sold design research to large companies. We faced similar challenges; fighting for more budget and time in the field to do more insightful work and wanting earlier involvement with designers so we could shape their work without compromise.
When we moved in-house, we faced new territory. Suddenly we had all the time we wanted, years of it. We had a research budget, sometimes a lot of it. We could work with designers from the minute they got their brief or in some cases, we were working to shape the design brief.
Yet we were also faced with some hard truths that we hadn’t anticipated. In our experience working for big consumer product companies means you are a small cog in a large machine, with objectives and dependencies that spread far beyond a specific research project. Couple that with a complex web of product owners and stakeholders and a design team to keep engaged, and you start to see why design research projects often come unstuck.
Often after spending budget on ethnographic research, design teams are still struggling later on, wanting insights that the research did not provide. And sometimes, no matter how clearly the external research agencies were briefed on project objectives, the deliverables unwittingly undermined the project vision, approach or relationships.
For both client and agency teams, keeping a research project on track is an art form in itself. However when we spoke with our colleagues and friends in research, it confirmed something we had suspected: nobody in industry or academia is openly discussing the process of buying design research. We can study project management styles but the topic of design research project management has been overlooked. The subject appears to be ‘taboo’, much to the detriment, we believe, of both client and agency research teams.
We decided to start that conversation by creating a workshop that explores “What We Buy When We Buy Design Research.” We ran the workshop at EPIC 2013 and drew attendees with a mix of agency and in-house experience, both buyers and sellers of design research. We aimed to chart the design research process, plotting stress points, strategies and skills along the way. The ultimate purpose of this was to provide a space for researchers to share experiences and best practice.
Running the workshop
To prepare for the workshop, we carried out interviews with experienced client and agency-side design researchers. We pulled apart the process of buying design research, and highlighted problem areas that would be pertinent to explore, such as maintaining a budget or deciding how to deliver findings. We used this analysis to create a framework that could bring structure and order to the workshop; exploring the timeline of a design research project, drawing out case studies with key questions and then identifying key problem areas and strategies for success.
We asked our participants to come primed with some cases of research projects that had either been a success or hadn’t met their expectations. Instead of language like “issues” and “pain points” that invites negativity, we asked participants to consider “stresses” and “strategies” as tangible but open concepts.
Participants were paired up with researchers from the opposite side of the client/agency divide or failing that, researchers with a different work experience background. In their pairs, they then discussed their cases, using headings from a template to tease out the details: the context; the project objectives; the research objectives; the outputs and impact of the research.
Our template also used three dimensions we had identified in our initial research as a way to categorise design research projects:
- Vision/Execution: how much is the project about creating a new design vision versus guiding the execution of an agreed vision?
- Drought/Saturated: how much information does the design team already have about their users? Are they in a drought or are they saturated with existing insights?
- Commercial/Laboratory: is the design project integrating with day-to-day commercial pressure of the organisation, or is it happening in a more isolated innovation environment?
This case study method seemed to resonate strongly with our participants. It’s very rare to be able to talk like this with a fellow professional from another organisation with confidentiality. Researchers don’t often get the opportunity for structured reflection; it is usually straight on to the next project. There was a great deal of energy and discussion, and it was hard to break the paired discussion up so we could move onto group work. One participant later said to us that this moment of talking and reflecting was the best moment of the conference for her.
However we did break up the discussion, and using time-honoured techniques of sticking ‘Post-its’ against a shared problem space, the last half of the workshop was spent sharing these discussed stresses and strategies with the group and identifying where they applied in the research buying process.
The results
We ran the workshop on Chatham House rules, which is a morally binding agreement that all the discussions are non-attributable and will remain confidential. This write-up respects that agreement while still trying to give you a flavour of the discussion, issues and hints towards conclusions that we reached.
We learnt a great deal during a small amount of time. Fieldwork and Deliverables were the topics that captured the most discussion and Post-its but they address only part of the bigger picture of design research projects. Looking closer, we could see that the topic of having an impact with research captured the greatest number of stresses for the group but attracted only a few strategies. This is the central disequilibrium that we and other in-house researchers felt when we first opened up the discussion about our responsibilities. There is so much shared wisdom in the design research field but so little of it addresses what stresses us the most: making an impact.
The highlights below come from some of the powerful thoughts around making an impact with design research:
1) Research can be a useful skill to tackle in-house challenges, but it can also be a limitation.
Research skills can be useful for teamwork in commercial situations, helping us understand colleagues, elicit ambitions and fears, and come up with new solutions that work for them. However we also have a tendency to think that we have a privileged position; that speaking to the user gives us authority over others. We may assume that teammates who disagree with us have good intentions and a valid perspective but during execution under commercial pressure, decision-making can be frantic, piece-meal and driven by compromise and limitation. Our passion to understand these views as well as representing the user voice can limit our ability to engage quickly with potential solutions. We can be blind to simple strategies that will keep a project moving forward and maintain momentum.
2) Managing energy levels over the duration of the project can be troublesome
The group spoke about maintaining momentum over a research project and the familiar issue of team resources peaking early in the process, in a few cases as early as the proposal stage. All too often the energy levels can wane during fieldwork as clients can appear to hand over responsibility until the results come in and agencies can feel like they’ve done the difficult part of setting up the project and getting into the field. Instead, the fieldwork should mark the point at which energies keep rising to one of the most important stages in the project: the analysis. It was agreed that this is an often neglected or poorly managed part of the process, perhaps due to the esoteric and often sheltered nature of the analysis stage.
3) Find what makes the SVP sweat, and then deliver on that
One common stress shared in the workshop was aligning research objectives and outcomes to the needs and mindsets of senior stakeholders. One strategy offered by a researcher was a metaphor for addressing this topic: look at the work of a good masseuse; they know specific pressure points over the body where a little pressure can address and relieve tension in their patient.
It’s useful to think of researchers as masseurs, where we can understand where the tension is within our senior stakeholders work lives. Often these won’t be about missing consumer research information; instead they may be about delivery, organisation or focus. If we accept this tension as an important target for our effort at work, then we can begin finding ways to inject user research data and insight into this domain. Of course, the metaphor breaks down because we can’t treat our stakeholders’ tension – but we can make sure our research is tightly aligned to their decision-making and that we, as researchers, are working strategically.
4) Is the client stealing my ideas as part of the proposal process?
An agency-side designer voiced a fear which was familiar to many participants: she saw clients asking for innovative methods, seeking clarification on their execution, and then awarding a project to a competitor. The response from the in-house researchers in the workshop can be summarised as “Well, what do you think would happen?”
This grew into a discussion over the ethical responsibilities of clients in the proposal process. What are the obligations of in-house researchers towards agencies who unsuccessfully propose? When there is a strongly preferred supplier, is it disrespectful to ask another agency to propose just to receive a second costing? For small agencies without dedicated business development teams, this can be a significant spend of time and resources. More specifically, is it OK to lift a good idea out of an otherwise mediocre proposal and give it to your preferred supplier? The discussion highlighted the fact that innovative methods are a differentiator both when evaluating agency proposals and when selling research internally on the client side. We also decided that we need many more workshops to untangle this topic alone.
5) Repeat business is all about how easy it is to work with individual consultants
We choose to work with an agency for the first time because we think they’ll do a good job. We choose to work with an agency for the second time because they are easy to work with and we like spending time with them. This suggestion was recognised by many of the more senior workshop members. When in-house researchers choose a new agency, we need to make sure that they can deliver credible and professional work at a price we can afford. But once this concern is assured, the reality is that over the course of a project we can spend months in each other’s company. No matter how good the results are, if there has been a disconnect in the relationship or we don’t enjoy the process it can be very hard to imagine working together again.
Tangibly, this played out that overt spend on hospitality came second to a shared work ethic and sense of humour. Other characteristics of good relationships that were mentioned included a dedication to helping in-house researchers deliver: keeping up the right level of communication to keep the project on track and explicitly delivering in a way that makes the in-house researcher look good. It was also seen as important that agency researchers have the energy to get in-house teams excited about the research. An example used here was that with final research finding presentations, strength of delivery goes further than scientific rigour.
Next steps
These findings are only a few highlights from the workshop discussion to demonstrate the rich vein of expertise and wisdom that we have tapped in to. We were really pleased to have started documenting the topic of buying design research after struggling to find evidence of this discussion in industry and academic literature. By generating and sharing best practice for managing design research projects we believe teams can close the divide between client and agency sides and provide solid management foundations on which to advance our research projects.
As John Dewey’s idiom goes: ‘a problem well defined is a problem half-solved’ and during the workshop we had extracted and explored a suite of stresses along with some strategies. The plan is to facilitate follow-on workshops to push further into the solution space and match strategies to every identified stress point. In doing so we can provide a complete toolkit for researchers both sides of the divide that will encompass everything needed to make the partnership work, from identifying the right partner to driving forward with passion and making an impact.
Perhaps the most encouraging sign of the workshop was that we planned to run for two hours, but instead the energetic discussion ran for three and a half hours. And then fittingly, the workshop ended where we it had begun: the pub.
_______________
Other posts in the EPIC 2013 theme:
- Why go to an ethnography conference?: Notes from the EPIC 2013 Conference, by Tricia Wang (@triciawang)
- I’m Coming Out: Four Awkward Conversations for Commercial Ethnographers, by Drew Smith (@drewpasmith)
- An Interview with the head of User Research at the UK Government Digital Services: Leisa Reichelt, by Leisa Reichelt (@leisa)
- Lessons Learned From EPIC’s Mobile Apps & Quantified Self Workshop, by Mike Gotta (@Mikegotta)
- Play nice: design ethnographer meets management consultant, an interview with Alicia Dudek from Deloitte Digital, by Alicia Dudek (@aliciadudek)
- A Psychologist Among Ethnographers: an interview with Beatriz Arantes from Steelcase, by Beatriz Arantes (@beatriz_wsf)
