Wednesday, February 18, 2009

User (experience) research, design research, usability research, market research, ...

A version of this post was published in UX Magazine.

I rather miss heading up a user research practice and managing and supporting user research personnel. Recently, I nearly accepted a position heading up a highly-respected user research consultancy looking to take things to the next level.

But should such a practice or offering be referred to as "user research" these days. The term is still in use (though the word "experience" often lies in the middle), but the word "user" can imply a much narrower conception of the practice than often intended. As I described in a much earlier blog entry, that was true when I was Director of User Research at Studio Archetype and Sapient; there, the label did not always communicate that we did more than only research of "users" and "use." And a recent conversation I had with an ethnographer who wanted to better understand "user research," something she said she did not do, revealed such preconceptions still exist even within the applied research community. (Use of ethnographic research methods was a big part of the "user research" we did at Studio Archetype and Sapient.)

In short, it is not always clear what label is best to apply to such a practice or consultancy. It is also not always clear what its ideal scope or focus should be or should become.

Lots of people conduct "usability research" these days, but the methods and approaches often used have lagged behind major changes that have occurred in the world of computing. In "Is usability obsolete?" -- an article we will be publishing in the May+June 2009 issue of interactions magazine, Katie Minardo Scott argues:
"Current usability work is a relic of the 1990’s: an artifact of an earlier computer ecosystem, out of step with contemporary computing realities. Usability can no longer keep up with computing: the products are too complex, too pervasive, and too easy to build. And in our absence, users and engineers are beginning to take over the design process. These trends demonstrate the growing gap between usability theory and commercial practice – the “new realities” of computing haven’t been truly embraced by the usability community. The trends are, at a minimum, making traditional usability more difficult, if not irrelevant in the new paradigm."
The label "design research" is used more and more these days. But when Yahoo! abandoned the label "user experience research" for "design research" two or three years ago, previous efforts -- some of which had been mine when I was in a management role at Yahoo! -- to involve user experience research in the early stages of product and service ideation and conception were undercut. As described by Yahoo!'s Klaus Kaasgaard, guest speaker during a user experience management course I taught last spring, the new label made people think that the research was only relevant to the later "design" phase of the product development process.

The narrow interpretations of the label "user research" at Studio Archetype and Sapient prompted us to extend the label to "user research and experience strategy." The narrow interpretations of the label "design research" at Yahoo! led Klaus to change the label back to "user experience research." But a much more significant change was made at Yahoo! more recently: a merger of the user experience research group and the market research group, yielding an organization named, "Customer Insights."

When I was in a management role at Yahoo!, we discovered that market researchers were encountering some of the same obstacles as our user experience researchers -- obstacles to being appropriately involved upstream in the process so to have a more beneficial impact on the company. So, we began to partner with market research in an effort to attain that involvement. During his guest appearance at my "User Experience Managers and Executives Speak" course, Klaus, now VP of Customer Insights at Yahoo!, spoke at length about the similarities and differences among goals and challenges faced by market researchers and user experience researchers, and about how important the merger has been to achieving such a strategic role. In an excellent article in UX magazine (Volume 7, Issue 2, 2008), Robin Beers paints a similar portrait regarding bringing together the market research and user research teams under the umbrella of Customer Experience Research & Design at Wells Fargo.

Is such a "coming together" of these two disciplines appropriate for every company? No, as implied by eBay's decision to split them up after they attempted to bring them together. There are multiple factors to consider when determining what is best for a particular company. But it is important to understand that great benefit can be achieved when the two work together.

In an October 2008 contribution to Jakob Nielsen's Alertbox, Christian Rohrer provides a mapping of a wide range of research methods, some typically thought of as "market research" methods, that can help you to better understand their similarities and differences.

In the November+December 2008 issue of interactions magazine, Liz Sanders provides different insight via her map of "design research" (see the map below right), which you can click to enlarge). Here is how Liz describes the map's organization:
The design research map is defined and described by two intersecting dimensions. One is defined by approach and the other is defined by mind-set. Approaches to design research have come from a research-led perspective (shown at the bottom of the map) and from a design-led perspective (shown at the top of the map). The research-led perspective has the longest history and has been driven by applied psychologists, anthropologists, sociologists and engineers. The design-led perspective, on the other hand, has come into view more recently.

There are two opposing mindsets evident in the practice of design research today. The left side of the map describes a culture characterized by an expert mind-set. Design researchers here are involved with designing FOR people. These design researchers consider themselves to be the experts and they see and refer to people as “subjects”, users”, “consumers”, etc. The right side of the map describes a culture characterized by a participatory mind-set. Design researchers on this side design WITH people. They see the people as the true experts in domains of experience such as living, learning, working, etc. Design researchers who have a participatory mind-set value people as co-creators in the design process. It is difficult for many people to move from the left to the right side of the map (or vice versa) as this shift entails a significant cultural change."
Yet another map of methods was developed during the Netherlands Design Institute's Presence project during the late '90s. The image to the left (click to enlarge) shows the map, which requires a legend in order to identify which method lies where. In this image, the location of "rapid ethnography" is revealed, along with helpful information about the method regarding required expertise, time, staffing, and cost. (This "methods lab" used to be online, but I am now able to find it only in the 1999 book, "PRESENCE: New Media for Older People.")

The ratings in the above image remind me of ratings developed by Luke Hohmann for individual "innovation games" -- a variety of research methods employing collaborative play. (See image at right for his ratings for a game called Speed Boat, and see "What is holding User Experience back or propelling User Experience forward where you work?" for a sense of what that game is about.)


Keith Instone said...

Thanks for the great, thought-provoking article. One comment -

"Customer insights" is a good label, but sometimes you may need something even broader. What if you are studying business partners, stockholders, employees, citizens, etc.: groups that are not "customers"?

I know, this is getting picky. You could say everyone is a "customer" - but then you open up the same can of worms as "design" - some will think narrowly and some will think broadly.

I do not have an answer. Maybe "people insights". That would work unless you work for a company in the pet industry, where understanding how dogs and cats behave is also important.

PS See you at the IA Summit, looking forward to your "show".

Anonymous said...

Hi Richard - Great article and thanks for the shout out to my article in User Experience Magazine last year! Robin Beers, Wells Fargo

Unknown said...

Keith, I really like Customer Insights -- and have used it since my stint at AT&T. I'd say there are two processes: Customer Insights and Business Insights. They use similar methodologies, sure, but the results are used differently. That doesn't solve the problem of what to call such a group (that is, if your group was involved in both customer side and business side). Maybe just Insights?

John Payne said...

Great post! This brings back some memories...

"In short, it is not always clear what label is best to apply to such a practice or consultancy. It is also not always clear what its ideal scope or focus should be or should become."

When it comes down to it, I don't feel that there is ANY good term for our particular mix of skills, even now. We're left with two choices: make something new up, or work with one of the exisitng terms in industry.

To this point, at Moment we chose to go with an existing term: "Experience Design and Strategy". Here's why:
1. because of our business context (we're a design firm) we thought we should name the practice after the value we produce rather than skills we possess. In practice this has been easier for me to explain/justify/market to our consulting clients,
2. Our group is made up primarily of folks who do user experience research but self-identify as designers. The outputs of the group are customer insights AND actionable strategies, services, or interfaces for our clients,
3. the industry has evolved to almost expect research/customer insights/user empathy as a part of any competency with "experience" in the title.

All that said, I'm not sold on the term. "Experience" anything to me has always seemed too vague and grandiose. We aren't actually creating "experiences" we're really only creating the possibility of them... and I don't like leaving out the object of our work (i.e. people). I should really stop here or there's a danger I'll write an essay on this. ;)