Friday, March 23, 2007

The internal consultancy model for strategic UXD relevance

Jim Nieters, Manager of Cisco's "central" User eXperience Design (UXD) Group, Subbarao Ivaturi, Technical Lead in that group, and Garett Dworman, a Senior Design Architect for Tec-Ed consulting to Cisco, will be presenting an "experience report" entitled, "The Internal Consultancy Model for Strategic UXD Relevance" at CHI 2007.

I interviewed the three of them about the topic of the report last month.

Richard: Why did you choose to write this report for the CHI conference?

Jim: We had seen a lot of discussions about different organizational models at CHI. What we hadn't seen was anybody talking about the internal consultancy model. For example, there was a tutorial done at CHI that talked about different organizational structures. They talked about forming a team around a centralized funding model -- a cost-center model. They talked about structuring a team around a client-funding model where you get money from business units to pay for your team. And they talked about a distributed model where teams sit in the business units. But they didn't talk about an internal consultancy model. And we've found at Cisco that this model has been very effective. So, we decided that it would be worth sharing information about it with the larger community.

Subbarao: We also wanted to educate the CHI community about what experiences we were having at Cisco in using different models -- what successes we've had, what issues we face, and how we are addressing those.

Richard: Describe the internal consultancy model.

Subbarao: We function as a consulting firm within Cisco. We provide services to product teams that request them, but when we work on projects, we go in as a team rather than an individual, emulating portions of what design firms often do.

Jim: A design firm would come into the company, they would bid on a project, and they would assemble a cross-functional team of visual designers, interaction designers, maybe developers, user researchers, usability engineers, ... and they would deliver great value. In our case, we saw that when we had one designer focused on many projects, it was difficult for that person to dig deep, and it really diluted their value. Because Cisco is a technology-led company without full understanding of user experience, we could not afford to continue to dilute our value by delivering incremental improvements. So, we decided to focus on only a few projects very intensively and provide great impact. So, for example, Subbarao has led many teams within the organization that have delivered tremendous value, and he has been allowed to own the user interface, which is a part of our price of entry. Those are projects where we've made millions of dollars for the company. In the previous model, we may have made only incremental improvements which were difficult to measure.

Richard: The title of your report includes the word "strategic" -- it states that this model is for "strategic" UXD relevance. Is that what you are beginning to talk about? Is that why you say the model is important strategically?

Jim: That is exactly it. The typical structure is where one person works on many design projects, and you can afford that in companies where the UXD function is embedded and is a standard part of the process, because teams expect that. Cisco has been a technology-led company rather than experience-led, and we found that we can only make incremental improvement (using that model), and when you only make incremental improvement, the product team can then say a year later, "Well, gosh, it is not a great product; UXD only did a good job, not a great job." We decided that to make sure we had a presence at the table with executives making strategic decisions and to gain the visibility necessary to grow the function within the company, we needed to create a big name for ourselves. And that big name is achieved by working on projects where we can deliver, for example, $100 million dollars worth of impact. At Cisco, we have a huge scale, so $100 million worth of impact is possible, because we go after the projects where there is a lot of revenue opportunity. What that has done has given us a voice at the table from a strategic standpoint.

Garett: I'll interject from an outsider's perspective that at Cisco, there is a need for the UXD team to prove themselves. Many of us have experienced usability as being difficult to bring into design process, since lots of people don't quite understand what user experience means. But some companies are more receptive to that than others. Over the past 3 or 4 years, Cisco has been readily accepting the terminology, and happy to say, "yes, we want user experience." But when it comes to doing it, a lot of the personnel I meet don't really know what it is and don't always understand why it is valuable. They've been told they've got to do it, but they think it gets in their way. They have timelines which are often agressive; they have a lot of other teams they need to get work done with. Why should they bother talkiing with user experience professionals who appear to be slowing down the process by wanting to talk with users, when the product team already knows what users want (or so they think)? So there has to be palpable evidence that this is really worth doing. I think this has forced Jim's hand. Incremental changes are good and important, but people don't notice them.

Richard: It sounds as if there might be times when the UXD group turns down requests for services?

Jim: Many times, in fact. We've changed the dialogue so that we don't have to sell our services. Because of the impact we've been having, we are able to turn people away and choose the projects that are the most valuable -- projects where the people really want to work with us and will partner with us, where there is a big business opportunity so we can show a big financial return, and where executives will be willing to give us some of the credit so that we can articulate that they made a big return on investment in UXD.

Richard: Is there a danger in saying "no" to some requests. Might that not hurt you strategically?

Subbarao: There could be that danger, but with the previous model, we were accepting all requests, and we couldn't achieve the kind of excellence we wanted to on each project because we were spread too thin.

Richard: The internal consultancy model is actually only a part of the overall Cisco model with respect to user experience. Talk a bit about where UXD is positioned elsewhere in the company.

Subbarao: At Cisco, we have three main user experience groups -- one for the Cisco intranet, one for the external-facing cisco.com, and our central UXD consultancy that works on revenue-generating products that ship to customers. There also are smaller user experience groups within business units that are starting to build their own programs.

Jim: Our goal really is to help those product organizations understand the need to build a user experience competency. So we augment those smaller teams. When an organization's UX team can't handle all of their projects, they call on us on a project basis.

Richard: Where is your consultancy positioned within Cisco?

Jim: We're in the Customer Advocacy organization -- an organization outside of Engineering. We've been positioned within Engineering in the past. We've also followed the centralized model and the distributed model -- we've had teams in the business units as dedicated teams. At this point, Customer Advocacy is the voice of the customer to a great extent within the company. So, from our perspective, it made the most sense to locate there. Maybe in the future that should change.

Richard: Say some more about the history that ultimately led you to implement this internal consultancy model.

Jim: When Subbarao joined the company around six years ago, we had a client funding model where business units provided us with some of their headcount. Cisco was growing so fast as a company, that we actually had promises from executives that we were going to be able to hire 60 people in our central organization. It was as if everybody was throwing requisitions at us. Then we had the downturn in 2001. At that point, executives said they had to pull back their funding for UXD headcount. During that transition, we were successful at converting those people to be centrally funded resources. But the challenge with that was that we grew to almost 60 people, and when you grow to almost 60 people, you become a cost center. And when you are a cost center, people target you; even if you are delivering value, it can become a political challenge no matter what company you are in. Hence, we decided that being a cost center wasn't going to be successful. So, we moved out of Engineering and into the Customer Advocacy organization, and decided that we were going to have a smaller group that we would expand and shrink as needed via access to a pool of outside consultants. When a project requires resources beyond what we have in the UXD group, the business unit pays for those resources.

An example of how we do that is the case of a product that Garett, an employee of an outside consulting firm, is working on now with some members of our UXD group. The product team is paying us about $200,000, but we're supplying money as well. Hence, it is a shared model, with the goal of partnering very strongly with the business unit. Neither organization is paying the whole cost. The business unit is trying to start earning $20 million per year more than they are today, and they feel that our help, costing them $200,000, will help them do that. From their perspective, the return on that investment is pretty significant. So, you can see that the business units are providing us with the dollars that enable us to expand our staff as we need to by hiring consultants.

Richard: How well has that worked for you, Garett?

Garett: It has worked very well. In some ways, it has made it much easier for Tec-Ed, my employer, to move from one project within Cisco to the next. We don't have to go through a complete sales cycle each time. Now, when I am working on a project that is coming to a close, we can approach Jim to discuss projects that are coming up for the UXD group that I might move right on to.

There is a slight issue with identity, in that the UXD group wants to expand their ability by hiring outside consultants, but they still want to be seen by the rest of Cisco as just the UXD group. They don't want to be seen as the UXD group and other consultancies. We've been working that out, as Tec-Ed does want it to be known that it is involved in the work.

Jim: The issue of branding -- Cisco-branded, partner-branded, or co-branded -- is something that has to evolve more, because leveraging partnership has to be sustainable for everybody. That is one area of challenge that we have to solve going forward.

Garett: But it hasn't been that much of an issue. And it has been easier for us to consult to Cisco, because I can look to the rest of the UXD group as partners. So, for example, if we needed a report creation module on the project I'm working on now here, I can find out whether someone in the UXD group has worked on such a module on another project, and then leverage that work, which is truly helpful. Before, when I was working on a project for Cisco that lasted a year and a half, I was very removed from the UXD group and was working from a more isolated position; that made it easy for me to clearly say I'm from Tec-Ed, but I couldn't use all the resources of the UXD group.

Subbarao: Another challenge is that the consultants have to be able to quickly learn both the technology and the UXD practices at Cisco. We've tried to address that challenge by developing a pool of consultants that we use a lot, so most of the consultants we bring in already know a lot about the technology and our processes.

Richard: Talk a bit more about the culture of Cisco, and about any other characteristics of Cisco, which make the internal consultancy model advisable here.

Subbarao: The key thing, as Jim and Garett said earlier, is the need to show significant impact to product teams and other decision makers.

Jim: That is right. In the HCI industry, we talk a lot about return on investment and whether you can measure it. As part of the engagement model for the consultancy, we measure the "before" and the "after," and we make sure we will get testimonials from the executives to support our ROI claims. Over the years, the team has had over $2 billion of impact, which is tangible, and we can get testimonials from executives saying that that has been the case. This has made it possible for us to say, "Look at the value; it is a clear return on your investment." With prior organizational models, it was more difficult for us.

Also important is the partnership we establish with the product teams up front. It is a matter of ensuring that we are integrated with the teams and considered relevant by the teams' executives at the very beginning. We get those agreements up front, and that makes a big difference.

Subbarao: And coming out of all this is better user experience. We show tangible value not just in terms of dollars but also in how the user experience has evolved.

Richard: In what (other) types of companies should this kind of model be considered?

Jim: Companies where you don't have company-wide, executive level buy-in but where at least some product teams know they need an improved user experience and are willing to work with your UXD team and allow the team some ownership over the user experience.

Complex systems companies. You can contrast that type of a company with volume operations companies like Yahoo! and Intuit, where user experience is a requirement from the beginning. Cisco is technology-led and was successful as a technology company. We're trying to show -- and I think we are doing that -- that user experience itself can be that next advanced technology, that it is that next thing that can get you a billion dollars.

Garett: Of relevance to that point is the Cisco product I'm working on now. We've learned from customers that they think that the product does what Cisco says it does, and they are happy with that. However, they have had to go to extra machinations to find out whether it was doing what Cisco says it does, because the user interface is not very good. And what Cisco is finding is that competitors are making tremendous headway against Cisco in this product field. There is no evidence that competitors' products work better, but there is evidence that these other products have a user interface that provides users with the information they want about what is going on. With the Cisco product, users are not sure what is going on. It has become pretty clear that Cisco's product would really take off if it had as good or better of a user interface.

Richard: Will the internal consultancy model be best for Cisco long-term?

Subbarao: I think all models have to evolve, but I think we have shown tangible results for this model. When we see significant change in the Cisco landscape around others' expectations or behavior, or regarding how well we can scale our own efforts by staffing more people in-house, at that point we should look at how we should change the model.

Jim: I agree. For example, if Cisco were to hire a Director of User Experience for every major technology group that we have in the Engineering organization, we may be able to become largely a central infrastructure team providing tools, processes, labs, education, career progression, etc. Presently, we don't have the governance across the company that supports user experience actively enough. We only have local governance -- that is, governance at the product team level. Garret is right that the executives are saying the right things about user experience, but until we see it more globally accepted across the company, I think this model works. Once user experience gets accepted across the company, it makes sense to consider a different model.

Garett: I agree. There are two factors. One is resources. The UXD group doesn't have enough resources to cover all of Cisco's needs for UXD, even if it keeps hiring out to firms like Tec-Ed. The second is moving beyond jumping on the bandwagon and using the cool terms that are alive today. Once people understand what those terms really mean and imply -- once user experience has a good hold in the culture, and once there are enough resources, then the UXD group can move on to the next model.

Richard: Thank you.

Saturday, March 17, 2007

Steven Pemberton

I was delighted when I received email from Steven Permberton a couple of weeks ago saying that he'd be visiting the San Francisco Bay Area and asking whether I'd be available to get together.

I was fortunate to have worked with Steven when we were both on the SIGCHI Executive Committee several years ago. Steven was Editor-In-Chief of SIGCHI Bulletin for several years when SIGCHI Bulletin was actually a substantive publication; he was subsequently Editor-In-Chief of interactions magazine (1998-2004). Steven also helped found SIGCHI.NL, SIGCHI's chapter in the Netherlands; hence, Steven also participated in workshops I gave when I was SIGCHI's Local Chapters Chair.

Steven was visiting the area for a meeting of one of the two W3C Working Groups he chairs. This work and some of his other "Projects Past and Present" are described on his home page, where he states:
"If there is one thread that runs through these projects, it is about people. In particular, what are the changes that need to be made to the system architecture to make the resulting system more human oriented."
Steven shared the following elaboration with me:
"One of the main problems with current systems is that they are not built to support usability. Designers are forced to add usability as a layer over the underlying system. And then time and again, for each program anew. Imagine if systems didn't support filestores but only the ability to write bytes to the disk. Then for each program you would have to write the code to deal with files, and you can be sure that each program would have its own filestore bugs, programs wouldn't be interoperable at the filestore level, and there would be acres of guidelines on designing filestores. Well, that's what we have with usability today but on a far grander scale. System architecture is designed by technicians who don't realise the far-reaching effects their design decisions are having. And as a result, usability is a band-aid over the top of bad system architecture."
I asked Steven about the challenges he encounters in his work for W3C in addressing this problem:
"W3C is, of course, essentially technological, though there are areas that are people-oriented, in particular accessibility and internationalisation. I think the problem is two-fold: firstly, W3C is balkanised along the major design axes. There are people thinking about accessibility, device independence, internationalization, and so on, but they are not in general embedded in the groups doing the actual designs, so that often the non-design groups end up writing guidelines - band-aids. Secondly, W3C is member-driven. This is a good thing in general, but it creates a vicious circle: if W3C doesn't do usability, no one from usability will join, and if no one from usability joins, there is no one to demand that it be done. As a result, there is no group responsible for usability within W3C and therefore not enough attention is paid to it."
Steven travels a lot giving keynotes and other invited talks about his perspectives and his work for W3C. Slides from many of his presentations are accessible via his home page, along with audio of various interviews (most in Dutch).

Are you familiar with the concept and experience of a Dutch auction? Over a drink, Steven told me about his first-hand experience. Wikipedia includes a description, but that description fails to mention the second stage that is included for auctions of residences, a process via which Steven and his partner Astrid purchased the residence next to theirs enabling them to expand their residence to accomodate their growing family (those are their two sons riding on the front of Steven's bicycle).

Perhaps Steven and Astrid should expand their award-winning guide to Amsterdam to address Dutch auctions in full. Be sure to access that guide should you be planning a trip to their wonderful city.

---
Top photo of Steven by Barbara Mensink.

Tuesday, March 13, 2007

Calculating return on investment

Much has been reported about the importance of analyzing return on investment (ROI) for user experience to gain influence in a corporate context.

For example, Tobias Herrmann, Head of Team User Experience at mobilkom austria, wrote in "Corporate UX -- Bringing value to the mobile industry" that "a tailored ROI model was the key to success":
"Our ROI model contained, among other things, the monitoring of user experience-specific key performance indicators (KPI), internal performance measurements, standardized product evaluations from a customer's perspective, and, of course, exemplary case studies with high customer and revenue impacts. Further, we included some major user experience KPIs in the Corporate Balanced Scorecard, and even in variable bonus systems for employees. As you might imagine, all these activities strongly supported organizational sustainability, but even more built up a common mindset on user experience and its relevance."
And my conversations with Justin Miller, now Senior Director of Product in Europe at eBay, confirm that estimating ROI has played a major role in moving user experience into a position of influence at eBay. Indeed, at CHI 2004, Jeff Herman described "A process for creating the business case for user experience projects" that enabled eBay's User Experience & Design group to achieve "significant success sponsoring successful user experience projects."

Yet, when he was VP of User Experience at Oracle, Dan Rosenberg wrote:
"…in my 20 plus years of experience, I have never been asked to produce an ROI analysis. Why has this never been necessary? Have I just been lucky in my choice of employers? Did these companies all have CEOs so enlightened about usability that no such analysis was necessary? I suspect not."
And just last month, David Siegel argued the following:
"Our field has been overly preoccupied with ROI as the basis for making the business case for user centered design (UCD). However, experience has shown that the most brilliant ROI analysis may often not win the day in the real world of business. Cost justification and ROI is often not persuasive, especially when we are talking to strategic level decision makers. At a certain point in the evolution of UCD, ROI arguments may have helped us gain credibility and get 'a foot in the door.' However, excessive dependence on ROI arguments can have some destructive effects. To be convincing, ROI analysis has to focus on easily measured variables that impact near-term outcomes. This can distort the way the value of our contribution is conceptualized and recognized, and artificially isolates UCD from other factors that affect the product’s ultimate success. Even more important, it can lock us into a peripheral tactical role where we address only modest incremental improvements. It can work against our field’s efforts to get involved earlier in the product planning process where we can have a more decisive impact and potentially contribute to strategic risk reduction."
Should you attend to calculating ROI where you work? Cannot such calculations contribute to strategic business planning?

This issue is among several that will be addressed by a group of people in senior management roles from a mix of companies during a session I'll be leading at CHI 2007 -- a session entitled, "Moving User Experience into a Position of Corporate Influence: Whose Advice Really Works?". Tobias Herrman (represented by Manfred Tscheligi) and Justin Miller -- both referenced above -- and Jeremy Ashley, Oracle's current VP of User Experience, will be among the session participants.

As I reported in earlier postings, "ownership of user-customer experience," "organizational positioning," and "documenting and evangelizing user experience work" are among other issues that will also be addressed during that session. And I'll address all these issues further as well as the CHI conference session itself in upcoming blog entries.

Wednesday, March 07, 2007

"Check your disciplines at the door" when beneficial

A version of this posting was published in UX Magazine.

During a user experience leadership seminar that I gave last summer, I asked about the extent to which the company's user experience personnel collaborated with each other as well as with others. I had just argued that such collaboration is important and can have a huge impact on the success of a multidisciplinary user experience group or organization.

For the only time during the seminar, the group of user experience practitioners responded with near silence.

I was surprised by this, since discussions about collaboration are often quite animated and can yield lengthy lists of frustrating obstacles encountered in the workplace. However, in this particular workplace, collaboration was seemingly not viewed as an important goal.

To some, collaboration means that nasty "designing by committee" or being forced to defend one's expertise.

To others, a process which facilitates contributions across discipline boundaries is often beneficial.

Consider what Dave Malouf wrote about his new work environment this past December:
"While most design projects seem to have a single designer at the helm, there are projects where different players (industrial designer, researcher, ux designer [read interaction designer]) are all engaged on a single project (Industrial Designer is usually the lead), no one really cares where ideas come from. 'Credit' seems to be shared across the team, thus encouraging more ideas to be generated because people have less fear about grasping ownership and care more about creating great things."
In a recent interview by Lisa Reichelt, Bill Moggridge calls this "checking your disciplines at the door," which he recommends for teams which also -- and which often should -- include personnel from additional disciplines, such as engineering and marketing.

Don Norman concurs, as quoted in "Ownership of the user-customer experience":
"Out of necessity, we divide ourselves up into discipline groups. But the goal when you are actually doing the work is to somehow forget what discipline group you are in and come together."
More from Bill:
"where the complexity is high enough, you're much better with a shared mind, and if you put people together from different disciplines, if they work successfully together rather than having some form of a conflct, then the result with the shared mind will be much greater than any individual mind could be"
Bill advocates "complete submersion in togetherness (particularly) in the main design creative innovation phase," which is reflected pretty well in the diagram to the right which comes from a paper I and two former colleagues wrote about some of the work we did at Viant. Among the many activities in which multiple personnel of the design, strategy, and technology disciplines were submersed in that work: rapid ethnographic research and its interpretation, and subsequent business and design brainstorming.

Yet, guidance and structure for the activities in which everyone was submersed were provided by those with the greatest expertise in the activities and in facilitating them. This was true as well for the "intensive, rapid, cross-disciplinary collaboration" I referenced a couple of years ago in this blog and which resulted in user experience personnel becoming partners in the development of business and product strategy in a particular business unit within Yahoo!.

And, as reflected in the center of the diagram, at other points in the process, the disciplines "spread out," as Bill puts it -- "people bring their individual expertise to the picture" when appropriate [e.g., when you've identified "an interaction design problem, the best person to (address) that would be someone who has a lot of experience with user conceptual models and screen design"].

However, as also stated by Bill:
"it is easy to sit around the table and argue the different case from the different discipline point of view; the big challenge is to work across disciplines fluently"
That might be the most important thing teams need to learn how to do.