RESEARCH NOTE C47

ROI ANALYSIS YOU CAN TRUST T M

Assessing the real ROI from Siebel

THE BOTTOM LINE Because the average Siebel deployment costs a company more than $18,000 per user per year, achieving a positive ROI means carefully assessing benefits and managing deployment costs. Sixtyone percent of customers interviewed do not believe they achieved a positive ROI from Siebel. SIEBEL SOLUTIONS Siebel Systems provides an integrated family of e-business applications software that enables multichannel sales, marketing, and customer service systems. Siebel solutions include the following: Sales, including Sales Management, Forecasting, Analytics, Incentive Compensation, and Mobile Sales functionality Marketing, including eMarketing, eEvents, and Market Analytics functionality Call Center and Service, including Call Center, Web Service, Field Service, Mobile Service, and Professional Services Automation functionality Interactive Selling, including eAdvisor, eSales, eConfigurator, eAuction, and ePricer components Partner Relationship Management Employee Relationship Management Analytics MidMarket Universal Application Network Industry Applications Many companies have invested in customer relationship management (CRM) solutions with the hope of achieving a positive ROI by increasing marketing share, reducing costs, and growing profits. Nucleus Researchs objective in looking at Siebel deployments was to assess the costs and benefits companies are actually experiencing and to provide Nucleus clients with information on best practices for achieving a positive ROI from Siebel. THE PROCESS For two weeks in June 2002, Nucleus searched the Siebel Web site to identify Siebel customers who could be included in the analysis. Nucleus did not select a particular customer vertical or Siebel solution, Nucleus attempted to interview every customer used as an example on the Siebel Web site. The assumption was that because Nucleus was examining customers that Siebel was itself promoting as examples of successful deployments, Nucleus would find customers that had received better-than-average returns. By analyzing these customers, Nucleus expected it could learn the common themes and best practices for achieving a positive ROI from Siebel so it could better advise its user clients.

2002 Nucleus Research, Inc. Reproduction in whole or in part without written permission is prohibited. Nucleus Research is the leader in the return on investment analysis of technology. Please visit www.NucleusResearch.com.

After a complete search, Nucleus found 66 customers Siebel had profiled on its Web site and contacted each one: Twenty-three agreed to participate in interviews. Twelve declined to participate for various reasons. Thirty-one never responded to repeated requests for interviews. Those who agreed to participate were asked questions about factors of their Siebel deployment that would impact ROI, including the following: How, why, and when did you select Siebel? How many users are using what functionality? What are the most significant areas of returns? How much did you spend on the project, and in particular, how much did you spend on software and consulting? What was your customization and deployment strategy? What have been your deployment challenges? Nucleus included data from all reference customers interviewed in this report. Most spoke on condition of anonymity. All phrases in quotation marks in this report, unless designated otherwise, are actual comments from the same Siebel customer spokesperson quoted in the customer case study on the Siebel Web site. Sixty-one percent of customers interviewed didnt believe they achieved a positive ROI from their Siebel deployment. THE BENEFITS FROM SIEBEL On its Web site, Siebel claims the average customer achieves increases of 12 percent in revenue, 20 percent in employee productivity, and 20 percent in customer satisfaction. However, 61 percent of the reference customers Nucleus surveyed did not believe they had achieved a positive ROI from their Siebel investment. Two factors that drive benefits from Siebel appear to be the willingness of the companys users to adopt Siebel technology and the ability of the company to customize the solution or use its standard functionality to support sales and customer service activities. In many cases, companies failed on one or both points: essentially, there were challenges in both entering information into and extracting information from the system. Some individual customers achieved a positive ROI from a specific Siebel component. More than one customer indicated that it had received positive returns from a call center application deployment but that other Siebel components purchased did not deliver the same level of returns. The few customers using other Siebel solutions that did achieve positive returns followed a deployment strategy targeting specific direct benefits. Goals such as headcount reduction or an increase in profits on revenues were planned from the beginning of their Siebel deployment. For example, one customer who was also a strategic alliance partner of Siebel said it calculated a 6-month payback period based on increased revenues from its Siebel Sales deployment.

Nucleus Research, Inc. www.NucleusResearch.com

Page 2

CUSTOMER ADOPTION EXPERIENCES Key to achieving productivity-based returns from any software solution is ensuring that users adopt and use it effectively. According to many Siebel reference customers, complexity and difficulty of use of Siebel solutions were significant stumbling blocks to achieving productivitybased returns from Siebel.

Seventy-eight percent of Siebel customers interviewed cited lack of user friendliness as a challenge to achieving positive returns from their Siebel deployment. Specific comments from customers included the following: The complexity of the product is a challenge. It is not as intuitive and user friendly as we had hoped. It required too much more training before we get the bang for the buck. It needs to be more user friendly. According to Siebels Web site, one customer said, Employees have been extremely positive about the system. But when interviewed by Nucleus, the same contact at the same customer site said, Despite a managed internal campaign to popularize the solution, they dont see this as an advantage. This is not a selected sample of customer responses; in fact, not one of the customers interviewed had a positive comment about the usability of Siebel. Companies can often resolve issues around software user adoption and training by customizing the application or interface to better align with user or corporate practices. However, for many of the reference customers Nucleus interviewed, further development and customization added time and cost without delivering the expected benefits. Sixty-five percent of Siebel customers interviewed cited difficulty in customization and performance as a challenge to achieving positive returns from their Siebel deployment. Specific comments from customers included the following: We needed to customize, and we found out that it is not so easy to customize. Our business environment is dynamic, but the rigidity of Siebel was an obstacle to customizing. Siebel cannot accommodate 100 percent of our customization and business needs its too rigid. There was some functionality customization we needed that we hadnt anticipated, so we had to keep some consultants on for longer, which increased costs. We had issues with performance of the solution. It needs to be a more multi-user environment in terms of data retrieval. In some cases we could not contact the person who was quoted and we were directed to interview other people associated with the project. Some of their comments follow: You have to keep investing to get the solution to try to work. The biggest challenge has been rigidity. None of the screens we use today have been out of the box. None of the existing screens get

Nucleus Research, Inc. www.NucleusResearch.com

Page 3

used at all; they all had to be customized and rewritten. We had to write some Visual Basic programs on our own. The challenge is getting reporting to work like you want. The data tables are complicated. As with the previous customer comments, these are representative samples from the customers interviewed; no customer had positive views about customizing or adapting Siebel. This result was surprising considering the number of customers interviewed who were quoted positively about functionality and appropriateness to business needs on the Siebel Web site. DEPLOYMENT EXPERIENCES Another option for improving adoption and usability issues is deploying portions of functionality in phases as users become more comfortable with the solution. This appears to be the strategy many Siebel reference customers have taken with their deployment. In fact, Siebel reference customers interviewed had, on average, been using Siebel solutions for more than two years. Respondents believe that, on average, they are using only 60 percent of the functionality of the solution. As one reference customer stated, We will always be deploying Siebel. Most customers purchased more than one Siebel component and, in many cases, slowed deployment of one component because of difficulties or changes in another. A better ROI-maximizing strategy would be to rapidly deploy the component that delivers the greatest returns and then evaluate and plan the deployment of other components. A phased deployment can be an effective strategy for a company to achieve a positive ROI from a complex technology. However, the phased functionality deployment approach not only slows the realization of benefits but also often increases and extends costs of consulting, training, and personnel. To determine whether a phased deployment is an appropriate approach, companies must consider whether the hopedfor benefits will ever be attainable and, if so, when. They must also consider the overall costs of the project and whether they are receiving enough value from the software they have purchased to justify the purchase price even if theyre using only half the expected functionality for the foreseeable future. THE COSTS OF SIEBEL In calculating the costs of customers Siebel deployments, Nucleus surveyed companies actual and projected spending on software, consulting, and personnel in the first three years of deployment. Because hardware and training costs varied widely, Nucleus excluded them from the calculation of the average cost of Siebel rather than skew the results. The total average cost of a deployment is therefore greater than the conservative figures calculated in Table 1. Nucleus found that the average cost of a Siebel deployment for the first three years was more than $6.59 million, or an average of at least $18,040 per user per year, based on the data customers provided.

Nucleus Research, Inc. www.NucleusResearch.com

Page 4

Table 1. The 3-Year Cost of a Siebel Deployment Category Software Cost $1.501M Data components Average initial license price = $1.01M Average annual maintenance price = $163,794 Average consulting expenditure of customers interviewed Average number of FTEs to support Siebel = 8.1

Consulting Personnel Total

$2.667M $2.422M $6.590M

In a normal enterprise software deployment, one would expect to have higher costs for the first few years and progressively lower costs over time. However, many customers Nucleus interviewed were continuing if not increasing their consulting and personnel investment to support Siebel after more than two years of deployment, so a dramatic reduction in total costs after the first three years is unlikely.

Table 2. The Per-User Cost of a Siebel Deployment Initial per-user costs Software license Consulting Ongoing per-user costs Software maintenance Support personnel Average annual peruser cost for the first three years. $1,689 $3,932 $893 $3,000 Average $11,315 $25,939 Median $4,190 $6,300

$18,040

$8,296

Thus, Nucleus believes it is not unreasonable to use the $18,040-peruser figure as an estimate for annual ongoing costs associated with a Siebel deployment, at least for the first few years.

Fifty-seven percent of Siebel customers interviewed said their deployment took longer than planned; fifty-five percent said they spent more money deploying Siebel than they had budgeted. Nucleus examined each of the three main cost areas software, consulting, and personnel in greater detail. SOFTWARE COSTS Nucleus found that the average initial license price spent for Siebel software was $1,009,696:

Nucleus Research, Inc. www.NucleusResearch.com

Page 5

 Based on actual number of users, the average license price per user for Siebel was $11,315, with a median of $4,190 per user, a high of $110,000 per user, and a low of $550 per user. The average annual license maintenance fee for Siebel ranges from 15 to 33 percent with an average of 18.2 percent, or $1,689 per user, for maintenance. Customers believe Siebels license price is high compared with that for their other software investments. Their specific comments about the price of Siebel software included the following: As a general rule, Siebel is expensive. In addition, the pricing structure is complex, and it is difficult to figure out what youre being sold, what youre getting, and what youll eventually not get. There are certain things that we now wish we hadnt licensed. The pricing structure is too much. Siebel is too much money. Our biggest challenge is pricing and pricing structures. Siebel was very expensive. It was expensive for us. On an ongoing basis, some of the new stuff were looking at it was like, Are you kidding me? Tom has very tight controls on pricing, so theres not a lot of individuality: here it is, take it or leave it. We pay around $100K a year in ongoing maintenance fees. Im very shocked about that it wasnt sold that way. Again, this is a representative sample of customer comments; no customers interviewed had positive comments about Siebel license costs. CONSULTING COSTS Almost all of the customers interviewed used Siebel Professional Services or a Siebel Alliance Partner to support solution development, customization, deployment, or testing. Seventeen percent of customers interviewed cited consulting or consultants as a primary challenge to achieving positive returns from their Siebel deployment. The average consulting spend for a Siebel deployment was $2,666,902, with a high of more than $14 million and a low of zero. The median consulting spend was $900,000. Specific comments from customers included the following: According to Siebels Web site, one customer said the deployment went smoothly and on time, but when interviewed by Nucleus, the same customer said, It took longer than we had planned. They couldnt finish it. We couldnt do it ourselves. The Siebel alliance partner didnt have adequate resources, and they ended up fighting with Siebel because the project wasnt going smoothly. Siebels rates for service are astronomical compared to other providers without providing greater value. I would like to see their TAMS [technical account managers] get better. We havent seen a wonderful experience there. In some cases we could not contact the person who was quoted and we were directed to interview other people associated with the project. One of them said, Our main problem was inexperienced consultants. They were a partner of Siebel and were recommended by Siebel, but they

Nucleus Research, Inc. www.NucleusResearch.com

Page 6

didnt have the experience to show us the maximum benefit and ways to exploit the solution. PERSONNEL COSTS Beyond the initial phase of the project, the average company has 8.1 people supporting its Siebel deployment, or one person for every 71 users. Companies should use this ratio as an estimate for ongoing personnel budgeting purposes and more than that number for the initial phase of the project. THE SIEBEL RELATIONSHIP In many cases, software companies have been forgiven for bugs and product limitations because their account managers and professional services teams worked closely with customers to resolve problems. The quotes on Siebels Web site were very different in tone and detail from the comments Nucleus received from customers interviewed. Nowhere was there more of a discrepancy than in the comments about the customers experience with Siebel. Following are a few examples: According to Siebels Web site, one customer said Siebel and its partners were clearly committed to its success and were working to ensure it was satisfied. When interviewed by Nucleus, the same contact at the same customer site said, We have had challenges with communicating with Siebel about product issues. We wish Siebel had been more forthcoming and more knowledgeable about issues. Their services area needs to be better developed. According to Siebels Web site, one customer said Siebel did an excellent job of making sure technical and business needs were met. When interviewed by Nucleus, the same contact at the same customer site said, Theyre very arrogant, very full of themselves. Their service staff is unresponsive to the point that I use their services as little as we can. According to Siebels Web site, one customer said Siebel was dedicated to providing 100 percent customer satisfaction. That person has since left the company. Nucleus interviewed the new project manager, who said, Every bit of support we receive from Siebel has price attached to it. The Siebel sales staff has been impatient with us that way. There is some basic level of support we expected that we shouldnt have to pay for. RECOMMENDATIONS Nucleus found many companies deploying Siebel solutions had higher costs and lower actual returns than expected because of usability and customization issues. It is imperative that companies carefully consider the potential ROI from their deployment and manage the development and deployment of the solution to ensure they achieve a positive ROI. Its interesting to note that Siebel does have an ROI tool on its Web site. As Nucleus advises all its clients, these automated tools both Web and non-Web based should be viewed more as marketing materials than as a means to objectively develop a business case for deploying technology. Calculating an ROI number is not as important as understanding the actual costs and benefits of a deployment. Instead of blindly believing a quick calculation, companies should complete a ROI evaluation that can then be used as a roadmap for

Nucleus Research, Inc. www.NucleusResearch.com

Page 7

deployment. This is particularly true in cases where careful management of a deployment phase is critical to ensure a positive ROI. Companies considering a CRM investment today should do the following: Carefully consider the ROI not just of Siebel but of all CRM options. You may find less well-known solutions that provide similar benefits with lower license and customization costs. If you choose to purchase a Siebel solution, aggressively negotiate with Siebel on price. One strategy is to negotiate value-based pricing and defer payment until expected benefits have been recognized. Consider non-technology alternatives that may deliver a higher ROI than an enterprise Siebel deployment, such as greater incentives for key personnel. Plan a phased deployment that targets the specific areas likely to produce returns first, and dont allow project creep or other components to delay delivery of benefits. Companies currently deploying or using Siebel today should do the following: Rapidly re-evaluate your deployments current ROI versus the expected ROI. Determine if direct attainable benefits are possible in the next six months. If so, limit all consulting and training spending to areas that will deliver those benefits; if not, consider stopping or delaying deployment. Consider renegotiating ongoing maintenance and consulting contracts based on specific deployment and benefit delivery milestones. CONCLUSION Nucleus was very surprised by the results of this analysis. Our goal was to learn from the customers Siebel considers its success stories about how they achieved a positive ROI and what common lessons learned could be applied to help Nucleuss clients. What we found were expensive deployments, limited benefits, and customers dissatisfied with their deployments. This analysis clearly shows that although CRM as a solution may provide benefits to companies, large-scale CRM deployments incur greater risk in cost and deployment time and should be carefully evaluated.

Nucleus Research, Inc. www.NucleusResearch.com

Page 8