Throughout the business intelligence software industry, product enhancements made by vendors have led to increasing functionality overlap amongst competitors. Product evaluations within IT organizations no longer examine the relative capabilities of point functionality such as reporting or OLAP, but discuss the ability of vendor product suites to provide all possible BI-related functionality. In this day and age of products that promise to be all singing and all dancing, IT organizations can be overwhelmed with the task of conducting effective product evaluations. This leads to the tendency of evaluators to pick a BI product based on past vendor relationships and marketing claim beliefs. IT organizations also face the question of what to do with their existing BI assets. These assets consist not only of licensed software, but also databases, meta-data, reports, integrations, Web sites, and so forth, that have been built around their third- party software. The scenario is often one where the existing deployment was only brought about through heroics ? not only by the organization s IT department but by the vendor s field consultants. So, with this as a background, organizations face a significant dilemma: do they stick with their current deployment, upgrade to new, perhaps untried, future versions of their current vendor s software or look at a new vendor? Any specific decision can mean the commitment of hundreds of thousands, if not millions, of dollars. It s a high stakes game. Ventana Research believes the application of some basic evaluation rules can assist the strategic BI acquisition decision-making process. Most important is that a business case be created that assesses the costs and returns so ROI be defined and critiqued. All possible scenarios should be considered, including do nothing and focus elsewhere. Estimating costs should be done using a total cost of ownership TCO- approach. Of course, the challenge with this approach is that portions of the cost assessment can be difficult to make accurate. Costs ascribed to ownership of BI solutions can come from various different budgets with varying levels of management oversight and politics. This also makes TCO harder to do. Nevertheless, without a TCO-based approach, overall cost optimization for the organization cannot be attained. Ventana Research recommends that organizations break TCO into hardware, software and personnel costs. The first two are relatively easy to estimate. Personnel costs can be further segmented into the following categories: Selection and purchase Installation and configuration Development, QA Deployment Administration Maintenance User evangelization, training and support Breaking down TCO costs into these categories will rapidly reveal which areas are most costly. Organizations should focus on the big-ticket costs, as their rounding errors will become noise that overwhelms the calculation of lesser costs. Assigning a dollar value to benefits can be even more difficult than assessing TCO. Ventana Research recommends creating intermediate measures of benefits to simplify the process. Benefits can be assigned to the following categories and then translated into dollar value of return: Improved speed: query response time, decision time, data massaging time, data publishing time, etc. Improved ease of use: requiring less training, skills, thinking, etc. Improved safety: lower risk of user error, lower risk of sabotage or theft, etc. Improved intelligence: better decisions, more decisions made, new opportunities found, etc. Lower cost: less personnel costs, less hardware/software costs, etc. These benefits should be evaluated from a user perspective first. Improved user benefits have the largest ROI potential. Not only can improved user efficiency contribute to lower organizational costs, but improved user effectiveness can contribute to improved organizational revenue. Benefits to the IT organization in terms of reduced cost should be considered as well, but these cost reductions can have a lower potential impact to the whole organization. Once costs and benefits are assessed and quantified, the ROI can easily be calculated along with potential error factors. Using ROI to evaluate BI technology strategies will significantly diminish both corporate and personal ambiguity and risk. Business intelligence technology is now a top-line priority for CIOs. Enterprise deployments of BI costing millions of dollars are now commonplace. Organizations can no longer use back-of-the-napkin-based assessments of BI technology, as we once did for departmental deployments. Ventana Research recommends using evaluation frameworks to understand ROI, key cost and benefit factors and estimation errors. TCO should be used to provide organizations with a complete cost assessment. Cost and benefit analyses should be simplified by using categorization and value translation to estimated dollar amounts. With IT costs now a larger-than-ever proportion of total organization expenses, CIOs can benefit from using similar evaluation frameworks as their C-level peers elsewhere in their organizations. Bron: www.intelligententerprise.coma>