Research shows demanding users and increasing data volumes are a deadly combination for the data war

Data warehouse performance is adversely affected by increasing user pressures, increasing volume of the data stored and lack of alignment between data value and storage. The architecture of data warehouse will have to change to accommodate the business demands being placed on it.

SAND Technology Inc., an innovator in analytic application infrastructure, released, at the Data Warehouse Institute TDWI- World Conference, the results of an independent research study by Dynamic Markets Limited entitled The Crumbling Foundations of Data Warehouses. The research reveals that data warehouse managers are coming under increased pressure as users demands grow at a time when large volumes of data are seriously decreasing the performance of data warehouses. According to the survey of data warehouse managers in the Top 2000 UK companies, 88 per cent of respondents think that the original architecture of their data warehouse will have to change to accommodate the business demands being placed on it. These demands come from two sources: the requirements of users within the business, and the pressures resulting from the way in which the data is stored within the architecture. Summary results show that fully 97 per cent of data warehouse managers feel that their users are becoming more demanding, and over three quarters 86 per cent- are having to restrict user queries in order to alleviate the strain on both the technology and their team. At the top of the list of areas in which users are being demanding is in requirements for query performance, with almost two thirds 62 per cent- of users asking for faster retrieval of data, followed by the growing complexity of requests 51 per cent-, and mounting interest in exploring older data 41 per cent-. In addition to user pressures, the volume of data stored, and the lack of alignment between data value and storage methods, is adversely affecting data warehouse performance, leading more than 60 per cent of respondents to expect they will increase the manpower allocated to their data warehouse in the next three to five years. The challenge of tiering the data according to business value in order to ease the pressures of retrieval requirements still resonates loudly with the sample: almost half 48 per cent- think that only a quarter of the total data stored is queried on a regular basis, yet less than 20 per cent tier their storage according to business value. Resource: businessintelligence.ittoolbox.coma>