Advanced content management, search, business intelligence and other big data technologies have armed companies with the ability to affordably analyse vast amounts of data. Given their potential to deliver massive value and competitive advantage, these technologies have not surprisingly seen tremendous growth and adoption over the past 18 months, according to new research conducted by the Association for Information and Image Management (AIIM).

The new AIIM report, 'Big Data and Content Analytics: measuring the ROI,' is a follow-up to a similar report issued in April 2012 and highlights the growth of big data over the past 18 months and exposes key trends. It also provides recommendations for how companies can more efficiently realise the benefits of analysing big data. See also: Analysts urge IT leaders to deal with 'dark data'

The latest research indicates that while big data analysis is increasingly seen as an essential capability, 60% of organisations have only limited capability in business intelligence (BI) reporting, and 65% have a 'somewhat disorganised' approach to content management.

This is not a good starting point for the complex world of big data analytics, warns AIIM, which fears that many organisations are finding that they have vast quantities of 'dark data' (data that lacks any control or classification) that could be holding valuable business intelligence.

'Big data potentially holds huge insight for organisations, but the mass of “dark data” could impact the ability to extract that insight effectively,' said Doug Miles, director of market intelligence at AIIM.

'Businesses should look to harness their information, and combine it across disparate systems as a precursor to their big data journey. Connecting structured transactional datasets to unstructured data or text-based content is a key way to unlock knowledge, but most are finding this to be a major challenge.' See also: Hadoop co-founder believes the future of big data is the Enterprise Data Hub

Beyond connectivity, having sufficient skilled users to work with big data toolsets is identified in the report as the next biggest challenge; perhaps no wonder then that 34% of early adopters either outsourced projects or brought in outside expertise (13%). A further third recruited or trained internal specialists, with the remaining third relying on existing in-house expertise.

Security is the third major big data adoption challenge – a potential show-stopper for nearly one in five organisations. Protecting personal data is the primary concern, but commercial and financial information is also sensitive. Security is also an inhibitor for cloud or SaaS deployment of big data tools. Automated cleaning and sanitising of content repositories is a big data application in its own right.

Source: information-age.com, 3 december 2013