To really make the most of big data, most businesses need to invest in some tools or services - software, hardware, maybe even new staff - and there's no doubt that the costs can add up. The good news is that big data doesn't have to cost the Earth and a small budget needn't prevent companies from stepping into the world of big data. Here are some tips and ideas to help keep costs down:
Think about your business objectives
Too many businesses focus on collecting as much data as possible which, in my view, misses the whole point of big data. The objective should be to focus on the data that helps you achieve your strategic objectives. The whole point of big data should be to learn something from your data, take action based on what you've learned and grow your business as a result. Limiting the scope of your data projects so they tightly match your business goals should help keep costs down, as you can focus only on the data you really need.
Make use of the resources you already have
Before you splash out on any new technology, it's worth looking at what you're already using in your business. Some of your existing infrastructure could have a role to play. Go through each of the four key infrastructure elements (data sources, data storage, data analysis and data output) and note what related technology or skills you already have in-house that could prove useful. For example, you may already be collecting useful customer data through your website or customer service department. Or you very likely have a wealth of financial and sales data that could provide insights. Just be aware that you may already have some very useful data that could help you achieve your business objectives, saving you time and money.
Look for savings on software
Open source (free) software, like Hadoop, exists for most of the essential big data tasks. And distributed storage systems are designed to run on cheap, off-the-shelf hardware. The popularity of Hadoop has really opened big data up to the masses - it allows anyone to use cheap off-the-shelf hardware and open source software to analyse data, providing they invest time in learning how. That's the trade-off: it will take some time and technical skill to get free software set up and working the way you want. So unless you have the expertise (or are willing to spend time developing it) it might be worth paying for professional technical help, or 'enterprise' versions of the software. These are generally customised versions of the free packages, designed to be easier to use, or specifically targeted at various industries.
Take advantage of big data as a service (BDaaS)
In the last few years many businesses have sprung up offering cloud-based big data services to help other companies and organisations solve their data dilemmas. This makes big data a possibility for even the smallest company, allowing them to harness external resources and skills very easily. At the moment, BDaaS is a somewhat vague term often used to describe a wide variety of outsourcing of various big data functions to the cloud. This can range from the supply of data, to the supply of analytical tools which interrogate the data (often through a web dashboard or control panel) to carrying out the actual analysis and providing reports. Some BDaaS providers also include consulting and advisory services within their BDaaS packages.
BDaaS removes many of the hurdles associated with implementing a big data strategy and vastly lowers the barrier of entry. When you use BDaaS, all of the techy 'nuts and bolts' are, in theory, out of sight and out of mind, leaving you free to concentrate on business issues. BDaaS providers generally take this on for the customer - they have everything set up and ready to go - and you simply rent the use of their cloud-based storage and analytics engines and pay either for the time you use them or the amount of data crunched. Another great advantage is that BDaaS providers often take on the cost of compliance and data protection - something which can be a real burden for small businesses. When the data is stored on the BDaaS provider's servers, they are (generally) responsible for it.
It's not just new BDaaS companies which are getting in on the act; some of the big corporations like IBM and HP are also offering their own versions of BDaaS. HP have made their big data analytics platform, Haven, available entirely through the cloud. This means that everything from storage to analytics and reporting is handled on HP systems which are leased to the customer via a monthly subscription - entirely eliminating infrastructure costs. And IBM's Analytics for Twitter service provides businesses with access to data and analytics on Twitter's 500 million tweets per day and 280 million monthly active users. The service provides analytical tools and applications for making sense of that messy, unstructured data and has trained 4,000 consultants to help businesses put plans into action to profit from them.
As more and more companies realise the value of big data, more services will emerge to support them. And competition between suppliers should help keep subscription prices low, which is another advantage for those on a tight budget. I've already seen that BDaaS is making big data projects viable for many businesses that previously would have considered them out of reach - and I think it's something we'll see and hear a lot more about in the near future.