The case for vertical integration in analytics platforms

The effective use of data and analytics is a challenge for most companies today. Too seldom do companies generate relevant insights as quickly as they desire and need to. Analytics content must be created in an iterative manner and it must not be impeded by the restrictions caused by data silos. The fast creation of reliable results and sharing them in a secure manner requires seamlessly integrated software that supports the complete analytics cycle from data ingestion to presentation.

Only a vertically integrated software delivers the required degree of flexibility for users through its end-to-end features. This is a clear shift from the paradigm promoting efficiency through horizontal integration. Creating all reports in a single tool and storing all data in a common data warehouse was meant to boost efficiency. Instead of curbing expenditure, horizontal integration curbed the innovative capacity of companies to use their data.

When BI started being industrialized in the 90s, it was largely considered an IT topic as it required fundamental technical expertise. As such, centralizing competence to provide stable service was the obvious move to make. By centralizing competence and standardizing tools, cost advantages were expected. What was lost along the way was the flexibility businesses required to freely visualize and analyze data in new ways that are fit for quick decision-making.

This brought about the rise of shadow BI, which was initially powered almost exclusively by Excel. Change came a decade later with the advent of user-friendly self-service BI and visual analytics. Early resistance was largely overcome when these tools were embraced as an opportunity to eradicate the bottlenecks created by cost-oriented BICCs. Unfortunately, a general lack of governing capabilities, originally perceived as guarantors of flexibility, and the dissemination of analytics into all corners of modern companies overstrained the approach. 

To overcome the defects of earlier generations of analytics and business intelligence software, vertically integrated data and analytics software couples the flexibility required for quick insights with governance features for scaling decentralized self-service and blending it with central delivery. This technology has been available for some time now and has penetrated many areas. Various developments over time have combined to make them the powerhouses of companies successfully deploying analytics to unearth the value of their data treasures.

Enhanced analytics agility: Vertical trumps horizontal integration

Mastery of data usage gives companies an edge over their competition. Converting their data into insights effectively throughout the organization enables them to support decision-making and drive innovation. To this end, a good share of companies sees huge potential in vertically integrated data and analytics software. While laggards have not fully bought into the benefits of the concept yet, leaders have already acted and are reaping substantial rewards. 

Today, these tools are quicker to implement through the cloud and easier to use than ever thanks to ML-based augmented guidance features. Powered by these technological innovations, the increasing scope of analytics requirements is successfully covered by tailored solutions. 

Advanced and predictive analytics, machine learning and AutoML are all prime examples of this increasing scope. While they are not yet supported by integrated data and analytics software as well as reporting and data preparation, they are among the top investment priorities for future implementations.

Boost time-to-insight with vertical integration and realigned processes

While the primary goal of horizontal solutions was to leverage synergies and cash in on economies of scale by serving the whole company, vertical solutions should increase agility. And in a dynamic world, flow is more important than scale. Therefore, it is not surprising that integrated data and analytics software is not widely considered as a tool for serving the whole company.

As with any tool following new paradigms, you must realign analytics processes, roles and responsibilities. This affects requirements and implementation processes as well as the responsibilities of dedicated developers at the center of analytics gravity. Increasing data literacy and intuitive tools empower business users. But to deliver reliable results, they need decent service from full-time experts.

Data democratization needs free flow of data and transparency around usage

There will always be room for improvement in the analytics process as demand and expectations rise constantly. And transparency is a must to democratize access to data in a company. Having comprehensive metadata providing a view on data lineage shows where data is used. Knowing where it is used and who uses it makes it easier to agree to share it. 

A major threat to the free flow of data and ideas are data silos, more often created by restrictive access policies than by incompatible software. With vertically integrated software, one has to be very cautious not to create more of them. Cataloging all available analytical assets helps to lower barriers by making visible what others achieve with data.

Better usability and tight integration propel effectiveness, speed and efficiency

Companies realize various benefits when deploying integrated data and analytics software. The clear number one benefit for companies of all sizes is better usability through unified interfaces, integrated metadata and augmented analytics features – but that is only a means to an end. 

It is an enabler to engage more business users to provide more relevant results in less time to inform and automate decisions. A look at the advantages of deploying integrated data and analytics software shows the stark contrast in satisfaction. While vertically integrated software is no magic wand, the satisfaction with results and creation are twice as high on average.

BARC Recommendations

• Analyze the potential for speeding up analytics and promoting it in business units. Vertically integrated analytics tools empower business users and developers to be more efficient and effective. Quick and intuitive data preparation, analysis and presentation are key in analytics.
• Only in tightly integrated software can experts refine, reshape and enrich their data and present actionable insights in the most suitable way. And with speed comes relevance and effectiveness.
• Identify clusters of requirements that can be covered within a unified platform. Be aware that additional tools must provide clear benefits and must fit into the architecture smoothly. They must not generate additional isolated data silos that limit the innovative power of analytics. Open interfaces and metadata exchange are the technological enablers for the required transparency that the organizational framework must follow.
• Define principles that guide collaboration between units and the BICC in a decentralized analytics environment. These principles must consider the paradigm enabled by vertically integrated analytics software. By dividing work along the data flow, huge benefits in speed and agility can be realized to improve a company’s innovative capacity.
• Reassign and upskill dedicated developers in the analytics organization to train, coach and advise users in business units and local entities to master their own analytics challenges. Together with curating data for shared use and providing guidance through best practices, this creates the foundation for evolving a cost center into a successful service shop. While business analysts and data scientists move to center stage in decentralized analytics, the contribution of developers and data engineers to smooth operations cannot be overestimated.
• Extend the reach of analytics with user-friendly business software for predictive and advanced analytics and machine learningthat includes leading data preparation facilities. Consider requirements of automating decisions even though they may not be crystal clear yet. Operationalizing, deploying and monitoring analytics and ML models in production will be relevant for all companies sooner or later and many are not prepared properly yet.
• Catalog all the analytics assets created throughout your organization, regardless of the tools used or the departments that created them. A comprehensive overview of reports, dashboards, data sets and analytics models is the oil to get the engine of your analytics processes running smoothly. 
• Appreciate the potential of integrated software with exhaustive metadata collection. These tools deliver common and extensible semantic models, collect compelling usage statistics and provide transparency into where data is sourced from, how it is transformed and where it is presented.
• Consider cloud-based analytics solutions when selecting vertically integrated software. Software as a service combines the advantages of practically unlimited scalability, quick set up and resource-efficient operation. For modern analytics, the cloud is a platform that can deliver on today’s and tomorrow’s needs.

Source: BARC (Business Application Research Center)

The role of Centers of Excellence in the development towards data-driven organizations

The CoE role is expanding to meet the demands of data democratization.

Centers of excellence (CoEs) have an important and evolving role that can make their businesses stronger. The role of a CoE was once limited to managing data and unearthing use cases for data and analytics. That changed as the pandemic brought forth a massive acceleration in the use of data and analytics and led to the growth of self-service business intelligence.

Data democratization that comes with this expanding use of data means data can finally be used by those equipped with the right skills and tools for more informed decision making. To provide the degree of excellence that organizations have come to expect, the role of CoEs must expand to focus on empowerment, education, and information sharing.

Ensure Access to the Right Data and Tools

Data democratization requires change management and a general understanding that the role of CoEs is maturing. CoEs can best serve an organization by ensuring that people have the access they need to the data that would be useful to them, as well as the necessary tools to make the most of that data. No doubt this requires a balance in governance -- it wouldn’t serve anyone well to open the floodgates and allow access to all data by all individuals. From security and regulations to legalities and the overall risks associated with open access, data needs to be governed as well as accessible.

This is part of a strategy of empowering employees to do more with data. Instead of building dashboards and specific point solutions, CoEs should focus on supporting an analytics ecosystem. A CoE must foster an analytics environment where the entire organization can participate (to various degrees) in self-service, on-demand analytics. This is a key outcome of successful data democratization and an essential step toward developing an organization that is data driven.

Share Information Across Departments

It is not uncommon for data silos to prevent teams from freely sharing information across departments. This can greatly limit the value of the data collected throughout the organization. Manufacturing, for example, may benefit from data gathered by sales and marketing and vice versa. R&D may be able to unearth valuable insights from the data collected by the firm’s accountants. The potential is only limited by the amount of data shared.

Good governance is crucial here to ensure that sensitive data is not exposed, and this is yet another area where CoEs can demonstrate their wealth of experience and unparalleled level of expertise. By shifting their role to one of leadership and empowerment, a CoE can play an integral role in how data is collected, shared, and used. It can help shape the guidelines of a governance strategy that protects data without preventing it from being used by those with a genuine business analytics need.

Improving Worker Skills Enhances Future Growth

Data scientists and other experts who make up a CoE could work on a data team and have a full-time job simply moving data around from the latest container technology to the next container technology. In the past, that would have kept them remarkably busy. However, today’s modern, data-driven enterprise is moving to the cloud, which can free up time that allows the CoE to pivot to an enablement role, especially to help colleagues become more data literate.

It may seem counterintuitive that CoEs should lead the “upskilling” revolution in its enterprise’s workforce -- after all, a CoE members are the data experts. However, it has become clear that when a CoE collects and disperses data that never gets used, it needs to stop and ask why. More often than not, data is dropped into a black hole because managers, directors, and even executive-level decision makers are unable to read, understand, and work with the data they are given. These skills are naturally built into the data scientist and analyst roles, but as organizations evolve, all employees must be sufficiently data literate to make effective business decisions. By advocating for and helping employees elevate their skills, CoEs can build stronger teams and, as a result, stronger organizations.

Empower, Educate, and Share

Digital transformations have been accelerating at an unprecedented pace. Businesses need to quickly embrace solutions that allow them to move faster and handle unanticipated events, which ultimately means they must fully embrace the idea of becoming data-driven.

Instead of their historical role as gatekeepers to the company’s most valuable information, CoEs need to evolve into champions of data democratization. The potential value of data increases every day; as more employees search for ways to use it to their advantage, CoEs will become more important than ever. A CoE can empower and educate staff and improve information sharing, which will help reshape enterprises and build data-driven organizations.

Author: Mike Potter

Source: TDWI

Using data successfully: the role of data democratization

An effective culture to underpin your strategy

A business that looks to become truly data-driven knows that employees are more likely to back the wider strategy if they have access to data that helps them do their jobs better. Data democratization and the positive culture it can create is, therefore, critical to the long-term success of any organization.

According to a recent report, Data Strategy and Culture: Paving the Way to the Cloud, senior decision-makers are confident that they’re opening up access to data sufficiently.

So do your employees at all levels actually have adequate access to data to boost their decision-making? Does the data at their disposal and how they work with it turn employees into strong advocates for your organization’s data strategy? I seek to address these questions.

This blog focuses on the crossover between data strategy and deployment decisions. It covers:

• Why data democratization is critical to developing a positive data culture
• What are the main barriers to this
• What else you can learn?

Data democratization: room for improvement

Successful organizations identify the key capabilities that are required to execute their data strategy effectively. Infrastructure decisions are an important part of this as any limitations can cause frustration and poor engagement — ultimately, the wrong choice can restrict how well an employee can perform in their role.

Almost four out of five respondents to our survey say their current IT infrastructure makes it challenging to democratize data in their organization. This is a significant obstacle to be overcome. There are additional barriers, too, such as a lack of relevant data skills or too many new data sources.

At this point, businesses have to focus on which deployment model best meets their needs. On the topic of data democratization, many will naturally think of the benefits the cloud can bring. The right deployment model allows for data sharing in a secure and cost-effective manner across all levels and departments. It allows people, and therefore the company, to perform at their best.

Don’t limit your potential

Despite the importance of this, almost half (46%) of respondents to our latest research believe that the democratization of data isn’t feasible for them.

This could be a big risk. If your technology infrastructure doesn’t allow you to open up access to data across the whole business, you’re stopping your organization from becoming truly data-driven. This could ultimately mean that insights can’t be gathered quickly enough, projects could be stalled, and a competitive edge on competitors can be lost.

Make the data work

There is a clear need for organizations to carefully consider which deployment option gives them the freedom needed to effectively open up access to data. Yet, the story doesn’t end once a decision has been made.

Teams must constantly monitor whether employees are able to work with the data at their disposal effectively. Can they get the insights they need from the data? Is there an ambition to increase the spread of data democratization within the organization?

So when it comes to optimizing the success of your data strategy, data democratization is an important and key step in the process — and your company needs to get it right.

The report investigates all of the key points raised in this blog and explains how developing a positive data culture starts with data democratization. This is the point when you secure your employees’ backing of the project. Only then are you truly ready to choose the right deployment model.

Author: Mathias Golombek

Source: Dataversity