Monitor, evaluate and act to improve with DevOps analytics

Today, most companies can say that they have integrated some form of DevOps collaboration between their development and operations teams. They are breaking down the silos, communicating better, and making the company more efficient as a result. But is that really true? Is your DevOps movement doing what it was set out to do?

DevOps first came about in 2007-2008 to fix problems in the software industry and bring with it continuous improvement and greater efficiencies. The idea was that to be more successful, development and operations teams that had never worked together before were now expected to share responsibilities, communicate with each other, and be transparent. Thus forming a highly performant DevOps team.  

For most companies, especially analytics businesses, the word 'efficiency' is what sparks interest. If the main goal is to bring about efficiencies, shouldn’t there be some measurement available to make sure the target is being met? 

Of course, there should be. And it’s called DevOps analytics.

DevOps analytics is the analysis of machine data to find insights that can be acted upon. In this case, insights that can be responded to in order to optimize a sequence or a larger process quickly. DevOps data analytics can be set up and measured at any time during your DevOps journey. Of course, the sooner you do it, the sooner you will be able to measure your successes and failures and make necessary adjustments. Every day is critical, especially in a DevOps environment where teams are working faster and more reliably. 

Getting started

Start by listing your KPIs. You should have at least one KPI for every part of your product cycle; planning, development, testing, deployment, release, and monitoring. Don’t overlook adding a few KPIs for the active cooperation between different teams (since this is the heart of the DevOps culture). Think in terms of developers working with their operations counterparts to deliver operable applications, or operations specialists monitoring applications delivered by development.  

Useful KPIs should be obtainable, reviewable, and actionable. With the ‘actionable’ KPI playing a significant role in analytics for DevOps. We’ll circle back to the actionable part later on in this blog. 

If you’re managing a team, you might want to consider adding usage analytics to your routine. This is an excellent way to see who is looking at their dashboards and using the data to move things forward.  

With DevOps touching on so many different aspects of the entire company (teams, people, processes, etc.), there are multiple DevOps dashboards that can be created. Most of the metrics you will want to measure will fit within three categories:

1. The process

This is the ultimate measurement. With DevOps teams working together to improve the entire service lifecycle, from design through development to production and support, this is where DevOps data analysis will drive efficiency and effectiveness. 

A good time-to-market dashboard can be used to measure and optimize the product development cycle. Phases of the product that are in delay can be identified quickly, and the overall delivery of the project can be minimized by acting on those insights quickly. 

Quality assurance dashboards can help i managing the release of the project. Detailed information on the cases in progress and the number of completed and failed cases all contribute to the decisions that need to be made.  Bottom line, if the quality assurance manager decides the release isn’t up to par, there better be data to back it up. 

It is recommended that you set alerts for key milestones that you want to reach, and get notifications if you are under or over the target. This will give you time to adjust the process and get those sequences back on track before it affects the entire process.

Set up your dashboard to include milestones, and then monitor the progress daily, even hourly if necessary. Just imagine what your daily scrum meetings will look like when you pull up a visual dashboard showing the team’s progress for build duration or cycle times? Nothing beats the visual impact of a chart to display the hard work of the entire team. And at the end of the sprint, you can pull up the historical data and discuss the output, and what can be improved in the coming period.   

Once you execute a successful activity, you can use it again and again. With DevOps teams putting more emphasis on releasing software faster and more reliably, reusing good processes can speed up the frequency of releases without diminishing the quality.

2. The end user

The holy grail of measurements. If customers are not satisfied with the product or release, then it doesn’t matter how efficient or effective your DevOps process is. Customer success is the base of any business growth, and there are many metrics you can choose from to measure this: NPS (Net Promoter Score), customer satisfaction score, customer effort score, churn rate, expansion revenue, and more.

Set up a customer service dashboard to track the most crucial customer service metrics for your business. Some examples of the metrics this dashboard shows are customer churn, customer engagement, and customer experience, as well as metrics like help desk tickets, CRM tracker, and call center analytics. It may not be obvious, but all of these items can be indicators that something is off and in need of a quick fix to bring value back to your customers. 

It will also help to keep track of customer satisfaction by asking your customers straight-up if they would recommend your product/service to a friend or colleague. High or low scores here are a direct reflection of your DevOps processes, and how you deliver quality products to your customers.

3. The people

People metrics, or employees, are a great way to measure success. Happy employees tend to have a tremendous cascading effect on the entire company, and the accompanying processes.

If you’re doing DevOps right, then a majority of your employees will be experiencing modification (almost daily) in the way they are collaborating with other teams. HR teams can use analytics to keep their fingers on the pulse of these specific teams and stay ahead of any issues that may be swerving from the norm. 

Here are some of the main HR KPIs that can be associated with DevOps: employee turnover rate, productivity, satisfaction, and even employee compensation (check to make sure you’re paying your DevOps people what they’re worth). HR teams can use people analytics to provide insight into organizational performance and engagement.

Keeping your DevOps people happy will keep the spirit of this movement (cooperation, communications, and transparency) alive and kicking in your company.  

Make it actionable and gain a competitive edge

Once you’ve started to measure DevOps KPIs, the next step will be to start implementing changes in the process by becoming predictive with your analytics. Predictive analytics uses AI (Augmented Intelligence) and ML (Machine Learning) to identify other patterns and relationships that might be hidden or harder to realize. This is the actionable part of your KPIs. By using predictive analytics, you can detect certain anomalies, act on them, and gain a real competitive edge.

Now you’re making the most of your DevOps journey.

Author: Dana Liberty

Source: Sisense