Accelerating software delivery through observability at two very different organizations

Delivering value to customers quickly and efficiently is critical to the success of modern businesses. Understanding the process and timeframe during which an organization generates new ideas and then designs, develops and deploys them is vital to success. At the recent Illuminate User Conference, Drew Horn, Director of Business Development, held a discussion with Clara Ko, Director of Engineering at Sauce Labs, and Bryan Veselka, Director and Product Owner for Cloud and Automation at Vizient. They shared how they use observability to streamline and accelerate software delivery in their organizations.

Ko has a proven track record of building high-performing teams that use continuous delivery to support scalable cloud-based systems. Sauce Labs is known for its extensive automated testing platforms that include virtual machines, banks of mobile devices, and many other essential components that support comprehensive testing.

Vizient provides technology solutions and partnerships with medical organizations and professionals. Although they serve a very different customer base than Sauce Labs, Vizient also relies on DevOps principles to design, build and deliver technology solutions quickly.

Metrics that matter—DORA

Despite the stark contrast in the market segments served by Sauce Labs and Vizient, they use similar metrics to understand the health of their DevOps teams and identify opportunities to optimize their processes. Both organizations use metrics identified by Google Cloud’s DevOps Research and Assessment (DORA) team. The four key metrics identified by DORA are:

• Deployment frequency

• Lead time for changes

• Time to restore services

• Change failure rate

Here's how Sauce Labs and Vizient use metrics and DevOps practices to accelerate software delivery in their organizations.

Digital transformation and embracing DevOps culture

According to Ko, Sauce Labs sees its software development process as a “people system.” They use metrics to gain observability into that system and see how it relates to the flow of ideas. Ko reported that they are currently focusing on deployment frequency and looking for ways to improve this over time.

Vizient has similar goals. Veselka said that they’re focused on speed and finding ways to deliver their software as efficiently as possible. Like many companies, they’re actively seeking to improve their processes so that they can remain at the forefront of the digital movement within the healthcare industry.

Balancing conformity and flexibility

A common challenge faced by both Sauce Labs and Vizient is gathering and reporting standardized data across heterogeneous teams. At Vizient, this is due to the fact that each team uses different technologies for development and deployment. Recent growth at Sauce Labs inspired their leadership to find innovative ways to assess the health of new and existing teams across the organization to identify improvement opportunities. “Having a common view across all our teams and product lines and trying to set up a system where leadership can get signals about what is going well and not going well and be able to visualize it is important,” shared Ko. “We start by looking at the whole picture. How do we figure out what customer problems are there that we can help with?”

Both Vizient and Sauce Labs stress the importance of finding ways to introduce monitoring and observability that don't require the adoption of additional technologies or impose undue burdens. It’s vital to find that balance so that teams can begin experimenting with different ways to ensure continuous improvement.

Driving a DevOps culture

DevOps culture is less about technology and more about the people involved. When addressing how Sauce Labs implemented such a culture, Ko discussed the importance of ensuring that new teams work alongside existing teams that are focused on similar products. The architectures and systems that organizations build often reflect how the teams involved communicate and collaborate.

From an agile perspective, the value proposition of this approach as compared to traditional waterfall development lies in improving delivery based on visibility and shorter feedback loops. Vizient found that sharing information about delivery speed across their engineering teams in a standardized manner is a major challenge. To counter this, Vizient is looking at the bigger picture. Teams now investigate how long it takes for an idea to move from ideation through development and deployment and into the hands of their customers.

Ko added that true agile maturity involves expanding a team’s definition of “done.” Can the team go beyond just deploying a new feature into production and identify ways to measure the impact on consumers? Can they validate the intention and idea that initially drove the development of the feature? This approach requires that teams plan how to validate the effects of a feature in advance, as well as how to bake it into the design, development, and deployment process. This validation is essential as teams experiment with different features and product enhancements.

Gathering and correlating end-to-end metrics

This greater awareness and observability into the entire system comes with additional challenges. As Horn pointed out, this is something that Sumo Logic is working to improve as well. Often, many of the systems create their own data silos, making it difficult to collect and share data with observability and monitoring solutions. The challenge is to identify how these systems emit events and then map that data to a normalized schema for automated correlation and computation of insights to illustrate the entire lifecycle of a product idea.

Vizient is currently working on measuring the lead time for their new features. "It’s great to have an idea, but the time it takes for you to actually start working on that idea is incredibly important," says Veselka. They’ve gained a great appreciation for the flexibility to track metrics as DevOps tools change and improve. Vizient’s evolution was enabled in part by their use of multiple deployment tools. The ability to map data from Jenkins, Azure DevOps pipelines, and GitHub into Sumo Logic’s standardized platform has been invaluable.

In addition to reporting data, engineering teams have found that the ability to access their data and view a consolidated picture of how their processes impact the overall picture is very beneficial. Sumo Logic’s platform consolidates and displays data in an easy-to-view, extensible format that's helpful to Vizient’s engineering teams. Ko emphasized that Sauce Labs has found it extremely valuable to have a flexible, organization-level tool that aggregates and normalizes data from all of their engineering teams.

Final Recommendations and Insights

It's important to learn from those who are actively engaged in accelerating software delivery within their organizations. Ko and Veselka had a few final thoughts and recommendations.

Ko emphasized the importance of focusing on the wider organization rather than just the engineering team. It's also important to understand why a team might be underperforming. For example, aside from experiencing issues with engineering metrics, the team might be unclear about their roadmap or have a high volume of work that needs to be prioritized. Or, it may be something else that isn’t immediately apparent when looking at metrics alone. It’s important to understand the individual nature of each team and appreciate the challenges it faces.

Veselka recommended that teams take an iterative approach and stressed the importance of embracing the journey toward improvement. You don’t have to have visibility into every single system to realize value. He suggested that teams start with what they see as most valuable and take it one step at a time.

Despite being in different industries and having different customer bases, organizations now share a common framework for driving faster to deliver critical solutions through a set of industry-standard DevOps practices and delivery tooling.

If you're interested in learning more about optimizing your software delivery practices through better observability, check out Sumo Logic Application Observability and Software Development Optimization solutions.