Our last two releases focused on open-source capabilities and the reengineering of our mainframe trace streaming feature to support OpenTelemetry. In this article, we will share why we see these capabilities as sure steps into vendor-neutrality as well as essentials for evolved DevOps teams that depend on mainframe application services. Before we delve into OpenTelemetry, observability and application performance monitoring (APM), lets take a look at what DevOps experts Jez Humble, Gene Kim, Peter Richards and Jeffrey Snover say about DevOps and how it affects your business, in this short YouTube video created by Puppet (an industry-leading IT automation software provider).
What is DevOps, a Youtube video by Puppet
The Four Key Metrics to measure DevOps performance
Research by DevOps Research & Association LLC (DORA) identified four key metrics that indicate the performance of a software development team:
- lead time (from code committed to code deployed),
- deployment frequency (to production),
- change fail percentage (for production deployments) and
- mean time to restore a.k.a. MTTR (from a production failure)
Application Performance Monitoring (APM) as a DevOps tool
After software is planned, developed, tested and deployed, DevOps teams turn to APM products and software to help monitor and improve their change fail percentage and MTTR. Next level APM software can among other things: detect deployment changes to applications, measure transaction performance, system resource utilization and storage consumption, alert on application errors and provide warnings when user-experience is deteriorating due to slow response times. To achieve this, APM vendors create “agents” to instrument applications, virtual machines, databases, operating systems and more. The agent instrumentation produces traces and metrics which are sent to an APM server(on-premise or SaaS), where they are correlated and visualized in a web-based user-interface.
End-to-end all-in-one APM products are expensive commodities
What happens when DevOps teams manage applications that depend on services or systems that their APM software does not provide an agent for (e.g. mainframe)? The unsupported services are displayed as part of the network calls made by the application. This network call will show a response time recorded from the point of view of the calling application that could include actual network time, message encryption/decryption and multiple tiers of internal and external services and processes. If the response time slows down or an error occurs, DevOps teams are unable to identify the root cause of the issue (network, security, internal or external services, system failure … ) . This can have negative effects on the team’s ability to maintain low change fail percentages and speed up MTTR.
Cue Cloud Native Computing Foundation’s OpenTelemetry Project
CNCF’s OpenTelemetry project aims to solve the shortfalls of proprietary and open-source APM software by introducing standards that will facilitate end-to-end, vendor-neutral observability. OpenTelemetry is an observability framework, made up of a collection of tools, APIs, and SDKs, and it defines specifications for collecting and exporting telemetry (metrics, logs and traces) for analysis in any APM product.
So why is OpenTelemetry a big deal for APM vendors and users?
Industry leading APM vendors, open-source APM software providers and countless others, recognize the benefits of what OpenTelemetry offers. Many APM vendors provide native OpenTelemetry support in their commercial products. The ability to support OpenTelemetry traces and metrics as well as integrate their APM software with 3rd party tools will increase the business value of their product and improve customer satisfaction. Having a standard instrumentation capability that produces vendor neutral traces and metrics means that vendors can focus on enhancing and introducing more unique capabilities through user-interfaces, increase customizability and refine Artificial Intelligence support. As Mark Albertson reported in a SiliconANGLE article about how OpenTelemetry is attracting major enterprise interest, over 80 companies, including global technology industry leaders, are involved in this project which has become the most popular CNCF project after Kubernetes.
Since z/IRIS launched in 2019, we learned that DevOps teams need tools that can adjust to their business needs as they arise and change over time. The rise of self-service platforms in high-performing DevOps companies attests to this (Puppet’s State of DevOps 2020 Report). With z/IRIS, teams have access to mainframe observability in their APM products, new and old, proprietary and open-source. Even integrating mainframe metrics in open-source visualization software like Grafana is possible using the z/IRIS Metrics Streaming feature, which streams metrics to in-house data sinks (e.g. time series databases like InfluxDB) that feed self-service, open-source or cloud-native visualization software. z/IRIS lets your teams use the tools that best suits their needs and get more mainframe information where they need it.
Mainframe-inclusive observability meets sustainable vendor-neutrality
z/IRIS provides mainframe observability capabilities in APM products. Using OpenTelemetry’s Context Propagation specification (a.k.a. correlation), z/IRIS traces are correlated to the related distributed application service calls in any APM software. We also create labels that are readable and understandable to teams that do not necessarily have extensive mainframe knowledge. DevOps teams use z/IRIS to begin monitoring the impact of their business applications and services on mainframe costs and resources like processors and storage. They can also identify latencies that occur within the mainframe transactions used by their applications.
The z/IRIS Distributed Db2 for z/OS Tracing feature has helped teams identify when their applications are impacted by deadlocks on Db2 servers and whether their applications are blocking the Db2 resources and causing other applications to experience timeouts or slow response times. Stay tuned to our Blog to find out more about our z/IRIS features and how they have been designed to help teams improve their DevOps performance. Can’t wait, then dig into product documentation to find out more about z/IRIS in our knowledge base.
We are excited about the future of mainframe-inclusive APM
As OpenTelemetry support grows and improves, we hope to see more standardization and increased parallel support between proprietary and OpenTelemetry agents. Gartner’s 2020 Magic Quadrant for Application Performance Monitoring predicts that 50% of new cloud-native application monitoring will use open-source instrumentation instead of proprietary agents to improve interoperability by 2025. Gartner describes how this will simplify monitoring, enable interoperability between monitoring solutions and replace the current agent-based pricing with value driven pricing-models.
Our development will continue to build onto and expand on our OpenTelemetry support, as we investigate more ways to provide mainframe support to DevOps teams. We believe that the growth of z/IRIS and mainframe-inclusive observability will encourage DevOps teams to integrate their mainframe application programmers and system administrators to boost all four of the DevOps performance metrics in mainframe-backed organizations. Mainframe customers who achieve mature and high performing DevOps, will gain a unique edge over their competitors when it comes to software development, reveal the substantial support that mainframe systems provide to the business and improve overall customer satisfaction.