Book Club: The DevOps Handbook (Introduction)

The following is a chapter summary for “The DevOps Handbook” by Gene Kim, Jez Humble, John Willis, and Patrick DeBois for an online book club.

The book club is a weekly lunchtime meeting of technology professionals. As a group, the book club selects, reads, and discuss books related to our profession. Participants are uplifted via group discussion of foundational principles & novel innovations. Attendees do not need to read the book to participate.

Background on The DevOps Handbook

More than ever, the effective management of technology is critical for business competitiveness. For decades, technology leaders have struggled to balance agility, reliability, and security. The consequences of failure have never been greater―whether it’s the healthcare.gov debacle, cardholder data breaches, or missing the boat with Big Data in the cloud.

And yet, high performers using DevOps principles, such as Google, Amazon, Facebook, Etsy, and Netflix, are routinely and reliably deploying code into production hundreds, or even thousands, of times per day.

Following in the footsteps of The Phoenix Project, The DevOps Handbook shows leaders how to replicate these incredible outcomes, by showing how to integrate Product Management, Development, QA, IT Operations, and Information Security to elevate your company and win in the marketplace.

The DevOps Handbook

An Introduction to DevOps

“Imagine a world where product owners, Development, QA, IT Operations, and Infosec work together, not only to help each other, but also to ensure that the overall organization succeeds. By working toward a common goal, they enable the fast flow of planned work into production (e.g., performing tens, hundreds, or even thousands of code deploys per day), while achieving world-class stability, reliability, availability, and security.”

An Introduction to DevOps

In this world, cross-functional teams rigorously test their hypotheses of which features will most delight users and advance the organizational goals.

Simultaneously, QA, IT Operations, and Infosec are always working on ways to reduce friction for the team, creating the work systems that enable developers to be more productive and get better outcomes.

This enables organizations to create a safe system of work, where small teams are able to quickly and independently develop, test, and deploy code and value quickly, safely, securely, and reliably to customers.

By adopting Lean principles and practices, manufacturing organizations dramatically improved plant productivity, customer lead times, product quality, and customer satisfaction, enabling them to win in the marketplace.

Before the revolution, average manufacturing plant order lead times were six weeks, with fewer than 70% of orders being shipped on time.

By 2005, with the widespread implementation of Lean practices, average product lead times had dropped to less than three weeks, and more than 95% of orders were being shipped on time.

Most organizations are not able to deploy production changes in minutes or hours, instead requiring weeks or months. These same organizations are not able to deploy hundreds or thousands of changes into production per day. They struggle to deploy monthly or even quarterly. Production deployments are not routine, but instead involve outages and firefighting.

The Core, Chronic Conflict

In almost every IT organization, there is built-in conflict between Development and IT Operations that creates a downward spiral, resulting in slower time to market for new products and features, reduced quality, increased outages, and an ever-increasing amount of technical debt.

Technical Debt: the term “technical debt” was first coined by Ward Cunningham. Technical debt describes how decisions we make lead to problems that get increasingly more difficult to fix over time, continually reducing our available options in the future — even when taken on judiciously, we still incur interest.

Two competing organizational interests: respond to the rapidly changing competitive landscape and provide a stable service to the customer.

Development takes responsibility for responding to changes in the market, deploying features and changes into production. IT Operations takes responsibility for providing customers with IT service that is stable and secure, making it difficult for anyone to introduce production changes that could jeopardize production. Dr. Eli Goldratt called these types of configuration “the core, chronic conflict”.

The Downward Spiral

The first act begins in IT Operations, where our goal is to keep applications and infrastructure running so that our organization can deliver value to customers. In our daily work, many of our problems are due to applications and infrastructure that are complex, poorly documented, and incredibly fragile. The systems most prone to failure are also our most important and are at the epicenter of our most urgent changes.

The second act begins when somebody has to compensate for the latest broken promise—it could be a product manager promising a bigger, bolder feature to dazzle customers with or a business executive setting an even larger revenue target. Then they commit the technology organization to deliver upon this new promise. Development is tasked with another urgent project that inevitably requires solving new technical challenges and cutting corners to meet the promised release date, further adding to our technical debt.

The Third and final act, where everything becomes just a little more difficult, bit by bit—everybody gets a little busier, work takes a little more time, communications become a little slower, and work queues get a little longer. Our work becomes more tightly-coupled, smaller actions cause bigger failures, and we become more fearful and less tolerant of making changes. Work requires more communication, coordination, and approvals; teams must wait longer for their dependent work to get done; and our quality keeps getting worse.

Why Does the Downward Spiral Happen?

Every IT organization has two opposing goals, and second, every company is a technology company, whether they know it or not. The vast majority of capital projects have some reliance upon IT.

“When people are trapped in this downward spiral for years, especially those who are downstream of Development, they often feel stuck in a system that pre-ordains failure and leaves them powerless to change the outcomes. This powerlessness is often followed by burnout, with the associated feelings of fatigue, cynicism, and even hopelessness and despair.”

An Introduction to DevOps

A culture can be created where people are afraid to do the right thing because of fear of punishment, failure, or jeopardizing their livelihood. This can create the condition of learned helplessness, where people become unwilling or unable to act in a way that avoids the same problem in the future.

Counteracting the Downward Spiral

By creating fast feedback loops at every step of the process, everyone can immediately see the effects of their actions. Whenever changes are committed into version control, fast automated tests are run in production-like environments, giving continual assurance that the code and environments operate as designed and are always in a secure and deployable state. Automated testing helps developers discover their mistakes quickly.

High-profile product and feature releases become routine by using dark launch techniques. Long before the launch date, we put all the required code for the feature into production, invisible to everyone except internal employees and small cohorts of real users, allowing us to test and evolve the feature until it achieves the desired business goal.

In a DevOps culture, everyone has ownership of their work regardless of their role in the organization.

The Business Value of DevOps

High-Performing Organizers succeed in the following areas:

• Throughput metrics
• Code and change deployments (thirty times more frequent)
• Code and change deployment lead time (two hundred times faster)
• Reliability metrics
• Production deployments (sixty times higher change success rate)
• Mean time to restore service (168 times faster)
• Organizational performance metrics
• Productivity, market share, and profitability goals (two times more likely to exceed)
• Market capitalization growth (50% higher over three years)

When we increase the number of developers, individual developer productivity often significantly decreases due to communication, integration, and testing overhead. DevOps shows us that when we have the right architecture, the right technical practices, and the right cultural norms, small teams of developers are able to quickly, safely, and independently develop, integrate, test, and deploy changes into production.

Organizations adopting DevOps are able to linearly increase the number of deploys per day as they increase their number of developers

“The purpose of the DevOps Handbook is to provide the theory, principles, and practices needed to successfully start a DevOps initiative. This guidance is based on decades of management theory, study of high-performing technology organizations, work the authors have done helping organizations transform, and research that validates the effectiveness of the prescribed DevOps practices.”

An Introduction to DevOps

The reader is not expected to have extensive knowledge of any of these domains, or of DevOps, Agile, ITIL, Lean, or process improvement. Each of these topics is introduced and explained in the book.

The goal is to create a working knowledge of the critical concepts in each of the above listed areas.

Series Navigation