- Book Club: The DevOps Handbook (Introduction)
- Book Club: The DevOps Handbook (Chapter 1. Agile, Continuous Delivery, and the Three Ways)
- Book Club: The DevOps Handbook (Chapter 2. The First Way: The Principles of Flow)
- Book Club: The DevOps Handbook (Chapter 3. The Second Way: The Principles of Feedback)
- Book Club: The DevOps Handbook (Chapter 4. The Third Way: The Principles of Continual Learning and Experimentation)
- Book Club: The DevOps Handbook (Chapter 5. Selecting Which Value Stream to Start With)
- Book Club: The DevOps Handbook (Chapter 6. Understanding the Work in Our Value Stream, Making it Visible, and Expanding it Across the Organization)
- Book Club: The DevOps Handbook (Chapter 7. How to Design Our Organization and Architecture with Conway’s Law in Mind)
- Book Club: The DevOps Handbook (Chapter 8. How to Get Great Outcomes by Integrating Operations into the Daily Work of Development)
- Book Club: The DevOps Handbook (Chapter 9. Create the Foundations of our Deployment Pipeline )
- Book Club: The DevOps Handbook (Chapter 10. Enable Fast and Reliable Automated Testing)
- Book Club: The DevOps Handbook (Chapter 11. Enable and Practice Continuous Integration)
- Book Club: The DevOps Handbook (Chapter 12. Automate and Enable Low-Risk Releases)
- Book Club: The DevOps Handbook (Chapter 13. Architect for Low-Risk Releases)
- Book Club: The DevOps Handbook (Chapter 14. Create Telemetry to Enable Seeing and Solving Problems)
- Book Club: The DevOps Handbook (Chapter 15. Analyze Telemetry to Better Anticipate Problems and Achieve Goals)
- Book Club: The DevOps Handbook (Chapter 16. Enable Feedback So Development and Operations Can Safely Deploy Code)
- Book Club: The DevOps Handbook (Chapter 17. Integrate Hypothesis-Driven Development and A/B Testing into Our Daily Work)
- Book Club: The DevOps Handbook (Chapter 18. Create Review and Coordination Processes to Increase Quality of Our Current Work)
- Book Club: The DevOps Handbook (Chapter 19. Enable and Inject Learning into Daily Work)
- Book Club: The DevOps Handbook (Chapter 20. Convert Local Discoveries into Global Improvements)
- Book Club: The DevOps Handbook (Chapter 21. Reserve Time to Create Organizational Learning and Improvement)
- Book Club: The DevOps Handbook (Chapter 22. Information Security as Everyone’s Job, Every Day)
- Book Club: The DevOps Handbook (Chapter 23. Protecting the Deployment Pipeline and Integrating Into Change Management and Other Security and Compliance Controls)
- Book Club: The DevOps Handbook (Conclusion)
The following is a chapter summary for “The DevOps Handbook” by Gene Kim, Jez Humble, John Willis, and Patrick DeBois for an online book club.
The book club is a weekly lunchtime meeting of technology professionals. As a group, the book club selects, reads, and discuss books related to our profession. Participants are uplifted via group discussion of foundational principles & novel innovations. Attendees do not need to read the book to participate.
Background on The DevOps Handbook
More than ever, the effective management of technology is critical for business competitiveness. For decades, technology leaders have struggled to balance agility, reliability, and security. The consequences of failure have never been greater―whether it’s the healthcare.gov debacle, cardholder data breaches, or missing the boat with Big Data in the cloud.
And yet, high performers using DevOps principles, such as Google, Amazon, Facebook, Etsy, and Netflix, are routinely and reliably deploying code into production hundreds, or even thousands, of times per day.
Following in the footsteps of The Phoenix Project, The DevOps Handbook shows leaders how to replicate these incredible outcomes, by showing how to integrate Product Management, Development, QA, IT Operations, and Information Security to elevate your company and win in the marketplace.
The DevOps Handbook
Chapter 17
All too often in software projects, developers work on features for months or years, spanning multiple releases, without ever confirming whether the desired business outcomes are being met, such as whether a particular feature is achieving the desired results or even being used at all.
Before building a feature, teams should ask themselves: “Should we build it, and why?”
A Brief History of A/B Testing
A/B testing techniques were pioneered in direct response marketing, which is one of the two major categories of marketing strategies. The other is called mass marketing or brand marketing; it relies on placing as many ad impressions in front of people as possible to influence buying decisions.
In previous eras, before email and social media, direct response marketing meant sending thousands of postcards or flyers via postal mail, and asking prospects to accept an offer by calling a telephone number, returning a postcard, or placing an order.
Integrating A/B Testing Into Feature Testing
The most commonly used A/B technique in modern UX practice involves a website where visitors are randomly selected to be shown one of two versions of a page, either a control (“A”) or a treatment (“B”).
A/B tests are also known as online controlled experiments and split tests. Performing meaningful user research and experiments ensures that development efforts help achieve customer and organizational goals.
Integrate A/B Testing Into Releases
Fast and iterative A/B testing is made possible by being able to quickly and easily do production deployments on demand, using feature toggles and potentially delivering multiple versions of our code simultaneously to customer segments.
Integrate A/B Testing Into Feature Planning
Product owners should think about each feature as a hypothesis and use production releases as experiments with real users to prove or disprove that hypothesis.
Hypothesis-Driven Development:
- We Believe that increasing the size of hotel images on the booking page.
- Will Result in improved customer engagement and conversion.
- We Will Have Confidence To Proceed When we see a 5% increase in customers who review hotel images who then proceed to book in forty-eight hours.