DASA DevOps Fundamentals FR 1.3.0 - eBook-English-(en-US)
Courseware DevOps Fundamentals Premium_1.4.1_EN English (en-US) Global K, SA de CV DASA DevOps Fundamentals Trainin
Views 138 Downloads 4 File size 10MB
Recommend stories
- Author / Uploaded
- Maria Hincapie
- Categories
- Desarrollo de software
- Prueba (evaluación)
- Tecnología
- Informática
- Tecnologías de la información
Citation preview
Courseware
DevOps Fundamentals Premium_1.4.1_EN English (en-US)
Global K, SA de CV
DASA DevOps Fundamentals
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright and Disclaimer DASA DevOps Fundamentals | r1.4.1 Copyright Copyright © Devops Agile Skills Association LLC 2018. All rights reserved. This is a commercial confidential publication. All rights reserved. This document may not, in a whole or in part, be copied, reproduced, translated, photocopied, or reduced to any medium without prior and express written consent from the publisher. This course includes copyrightable work under license and is protected by copyright. No part of this publication may be reproduced, distributed, or transmitted in any form or by any means, including photocopying, recording, or other electronic or mechanical methods, without the prior written permission of the publisher, except in the case of brief quotations embodied in critical reviews and certain other noncommercial uses permitted by copyright law or further disseminated without the express and written permission of the legal holder of that particular copyright. The Publisher reserves the right to revoke that permission at any time. Permission is not given for any commercial use or sale of this material.
Trade Marks DevOps FundamentalsTM is a registered trademark of DASA Limited.
Disclaimer Information provided about the course, modules, topics and any services for courses including simulations or handouts, are an expression of intent only and are not to be taken as a firm offer or undertaking. The Publisher reserves the right to discontinue or vary or maintain such course, modules, topics, or services at any time without notice and to impose limitations on enrolment in any course. The course materials provided may have hypertext links to a number of other web sites as a reference to users. This service does not mean that the publisher endorses those sites or material on them in any way. The publisher is not responsible for the use of a hypertext link for which a commercial charge applies. Individual users are responsible for any charges that their use may incur. The publisher is also not responsible for the discontinued service if any, for example, the Web links in the course might not work in future. The information in this course is written using a blend of British and American English. Although every effort has been made regarding the usage of correct spelling, punctuation, vocabulary, and grammar with regard to the Standard English, the publisher accepts no responsibility for any loss or inconvenience caused due to the regional differences in the usage of the English language.
ii │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Contents Acknowledgements v Module 1: Course Introduction
1
Let’s Get to Know Each Other 1 Overview 1 Course Objectives 2 Course Agenda 3 Type of Activities 4 Exam 4 Module Summary 5 Module 2: DevOps Introduction
7
Module Objectives Module Topics Emergence of DevOps Core Principles of DevOps DevOps Agile Skills Association (DASA) Module Summary Module End Questions
7 8 8 21 30 40 41
Module 3: Culture
43
Module Objectives Module Topics Essence of a DevOps Culture Key Elements of DevOps Implementation of a DevOps Culture Module Summary Module End Questions
43 44 44 56 74 78 80
Module 4: Organization
81
Module Objectives 81 Module Topics 82 Organizational Model 82 Autonomous Teams 88 Architecting for DevOps 97 Governance 106 Module Summary 109 Module End Questions 111 Module 5: Processes Module Objectives Module Topics Process Basics DevOps in Relation to ITSM Agile and Scrum Optimizing Processes Using Lean Business Value Optimization and Business Analysis Using Story Mapping Module Summary Module End Questions
113 113 114 114 115 119 136 146 150 151
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
iii
Module 6: Automation Module Objectives 6A Automation Concepts Topics 6A Automation Concepts: Automation for Delivery of Software 6A Automation Concepts: Continuous Delivery Core Concepts 6A Automation Concepts: Continuous Delivery Automation Concepts 6A Automation Concepts: Continuous Delivery Automation Focus Topics 6B Data Center Automation 6B Data Center Automation: Emergence of Cloud Technology and Principles 6B Data Center Automation: Cloud Services Concepts in a DevOps Organization 6B Data Center Automation: Automated Provisioning Concepts 6B Data Center Automation: Platform Product Characteristics and Application Maturity Module Summary Module End Questions Module 7: Measure and Improvement Module Objectives Module Topics Importance of Measurement Choosing the Right Metrics Monitoring and Logging Module Summary Module End Questions Are you missing something? Exam Preparation Guide Module Learning Objectives Topics Covered in This Module 1. Qualification Learning Objectives 2. Learning Level of the Syllabus 3. Certification 4. Exam Instructions 5. Tips for Taking Exam
153 153 153 154 157 164 168 179 179 184 188 193 196 198 201 201 202 202 203 214 223 224 225 227 227 227 227 228 229 229 229
Mock Exam
231
Appendix A: Answers
251
Appendix B: Syllabus
257
Appendix C: Glossary
279
Appendix D: Release Notes
297
Appendix E: Participant Feedback Form
299
iv │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Acknowledgements We would like to sincerely thank the experts who have contributed to the development of the DevOps Fundamentals.
Lead Author Rik Farenhorst, Business Unit Manager at Xebia Rik has a PhD in software engineering and more than 10 years of experience in the business-IT domain. He has worked as Enterprise/IT Architect, Consultant, Trainer and Coach, and has later grown into sr. management positions. He guides organizations and individuals in raising their bar by making IT simpler, better, and of higher quality. Rik works as Business Unit Manager at Xebia, a specialized international IT consultancy, solutions and training company specialized in high-performance IT, digital transformation, and thought leader in DevOps. Rik is a member of the editorial board of the DevOps Agile Skills Association (DASA).
Michiel Sens, Solution Architect at Xebia Michiel Sens has worked in the IT Industry for over 18 years, is co-writer of the book ‘Continuous Delivery for Managers’ and works as a Principal Consultant/ Solution Architect at Xebia in The Netherlands. Michiel specializes in Continuous Delivery and full lifecycle software development programs and as such focuses on coaching Agile Teams, setting up central build environments, automating tests, automating deployment over multiple environments and implementing fully automated PaaS platforms. He advocates the use of Continuous Delivery at seminars and meetups and remains in touch with details by performing actual implementations as well.
Frederik Schukken, Managing Consultant at Quint Wellington Redwood Frederik is a Managing Consultant at Quint Wellington Redwood where he and his team help businesses to adopt disruptive thinking and create highperformance IT. He is passionate about helping others to adopt Lean and DevOps methodologies and break out of rigid, traditional ways of working.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
v
Thomas Kruitbosch, DevOps Consultant at Xebia Thomas is specialized in Continuous Delivery and optimizing IT for fast adoption of new technologies. Thomas has extensive experience and in-depth knowledge of both development and operations. As Consultant, Trainer, and Conference Speaker, Thomas advocates the use of Agile and Automation Delivery principles. Also, he performs technical implementations of Continuous Delivery Automation and (cloud) platforms.
Niels Loader, Managing Consultant at Quint Wellington Redwood As an advisor to 10 of IT organizations, Niels has extensive knowledge and experience in implementing IT Service Management, IT Performance Management, Lean IT, and DevOps within IT organizations. In 2010 and 2011, he was one of the initiators of the Lean IT Foundation certification and spent four years as the Chief Examiner for the APMG Lean IT certification. He is the Lead of the Content team of the Lean IT Association.
vi │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
1 Course Introduction Let’s Get to Know Each Other Introduce yourself in the following format: Name Company Role and background Familiarity with DevOps concepts and their practice Experience in application development, infrastructure development, and/or operations Expectations from this course
Overview This 3-day course provides learners an extensive introduction to the core Agile DevOps principles. It covers all 12 key knowledge and skill competences that have been defined by the DevOps Agile Skills Association (DASA). With the help of key DevOps concepts and terminology, cases or scenarios, group discussions, and examples included in the course, you will acquire fundamental knowledge of DevOps. DevOps Fundamentals is the starting point for anyone involved in an Agile and/or DevOps team. Improved workflows and faster deployment start with a core understanding of DevOps fundamentals by all team members. This course is designed to provide the core education necessary to build your DevOps vocabulary and to understand its principles and practices. The course will inspire you to serve as a Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
1
Course Book | DASA DevOps Fundamentals
change champion by sharing and using what you have learned, and continue to learn, about DevOps to lead and mentor others.
Course Objectives
Course Objectives DevOps relation to Lean and Agile methodologies
Critical success factors for DevOps implementation
Key concepts and principles of DevOps
Service Delivery process
Drivers responsible for the emergence of DevOps
Popular DevOps tools
Business benefits of DevOps and continuous delivery
Test automation, infrastructure automation, and build and deployment automation
At the end of this course, you will be able to:
Copyright © 2018 | 3
Explain the drivers responsible for the emergence of DevOps. Define and discuss the key concepts and principles of DevOps. List and explain the business benefits of DevOps and continuous delivery. Describe the Service Delivery process. Explain the concepts of test automation, infrastructure automation, and build and deployment automation. Describe how DevOps relates to Lean and Agile methodologies. List the most common and popular DevOps tools. Discuss the critical success factors for DevOps implementation.
2 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 1 | Course Introduction
Course Agenda
Course Agenda DAY 1 Module
Subject
01
Course Introduction
02
DevOps Introduction
03
Culture
04
Organization
DAY 2 Module
Subject
04
Organization (Contd.)
05
Processes
06
Automation
Course Agenda (Contd.) DAY 3 Module
Subject
06
Automation (Contd.)
07
Measure and Improvement
Copyright © 2018
Recap
Mock Exam Certification Exam (Optional) Though the Mock Exam is provided in the Course Book, you need to visit the DASA website, http://www.devopsagileskills.org/, for the latest version. Therefore, it is possible for the Mock Exam provided in the Course Book to be different from the one provided on the website. Note: It is not necessary to appear for the Certification Exam on Day 3. You can attempt the exam later as well.
Copyright © 2018
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
3
Course Book | DASA DevOps Fundamentals
Type of Activities Group Discussions The course contains group discussions, spread out in all modules, with the intent of enhancing participants’ understanding, adding context to the content, broadening participants perspective, reinforcing knowledge, and building confidence. By interacting among themselves and responding to the varying viewpoints, participants tend to learn continually. These discussion allow the participants to come across the thoughts of their peers, which help them know about each other’s past experience, perspectives, and opinions in the context of the topic in discussion. Caselets A case (or scenario) with related exercises and activities is used in the course. These exercises can include: Brainstorming Discussion Forum Group Discussion
Exam At the end of the course, an exam will be conducted. The exam details are: Bloom Level: 1 and 2 Question Type: Multiple Choice Questions (MCQs) Question Number and Passing Mark: 40 questions with a minimum passing rate of 65% (26 correct out of 40) Time: 60 minutes (15 minutes extra for non-native English speaker) Exam Type: Closed book Suggestion: Recommended that participants take the exam after completion of the course
Activity: Group Discussion Activity Time: 10 mins Write down your expectations from this training on a sticky note and attach it to a wall.
4 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 1 | Course Introduction
Module Summary In this module, you learned that: DevOps Fundamentals is a 3-day course that is designed to provide the basic education required to build your DevOps vocabulary and understand its principles and practices. What are the various objectives that this course will help you accomplish? What is the 3-day schedule of the training? The course contains group discussions and activities for better understanding of the concepts. The exam of this course will have 40 MCQs, and its duration will be 60 minutes.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
5
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
vOps
als
2 DevOps Introduction Module Objectives Module Objectives
Emergence of DevOps
The central idea behind and the business case for DevOps
When and why traditional ways of working in IT fall short
Meaning of DevOps for you as a professional and for your organization
Core principles and concepts of DevOps
The intent and scope of the DevOps Agile Skills Association (DASA) Required knowledge and skill areas for DevOps professionals Copyright © 2018 | 1
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
7
Ops
s
Course Book | DASA DevOps Fundamentals
At the end of this module, you will be able to: Describe the emergence of DevOps. Explain when and why traditional ways of working in IT fall short. Define the central idea behind and the business case for DevOps. Explain the core concepts and principles of DevOps. Define the required knowledge and skill areas for DevOps professionals. Explain the intent and scope of the DevOps Agile Skills Association (DASA). Discuss what DevOps means for you as a professional and for your organization.
Module Topics Emergence of DevOps Core Principles of DevOps DevOps Agile Skills Association
Emergence of DevOps Typical Challenges Traditional IT Organizations Face Typical Challenges Traditional IT Organizations Face Low Quality
Manual Release
ence of DevOps
iples of DevOps
e Skills ociation
Product Backlog
Infrequent Releases
8 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 |
Module 2 | DevOps Introduction
In many organizations, IT is more a bottleneck than a strategic enabler or differentiator. In times in which “software is eating the world”, it is a necessity to improve the overall quality of the IT capability. In many organizations nowadays, software is still delivered late, with many errors, released only a few times per year, and costs and processes are far from being Lean and mean. There is no option to sit still and do nothing about this. If organizations want to survive the ever fasting pace competitors enter their market, customer requires new services/ products as existing products become obsolete. Therefore, they need to deal with these typical challenges. The business demands faster and continuous delivery. However, it is not easy due to various challenges that occur due to contracting goals of the various teams, especially the Development and the Operations teams, involved in the software development and delivery. The job of the Development team is to build software, apply changes to incorporate new features, and fulfill the internal as well as the external requirements. On the other hand, the Operations team focuses on stability, reliability, and performance of the systems maintained by them. The two competing contradicting goals of the two teams result in a wall of confusion.
ASA DevOps
ndamentals
Need for Change and Stability Causes Tension
Need for Change and Stability Causes Tension
The typical challenge that traditional IT organizations face is to deal The typical challenge that traditional IT organizations face is to deal with the wall of confusion between withDevelopment the wall of confusion between Development and Operations. and Operations. Is the Development and the Operations teams connected to each other?
Emergence of DevOps Core Principles of DevOps
Wall of Confusion Think of the wall of confusion as a solid brick wall where no communication is possible between the people standing on either side. The traditional way of developing software is negatively impacted by this wall of confusion. If the Development team is working on an application, they hand it over to the Operations team only when it is complete. This approach has many loopholes and ultimately results in severe problems in production that causes blast like situation.
Wall of confusion Stability
Change
DevOps Agile Skills Association
Development
Operations
What is the key goal of the Development team?
What is the function of Operations team? Copyright © 2018 | 5
The wall of confusion is a psychological and procedural barrier that obstructs the flow of communication between the Development and the Operations teams and can result in severe problems in production, such as: No methodical hand-over to the Operations team is done. Consequently, the Operations team faces problems in production Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
9
Course Book | DASA DevOps Fundamentals
that they are unable to solve and look to the Development team for resolving the problem. Such a feedback loop delays problem resolution. In the absence of required discussions between the Development and the Operations teams during earlier phases of development, a lot of useful information is not shared between the two teams. Such information is crucial for the Operations team to get ready for the upcoming changes to the applications under development. The Operations team can share valuable information from their experience of managing the Production environment. This information can help the Development team design and develop more robust applications. However, due to lack of communication between the two teams, this information sharing is missed. A critical part of the transition between the Development and the Operations teams is knowledge articles. These articles help the Operations team to solve known problems. In the presence of the wall of confusion, these knowledge articles are missed. As a result, the Operations team spends valuable time solving trivial problems for which the solution is already known.
Activity: Group Discussion Wall of Confusion
Activity Time: 10 mins What can be the possible problems that can arise due to the wall of confusion between Development and Operations?
Problems Although the wall of confusion plays a significant role in the problems IT organizations face, it should be clear that dissolving the wall of confusion requires tackling a variety of underlying problems, such as: Organizational Silos: In many IT organizations, the Development and Operations teams tend to work in isolation from one another. This ensures that they are not confronted with each other until the critical moments of the delivery of IT products and services. The stress of these moments tends to lead to irritation rather than understanding. Different Mindsets: The Development team aims to incorporate new techniques or features to do their work efficiently. On the other hand, these changes not only to the IT service but also to the underlying components make the work of the Operations team difficult as such changes result in instability. Therefore,
10 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 2 | DevOps Introduction
the Operations team sees these changes as a challenge in maintaining the integrity of the Production environment. Different Implementations: The different implementations to perform the same work by the two teams result in incompatibility and lead to various bugs in the QA and the Production environments. Different Tools: The different tools used by the two teams lead to various errors and bugs in the Production environment. As an example, Development might deploy to a Test environment using a dependency management tool, while Operations might use a self-made script for the process. Lack of Interest in Learning Other Tools: Each team considers its tool or style of working to be the best and does not want to learn a new tool. Different Environments: The different environments, such as Development, Test, and Production, are one of the largest sources of errors and bugs raised by the different teams. Loss of Work: The various errors and bugs result in loss of valuable efforts. Blame Game: For each error, bug, or resulting incident, each team tries to ensure they are not identified as the cause of the issue. Therefore, they tend to pass on the blame of delayed delivery or errors to each other, leading to further irritation, lack of understanding, and intensifying of the wall of confusion. Build Rollback: Many times the build rollbacks are required as a result of a variety of causes, such as incorrect client requirements, incorrect database in the QA or Production environment, incompatible tools and others. Disintegrated Processes: The traditional structure of the various processes of the two teams are generally based on different frameworks, such as ITIL, ASL, COBIT, and Scrum. Therefore, development processes do not integrate well with operations processes leading to disjointed processes and different vocabulary, again intensifying the wall of confusion. No Feedback Loop: In many IT organizations, there is in fact a feedback loop. However, it is generally negative and strongly affected by the blame game. Lack of a feedback loop that is both positive and continuous in nature between development and operational processes causes the lack of understanding to increase. There are many symptoms and causes that enhance the wall of confusion to a point that a truly concerted effort is required to bring it down. DevOps encompasses a series of ideas, concepts, and concrete actions that focus on removing these symptoms and causes.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
11
Course Book | DASA DevOps Fundamentals
DASA DevOps Fundamentals
A Brief History of DevOps
A Brief History of DevOps Food for Thought The problems due to the wall of confusion have been encountered by many IT Learn about the different organizations. DevOpsdue tookto birth wall of confusion between Development The problems thedue walltoofthe confusion have been encountered by and frameworks, such as ITILofand COBIT, Emergence Operations. DevOps and try to understand the relevance many IT organizations. DevOps took birth due to the wall of confusion between Development and Operations. of processesCore in these frameworks. Principles of DevOps
DevOps Agile Skills Association
Dev
Ops
Patrick Debois, 2007
According to Damon Edwards, co-founder of DTO Solutions, the DevOps movement was germinated in Belgium back in 2007. Patrick Debois, an IT consultant, was frustrated by struggle, lack of communication, and disconnection between Development and Operations departments. He found himself straddling between the two teams, while working on a huge data center migration project for the Belgium Government Ministry. Patrick was performing the dual role of firefighting in the unpredictable world of IT operations and working on the Agile development. He was confident that there was a better way of working which would allow bridging the substantial gap between the two teams. At the Agile 2008 Conference in Toronto, Canada, Andrew Shafer (a partner at Reductive Labs) proposed a discussion topic for an ad hoc session entitled ‘Agile Infrastructure’. However, the session got cancelled due to lack of interest and feedback. Patrick Debois was the only to follow the discussion and eventually tracked down Andrew at the conference, and then they had an in-depth discussion about their mutual frustrations. The discussion gave rise to the Agile System Administration group on Google Groups by Andrew Shafer and Patrick Debois. Although the group was not overly popular, but it leads to some fascinating discussion. Moving forward to Velocity Conference, San Jose, the famous talk was given by John Allspaw and Paul Hammond of Flickr in 2009. DevOpsDays was born, and the first event was conducted from October 30th to 31st , 2009, Ghent, Belgium. The event attracted administrators, developers, and managers from all around the world. The event’s success inspired other DevOpsDays events in different countries. These events acted like a catalyst for the conversation and a grassroots movement. Although vendors, analysts, and traditional enterprise IT shops mostly ignored the movement, but it ignited the passions of those who were concerned. As a result, the movement began to flow and spawned tools, such as Vagrant, Puppet, Chef,
12 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 | 8
Module 2 | DevOps Introduction
and FPM. The movement also started running circles around legacy enterprise IT systems. Reference Reading: https://blog.newrelic.com/2014/05/16/devops-name/
ps
Benefits of DevOps Fast movers displace traditional companies in all industry domains. To
aditional companies in companies all industryneed domains. To survive, survive, to radically rethinkcompanies their IT strategy. Where their IT strategy. Where will your company be in five years? will your company be in five years? Deliver Faster
elopment
ery well delivery!
Improve Continuity/Stability
omation, ment, and models.
Room for Innovation
erentiator.
Reduce Cost
What is commonality?
Copyright © 2018 | 9
IT is the strategic differentiator. Fast movers use Automation, Continuous Improvement, and simplified operating models. Operations and Development are in sync. Software lends itself very well for fast and dynamic delivery! Many organizations have started to tear down the walls between business and IT, and the even thicker walls between technical departments within IT. They have replaced their technical departments with organizational forms that ensure quick feedback loops and short iterations. The leaders of such organizations are now starting to realize that IT is a strategic differentiator. Market conditions demand higher responsiveness and moving slow is not an option as the competition is eager to grow their market share. CEOs read almost daily in the media, the stories about organizations that have been dramatically transforming themselves by adopting an engineering culture and moving towards a new world of IT. The success of extremely fast concept-to-cash or low time-to-market, and much lower operating and capital expenditures are enticing stories. Another Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
13
Course Book | DASA DevOps Fundamentals
type of story, which offers equally interesting lessons learned, focuses on organizations that have either gone bankrupt or lost huge parts of their market share as they have been replaced by a startup or an App. In all industries, this digital disruption is happening. The examples, such as AirBNB, Spotify, Uber, WhatsApp, and Apple Pay, give a good impression of how industries are being shaken up or becoming redundant because of fast movers, who are able to develop and release new or enhanced services to customers on a frequent basis.
Tips Marc Andreesen said: “Software is eating the world”. The pressure to deliver faster, better and cheaper software products has considerably increased the focus on DevOps principles.
The common factor among all these fast movers is that they have triggered many of the traditional organizations to look at the world differently. Embracing a digital transformation is now key to survival and the four key goals listed on the preceding figure are now placed high on the agenda of top management. As Marc Andreesen said: “Software is eating the world”. Therefore, making the right software in the right way matters a lot. The pressure to deliver faster, better and cheaper software products has also increased the focus on DevOps principles. As a result, considering the community-driven vision on collaboration and culture, DevOps has become a necessity for staying in business.
Cycle Time Reduction Jack Welch, former CEO of General Electric and the writer of the book ‘Winning’, touches upon the essence why companies need to reduce cycle time reduction. “If the rate of change on the outside exceeds the change on the inside, the end is near.” Jack Welch
Antifragility In order to stay in business, digital enterprises need to be antifragile organizations. Antifragility is the ability of systems (or organizations) to get better as a result of shock, disruptions or disorder. Antifragility is about being the opposite of fragile. It means thriving on stressors. Source: Design to Disrupt Whitepaper, Mastering Digital Disruption with DevOps, Sogeti VINT, March 2016
Antifragile systems love randomness and uncertainty, going beyond resilience or robustness. Such systems get stronger with stress and volatility. Startups tend to be antifragile. However, large, successful organizations tend to be fragile.
14 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Ops
ls
Module 2 | DevOps Introduction
IT organizations need to take on more antifragile characteristics as the world of IT is changing rapidly. Businesses need their IT organizations to be responsive and need their IT architectures to be adaptable to shocks or disruptions. The whole concept of DevOps is aimed at creating this kind of environment; creating organizational units that can autonomously act on the software and hardware required to deliver the IT service. DevOps requires short delivery times to ensure disruptions (in the form of different choices resulting from market changes) can readily be accommodated and used to improve the IT service. In other words, creating highly automated environments that Building Antifragile Using DevOps can be easily reprogrammedOrganizations if necessary.
Building Antifragile Organizations Using DevOps
Food for Thought Core ingredients of antifragile organizations are Management Innovation, Lean Startup, Core ingredients of antifragile organizations are Management Are “antifragile organizations” sameand as DevOps. Innovation, Lean Startup, and DevOps. “robust organizations”? Think about it!
gence of DevOps
ciples of DevOps
ile Skills sociation
Human component back into focus
Stop wasting people’s time
Source: to Disrupt Whitepaper, Mastering Digital Disruption with Source: Design toDesign Disrupt Whitepaper, Mastering Digital Disruption with DevOps, Sogeti VINT, March 2016 Sogeti VINT, March 2016
Accelerate innovation
DevOps, Copyright © 2018 | 12
DevOps is one of the central pillars on which many of the new breed of IT organizations realize a new modus operandi for delivering IT services. Using DevOps across the entire organization, sometimes dubbed “enterprise DevOps” or “BusDevOps”, organizations redesign their business and IT departments using a new operating model that replaces traditional demand-supply models, centralized IT operations, and complex value streams with an excess of handovers, waste, and error-prone manual activities. High-performing IT organizations have significant advantages over their competitors as a 2014 State of DevOps Survey by PuppetLabs indicated: high performing IT organizations deploy 30 times more frequently, have 200 times shorter lead times, have 60 times fewer failures, and recover 16 times faster! Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
15
Ops
s
Course Book | DASA DevOps Fundamentals
Management Innovation, Lean Startup, and DevOps are three manifestations of the same phenomenon, a different way of working that is in line with modern practices. There are, however, minor differences. DevOps is the most holistic and more likely to take cultural aspects and the existing operation into consideration. Lean Startup tends to focus more on a method for product development. Both of these explicitly work on Management Innovation and put up a vigorous fight against bureaucracy, make teams and staff responsible, and urge the customer to get involved when it comes to digital innovations. This is how speed, staff engagement, and customer obsession come within reach of any organization.
Business Case: The High-performing IT Organization Business Case: The High-performing IT Organization
ence of DevOps
iples of DevOps
e Skills ociation
46X More Frequent Deployments
96X Faster Recovery From Failures
5X Lower Change Failure Rate
440X Shorter Lead Times
Source: State of Devops report 2017 Source: State of Devops report 2017
Tips The current State of DevOps report can be downloaded from https:// puppet.com/resources/whitepaper/ state-of-devops-report.
The preceding numbers are taken from the State of DevOps 2017 report. Over the past 5 years, they have surveyed more than 25,000 professionals worldwide to better understand how DevOps practices impact IT and organizational performance. There is a tangible difference between high-performing organizations and their low-performing counterpart. One of the reasons that enable high-performing organizations to perform better is their style of working according to a set of patterns. This set of patterns is known as DevOps.
16 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Cop
Module 2 | DevOps Introduction
s Case:Business Seven Reasons for DevOps Case: Seven Reasons for DevOps
C
1. Improved speed to market: Increased speed to market allow an organization to gain a competitive edge in an industry. However, software and tools get outmoded almost as quickly as these are released. Introducing a DevOps approach enable an organization to go from an initial concept to a viable product in a shorter timescale than was traditionally acceptable with a Waterfall approach. 2. Continuous Integration and delivery: Continuous Integration is a development practice that involves deploying code to a shared repository several times per day. Using an automated build process combined with automated testing helps verify each Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
17
Course Book | DASA DevOps Fundamentals
check-in, which produces more stable software. The updated 2015 State of DevOps report confirmed the finding of the 2014 report that a DevOps approach and culture allows organizations to deploy code more frequently, with shorter lead times. 3. Higher quality, fewer failures, and higher stability:
Functional and Nonfunctional Requirements Functional requirements are associated with the tasks a product/service should do. Nonfunctional requirements refer to the attributes of a product/ service, such as autonomy, resiliency, maintainability, testability, scalability, and reliability.
a) Higher quality: The systems not only entertains the functional requirements (what the system will do considering what the customer wants) but also meets the nonfunctional requirements (unsaid customer requirements but expected from the system; how the system will do a particular task?), such as robustness, reliability, maintainability, and security. b) Fewer failures: The 2014 State of DevOps report showed that high-performing organizations had 50 percent fewer failures. The 2015 State of DevOps report showed a continuation in trend by revealing that the organizations who adopt a DevOps mindset and culture have 60 times fewer failures than those not implementing a DevOps approach. c) Higher stability: DevOps allows a single team to handle both, new functionality and the stability of the system. Each team member takes ownership of the business goals. Deploying often and in smaller, indivisible groups allows engineers to troubleshoot and resolve issues faster. The combination of tools and best practices, along with automation allows a DevOps team to increase overall stability. 4. Innovation and creativity: Continuous Integration, standardized production environments, and automated deployments allow practitioners to focus on the more inventive and creative side of their role. The environment and culture of DevOps encourage a deeper understanding and implementation of best practices in an organization. 5. Increased employee engagement and job satisfaction: DevOps provides a collaborative and multi-skilled environment, which contributes heavily to job satisfaction. DevOps practices and culture increase employee satisfaction, which leads to better business outcomes. 6. Breaking down silos and eliminating waste; It is all about collaboration!: a) Breaking down silos and eliminating waste: Combining multiple teams and disciplines into a single DevOps team that has a cross-functional skill set and communicates efficiently helps removing the organizational silos. In addition, applying the Lean focus on removing waste ensures that DevOps allows teams to complete their tasks quickly and efficiently while maintaining stability and quality.
18 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 2 | DevOps Introduction
b) It is all about collaboration!: Every individual and every single team, whether it is Development, Operations, QA, or Support, all come across challenges on a daily basis. These challenges impact the business and its operations, and an incident or impediment in one department can resonate in another. Regardless of who is responsible, the problem needs to be solved. It is unnecessary placing blame and pointing fingers. The key is to use the available time and resources to solve the issue. Without collaboration, this process takes longer and can create further problems that might not be immediately apparent. Working together and communicating efficiently allow teams to implement solutions that can help prevent similar incidents in the future. In complex enterprises with many fast-paced IT innovations, investing in collaboration helps to leverage all knowledge and skills available. It helps to work faster and smarter than ever before. Moreover, it raises awareness of the challenges that each department faces on a day-to-day basis and helps to thoroughly understand the business needs. 7. Resource and cost reduction: By implementing a DevOps approach, an organization is able to significantly reduce the costs and demand for resources associated with traditional IT implementations. When the organizations use continuous delivery and Lean Management practices, higher quality results and shorter cycle times are achieved, which further reduce costs. Other factors that help to reduce cost and resource requirements include minimal project startup and ongoing operational costs, increased collaboration, increased data availability and accessibility, and improved security. Note: Cost reduction is not measured as part of the State of DevOps Survey. It is not a goal on its own and seen as a byproduct. RESULT: Increased Performance Standardized production environments and automation tools help make deployments predictable. These processes free people from routine tasks, allowing them to concentrate on the more creative aspects of their role. Consequently, it leads to increased performance of the people.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
19
Course Book | DASA DevOps Fundamentals
Research Confirms theResearch Benefits of DevOps Confirms the Benefits of DevOps The following Prediction model is based on the State of DevOps report It shows the State relationship betweenreport continuous culture, The following Prediction model 2016. is based on the of DevOps 2016.delivery, It shows the and IT performance. relationship between continuous delivery, culture, and IT performance.
Source: State of DevOps Report 2016
Source: State of DevOps Report 2016
Copyright © 2018 | 15
This model is a result of research done on the data from the state of DevOps reports. The model highlights the interesting relationships between continuous delivery and burnout (engineers getting frustrated with the development and the release processes). The left part of the model contributes to continuous delivery and positively or negatively impacts the right part. R2 is the percentage that the influencer impacts / has influence on the variance of the topic. Continuous delivery explains 5.9% of the variance in the change failure rate. As the research has shown, based on data gathered from a large amount of organizations, continuous delivery and organizational culture have the most effect on IT performance! Reference Reading: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2681909
20 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 2 | DevOps Introduction
Core Principles of DevOps Activity: Group Discussion Activity Time: 10 mins What do you think are the core principles when “going DevOps”?
Some DevOps Definitions “DevOps isn’t a thing. It’s not a product, standard, specification, framework or job title. DevOps is about experiences, ideas and culture. It’s about the close communication and collaboration between IT operations and development, and how they can improve the products and services that they produce by thinking differently about how they work together, using a new mentality.”
Food for Thought Find out some other definitions of DevOps and ponder over the similarities and difference between these definitions.
Gareth Daine, Devops Evangelist
“Fundamentally, DevOps is the activity of optimizing the development-to-operations value stream by creating an increasingly smooth, fast flow of application changes from development into operations, with little waste. Optimization of the value stream takes place continuously using various continuous improvement techniques like the Toyota Kata.” Dave Roberts, Executive Advisor at BMC Software
Hundreds of definitions for DevOps exist. Some essential elements of the preceding definitions are highlighted. These elements stand for something larger than a ‘thing’, such as a product, standard, or framework. This makes DevOps intangible but also applicable to enterprise-wide IT improvement and continuous innovation of the IT capability. Considering the preceding three definitions, the four key points are: DevOps is about communication, helping tightly integrated disciplines to understand each other’s roles and challenges in a better manner. DevOps is about working together toward a common goal. It enables businesses to succeed by helping them release quality and stable products and services, faster, and with fewer failures and bottlenecks. There is no overall authority on DevOps. It is a movement, inspired by practitioners for practitioners.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
21
Course Book | DASA DevOps Fundamentals
DASA DevOps Fundamentals
Emergence of DevOps Core Principles of DevOps
No one owns DevOps. It is a culture about improving how people work together.
DevOps is the Culture of High-Performance IT DevOps is the Culture of High-Performance IT
DevOps is a CULTURAL and OPERATIONAL model that fosters COLLABORATION to ENABLE high-performance IT to ACHIEVE business goals.
DevOps Agile Skills Association
Copyright © 2018 | 19
DASA DevOps Fundamentals
Emergence of DevOps Core Principles of DevOps
To survive, organizations need to adopt DevOps practices to create a culture of high-performance IT. The cultural and operational model underlying the core vision and principles of DevOps is the key to arriving at this high-performance level. The focus on culture means people, like you, need to help investigate what works and what does not. Therefore, people need to bring the right mentality and drive to continuously improve the state of the practice, together with colleagues and peers. Unfortunately, there is no silver bullet for DevOps, but DevOpsthe is Tightly Intertwined with and Leanteams IT and IT core principles of DevOps canAgile help individuals, organizations get started in the right direction. The underlying principles of Agile and Lean ITwith are at the core of DevOps DevOps is Tightly Intertwined Agile and Lean IT and the three are tightly intertwined. Therefore, DevOps practitioners often choose to use many of the same underlying of Agile and Lean IT are at the core of techniques The as found in Agileprinciples and Lean IT. DevOps and the three are tightly intertwined. Therefore, DevOps practitioners often choose to use many of the same techniques as found in Agile and Lean IT.
DevOps Agile Skills Association
Agile Methodologies
22 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Lean IT
Copyright © 2018 | 20
Module 2 | DevOps Introduction
DevOps is an idea that covers all facets of working in an IT organization. It builds on a different paradigm than the traditional way of organizing and running IT. Such a paradigm is already present in many IT organizations in the form of Lean IT and Agile. Lean IT and Agile both originated from the application of Lean principles to the whole of an IT organization (in the case of Lean IT) or software development (in the case of Agile). DevOps makes good use of Lean IT and Agile practices and methods that have already proven to be successful and takes the delivery of IT services to the next level. Before getting into DevOps in more detail throughout the next modules, understanding its two underlying principles, Lean IT and Agile, is essential. Reference Reading:
evOps
Agile: Satisfy the Customer http://theagileadmin.com/what-is-devops/
ntals
Agile: Satisfy the Customer Agile movementprovides provides alternative to traditional development. Agile movement alternative to traditional projectproject development.
mergence of DevOps
Plan Driven (Traditional)
Resources
Functionality
rinciples of DevOps
Fixed Quality
Agile Skills Association
Value Driven (Agile)
Traditional Development Start with a complete design
Resources
Time
Quality Estimated
Time
‘Activity-Focused’ (siloed): Traditional
Building is followed by testing the final product
Functionality ‘Product-Focused’ (team): Agile
Finally, testing in practice No feedback loops Plan driven
Specialty Oriented
Work Oriented
Functionally Organized
Team Organized
Project Focused
Product Focused
Work with Individuals
Work with Teams
Agile Development Every Sprint delivers working software that can be used in practice Starts with delivering basic functionality to which features are added Value driven
Features of the product-focused approach:
Responsibility of the Product team extended all the way into production. All are responsible and accountable for a fully working product.
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Copyright © 2018 | 21
Agile breaks the development of new functionalities into smaller functional units according to user stories. The units are continuously prioritized to deliver the software in short cycles known as iterations. DevOps in many ways fits uniquely well with Agile. The two unique approaches are almost synonymous except for prevue in scope. The convergence of development methodologies and delivery solutions has paved the way for numerous companies to reap huge success and improvements. Reference Reading: “The Art of Agile Development” by James Shore Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
23
Course Book | DASA DevOps Fundamentals Lean: Customer Value at the Center
Lean: Customer Value at the Center
Lean focuses on creating the value for customer.
Lean focuses on creating the value for customer.
Assess if all the activities in the process add value in the eyes of the customer Focus on first time right and quality prevention of defects
Demand triggers the process chain in order to reduce stock
Create a continuous flow in production with the Just-in-Time approach and reduce peak and low volumes
Lean is delivering value to customers and continuously improving the ability to do this, by removing waste from the entire system that produces the value.
Tips Value Stream mapping is a method for analyzing the current state and designing a future state for the series of events that take a product/ service from its beginning through to the customer. At Toyota, it is known as “material and information flow mapping”. Food for Thought Learn about the reasons that led to the need of Lean adoption at Toyota.
Lean principles are the basis of DevOps. Lean IT and Agile, the principles behind DevOps, clearly show the roots of Lean that provide an excellent framework for organizations to start with improvements. For example, Lean provides tools to visualize the DevOps value stream and measure it. Such measurements help improve the delivery pipeline by eliminating bottlenecks, and making it more efficient and productive. Organizations can apply Lean principles in many contexts, tools, and methods with multiple sources. However, many of the iconic elements of Lean come from Toyota Production System. Starting in the 1930’s and intensifying after World War II, Toyota realized a series of innovations could provide continuity in the process flow and a wide variety of product offerings. It was crucial for Toyota to catch up with the rest of the world, particularly the car manufacturers in America. Considering the scarcity of various resources, Toyota focused on minimizing the amount of raw materials required to produce cars and the time between purchasing raw materials and sending an invoice to the customer. Such ideas became known as the Toyota Production System. You will get to know about some of the elements of Lean later in this course.
24 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 | 22
Module 2 | DevOps Introduction
ps Core DevOps Principles Core Principles Customer-Centric Action
Create with the End in Mind End-to-End Responsibility Cross-Functional Autonomous Teams Continuous Improvement Automate Everything You Can
Copyright © 2018 | 23
Many definitions of DevOps exist. Many of these adequately explain one or more aspects important for finding flow in the delivery of IT services. Instead of trying to create a comprehensive definition on your own, refer to the following six principles when adopting or migrating to a DevOps way of working: 1. Customer-Centric Action: It is imperative nowadays to have short feedback loops with real customers and end-users. Therefore, all activities involved in building IT products and services should revolve around customers. 2. Create with the End in Mind: The principle focuses on understanding the real needs of customers and working towards creating products and services that solve the problems. In other words, the principle considers taking a holistic view of both the creation and use of the IT product/service. 3. End-to-End Responsibility: In a DevOps organization, teams are vertically organized so that they can be fully accountable for the products and services they deliver. End-to-end responsibility means that the team holds itself accountable for the quality and quantity of services it provides to its customers. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
25
Course Book | DASA DevOps Fundamentals
4. Cross-Functional Autonomous Teams: In product organizations with vertical, fully responsible teams, the teams need to be fully autonomous throughout the entire lifecycle of the product. Therefore, the teams should possess all the necessary expertise to take on the end-to-end responsibility. 5. Continuous Improvement: In a DevOps culture, a strong focus is put on continuous improvement to enhance the products/ services offered to customers. Some of the improvement activities include minimizing waste and optimizing speed, costs, and ease of delivery. 6. Automate Everything You Can: Automation is synonymous with the drive to renew the way through which the team delivers its services. Extensive automation means having a deep understanding of the processes needed to develop and deliver the IT services. Reference Reading: White Paper: Embracing Digital Disruption by Adopting Devops Practices (https://www.devopsagileskills.org/resources/document/white-paperembracing-digital-disruption-by-adopting-devops-practices/)
DevOps Principle #1: Customer-Centric Action
Tips To meet customers’ requirements, DevOps organizations need to act as Lean startups that: Innovate continuously. Adjust when a certain strategy is not working. Constantly invest in products and services that receive a maximum level of customer delight. Rapidly respond to changing or emerging customer needs.
DevOps encourages an open culture that has the following characteristics: No bureaucracy No fear of asking questions Risk taking
Courage Innovate
Innovating In an open culture, teams are open to feedback. There is no hesitation or barrier to ask questions and calculated risk taking is encouraged. It is imperative nowadays to have short feedback loops with real customers and end-users. Therefore, all activities involved in building IT products and services should revolve around customers. To meet customers’ requirements, DevOps organizations need to have courage to act as Lean startups that: Innovate continuously. Adjust when a certain strategy is not working. Constantly invest in products and services that receive a maximum level of customer delight. Rapidly respond to changing or emerging customer needs. The focus on customer-centric action is not always easy. Customers are notorious for not knowing exactly what they want. In a DevOps environment, teams and individual have both the courage and the
26 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 2 | DevOps Introduction
authority to act to fulfill customers’ needs. Therefore, the team can perform new tasks that they have never done before in the cause of creating the product/service that best fulfills the needs of the customer.
DevOps Principle #2: Create with the End in Mind If you do not know where you are going, you will not know whether you lost the way. Create with the end in mind brings focus on the end results. This will foster the product, service thinking, and collaboration, which is one of the key ingredients to DevOps. However, it requires an engineering mindset and mutual trust among different teams and team members. DevOps teams need to act as “product companies” that explicitly focus on building working products sold to real customers. All employees need to share the engineering mindset required to envision and realize those products. DevOps IT organizations tend to avoid Waterfall and process-oriented models. Such models encourage units or individuals to work on a particular area rather than oversee the complete picture.
Product and Service Thinking Engineering Mindset
Collaborate
Teams that adopt Lean thinking focus on creating flow and become experts in problem solving (the Lean Kaizen mindset). They always strive towards delivering a Minimal Viable Product (MVP), the smallest amount of functionality having value to the customer, as fast and reliably as it can. In addition, they focus on maximizing flow by minimizing unnecessary hand-offs. The reason being more time gets lost during handoffs than the actual time required to perform the desired work. These teams deliver value in an Agile way by delivering single changes or in small batches to customers so that potential issues can be identified immediately. The key is to get feedback quickly about what their customers find most valuable. Transparency regarding progress helps everybody within and around the team to see where they are at any point of time and also makes potential risks more visible.
DevOps Principle #3: End-to-End Responsibility Caring about the end-to-end responsibility might be the most crucial ingredient for DevOps. When people care and have the required skills, knowledge, and resources, they can and will collaborate to live up to their responsibility. If they care, they will learn, adapt, improve, and provide great services and value. Traditional organizations develop IT solutions that are handed over to operations to deploy and maintain. However, in a DevOps organization, teams are vertically organized so that they can be fully accountable for their services. Therefore, all employees need to work in teams. The teams are kept stable to ensure effective working habits can be embedded. These teams provide performance support to products until these reach end-of-life. Such an end-to-end support enhances the level of responsibility and the quality of the engineered products.
Live Your Accountability Concept to Grave Performance Support
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
27
Course Book | DASA DevOps Fundamentals
T-Shaped Profiles Complementary Skills “There’s no such thing as a DevOps Hero”
Tips Think about how an organization can flourish by establishing crossfunctional teams in contrast to competence-based teams or a culture of heroes.
DevOps Principle #4: Cross-Functional Autonomous Teams Cross-functional teams consist of representatives from all disciplines responsible for developing and deploying an IT service. These teams are fully empowered and self-sufficient to design, build, test, deploy, and run the software. To be able to do this, a team needs team members with T-shaped profile and complementary skills. The product organizations with vertical, fully responsible teams need to be fully autonomous throughout the lifecycle of a product. For teams to be autonomous, a balanced set of skills and team members with teams having in-depth knowledge and collaboration skills are required. These teams become a hotbed of personal development and growth. Patrick Debois, the godfather of the DevOps movement, always says DevOps is a human problem. Doing DevOps requires a culture that brings development and operations people together so that they can understand each other’s perspectives and concerns. The aim is to enable both the teams to build and deliver resilient IT services that are production ready, in a timely manner. DevOps is synergistic. It requires people to collaborate effectively by: Ensuring they have overlapping skills and knowledge, combined with complementary specialist skills and knowledge (T-shaped profiles). Giving feedback to each other. Avoiding blame evaluations (no blame game or finger pointing). Trusting each other (Having a high-trust culture has a strong impact on both IT performance and organizational performance.).
If it Hurts, Do it more Often Fail Fast Experiment “What cannot be measured, cannot be improved”
Measurement Through Metrics DevOps seeks measurement of processes, people, and tools through following metrics: Performance Metrics Process Metrics Innovation Metrics Culture Metrics Support Metrics
28 │ Copyright © 2018
DevOps Principle #5: Continuous Improvement Continuous improvement is an approach to identify opportunities to streamline work and reduce waste. Continuous improvement is a dominant concept borrowed from the Lean movement that focuses on adaptability and learning through structured problem-solving. It requires teams to focus on experimentation, minimizing waste, optimizing for speed, cost, and ease of delivery to continuously innovate and help their organization beat the competition. It is, therefore, important to experiment and learn from failures (try to fail as fast as possible) To continually improve a product/service, it needs to be measured. A DevOps implementation should be designed to measure everything, such as processes, people, and tools.
DevOps Principle #6: Automate Everything You Can Regardless of the technology platform or development practices, every organization uses a process to build new software and IT services. This process can be manual or automated. Moving away from manual efforts to automation, derives efficiency and consistency in the process. Automation helps to:
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
of ps
of ps
ls on
Module 2 | DevOps Introduction
There are several ways that can help increase the speed, reduce the cost, and enhance the quality of IT, such as:
Food for Thought Think about how automation leads to enhance quality and maximize the flow of value to customers.
Continuous improvement (waste reduction) is greatly helped by automation. There have been various IT innovations and trends that help achieve this goal. Continuous delivery focuses on bringing software into production through a fully automated process, multiple times per day without problems. Cloud-native platforms replace traditional data centers. Container-based infrastructure help treating Infrastructure as Code (IaC). DevOps Principles and Aspects of IT
DevOps Principles and Aspects of IT The DevOps Core Principles Define All Aspects of IT IT Aspects
Principles Customer-Centric Action
Culture
Create with the End in Mind Organization
End-to-End Responsibility
Defines
Cross-Functional Autonomous Teams
Processes
Continuous Improvement Automation
Automate Everything You Can
Measurements and Improvements
The core DevOps principles impact the various aspects of IT, such as culture, organization, processes, and automation. You can use the DevOps principles to select, implement, and evaluate best practices for IT (culture, organization, processes, and automation) optimization. In addition, you can perform measurements and improvements to implement a continuous feedback or continuous improvement loop. For example, “Automate Everything You Can” need a culture and mindset that promotes automation and optimization of the organization, processes, and technology used.
Copyright © 2018 | 30
Activity: Group Discussion Activity Time: 15 mins One of the principles of DevOps is to automate everything you can. How much of the delivery of IT services through the entire lifecycle is automated in your organization? Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
29
Course Book | DASA DevOps Fundamentals
Key Roles in a DevOps Team DevOps Agile Skills Association (DASA) Key Roles in a DevOps Team
Scrum Master/Team Manager Business Representative/ Product Owner
User Experience
DevOps Operations Engineer
Technical Architect
Tester
Developer
One of the most significant aspects of the paradigm change from traditional IT to DevOps-based IT is the effect on the roles within IT. In the traditional IT organization, becoming a specialist was strongly encouraged. This had benefits as there was always someone who had incredible in-depth knowledge of a particular area, system, or tool. The downside was that when this person was on holiday much of the knowledge was gone. Moreover, when the area of expertise became obsolete or redundant, the person went the same way. As DevOps teams develop their product and associated services, the role of the IT Engineer become more generic. The key to working in a DevOps environment is to recognize a skills and knowledge set that needs to be present in every DevOps team. The distribution of these skills and knowledge vary from team to team. Each team needs to ensure they have the required skills and knowledge to deliver the service required by the customers, autonomously. Over the past couple of years, DevOps teams have been seen in various phases of development. Many of DevOps practitioners confirmed that there are specific skills and knowledge that can be discerned.
30 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 2 | DevOps Introduction
Does this mean that specialists are no longer required? No. What is necessary is that each specialist adds a number of knowledge areas to their capabilities to ensure the continuity of support and development of the product/service when someone more specialized in the task is unavailable.
A DevOps Competence Model DASA DevOps Competence Model
mpetence model is the core body of knowledge for DASA. They provide 8 knowledge The competence model in is the body of figure. knowledge for DASA. nd 4 skill areas, as mentioned thecore following They provide 8 knowledge areas and 4 skill areas, as mentioned in the following figure.
Knowledge Areas
Skills Areas
Business Value Optimization
Courage
Business Analysis
Teambuilding
Architecture and Design
DevOps Leadership
Programming
Continuous Improvement
Continuous Delivery Test Specification Infrastructure Engineering Security, Risk, and Compliance
All 12 capabilities (or areas) need to be mastered at the Expert level by the team in order to perform well. Not everyone needs to operate at the Expert level, some people are better at specific capabilities than others, but as a team they need to master the Expert level.
Copyright © 2018 | 34
DASA DevOps Competence Model: Skill Areas The skill areas are the fabric of the DevOps culture. The following table lists the various qualities required to be proficient in each skill area. Courage
Evangelism, Coaching, Self-confidence, Proactivity, Reflection, Trust, Open discussions, Experimentation, Fail fast, Courage to change
Teambuilding
Understand the other’s point of view, Collaboration, Mutual accountability, Common purpose, Ability to integrally support the service/product
DevOps Leadership
Facilitating teams to high performance, Humility, Service lifecycle mindset, Stakeholder management
Continuous Improvement
Today we do our work better than yesterday, Kaizen mindset, Quality at the source, First time right, Knowledge-sharing, Adaptiveness Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
31
Course Book | DASA DevOps Fundamentals
Courage covers more than just having the confidence to act. It is also about trusting team members, being open, and being prepared to speak out. Teambuilding focuses on ensuring the team bonds to a set of people that interact effectively in a common cause. In the course of doing so, they hold each other accountable and support each other. DevOps Leadership is not reserved for management or other formal leadership positions. It is fundamentally about facilitating teams to reach a high performance through formal and informal leaders. Continuous improvement is a built-in desire to do better today than the team did yesterday. At the same time, it includes recognizing successes and failures, celebrating both for what they are or opportunities to learn. This skill is also the basis for creating an autonomous team with all the knowledge necessary to serve the customer.
DASA DevOps Competence Model: Knowledge Areas The eight knowledge areas represent the entire knowledge set, a DevOps team needs to be effective for its customers. The following table lists the various qualities required to be proficient in each knowledge area. Business Value Optimization
Use of the IT service in real life, including direct feedback loop of user comments to team, Service level management, Definition of done, Business activity/performance monitoring, Business case management
Business Analysis
Functional requirements, Non-functional requirements, Longer term development of business process (based on translation of market developments), Data analysis, Refinement
Architecture and Design
Ensuring fit between developments and current situation, Overall service design, Patterns and styles
Programming
Software engineering mastery, Everything as code, Data management
Continuous Delivery
Automated testing, Deployment and release management, Configuration management, Version control, Cloud, Containerization, Feature-driven delivery
Test Specification
Design of test cases, Test concepts
Infrastructure Engineering
Technical monitoring, Performance management (for example, Load balancing), Capacity and availability management, Reliability engineering, Cloud, Containerization
Security, Risk, and Compliance
Security, Service continuity planning Business Value Optimization is to look at the use of the IT service in real life. The DevOps team should know about the usage of their IT service in practice and its impact on the further
32 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 2 | DevOps Introduction
development of the service. It is vital to understand the business process the IT service supports to be able to build a business case for improvements. Business Analysis is to analyze data and processes to determine what improvements have the maximum effect on the business. It is about detailing the requirements but also doing Value Stream Mapping. It also includes the ability to refine the backlog of desired improvements. Architecture and Design encompasses, principally, the overall design of the IT service including creating a view of the direction in which the service is developing. In other words, it involves defining architectural scenarios. Programming is the core knowledge area for all DevOps team members. Programming is essential to develop applications. However, with the creation of infrastructure as code, programming has found its way into the infrastructure domain. In both the cases, application and infrastructure, it is crucial for people managing the Production environment to have strong programming knowledge. Continuous Delivery covers the central DevOps process of getting the newly developed code from the development environment into production using a solid set of tests with reliable version control. Test Specification is closely related to Business Value Optimization and Business Analysis. Developing a set of tests is a good way to specify the functionality to be developed. Test Driven Development (TDD) is an example of a method that promotes this way of developing applications, i.e. describe the required business value in terms of the tests that need to be passed and then develop the code. Infrastructure Engineering includes knowledge of load balancing and other such performance management aspects. It also means building effective monitoring solutions that work as predictors for incidents or problems. Capacity and availability management are part of this knowledge area. Security, Risk, and Compliance is vital for all IT products and services. It is essential to integrate this knowledge into the DevOps team rather than relying on outside knowledge. This knowledge area plays a vital role in building Quality at the Source. Note: The eight knowledge areas are highlighted throughout modules 4 to 7.
DASA DevOps Competence Quickscan™ DASA DevOps Competence Quickscan™ will help you assess your DevOps competences. Would you like to assess your readiness to engage in a DevOps team before continuing with the course? Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
33
Course Book | DASA DevOps Fundamentals
We strongly recommend you to please devopsagileskills.org/ and complete the scan.
DevOps
mentals
visit
https://scan.
DASA DevOps Certification Scheme
re Principles of DevOps
DASA DevOps Product Owner
DASA DevOps Leader
DASA DevOps Coach
DASA DevOps Professional Enable and Scale
DASA DevOps Professional Specify and Verify
DASA DevOps Professional Create and Deliver
FOUNDATIONAL Know
Ops Agile Skills Association
LEADERSHIP Lead and Enable
Emergence of DevOps
PROFESSIONAL Know and Apply
DASA DevOps Certification Scheme
DASA DevOps Fundamentals
34 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 | 38
Module 2 | DevOps Introduction
DASA offers certifications to all key profiles and comes up with the certification program, as shown in the preceding figure. It includes the following three levels, ensuring the right certification for the right audience: The Foundational level focuses on ‘knowing’ and helps individuals to build an understanding of DevOps, its principles, DASA’s approach to DevOps, and puts DevOps into a business perspective. The Professional level certification level provides capability oriented certifications. The certification help professionals learn the key traits of their job and how to apply DevOps in real life. This level includes three certification programs, one for each of the professional profiles that we identify in a team. The Leadership level focuses on the ability to lead and enable. This program does not focus on building capabilities. It also does not explain the tools that are there in the toolbox for the professional. It focuses on how the professionals can best operate in his/her role, such as managing the process, removing barriers for people, and leading the team.
DASA DevOps Fundamentals Certification Teambuilding 5
DevOps Leadership
Courage 4 3
Architecture and Design
2
2
Continuous Improvement
2 1
2
2
DASA DevOps Fundamentals
2
Business Value Optimization
2
Infrastructure Engineering
2 2 2 2
2
Security, Risk, Compliance
Business Analysis
Continuous Delivery
Test Specification Programming
1. Novice / 2. Competent / 3. Proficient / 4. Expert / 5. Master
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
35
Course Book | DASA DevOps Fundamentals
The DASA DevOps Fundamentals certification provides an extensive introduction to the core agile DevOps principles covering the essential knowledge and skill competences that have been defined by DASA. It is the starting point for an organization going on a DevOps journey. Improved workflows and faster deployment starts with a core understanding of DevOps fundamental concepts by anyone involved in an agile and/or DevOps team. DASA DevOps Fundamentals brings all Knowledge and Skill Areas to the Competent level. Having said this, the Fundamentals course does not provide, for example, practical Business Analysis or Programming experience. Such practical knowledge needs to be acquired through specific training.
DASA DevOps Professional Certifications DASA DevOps Professional: Enable and Scale Teambuilding 5
DevOps Leadership
Courage 4 3
3
3 2
Architecture and Design
1
2
3
DASA DevOps Professional Enable and Scale
2
Business Value Optimization
Continuous Improvement
2
Infrastructure Engineering
2 2 2 2
2
Security, Risk, Compliance
Business Analysis
Continuous Delivery
Test Specification Programming
1. Novice / 2. Competent / 3. Proficient / 4. Expert / 5. Master
DASA DevOps Professional: Enable and Scale brings the Skill Areas to the Proficient level. Enable and Scale is about ensuring that a
36 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 2 | DevOps Introduction
DevOps team can be successful in its environment. The Enable and Scale certification: Validates the candidate has a practical understanding and experience in leading DevOps teams. Enables members to become effective ensuring the team works optimally. Covers the effective coexistence and cooperation of multiple DevOps teams. In other words, it answers the question “How do you turn a traditional IT organization into a DevOps organization?” DASA DevOps Professional: Specify and Verify Teambuilding 5
DevOps Leadership
Courage 4 3
Architecture and Design
2
2
Continuous Improvement
2
3
1 2
DASA DevOps Professional Specify and Verify
3
Business Value Optimization
2
Infrastructure Engineering
2
3 2 3
Business Analysis
2
Security, Risk, Compliance
Continuous Delivery
Test Specification Programming
1. Novice / 2. Competent / 3. Proficient / 4. Expert / 5. Master
DASA DevOps Professional: Specify and Verify focuses on the following competencies: Architecture and Design Business Value Optimization Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
37
Course Book | DASA DevOps Fundamentals
Business Analysis Test Specification According to estimates, approximately a third of the team is involved in these areas of expertise. The focus of these areas is to ensure customers’ requirements are understood and translated into a team. In other words, these knowledge areas confirm that the requirements can be integrated into an IT service. The ultimate responsibility of the role, Specify and Verify, is to ensure the design of the service is future proof, both technologically and functionally. In addition, the role confirms that the ability to test any new functionality is optimally facilitated by Test Specification that takes both the customer usage of the system and the need for speed into account. DASA DevOps Professional: Create and Deliver Teambuilding 5
DevOps Leadership
Courage 4 3
Architecture and Design
2
2
Continuous Improvement
2 1
2
2
DASA DevOps Professional Create and Deliver
2
Business Value Optimization
3
Infrastructure Engineering
2 3
2
Business Analysis
Security, Risk, Compliance
3 3
Continuous Delivery
Test Specification Programming
1. Novice / 2. Competent / 3. Proficient / 4. Expert / 5. Master
38 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 2 | DevOps Introduction
The majority of a DevOps team falls into the competency area of Create and Deliver. The core knowledge areas of this competency are: Programming Continuous Delivery Security, Risk, and Compliance Infrastructure Engineering This competency is really the heart of the team’s capabilities. The exact balance of skills, particularly Programming vs. Infrastructure Engineering, depends on the total area of responsibility of the team. DevOps is being used to deliver infrastructure services, which in turn deliver application services. A vital aspect that helps deliver a successful service is monitoring. In practice, full stack DevOps teams responsible for everything from user interface to server and network hardware are rarely seen. It is the technology stack that ultimately determines the balance of knowledge required in a team.
DASA DevOps Leadership Certifications DASA DevOps Leadership Certifications
DASA offers three distinct Leadership part of the DASA offers three distinct Leadership certificationscertifications as part of theas certification scheme. The certification The leveland does not cover the practical Leadership level doesscheme. not cover theLeadership practical tools capabilities covered in the tools and capabilities coveredon in leading the Professional Certifications, Professional Certifications, but focuses and enabling. It helps thebut individuals in focuses on leading andorganization enabling. It helps the individuals in these roles these roles navigate within their and drive the best decisions forward. navigate within their organization and drive the best decisions forward.
DASA DevOps DASA DevOps Product Owner Product Owner
DASA DevOps DASA DevOps Leader Leader
DASA DevOps DASA DevOps Coach Coach
Let us discuss how these certifications help you in gaining the required skills: In a DevOps environment, the Product Owner is a critical leadership role and responsible for managing the full lifecycle of a product from concept to the grave. This certification program helps the Product Owner realize maximum business value, engage with stakeholders, and deal with future requirements as well as operational challenges.
Copyright © 2018 | 43
The DevOps Leader is responsible for leading the DevOps initiative and creating the framework for teams to scale and achieve maximum business value. This certification program helps leaders understand leadership in the context of DevOps, discusses leadership development models, building teams, and transforming the organization. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
39
Course Book | DASA DevOps Fundamentals
The DevOps Coach helps team members and other stakeholders in the organization to apply DevOps concepts and principles within their organization. The coach oversees the transformation and guides the organization through their journey in building high-performing teams.
Activity: Group Discussion Activity Time: 2 mins What do you think would be your profile?
Module Summary In this module, you learned that: Emergence of DevOps The wall of confusion is a psychological and procedural barrier that obstructs the flow of communication between the Development and the Operations teams. Dissolving the wall of confusion requires tackling a variety of underlying problems, such as organizational silos, different mindsets, blame game, build rollback, disintegrated processes, and no feedback loops. Antifragility is the ability of systems (or organizations) to get better as a result of shock, disruptions or disorder. Management Innovation, Lean Startup, and DevOps are three manifestations of the same phenomenon, a different way of working that is in line with modern practices. There are seven apt reasons which buttress the need for building a business case for DevOps. Core Principles of DevOps DevOps is a cultural and operational model that fosters collaboration to enable high-performance IT to achieve business goals. The underlying principles of Agile and Lean IT are at the core of DevOps and the three are tightly intertwined. The six DevOps principles are Customer-Centric Action, Create with the End in Mind, End-to-End Responsibility, CrossFunctional Autonomous Teams, Continuous Improvement, and Automate Everything You Can.
40 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 2 | DevOps Introduction
DevOps Skills Agile Association (DASA) The key roles required in a DevOps team to make it autonomous are Scrum Master/Team Manager, Business Representative/ Product Owner, Technical Architect, Developer, Tester, Operations Engineer, and User Experience. The DASA DevOps Competence Model identifies eight knowledge areas and four skill areas that are required for DevOps professionals, teams, and organizations to be successful. DASA DevOps certification scheme recognizes maturity levels: Foundational, Professional, and Leadership.
Module End Questions Q1. Match each statement/example with the related DevOps principle. a) DevOps help us to rapidly respond to changing or emerging customer needs.
i End-to-End Responsibility
DevOps teams need to act as “product companies” that b) explicitly focus on building working products sold to real customers.
ii Cross-Functional Autonomous Teams
In a DevOps organization, teams are vertically organized so c) that they can be fully accountable for their services
iii Customer-Centric Action
This principle is a dominant concept borrowed from the d) Lean movement that focuses on adaptability and learning through structured problem-solving
iv Automate Everything You Can
These teams are fully empowered and self-sufficient to e) design, build, test, deploy, and run the software. To be able to do this, a team needs team members with T-shaped profile and complementary skills.
Create with the End in Mind v
The principle focuses on bringing software into production f) through a fully automated process, multiple times per day without problems.
vi Continuous Improvement
Q2. What is the primary goal of Agile? a) Creating the culture of high-performance IT b) Creating the culture of collaboration and automation c) Creating roadmaps and a structured way of working d) Satisfying the customer
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
41
Course Book | DASA DevOps Fundamentals
Q3. What are key reasons you should include when building a business case for DevOps? 1. Eliminate waste by breaking down silos. 2. Expand the product portfolio to customers. 3. Increase performance. 4. Spur innovation and creativity of your employees. a) 1, 2, and 3 b) 1, 2, and 4 c) 1, 3, and 4 d) 2, 3, and 4
42 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
3 Culture Module Objectives
Module Objectives
Essence of DevOps culture
Key elements of a DevOps culture
Important aspects to create a DevOps culture
Copyright © 2018 | 1
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
43
Course Book | DASA DevOps Fundamentals
At the end of this module, you will be able to: Describe the essence of a DevOps culture. Explain the key elements of a DevOps culture. Identify the important aspects to create a successful DevOps culture.
Module Topics Essence of a DevOps Culture Key Elements of DevOps Implementation of a DevOps Culture
Essence of a DevOps Culture Build Around Teams: Facilitated Lean Product ‘Companies’ Organized around products or services Self-organized (Always) Driven by the DevOps principles
Emerged as Lean startups Teams: Facilitated Lean Product ‘Companies’
Discharged when a product or service is no longer sustainable Facilitated by management or ‘traditional’ staff functions
nd products or Innovate
4. Milk it.
Revenue
by the DevOps
$$$
3. Scale it.
an startups
en a product or nger sustainable
anagement or functions
5. Kill it.
2. Nail it. 1. Pilot it.
Profit
Minimal Viable Product (MVP)
Introduction
Growth
Maturity
Decline
44 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 | 4
Module 3 | Culture
DevOps is a way to run a business, not just a way to organize a few departments. DevOps teams are responsible not only for feature development but also for testing, securing, and deploying code into production, and production operations and availability. You can organize your business around DevOps products/services, which are facilitated by management and ‘traditional’ staff functions.
The Boston Consulting Group (BCG) Matrix The Boston Consulting Group (BCG) Matrix
H2
H3
H1
Pets
Tips Innovation and ability to innovate quickly is a sustainable strategy. On the other hand, reducing the cost to increase profit is a shortterm strategy that results in lower adaptability. It makes the organization more vulnerable to (negative) market forces.
Source: http://www.valuebasedmanagement.net/methods_bcgmatrix.html
Source: http://www.valuebasedmanagement.net/methods_bcgmatrix.html
In a DevOps organization, teams are End-to-End responsible for their product. They are responsible for the concept to the grave of their product or the product lifecycle. The products in a DevOps team go through the different stages as shown in the BCG matrix. They can use the matrix as a guide to decide whether to implement changes or not.
Copyright © 2018 | 5
The following ideas apply to each quadrant of the matrix: Stars: The business units or products that have the best market share and generate the most cash are considered stars. Monopolies and first-to-market products are frequently termed stars. However, because of their high growth rate, stars also consume large amounts of cash. This generally results in the same amount of money coming in that is going out. Stars can eventually become cash cows if they sustain their success until a time when the market growth rate declines. Companies are advised to invest in stars. Cash cows: Cash cows are the leaders in the marketplace and generate more cash than they consume. These are business units or products that have a high market share, but low growth Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
45
ps
Course Book | DASA DevOps Fundamentals
prospects. According to NetMBA, cash cows provide the cash required to turn question marks into market leaders, to cover the administrative costs of the company, to fund research and development, to service the corporate debt, and to pay dividends to shareholders. Companies are advised to invest in cash cows to maintain the current level of productivity, or to “milk” the gains passively. Dogs: Also known as pets, dogs are units or products that have both a low market share and a low growth rate. They frequently break even, neither earning nor consuming a great deal of cash. Dogs are generally considered cash traps because businesses have money tied up in them, even though they are bringing back basically nothing in return. These business units are prime candidates for divestiture. Question marks: These parts of a business have high growth prospects but a low market share. They are consuming a lot of cash but are bringing little in return. In the end, question marks, also known as problem children, lose money. However, since these business units are growing rapidly, they do have the potential to turn into stars. Companies are advised to invest in question marks if the product has potential for growth, or to sell if it does not. Source: http://www.businessnewsdaily.com/5693-bcg-matrix.html
The Three Horizons Model
The Three Horizons Model
The Three Horizons Model High Growth Businesses
nts of vOps
n of a ulture
Accumulated Total Returns
vOps ulture
Today’s revenue growth + tomorrow’s cash flow
Current Businesses
Leadership mindset:
Inventors
Horizon 3 36 to 72 months
Leadership mindset:
Builders
Generate today’s cash flow
Horizon 2 12 to 36 months
Leadership mindset:
Operators
Horizon 1 0 to 12 months
Expected Window of Returns
Growth Options Options on furure highgrowth business
“explore” (innovations) Horizon 3: Far away Goal: Create new business Key metrics: No. of potential validated growth ideas Horizon 2: Near future Goal: Start creating revenue Key metrics: Revenue growth Horizon 1: Now Goal: Maximize returns Key metrics: Optimize margin “exploit” (operations)
Source: The Three Horizons Model by Mehrdad Baghai, Stephen Coley, and David White
Source: The Three Horizons Model by Mehrdad Baghai, Stephen Coley, and David White
Copyright © 2018 | 6
In an ever changing world, one must continuously invest in new opportunities and growth ideas. In a DevOps organization, one wants to try out new ideas as quickly and as cheaply as possible. Teams
46 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 3 | Culture
can have multiple products, and they must always be on the lookout for new opportunities to stay relevant as their current products can become a pet. From a process perspective, this type of work is performed over several “horizons”, each having different properties. This type of mechanism works as follows: Horizon 3: This is where work is performed to innovate and try out many new ideas which possibly might become profitable over an extensible amount of time (for example, 36 to 72 months). The goal of Horizon 3 is to create new business through trying and testing new ideas. The key metric for work executed in this area is the number of potentially validated growth ideas. {{
{{
Typically 98% of the ideas generated in this horizon will fail, but it is about the 2% which are good and feasible ideas. Ideas are tested through the use of MVPs, where MVPs are build as cheaply as possible. Inventors do not have to worry about engineering edge cases. It is all about testing and validating new ideas. Remember, to have one good idea, one must have many ideas. The leadership mindset and processes/frameworks in this area are based upon facilitating “inventors”. Horizon 3 is about exploration.
Horizon 2: This is where potential validated growth ideas arrive once they have been properly tested for feasibility and possibly success. The goal of Horizon 2 is to grow these ideas and its key metric is revenue growth. This horizon is about expansion, marketing the idea, starting to create real revenue, and requiring capital investments from (external) investors believing in the product at hand. The story of the product must be explained to communities, populations, and stakeholders as much as possible. From an engineering perspective, the product is to be made scalable, sustainable, maintainable, and new features should be added on a daily basis. Customer usage is closely monitored in such a manner that new features really add value to the product from a customer perspective. Typically this is done along the contours of a story board. The leadership mindset focuses on facilitating builders, highly skilled engineers, and entrepreneurs. Roles, that aggressively focus on expansion. Horizon 1: It is all about operating the product against as low a cost as possible. This is where the product will reside, once optimal growth has been established Horizon 2. The goal here is all about maximizing returns and its key metric is to optimize margin. Typically, this is the horizon to really exploit the idea that was once established in Horizon 3. From an engineering perspective, this is where the product will be further maintained while of course new features are added, at the same time keep in mind that margin needs to be optimized as much as possible. Leadership mindset is about facilitating operators and operating as cheaply and efficiently as possible. Horizon 1 is about exploitation. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
47
Course Book | DASA DevOps Fundamentals
Considering this model, four typical mistakes which are often made: 1. An organization forgets about investing in Horizon 3 (and 2 for the same reason). An example might be record businesses, for a long time record companies kept focusing on the cash cow, resisting innovation and jumping on the bandwagon themselves. 2. An organization wants to innovate, but starts doing this in Horizon 1 where the goal is to ”maximize returns” and “optimize margin”. This poses an issue as innovation requires investments. Also, innovating in an environment where daily operation takes place is far from optimal: i. The surroundings are all about current operations, which does not really free the mind for new ideas. ii. A potential idea might interfere with day-to-day operations resulting in corporate resistance to work on the idea. iii. What about when things go sour in Horizon 1? All resources will be pulled back into day-to-day operations. 3. Overswing/Underswing is when ideas are pulled from Horizon 3 to Horizon 2 far too quickly without properly measuring whether the idea will really resonate in the market. Management might have been too fond of their baby and start investing in something which is really not interesting. Underswing is where ideas take a longer time to develop and are killed in the process. Something that happens often when dealing with external investors that focus on short-term results. 4. An organization forgets to think about how to strategically embed new ideas into the current operating model. Innovative ideas, wherever they come from, even when invented inside a company, might conflict with the current operating model.
Team Activities: Example Activities for DevOps teams are not bound to traditional development activities. They include: Defining, designing, building, evaluating features/patches
validating,
releasing,
and
Defining and documenting hiring profiles or conducting interviews Conducting market research Analyzing application usage
DevOps Safari DevOps safari is an activity where one team visits another team to learn from one another’s practices and experiences.
Analyzing production incident Documenting and sharing incident Post Mortems/lessons learned Conducting DevOps Safari Improving delivery process effectiveness
48 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 3 | Culture
Researching a new technology/tool Refining/aligning product vision Supporting customers/application users and answer their questions Conducting standby shifts (pager duty) Performing Gatekeeping In a traditional organization, team members are task-oriented. They perform specific tasks they were hired for. On the other hand, DevOps teams have a broader scope of responsibility. They are also involved in other activities. The preceding list mentions some of these activities. It includes a much broader range of tasks varying from hiring team members to working standby shifts (pager duty).
DevOps
What is culture?
ntals
What is culture?
Tips
of a DevOps Culture
Organizational culture is about the characteristics of a particular Changing/establishing a culture is Organizational culture is about the characteristics of a particular set of people, which forms set of people, which forms the distinctive social and physiological done through changing the mindset, the distinctive social and physiological environment of an organization. The following figure environment of an organization. The following figure lists some of which consequently influences lists of those characteristics. thosesome characteristics. behavior.
Elements of DevOps
Characteristics of an Organizational Culture
entation of a Ops Culture
Knowledge Behavior Values
Transparency Mindset
Beliefs
Attitude Social Habits
Trust
Culture is the sum total of behavior and mindset of an organization, supported and enhanced by the values and beliefs of that organization. As an organizational unit, each team develops its own culture with the ultimate goal to achieve high performance. This counts for each DevOps team. The module discusses the cultural elements that form the basis of successful teams.
Copyright © 2018 | 8
Culture is shared and maintained through role-model behavior. This means that within and across teams people must exhibit consistent behavior to reinforce the strengths necessary for their teams to be successful from the cultural perspective.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
49
s e
f s
a e
Course Book | DASA DevOps Fundamentals
Reference Reading:
Typical TipsQuotes
Typical Quotes Collect quotes on your workplace The attitude and behavior of people shows the culture of an and write them on post-its. Identify The attitude and behavior of people showsThe thefollowing culture of an organization. following quotes are an outingThe of the behavior and which ones show the organization is organization. quotes are an outing of the behavior and can be used to visualize the culture: can be used to visualize the culture: ready for DevOps. If it is not in version control, it does not exist.
Code is not released without a review.
Our application usage data shows that customers have trouble using feature x of our product. How can we improve this feature?
Let’s explore how we can test, validate and evaluate this new feature.
I don’t like workarounds. We should solve problems instead!
Operational requirements are as important as functional ones.
Core of DevOps Culture
Quality is a first class citizen!
If it’s not in production, it has no value.
Performed a manual task 3 times? Automate it!
Copyright © 2018 | 9
Core of DevOps Culture The core of the DevOps culture is the emphasis on service. The following figure shows how The core of product. the DevOps culture is the emphasis on service. The this mindset help to deliver a high quality following figure shows how this mindset help to deliver a high quality product.
s e
f s
https://hbr.org/2013/05/what-is-organizational-culture
Core of DevOps
Product Usage
Product Lifecycle
Integral Responsibility
Service Mindset
High Quality Product
Continuous Support
Overall Health of the Product
a e
The focus on service is not just limited to ensuring a product is Copyright © 2018 | available for use by the customer. A service mindset ensures that a
50 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
10
Module 3 | Culture
high quality product is not only available but also meets the needs of the customer. Therefore, it is essential for a DevOps team to be regularly updated about how their product is being used. In addition, the team should analyze the product throughout its lifecycle to know whether it continues to be useful to the customer. Such an analysis helps the team to make necessary alterations or improvements, if required, to support the product. You can say that a DevOps team is completely responsible for the overall health of the product/service as: It fulfills the needs of the business process it supports. It works as intended without problems. It does not develop problems as the result of its further growth and development through the product’s lifecycle.
Typical Cultural Aspects of a DevOps Team Team Cultural Aspect
Description
Continuous Learning & Continuous Improvement
We have the desire to explore and learn in all activities we do. We strongly believe that working together, transparency, and sharing knowledge is vital. We care about our job enough to not pass the buck, we want to learn all the parts as a whole and not just our little world.
Experimentation & Risk Taking
We always conduct experimentation using solid methodologies to ensure ideas are evaluated on the real value instead of the assumed value.
Build Quality in
Quality is built in from the initiation of the teams up to discharge. It is at the heart of every activity. It is never compromised. We value full transparency.
An Engineering Culture
We have the desire to utilize our knowledge, skills, and creativity to solve problems, implement product features, and optimize our delivery process. We do not settle for the current status quo. We strive to improve our craftsmanship.
A Culture of Effectiveness
We continuously improve our delivery to improve our effectiveness. We define effectiveness as our ability to adapt to “market” circumstances and the success (value) of the product features delivered. Note that this also includes the effectiveness of activities, such as backlog prioritization.
A Culture of Product Thinking
Our application is our product. It must deliver value when it runs in production. We need continuous improvements to ensure the application delivers value now and in the future.
A Culture of Taking Responsibility
All members of our team are responsible for the complete product, which includes the full delivery cycle as well as operating/providing customer support throughout the lifecycle of the product.
Inspirational and Fun Environment
An environment in which people perform at best, where they feel inspired, where they want to be, feel welcomed and are encouraged to think out of the box. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
51
ulture
Course Book | DASA DevOps Fundamentals
e imposed from top to down. A culture is ‘grown’. As soon as you based try to upon DevOps principles share some Many organizations by telling people what to do, it will result in a culture of obedience with important cultural aspects. Some of these are described in the is something which is grown from within. If one wants to establish a table. important to provide the right context for preceding such a culture to be
Growing a Culture A culture cannot be imposed from top to down. A culture is ‘grown’. As soon as you try to amplify behaviors by telling people what to do, it will result in a culture of obedience with resistance. Culture is something which is grown from within. If one wants to establish a certain culture, it is important to provide the right context for such a culture to be established. Take for instance a campfire culture. In order to establish a campfire culture, ensure the availability of wood, match boxes, bags of marshmallows, and sticks. You can even look for a guitar and some drinks. As soon as people see this context, they will start to apply certain practices, such as putting marshmallows on the stick, start a campfire, and play the guitar. When many people start applying Copyright © 2018 | 12 certain practices, they give rise to what is called a “culture”.
What context to provide to facilitate growth areas for teams? Team Growth Areas
Tips to Facilitate
Continuous Learning and Continuous Improvement
Organize by teams, facilitate knowledge sessions, conduct hackathons, make people responsible. Use concepts of MVP. Set a clear DoD and coach teams in Agile way of working.
Experimentation and Risk Taking
Provide time, use instant sandbox environment, remove hurdles, applaud ideas, fail safely. Award failure!
Build Quality in
Advocate end-to-end responsibility; Encourage the team to utilize their skills as they know the best; avoid cutting corners; practice automation (test, deploy, and provision), continuous improvement, and transparency (monitors).
An Engineering Culture
Hire people with matching ambitions, move away from function-title model, support experimentation, support automating manual tasks, do not focus only on utilization.
A Culture of Effectiveness
Begin with end in mind, in retrospectives: ask people why? people are responsible to choose activities in teams, avoid or eliminate handover moments, move away from rigid processes, achieve results in small batches, generate awareness by Lean.
A Culture of Product Thinking
Encourage customers to attend demos, implement user feedback into a storyboard, allow customers to write about product and respond, organize people around the product, encourage the team to write blogs about product (you build it; you run it).
A Culture of Taking Responsibility
Do not explain ‘how’ to do, ask ‘what’ is required, address derailment openly, let people figure out how to do things, reward responsibility, reward failure, bring transparency in what everyone is doing.
Inspirational and Fun Environment
Provide good coffee, create nice and open environments, invest in additional facilities like games, conduct contests or arrange drinks at the end of the week, allow teams to modify/tailor the office to their needs.
52 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 3 | Culture
We now know a culture is established as soon as many people apply certain practices or exhibit certain behaviors. Practices and behaviors are encouraged by the context in which we operates. To amplify required behaviors many thing can be done. Some of the examples are listed below. Tips for ‘growing’ a culture in which people will learn! “Continuous Learning”: Organizations should facilitate multidisciplinary teams rather than resource pools, which creates shorter feedback loops so one can understand the implications of an action. Facilitated by knowledge sessions which are well funded and held (partly) during work time. Organizations need to show this is important for them and they must invest in knowledge sessions. These sessions must accompany with good food and fun, so that participants find them interesting to attend. Conduct hackathons/innovation days. Again, this means investing in ‘time’. Organizations put efforts to make sure you understand/see that innovation is vital to the business. Make people responsible, as when people feel responsible, they will do what is required to do the job. People in the organization need a sense of responsibility. Apply the concepts of a Minimal Viable Product (MVP) when trying out new ideas. Tips for ‘growing’ a culture in which people will try out new ideas! “Experimentation”: Provide resources the time and means to explore new ideas. Make sure that obtaining infrastructure/playground/sandbox does not pose a hurdle to try out new ideas, systems need to be made instantly available as soon as needed, just like in cloud. Applaud new ideas, write about new ideas and if people want to support the ideas, provide them room to do so. Trust people to do the right thing. Provide a platform for people where they can showcase their new ideas. Consider using the knowledge sharing sessions for this. Create an environment in which it is safe to fail. Tips for ‘growing’ a culture in which people will pay attention to levels of quality: Make people responsible for the complete end result, not just an activity. When delivering new software, ask team to provide insight into the amount of functionality that can be delivered. Do not push or cut corner as team knows best the insight of handling a software development. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
53
Course Book | DASA DevOps Fundamentals
Provide people with the means to measure what they are doing, it is about feedback loops, therefore, instill open monitors on the work floor, build monitors (provides a detailed view of the status of selected jobs), and display test results in open. Transparency is key. Test and deliver often, it is easier to correlate test results to tasks that have been performed over the last couple of hours than over the last couple of weeks. Build quality in, as per Scrum terminology, add to the Definition of Done (DoD) when delivering a software, the software should be verified while moving through all stages of test. Reserve 10% of time for continuous improvement, this practice not only improve product, but also the way of working (in Scrum this is done during retrospectives). Automate manual tasks as often as you can, to make sure the consistency of results can be expected, automate your tests, automate deployments, and automate provisioning of new machines. Tips for ‘growing’ an “engineering culture”: When interviewing new candidates, make sure their ambitions match the ambitions of the company. Attach people to products based upon ‘added skill or knowledge’, move away from adding people based on ‘function-title’ model. Reserve 10% of time for automation of slow and costly manual steps as manual activities are a source for error. Allow/support people to help one another in solving issues by just not managing on utilization alone. Support experimentation by supplying a common playground/ sandbox in which people can experiment. Tips for ‘growing’ a “culture of effectiveness”: Begin with the end in mind and work your way back by only doing the things which are really required. This might alleviate you from getting distracted by following process steps which do not add value. Help people build quality in by not cutting corners, by automating tests and by automating manual (often boring) activities. Eliminate handover moments (a source of inefficiency) as much as possible. Strive towards autonomy on every level. Teams should be able to operate independently. Leave the responsibility to choose activities to achieve results to the team as each situation might be different. Stop following rigid processes without introspection. Do things in ‘small batches’, start small and assess whether you are doing the right things. Iterate when working on products.
54 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 3 | Culture
Grow awareness that people should start to think about which activities really add to the end results. Educate, for instance, by introducing Lean training/principles. Tips for ‘growing’ a culture which support “product thinking”: Involve the whole team with customer. Not just the Product Owner, but involve engineers as well. Make sure customers attend the demo after each sprint. Shorten the feedback loop by incorporating customer feedback/ user feedback straight into the user storyboard. Allow your customers to write about your product and enable team to respond to these comments. Make teams understand about what they are working. Organize teams around products and make teams responsible from start to end, abandon resource pools in which people are attached to multiple ‘projects’. Strive towards complete autonomy of the teams and thus the products. Ask teams to write about the product they are working on. For instance, the company blog or speak about the product at seminars/customer engagements. Tips for ‘growing’ a culture which people take end-to-end responsibility: Do not explain how things should be done, instead support resources in thinking new ideas/ways of working for themselves. Give more authority for decision making to teams. For instance, let the teams decide when to go live and when not. When possible, allow teams access to systems, they usually should request somewhere else. Make people responsible for a product from start to finish. Let people speak out to one another when they see someone ducking responsibility. You can do so by establishing transparency (for example, by standing in front of the Scrum board). Address derailment of the process. Create an environment in which it is safe to fail. Reward taking responsibility and also failure. Reward people speaking out when something does not seem to work. All in good health, of course. Tips for ‘growing’ a culture which is inspirational and fun for people: Provide good coffee in the office. It is the thing they drink the most during the day. Create nice and open environments to work in. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
55
Course Book | DASA DevOps Fundamentals
Invest in additional facilities, such as games for people to add some fun to the workplace and an outing at the end of the week so people can get to know each other outside their work environment. Allow the team to create their own space to help them create a sense of ownership.
DevOps
entals m
DevOps SkillsKey Elements of DevOps DevOps Skills
of a DevOps Culture
y Elements of DevOps
mentation of a vOps Culture
DevOps is all about people, processes, and tools. People across the DevOps is all about people, processes, and tools. People across the groups working groups working together determines DevOps success. The primary together determines DevOps success. The primary goal of DevOps is to build a goal of DevOps is to build a collaborative and respectful culture across collaborative and respectful culture across ITpiece organization to bringdelivery. a single piece of flow IT organization to bring a single of flow in software software delivery. A team cannot become a DevOps team just by using a set of tools. They
A team cannot become DevOps justDevOps by using a set of tools. They need to ensure need toaensure toolsteam support requirements and workflows. A tools support DevOps requirements andideals workflows. A culture that supports culture that supports the of the DevOps movement is crucial.the ideals of t However, some of the cultural elements that can help you develop DevOps movement is crucial. However, some of the cultural elements thatancan help you effective and successful DevOpsculture culture are: develop an effective and successful DevOps are:
Teambuilding and Collaboration
Leadership and Feedback
Continuous Improvement and Problem-Solving
Courage and Experimentation
Copyrigh
The following pieces of information help you know the preceding cultural elements: Teambuilding and Collaboration: Collaboration among crossfunctional teams is a key to building a successful team. When teams collaborate with each other, they work more effectively, spend less time firefighting, and deliver products faster. A key to collaboration is open communication with clear transparency of information among people including the senior management. Open communication occurs when everyone is able to express and share ideas to one another without any fear. DevOps encourages collaboration and open conversations throughout the lifecycle of a product to discuss its requirements, features, schedules, resources and others.
56 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 3 | Culture
Continuous Improvement and Problem-Solving: Continuous improvement helps teams to think of new ideas of operating. It forces teams to continually think of new ways of working, build better quality products, improve productivity, and reduce costs. An effective continuous improvement process requires teams to stay bold and curious to innovate. To implement continuous improvement, DevOps needs teams to shed unproductive methods and invent more optimized methods. DevOps teams need to adopt automated processes and techniques to solve problems and reduce waste. Courage and Experimentation: DevOps, being a cultural shift, requires its teams to leave old ways of working and adopt best practices to optimize the existing processes of the organization. The ultimate goal of this optimization is to deliver added value to the customers. However, it requires courage, as the boldness described in continuous improvement and problem-solving. Organizations that are open to experiment new changes for improvements have stronger teams that value customers’ needs and strive hard to meet those needs. The key activity for an effective DevOps team is experimentation and the cultural space to fail.
A DevOps
amentals ium
Leadership and Feedback: DevOps leadership is to encourage and facilitate ownership and responsibility for the work. When each individual of the team takes complete ownership of the work, the collective efforts bring bigger benefits to the organization. DevOps leaders realize that they need to be both well-informed of the state of the team and the work, and leave space for the team to act on their mission. This requires excellent (real-time) feedback to fuel continuous improvement.
Teambuilding and Collaboration
You can easily map these areas to the four skill areas of the DASA Team competence framework.
Visual Management Teambuilding and Collaboration
ence of a DevOps Culture Team
Collaboration
Visual Management
Key Elements of DevOps Collaboration
plementation of aWhat is a team? DevOps Culture
Before getting into the culture of DevOps teams, take a brief look at the definition of a team. “A team is a small number of people with complementary skills who are committed to a common purpose, set of performance goals, and approach for which they hold themselves mutually responsible.” Katzenbach & Smith, The Wisdom of Teams, 1993
Tips Team building is so important to DevOps that you could sum up most of the methodology’s goals for improvement with two Cs, Collaboration and Communication. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
57
Essence of aEssence DevOpsof a DevOps Culture Culture “A team
is“Aa team smallisnumber a small of number peopleofwith people complementary with complementary skills whoskills are committed who are committed to a to a commoncommon purpose,purpose, set of performance set of performance goals, and goals, approach and approach for which for they which hold they themselves hold themselves Key Elements Key of Elements of mutually mutually responsible.” responsible.” DevOps
DevOps
Course Book | DASA DevOps Fundamentals Katzenbach Katzenbach & Smith, The & Smith, Wisdom The ofWisdom Teams, 1993 of Teams, 1993
Implementation Implementation of a of a DevOps Culture DevOps Culture
The following difference between two types of cohesion further helps
Cohesion The following The following difference difference betweenbetween two types two types cohesion of cohesion further helps further you helps understand you understand the the you understand the meaning ofofteams. meaning meaning of teams. of teams. Cohesion refers to the degree to which the elements inside a Logical Cohesion Logical Cohesion Functional Functional CohesionCohesion module belong together. ThereIt results inIt results Those parts Those of aparts module of aare module grouped are grouped “task-oriented in “task-oriented silos” as the silos” as the that contribute that contribute to a singletowell-defined a single well-defined components are logically arecategorized logically categorized are many different levels ofcomponents task of thetask module. of theSuch module. a grouping Such a grouping and to grouped do the to same do the tasksame eventask even Cohesion used in different levelsand grouped results in results formingin'teams.' forming 'teams.' though they though are different they areby different nature.by nature. of an module. Copyright © 2018 Copyright | 17 © 2018 | 17
Logical Cohesion Logical cohesion implies that there is some Logical relationship between the elements of the module.
The preceding definition matches the situation of a DevOps team. People joining a DevOps team often come from different technical teams. Therefore, they do not really fit the definition of a team. These technical teams have people with similar skills. However, they do not always have the same purpose or goals. They might be working together with other people (outside their group) to achieve the enterprise’s goals. Functional Cohesion Therefore, when transitioning from a technical team to a DevOps team, Intrinsically Motivated Teams Functional cohesion implies that it is essential to give proper attention to teambuilding.
all the elements of the module Intrinsically Motivated Teams are related to performing a single In 2011, function.Daniel Pink published a book on motivation called Drive. According to him, there
2011, Daniel Pink as published bookfollowing on motivation called Drive. are three key aspects that canInmotivate anyone, listed ina the figure.
According to him, there are three key aspects that can motivate anyone, as listed in the following figure. Aspects of Motivation
Autonomy
The need of people to have control over what they do, when they do it, who they do it with, and how they do it.
Daniel Pink Daniel H. Pink is the author of several provocative, bestselling books about business, work, and behavior. His books include To Sell is Human, Drive, A Whole New Mind, The Adventures of Johny Bunko, and Free Agent Nation. His latest book is When: The Scientific Secrets of Perfect Timing, which to be published in January 2018.
Mastery
The desire of people to become good at specific task(s). This mindset requires dedication and hard work but also the space to learn and practice.
Purpose
People are motivated by the fact that they are doing any task for a reason, often an altruistic (selflessness) reason.
If you plot the three aspects on the intention of a DevOps team, you Copyright © 2018 | will find that the DevOps teams are intrinsically motivating. DevOps teams are set up organizationally and technologically so that they are as independent as possible. Such an independency helps them gain significant autonomy to act on behalf of their customers. DevOps teams work closely with their customers, therefore, their sense of purpose is strongly triggered on a day-to-day basis. The independent nature also forces the DevOps team members to even become the masters to act more timely and effectively. There are ample opportunities for engineers to develop their mastery. Not only
58 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
18
Module 3 | Culture
the deeper knowledge but also the broader knowledge is necessary. Consequently, the need for engineers joining a DevOps team is quite evident these days. As you progress through the course, it will help you become clearer.
Visual Management: A Key Tool of Teambuilding Visual Management is one of the strongest tools to stimulate collaboration and ensures that the pitfalls are uncovered. The tool helps ensure the work done by a team is constantly visible on manual or electronic boards. It also helps:
Identify work and impediments. Key Tool of Teambuilding
Communicate important information. Show how to perform a task.
strongest tools stimulate collaboration and ensures that to Show planning and priorities. ol helps ensure the work done by a team is constantly In its most basic form, Visual Management includes three boards, as oards. It alsoshown helps: in the figure.
Tips Some of the possible effects of Visual Management on the behavior and mindset within a DevOps team: Focusing on better communication Taking end-to-end responsibility Encouraging awareness of flow of value to customers Providing feedback Offering support Building discipline
s.
mation.
Week Board
Day Board
Problems
Problems
Improveme Improvement Board nt Board
Progress on Problem-Solving Each board has its own role, such as:
Copyright © 2018 | 19
The Day board is used to manage the progress of work on a daily basis. The management can be done using the simple “To Do-Doing-Done” setup or a more complicated approach. The Week board is used to ensure the work is planned and agreed. It is also a place where a team should discuss their Key Performance Indicators (KPIs). Such a discussion is preferred on a daily basis. However, if it is impossible, the discussion should happen at least once a week. The board also serves as a repository for comments from customers and employees. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
59
Course Book | DASA DevOps Fundamentals
The Improvement board is where all the problems or improvements initiatives are collected that need to be solved or carried out. It is the place where the team members should communicate and share solutions and learnings. It also shows the progress on problem-solving. Using the three boards, a DevOps team can manage long-term trends, short-term planning, and improvements. Some of the characteristics of Visual Management include: Instruction: guides you how to perform a specific task, such as work instructions and standard operating procedures. Informative: gives you important information on the status of work. Planning: helps you plan the various activities using tools, such as Gantt charts and white board walls, and let others to see and know the plan.
Kanban Kanban is Japanese for “visual signal” or ‘card.’
One of the popular tool of doing Visual Management is Kanban. Back to presentation
Back to presentation
60 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
ps
vOps lture
nts of vOps
Module 3 | Culture
Kanban is a signalling system that identifies when a new work can enter the system. It helps establish single-piece-flow to ensure flow of value and keep the team free from getting overloaded with work. You will learn about Kanban in more detail In Module 05, Processes.
Collaboration: A Success Factor of a Team
Collaboration: Success Factor of a Team The successAkey of any team is teamwork, and the most important
behavior of teamwork is collaboration. The meaning of collaboration is working together to achieve a goal. It is the central theme for DevOps The success key of any team is teamwork, and the most important behavior of teamwork is teams. collaboration. The meaning of collaboration is working together to achieve a goal. It is the central Collaboration theme for DevOps teams. offers a number of benefits but also brings with it a number implemented effectively, listed in the Collaboration offersofa pitfalls numberifofnot benefits but also brings with it as a number of pitfalls if not following figure. as listed in the following figure. implemented effectively,
n of a lture
Benefits
Combines different perspectives Encourages creativity Takes advantage of synergies Brings balance to decision making Improves delivery times
Irrational decision-making due to group thinking Ambiguity in roles and responsibility High cost of collaboration Longer decision times Conflict within the group
Pitfalls
Copyright © 2018 | 21
In order to achieve the benefits, the team should learn to accept or avoid the pitfalls. The discussions can take longer to come to a decision. DevOps teams aim to ensure that team members have overlapping skills and knowledge. However, the overlapping skills might lead to some ambiguity in responsibilities. As a result, it is crucial for a DevOps team to have open communication to ensure thethat Source the pitfalls are uncovered and addressed.
ntinuous Improvement and Problem-Solving
uality at
roblem-Solving Continuous Improvement and Problem-Solving
aizen Mindset
Quality at the Source
Problem-Solving Kaizen Mindset
Importance of Quality at the Source Doing tasks right in the first go always pay back later in the process. Therefore, it is essential for organizations to have quality at the source to avoid future issues and the corresponding efforts. The following graph shows the cost effect of not building in quality at the source.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
61
Doing tasks right in the first go always pay back later in the process. Therefore, it is essentia Course Book | DASA DevOps Fundamentals for organizations to have quality at the source to avoid future issues and the corresponding efforts. The following graph shows the cost effect of not building in quality at the source. Cheap to find issues here and fast feedback
$16,000
Percentage of bugs
85% % Defects introduced in this phase % Defects found in this phase
$1,000
$ Cost to repair defect in this phase
$250
$25 Coding
$100 Unit Test
Source: Applied Software Measurement, Capers Jones, 1996 Source: Applied
Function Test
Field Test
Post Release
Software Measurement, Capers Jones, 1996
Some of the characteristics of having quality at the source include: Focus on strong correlation: It is easy to fix issues right after these have been created. The strong correlation between the error and the cause of error during initial stages makes it easier to trace the issue(s) and apply the required solution(s). Prevent issues to stack upon one another (House of cards): It is easier to find the source of the problem if you avoid accumulating new features and/or issues on top of this problem. One can view this like a house of cards where it is best to first ensure all cards are at their correct places before moving on to the next level. Respect the flow of work: Accumulating work results in waste and inefficiencies in the process. Consequently, it breaks the flow of work. This is very much related to a Lean principle where it is a paradigm “Work in Progress should be kept to an absolute minimum in any process”. You will learn more about this in Module 5. Require less testing efforts: Consider an application consisting of 3 layers, where each layer is supporting 5 different possible flows. Using unit tests (testing each layer separately), the number of tests involved to test all possible flows will be 5 * 3 = 15 tests. On the other hand, testing all possible flows using functional tests (by testing the complete system from front-end) will require 53= 125 functional tests.
62 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 201
Module 3 | Culture
Cost of Accumulating Technical Debt Accumulating technical debt is a real threat to your business. For example: Customer responsiveness reduces over time as a result of accumulating technical debt. The Cost of Change (CoC) increases at a faster rate if technical debt goes unchecked over time. In applications with high technical debt, estimation is nearly impossible.
Technical Debt Technical debt is a concept in software development that reflects the implied cost of additional rework caused by choosing an easy solution now instead of using a better approach that would take longer.
There are three strategies to deal with technical debt: Do nothing, it gets worse
Replace, high cost/risk ulating Technical Debt
Incremental refactoring, commitment to invest
al debt is a real threat to xample:
iveness reduces over time mulating technical debt.
ge (CoC) increases at a cal debt goes unchecked
h high technical debt, y impossible.
egies to deal with technical worse
/risk
oring, commitment to Source: ©2008 information, Inc. Copyright © 2018 | 24
Maintainability and extensibility are important aspects when working in an environment where a product is shaped over time. Accumulating or avoiding technical debt has an increased impact on the cost of every change applied. Your organization might end up in the situation that one cannot respond to customers in a timely fashion resulting in loss of clients. Therefore, it is important to focus on quality at the source.
Continuous Improvement is About Problem-Solving Continuous improvement is to solve problems to achieve one or more of the following goals: Deliver better value faster and cheaper to the customers Bring more meaning to your work Leave a healthier environmental footprint Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
63
Course Book | DASA DevOps Fundamentals
Problem-solving is synonymous of an engineering culture, which means emphasizing:
Kaizen Kaizen is a Japanese word for “continuous improvement.”
Data and Facts Feedback Kaizen “Engineers build things that solve problems. You don’t have to be a computer scientist or have any particular degree to be an engineer. You just have to speak up when things aren’t right, evaluate ideas on their merits, and build things that fix what’s broken.” Palantir Technologies, USA
Continuous improvement is integral to an engineering culture. To implement continuous improvement, the culture within a DevOps team should be based on facts, data from the technical components, and the use of a service by its customers. In other words, a culture of seeking out feedback from customers, suppliers and, of course, the systems. You can have such a culture through small incremental improvements, though sometimes breakthrough changes are necessary. Problem-solving is integrating Kaizen into the behavior and mindset within a team. Kaizen is all about building continuous improvement into the way of working. It works in two ways: Daily Kaizen, looking for small improvements every day.
evOps
tals
Kaizen events, handling larger problems using the Kaizen structured problem-solving method of the Define, Measure, Analyze, Improve, and Control (DMAIC) steps.
Structured Problem-Solving Food for Thought
DMAIC is a tool for continuous Structured Problem-Solving improvement and problem solving. The key tool for continuous improvement is a structured problemfor continuous improvement is aAstructured problem-solving wellDo youThe knowkey othertool problem-solving solving method. well-known method to method. facilitateA continuous known method to facilitateimprovement continuous improvement is DMAIC. methods? is DMAIC.
a DevOps Culture
Problem-Solving to Facilitate Continuous Improvement
lements of DevOps
Define
ntation of a ps Culture
Anchor the change in the organization Share lessons learned
Control
Describe symptoms and define the problem Ensure stakeholders agree on scope of problem
Measure
DMAIC Define alternative solutions Decide on and implement improvements
Improve
Analyze
Collect data and facts about the problem Validate the data
Analyze and structure the data Define and test hypotheses regarding the problem
64 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 | 26
Ops
s
DevOps Culture
ents of DevOps
on of a Culture
Module 3 | Culture
You cannot solve a problem, unless you first Define it. Defining a problem includes describing the current situation and why it is not acceptable. Therefore, you should ensure the on-board stakeholders agree with the statement defining the problem. You should then Measure the variables that can influence the problem by collecting the related data and facts. It is vital to ensure that the data is correct. The next step is to Analyze the gathered data. Such an analysis includes structuring and visualizing the data into a format that will allow you to understand what the data is all about and converting it into information. You can then use the information to test hypotheses regarding the problem. Having understood the dynamics of the problem, you can now move on to the Improve phase, which defines potential solutions to the problem. Once a number of solutions are generated, decide the improvement to implement. After ensuring whether a particular solution works, embed the solution into the organization’s way of working in the Control phase and share any lessons learned. Some of the commonly used problem-solving techniques are PlanDo-Check-Act (PDCA), 5 Whys, and Kepner-Tregoe (KT).
The Kaizen Mindset: Tackling the Root Cause of Problems The Kaizen Mindset: Tackling the Root Cause of Problems The Kaizenmindset mindset means means integrating thethe following threethree behaviors The Kaizen integrating following behaviors into the DevOps team: into the DevOps team:
Seeing and prioritizing problems
Solving problems
Sharing lessons learned
Be truly prepared to: Uncover problems Accept them as a part of daily life Initiate an action to identify the problems that need immediate solutions
Be prepared to: Invest time and other resources Understand the root causes of problems Resolve the problems completely
Be driven to: Share the lessons learned with others in the IT organization, so they can benefit from it
Kaizen and the Kaizen mindset are about seeking the underlying or root causes of the problems in a team. These problems are not limited to technology. These can relate to processes, interactions between team members, or cooperation with other parties.
Copyright © 2018 | 27
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
65
Course Book | DASA DevOps Fundamentals
Courage and Experimentation Courage Experimentation Experimentation Meetups
Experimentation
Tips
The third key cultural skill within DevOps is courage. It is the ability to act in the face of uncertainty or danger while understanding and minimizing the risks. Courage is to take risks, and the underlying key behavior is experimentation.
Copyright © 2018 | 28
One of the catchphrases of DevOps teams is “If it hurts, do it more often.” The phrase relates to the Daniel Pink premise that motivation is based on autonomy, mastery, and purpose.
Experimentation is to test a hypothesis. In other words, trying something new based on a need is known as experimentation. DevOps teams have the courage to experiment, with the potential of failure. They know how to fail fast, take the next step, or go back a couple of steps under time pressure to ensure there is quality at the source. Much of DevOps is uncharted territory. However, for the coming years, it will be the domain of early adopters (the innovators are already there). There will come a time when the DevOps way of working will be the norm, but still IT engineers will need to have the courage to take the next step.
Marshmallow Challenge: Being experimental works!
Marshmallow Challenge: Being experimental works!
Source: marshmallowchallenge.com
Source: marshmallowchallenge.com
66 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 | 30
Marshmallow Challenge: Being experimental works! (Contd.)
Module 3 | Culture
Specialized Skills + Facilitation Skills 30
= Success
20
10
0 Height
Average
(Inches)
Business School Students
Lawyers
Kinder- Architects & garten Engineers
CEOs
CEOs & Executive Admins
Source: Adapted from marshmallowchallenge.com
rce: Adapted from marshmallowchallenge.com
Copyright © 2018 | 31
The Marshmallow Challenge is an activity done to help business leaders realize the power and challenges of team problem solving. Thousands of groups have done the activity and there have been some interesting statistics that have come out of these studies. 1. One of the poorest groups on average are College graduates with Business Majors (an average of 20 inches.) The reason… they have been told that problem solving is a linear solution where you plan, and then execute a plan. They work to the very end, place their marshmallow on top and have either an “aha moment” or more often an “oops moment”. 2. One of the best performing groups is another group of graduates…graduates from Kindergarten. Kindergarteners average 30 inches. Why, because they have a natural instinct to prototype. They start with the marshmallow and build up. Plus they don’t have the natural power struggle within their teams that adults develop. Source: Wujec, T. (2010). The marshmallow challenge
Generalising this research, we see that: Prototyping matters. By repeatedly prototyping and refining give a better result than testing at the end. Diverse skills matter. By combining multiple disciplines, you can achieve a better result.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
67
Course Book | DASA DevOps Fundamentals
Food for Thought To learn about an example of experimentation, learn about the “Marshmallow Challenge”. For this challenge, the teams need to build in eighteen minutes, the tallest ns freestanding structure out of 20 sticks of spaghetti, one yard of tape, one yardisofto string, and one marshmallow. erimentation understand and ork the best. order to experiment For Inmore information, visit https:// to test. www.playmeo.com/activities/teambuilding-problem-solving-activities/ marshmallow-challenge/. Fail Fast
Experimentation Comes with Complications Experimentation is not about “just trying something.” Experimentation is to understand and analyze a problem, and determine which solutions will work the best. In order to experiment with low risk, you should have hypotheses that you want to test. Hypotheses are tested in a running business when: Business requirements tend to evolve over time. Gaining consensus on requirements might be difficult. Hypotheses are tested in a changing IT environment when: Parts of the IT service are changed or the service itself is changed. The baseline changes thereby create new possibilities or impossibilities. New tools and capabilities of IT make actions that were previously difficult.
n:
f is
e
Copyright © 2018 | 32
The courage required in a DevOps team comes largely from the fact that experimentation should occur in a live business environment with varying requirements. However, the associated IT environment is also changing on a daily (even hourly) basis. It is, therefore, difficult to run longer term experiments if the baseline is continuously changing. Considering the changing requirements, the aim of a DevOps team should be to take calculated risks, fail fast in a small way, and learn Act: Asolutions Key Behavior a DevOps Team which will work theof best.
SA DevOps Courage to Minimal Viable Product damentals Minimum Viable Product is a mium Courage to Act: A Key Behavior of a DevOps Team development technique in which The DevOpsinteamThe can take a number of steps embedded a new product is introduced DevOps team can take toa ensure numberthat of experimentation steps to ensure isthat in the way of working. The four actions that a DevOps team can take the market with basicculture features,and the experimentation is embedded in the culture and the way of working. to their actions arefour accepted are: a DevOps team can take to ensure their actions ence of a DevOps but enough toensure get feedback from The actions that Culture customer. are accepted are: Key Elements of DevOps
mplementation of a DevOps Culture
Ensure customer buy-in
Define and deliver a Minimal Viable Product (MVP)
Focus on the goal “customer value is delivered first time, right in flow”
Take small steps and do not carry out large experiments
One of catchphrases of DevOps teams is “If it hurts, do it more often.” The phrase relates to the Daniel Pink premise that motivation is based on autonomy, mastery, and purpose. In the quest of mastery, Pink
68 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 | 33
Module 3 | Culture
states that we need to deal with pain as we learn to do things better. This is certainly the case for DevOps teams.
Courage and Experimentation Not only product, try out new ideas Experiment on how the delivery process can be improved. Measure/experiment with new technologies for provisioning new systems. Measure/experiment different types of organizations when it comes to operations.
vOps
als
DevOps Culture
Measure/experiment the usage of the existing features. (The customers might exhibit some cyclic behavior.) Courageous Behavior Requires Safety
Courageous Behavior Requires Safety Businesses need innovation, but but people cannot innovate in an environment due to Businesses need innovation, people cannot innovate in unsafe an unsafe apathy, fear, and complacency, as shown in the following figure. environment due to apathy, fear, and complacency, as shown in the following figure.
ation of a s Culture
Through: Communication Mission Support Tools …
Provide Safety
ements of DevOps
Complacency
Apathy
Innovation
DevOps teams need to be here
Fear
Urge Courageous Behavior
It is quite easy to say people need courage and they need to exhibit courageous behavior, such as experimenting, failing fast, and taking many small steps quickly. However, one key condition that needs to be fulfilled before this happen is a safe environment for the team, where people can talk and act without fear of reprisals. Therefore, the primary role of the organization’s leadership is to develop a safe environment, within teams, between teams, and across the hierarchy.
Copyright © 2018 | 35
Leaders can develop a safe environment through several activities, such as: Providing safety by clear communication Sharing a good mission statement Offering genuine support to people Providing tools that enable employees to exhibit the wanted behavior Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
69
ps
Ops ture
Course Book | DASA DevOps Fundamentals
Experimentation Meetups: A Key Tool of Courage Experimentation Meetups: A Key Tool of Courage A key tool to encourage experimentation meetings where new actions or initiatives A key tool to are encourage experimentation are meetings whereare new tested. Experimentation meetings, such as hackathon, is a great way to simulate creative actions or initiatives are tested. Experimentation meetings, such as actions. hackathon, is a great way to simulate creative actions.
s of Ops
of a ture
k
Organizations use up to 20% of employees time to work on innovations or improvements.
Hackathon, Codefest, Hack Day, or Hackfest Source: http://venturebeat.com/2013/10/23/running-a-hackathon-challengepost-willhelp-you-organize-it-for-free/ Source: http://venturebeat.com/2013/10/23/running-a-hackathon-challengepost-will-help-you-organize-it-for-free/ Copyright © 2018 | Hackathon is an event on software projects that involves developers and other people involved in the development and delivery of IT services. The primary aim of such an event is to collaborate and develop a usable software. However, it can be used for academic or social purposes as well. The duration of a hackathon varies from one day to one week. A hackathon is also known as codefest, hack day, or hackfest.
Leadership and Feedback Empowerment Facilitating Leadership Feedback
Food for Thought Some of the leadership traits required for DevOps teams are clear expectations, challenging the team, leading by example, being humble, and learning through own observation(s). Think of some other formal (management) and informal (such as technical and behavioral) leadership traits required for a DevOps team.
Leadership in a DevOps Environment Copyright © 2018 | 37
One of the behaviors to make a DevOps team successful is leadership. We often equate leadership with management. In DevOps teams, this is only part of the story. Every member of the DevOps team must be encouraged to lead. In this context, leadership is about taking decisions, acting in accordance with the goals of the team, and being accountable for the actions taken by the team.
70 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
36
of a DevOps Culture
Elements of DevOps
entation of a Ops Culture
One of the behaviors to make a DevOps team successful is leadership. We often equate leadership with management. In DevOps teams, this is only part of the story. Every member Module 3 | Culture of the DevOps team must be encouraged to lead. In this context, leadership is about taking decisions, acting in accordance with the goals of the team, and being accountable for the actions taken by the team. In In DevOps, DevOps, leadership leadershipisistotofacilitate facilitateteams teamswith witha afocus focusononthe the following two aspects: following two aspects:
Mission Command
Ensure the DevOps team knows its mission and goal, and is free to act in accordance with achieving the mission.
Empowerment
Ensure the DevOps team is capable (have sufficient knowledge) and authorized to take the required decisions.
Facilitating Leadership Management should ensure that DevOps teams continue to be clear on their mission and empowered to achieve it.
DevOps
entals m
In a DevOps team, there must be a focus on mission command. In other words, knowing the mission of the team and working towards achieving it. In successful DevOps teams, team members feel empowered to take decisions. They are both authorized to act and have sufficient knowledge and capability to action the decisions taken. Therefore, management needs to assume a facilitating role, essentially to removeMission the blockages impeding the team’s progress. Command Leadership: Command vs. Central
Copyright © 2018 | 38
Leadership: Mission Command vs. Central Command
of a DevOps Culture
Central command alsoalso known as detailed command is to leadisa to team Central command known as detailed command lead a team through detailed through detailedOn instructions. the other hand,command mission command is a team through vision and instructions. the otherOn hand, mission is to lead toempowerment. lead a team through vision and empowerment. The following table The following table lists the characteristics of the two ways of leading a lists the characteristics of the two ways of leading a team. team.
y Elements of DevOps
mentation of a vOps Culture
Mission Command
Today’s world!
What we need!
Probabilistic Unpredictable
Decentralization Spontaneity Informality Loose Rein Self-discipline Initiative Cooperation Faster Acceptable Decisions Echelons Ability Higher Tempo
Implicit Vertical and Horizontal Interactive Delegating Transformational
Versus Assumptions
Behavioral Trends
Communication Styles Leadership Styles
Central Command Deterministic Predictable Centralization Coercion Formality Tight Rein Imposed Discipline Obedience Compliance Optimal Decisions (but later) Ability Focused at the Top
Explicit Vertical Linear Directing Transactional Copyright © 2018 |
Mission command relates very much to the world of DevOps. The DevOps teams exist in a world that is unpredictable and requires them to accept uncertainty. The behaviors and mindsets that can help the Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
71
ps re
of ps
Course Book | DASA DevOps Fundamentals
Leadership: Five
teams to be successful under such conditions are very much similar to those that captured in the DevOps cultural elements. Barriers of are Effective Collaboration
Leadership: Five Barriers of Effective Collaboration Tips has one clear role regarding a DevOps team “to ensure the team functions Management Management has one clear role regarding a DevOps team “to ensure and correctly.” Lencioni (2002) identified the followingPatrick five dysfunctions Basic collaborates ingredients to overcome the Patrick the team functions and collaborates correctly.” Lencioni (2002) of a team or barriers to effective collaboration. An effective leadership helps DevOps teams to preceding barriers include trust, identified the following five dysfunctions of a team or barriers to overcome theshared five barriers. open communication, values, effective collaboration. An effective leadership helps DevOps teams and a common goal. to overcome the five barriers. Barriers to Effective Collaboration
a re
Lack of Trust
Fear of Conflict
Avoidance of Accountability
Lack of Commitment
Inattention to Results
The following pieces of information explain how the preceding barriers result in dysfunction of a team: Lack of trust between the team members is like a team Copyright © 2018of | 40 individuals working together. Such a team often results in disappointed progress. Fear of conflict means the differences of opinion are not sought out rather avoided and left undiscussed. Avoidance of accountability can be found in finger-pointing, for example, developers do not take responsibility for the effect of their work in the production environment. Lack of commitment means the team members do not support each other. Inattention to results leads to a lack of focus on the goals of the team and whether these are being achieved. Overcoming these barriers requires the leadership to exhibit the following behavior: The leaders need to demonstrate genuine vulnerability first, and create an environment that does not punish admissions of weakness or failure. It is key that leaders demonstrate restraint when team members engage in conflict, and allow resolution to occur naturally. The leader must be comfortable with the prospect of making a decision that turns out to be wrong, and must be constantly pushing the group for closure around issues.
72 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 3 | Culture
The leaders need to create a culture of accountability on the team by encouraging and allowing the team to serve as the first and primary accountability mechanism. The leader must set the tone for a focus on results. If the team members sense that the leader values anything other than results, they will take that as permission to do the same. Source: https://medium.com/the-mission/part-2-overcome-the-5-dysfunctions-of-a-
A DevOps
team-ef922309f8b5 Leadership and Feedback
amentals um
Leadership and Feedback
One of the main reasons for conflict …. does anybody know?
nce of a DevOps Culture
Key Elements of DevOps
One of the main reasons for conflict …. does anybody know?
No shared outcome, having different interests …
(This happens when we are made accountable for tasks only: local optimization!)
plementation of a DevOps Culture
eadership: Style
Copyright © 2018 | 41
Leadership: Style
he six words (Go, See, Ask, Why, Show, and Respect) embody the way in which Lean The six words (Go, See, Ask, Why, Show, and Respect) embody the adership contributes to the development of a DevOps culture: way in which Lean leadership contributes to the development of a DevOps culture:
Go See “Senior management must spend time on the plant floor.”
Ask Why “Use the “Why?” technique daily.”
Show Respect “Respect your people.”
Chairman Fujio Cho, Toyota Corporation Copyright © 2018 │
73
Copyright © 2018 | 42
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
s
Ops ure
s of Ops
of a ure
Course Book | DASA DevOps Fundamentals
In a DevOps environment, it is vital that there is a facilitating style of leadership. The management and other leaders must ensure that they support teams in the development of effective behavior and habits. From the Lean style of Leadership, we learn that it is important to see what is happening on the work floor, challenge the way work is done by investigating why activities are carried out the way they are and always deal with people in a respectful way. This way of working encourages the team to take responsibility for their work.
Feedback: A Key Leadership Tool The key tool that leaders should use and stimulate is feedback. Giving and receiving feedback is the basis for all improvement Feedback: A Key Leadership Tool of teamwork. It is vital for both leaders and team and development members to learn and practice giving and receiving feedback in a respectful manner. The key tool that leaders should and stimulate is feedback. Giving and receiving Theuse following figure shows the steps that anyone should take to give feedback is the basis for all improvement and development of teamwork. It is vital for both or receive feedback in an effective manner. leaders and team members to learn and practice giving and receiving feedback in a respectful manner.
The following figure shows the steps that anyone should take to give or receive feedback in an effective manner.
4 Give concrete suggestions OR recognition/incentive
3 Wait and listen to clarify questions
2 Explain what it does to you
To give feedback
1 Describe concrete observations
Check if there is clear understanding
Recognize other
person’s position
Thank him/her Determine whether the feedback is applied
Avoid
discussions or excuses
To receive Listen without interruption feedback
The Feedback Model Copyright © 2018 | 43
An effective feedback helps foster an open atmosphere in which experimentation, teambuilding, and improvement become part of the culture.
g a DevOps Culture There is no standard recipe to build a DevOps culture.
DevOps Culture Cookbook
74 │ Copyright © 2018
Implementation of a DevOps Culture Building a DevOps Culture Creating a DevOps culture is a step-by-step improvement of the four elements: teambuilding, courage, continuous improvement, and leadership. There is no specific order or set of rules that you can follow. Each team has its own development paths. The key is to consider the end result in mind and ensure that the steps the team takes encourage the improvement of the cultural elements. Copyright © 2018 | 45
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
evOps
tals
Module 3 | Culture
Impact of Treating Change as a Program
Impact of Treating Change as a Program Treating change as a program results in failure, as depicted in the following figure. a DevOps Culture
Treating change as a program results in failure, as depicted in the following figure.
ntation of a ps Culture
Ops
s
DevOps Culture
Performance
lements of DevOps
Business as usual
Change program
Business as usual
Change program
Business as usual
Time Source: From the book ‘lean enterprise’, by Jez Humble, Joanne Molesky & Barry O’Reilly
A program has a beginning and an end. When the program ends, the possibilities are that the organization will fall back into its old habits and behaviors. Change needs to come from within an organization where From the book ‘lean enterprise’, by Jez Humble, Joanne Molesky & Barry O’Reilly aSource: culture needs to grow as an integral part when doing business. A change should not be viewed and treated as a separate activity.
Copyright © 2018 | 46
Growing Culture: Experimenting, Measuring, and Probing Growing Culture: Experimenting, Measuring, and Probing
Just like any other product, culture should grow by experimentation,
Just like any other product, culturetheshould growtoby experimentation, measuring, probing, and measuring, probing, and deciding next steps take. Steering the deciding the next steps to the development of culture is a part of daily business. development of culture is take. a partSteering of daily business. The development Theofdevelopment culture is path, not aasstraight-line as shown culture is not aof straight-line shown in thepath, following figure. in the following figure.
ments of DevOps
Desired Target Condition
Performance
ion of a Culture
Experimentation
Desired Target Condition
Experimentation
New Current Condition
Set as ‘Standard’
New Current Condition Set as ‘Standard’
Current Condition Iteration
Iteration
Iteration
Time Source: From the book ‘lean enterprise’, by Jez Humble, Joanne Molesky & Barry O’Reilly
Source: From the book ‘lean enterprise’, by Jez Humble, Joanne Molesky & Barry O’Reilly
Copyright © 2018 | 47
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
75
Course Book | DASA DevOps Fundamentals
As the world is always in a dynamic state, the culture of an organization should be able to adapt to the changing environment. Be aware that apart from continuous experimentation, it is equally important to concentrate on factors that produce results and treat these as a part of next improved version.
Importance of Tracking the Movement Towards a DevOps Culture It is important to track how the organization is progressing towards its goal. Define the behaviors and mindset required in your organization and take necessary steps towards achieving these. The following table shows a sample sheet that can be used to monitor the DevOps culture. Cultural Element
Behavior
Continuous Learning and Improvement
Retrospectives, facilitate knowledge sessions, hackathons, make people responsible
Experimenta tion
Provide time, Instant sandbox environment, remove hurdles, applaud ideas, safe to fail
Build Quality in
Test automation, responsibility for endresult, team knows best, root-cause analysis, process automation (deploy, provision), continuous improvement, transparency (monitors)
An Engineering Culture
Hire people with matching ambitions, every repeated task is automated, support experimentation, support automating manual tasks, not only focus on utilization
A Culture of Effectiveness
Begin with end in mind, ask why? people broaden activities, handover moments avoided, move away from rigid processes, achieve results in small batches, Lean
Strongly Disagree
Disagree
Neutral
76 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Agree
Strongly Agree
Module 3 | Culture
Cultural Element
Behavior
Strongly Disagree
A Culture of Product Thinking
Customer to attend demos, user feedback onto storyboard, allow customers to write about product and respond, organize people around product, team to write blogs about product, you build it, you run it
A Culture of Taking Responsibility
Address derailment openly, let people figure out how to do things, rewards responsibility, reward failure, transparency in what everyone is doing
Disagree
Neutral
Agree
Strongly Agree
Sample Tracking Sheet Remember that developing a DevOps culture is not a straight line. There will always be ups and downs. Even though DevOps advocates stable teams, team members will change. As a result, teams will experience setbacks as a new equilibrium is obtained with every change. Each team should use an overview of intended behaviors to drive and track their development. The preceding table shows an example of how the DevOps culture can be monitored. The team members should complete the table individually accompanied by a healthy discussion. They should then define actions to increase their scores in each of the behaviors they are aiming to embed in the team. They can define new behaviors as the team improves.
amentals um
nce of a DevOps Culture
Key Elements of DevOps
plementation of a DevOps Culture
Changingthe the Culture: A Collective Movement Changing Culture: A Collective Movement
Food for Thought
Think of some ‘To’ characteristics Changing the culture is a collective movement in which behavior, that you believe Changingand the culture is are a collective movement in which behavior, attitude, and mindset are are part of a DevOps attitude, mindset adjusted through feedback. Using feedback, adjusted feedback. Using feedback, can move from its current status team andtoculture. the teamthrough can move from its current statusthe toteam the desired situation, as the desired situation, as shown shown in the following figure.in the following figure. From (Current Culture)
To (DevOps Culture)
Group
Team
Separate technical departments
Integrated multidisciplinary teams
Turf conflicts regarding responsibility
Integral responsibility for a service
Large distance from the customer Specialist roles Individual thinking
Feedback
A DevOps
Close to the customer Generalist roles with specialism Creative Action
Problem avoidance
Fail fast
…
… Copyright © 2018 │
Copyright © 2018 | 49
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
77
Course Book | DASA DevOps Fundamentals
The cultural development of a team (or organization) starts with the end in mind. The key is to describe the “To” situation. The desired situation is, of course, a DevOps culture. The aspects in the “To” column are descriptions of a DevOps team. The “From” column describes the current situation, which might not be exactly what is required. However, it is important to give a true description of the current situation. In fact, this is an opportunity for the team to work on: Giving feedback to each other Understanding and solving the problems
Activity: Group Discussion Activity Time: 20 mins Answer the following questions: What are the key characteristics of a DevOps Culture?
How will you build a DevOps culture?
What challenges do you think you will encounter moving from your current situation to DevOps?
Module Summary In this module, you learned that: Essence of a DevOps Culture Culture is the sum total of behavior and mindset of an organization, supported and enhanced by the values and beliefs of that organization. The core of the DevOps culture is the emphasis on service. A perfect service mindset ensures that a high quality product is not only available but also meets the needs of the customer. The typical cultural aspects of a DevOps team are Continuous Learning, Experimentation, Build Quality in, An Engineering Culture, A Culture of Effectiveness, and A Culture of Product Thinking. Key Elements of DevOps Some of the cultural elements that can help you develop an effective and successful DevOps culture are Teambuilding and Collaboration, Continuous Improvement and Problem-Solving, Courage and Experimentation, and Leadership and Feedback.
78 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 3 | Culture
A team is a small number of people with complementary skills who are committed to a common purpose, set of performance goals, and approach for which they hold themselves mutually responsible. The three key aspects that can motivate anyone are Autonomy, Mastery, and Purpose. Visual Management is one of the strongest tools to stimulate collaboration and ensures that the pitfalls are uncovered. The success key of any team is teamwork, and the most important behavior of teamwork is collaboration. It is essential for organizations to have quality at the source to avoid future issues and the corresponding efforts. Continuous improvement is to solve problems in order to deliver better value faster and cheaper to the customers, bring more meaning to your work, and/or leave a healthier environmental footprint. A well-known method to facilitate continuous improvement is DMAIC. The Kaizen mindset means integrating the three behaviors: Seeing and prioritizing problems, solving the problems, and sharing lessons learned, into the DevOps team. The four actions that a DevOps team can take to ensure their actions are accepted are ensure customer buy-in, define and deliver an MVP, focus on the goal, and take small steps. Organizations should provide safe environment to people to innovate, where they can talk and act without fear of reprisals. Leadership is about taking decisions, acting in accordance with the goals of the team, and being accountable for the actions taken by the team. Central command is to lead a team through detailed instructions. Mission command means leading a team through vision and empowerment. An effective leadership helps DevOps teams to overcome the five barriers: lack of trust, fear of conflict, avoidance of accountability, lack of commitment, inattention to results. Feedback is the key tool that leaders should use and stimulate. Implementation of a DevOps Culture Culture should grow by experimentation, measuring, probing, and deciding the next steps to take. Changing the culture is a collective movement in which behavior, attitude, and mindset are adjusted through feedback.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
79
Course Book | DASA DevOps Fundamentals
Module End Questions Q1. Which aspect of a DevOps Team encourages to utilize the knowledge, skills, and creativity to solve problems, implement product features, and optimize the delivery process? a) Experimentation & Risk Taking b) Build Quality in c) An Engineering Culture d) A Culture of Effectiveness Q2. Which cultural element helps the DevOps teams to overcome the barriers of communication? a) Teambuilding and Collaboration b) Continuous Improvement c) Leadership and Feedback d) Courage and Experimentation Q3. Which characteristic can help you develop a DevOps culture in your organization? a) Integrated multidisciplinary teams b) Separate technical departments c) Specialist roles d) Turf conflicts regarding responsibility Q4. Which phase of the DMAIC cycle includes structuring and visualizing the data into a format and converting it into information? a) Define b) Measure c) Analyze d) Improve
80 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
ps
4 Organization Module Module Objectives Objectives Impact of DevOps on architecture Microservice Architecture
Building a DevOps team
The need for an autonomous team
Difference between Business System teams and Platform teams
Governance within teams, between teams, and between organizations doing DevOps
At the end of this module, you will be able to:
Copyright © 20
Discuss the DevOps organization. {{
Explain the difference between System teams and Platform teams.
{{
Explain why DevOps has a product focus. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
81
Course Book | DASA DevOps Fundamentals
Explain the need for an autonomous team. {{
Why autonomous teams?
{{
What is required to make teams autonomous?
Explain the impact of DevOps on Architecture. {{
Building (other) qualities in simultaneously.
{{
Why use microservice architecture?
{{
How to build systematic resilience?
Discuss the governance. {{
Within DevOps teams.
{{
Between DevOps teams.
{{
Between organizations doing DevOps.
Module Topics Organizational Model Autonomous Teams Architecting for DevOps Governance
DevOps
mentals m
Organizational Model
Impact of DevOps on the Organization Impact of DevOps on the Organization
zational Model
omous Teams
Architecting for DevOps Governance
TheThe Organizational model refers to your business structure or the way Organizational model refers to your business the business or organization is structured in departments, units, and structure or the way the business or organization teams.
82 │ Copyright © 2018
is structured in departments, units, and teams.
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 4 | Organization
An organizational model describes how the organization is structured. In an organizational model, people spread across all over an organization in technically-oriented silos, which result in communication problems. However, DevOps organizations aim to work with an organizational model that facilitates collaboration between people to provide the customer with the best products and services. In traditional organizational models used within IT, there is a substantial need for coordination overhead in the form of Service Delivery Managers and Project Managers responsible for bringing people together to create a value to the customer. On the other hand, the DevOps organizational model focuses on a single (mostly colocated) team doing the work, hence, reducing communication and coordination issues. In the traditional organization, there is often one department responsible for running the applications (operations) and one department for developing the applications (development).
Organizational and Operational ‘Organizational’ refers to your business structure, while ‘operational’ refers to how you get things done.
The operational model describes how work gets done. In the traditional IT organization, the operational model is dependent on coordination roles, which ensure the people in technical silos are brought together to deliver a service. The operational model of a DevOps IT organization focuses on delivering work through autonomous teams containing the knowledge required to develop, deliver, and maintain the IT service. These are two distinctly different operational models. The organizational model and the operational model are closely related but distinct entities. Both need to be addressed when organizing DevOps teams. For example, it is conceivable to describe a DevOps environment in which the Project Manager has an important role in realizing the IT service (not a preferred solution). This means the organizational model can have autonomous teams, but the operational model is project-based.
Aligning Organizational Model with IT Services: Crucial for IT Services Many organizations are not aligned with the way the IT services are structured and organized, with multiple or distributed teams working on producing and delivering the same IT service. Lots of different people in different departments need to cooperate closely to deliver IT services. These people and departments have different priorities, goals, time frames, and cultures, making successful cooperation difficult. Melvin E. Conway described these issues in Conway’s law as: Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.
Melvin Conway Melvin Edward Conway is a computer scientist, computer programmer, and hacker.
Source: Conway, Melvin E. (April 1968)
The same law applies to the design and delivery of IT services. Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 │
83
Course Book | DASA DevOps Fundamentals
The Conway’s law, in other words, means that complex organizations ends up with complicated and convoluted systems. To overcome the problems, organizations have been embracing new organizational models. Companies, such as Netflix and Amazon, organize themselves around multiple small teams that are responsible for (part of a) services. These teams are set up to be autonomous and independent and own the entire lifecycle of the services they create. The features of such teams help them to achieve a greater degree of autonomy than is possible for larger or multiple teams with responsibilities for large complex services. To aid the autonomy of the teams, the small services are designed with the ability to be changed independently of other services, resulting in the ability to deliver changes faster. The siloed organizational structure and the design of larger services do not provide the ability for teams to do experiments. In addition, the structure and design do not allow fast and reliable changes, resulting in less happy customers.
Needs all Knowledge DASA DevOpsand Skills Fundamentals Premium Consider a typical IT organization divided into departments. Each department is Organizational aligned Model with a specialty or profession. You might see separate departments Autonomous Teams with network specialists, storage Architecting for specialists, Unix administrators, DevOps VMware administrators, DBAs, Governance application managers and others. In such an organization, each application and system depend on all other departments to function correctly.
Organization Model: Taking DevOps “Literally” is No Solution Organization Model: Taking DevOps “Literally” is No Solution
To create independent and autonomous teams around services, you might be tempted to build teams with full stack expertise as depicted To in create and autonomous teams services, you might be tempted to theindependent following figure. Although thisaround might be a better setup, it takes build teams with full stack expertise as depicted in the following figure. Although this might approach too literally. Each needs allneeds knowledge be DevOps a better setup, it takes DevOps approach too team, literally. now, Each team, now, all knowledge and skills, reuseisisalmost almost zero. and skills, andand reuse zero.
PO, Storage Developers, Testers, DBA’s, Sysadmins, Network
PO, Storage Developers, Testers, DBA’s, Sysadmins, Network
PO, Storage Developers, Testers, DBA’s, Sysadmins, Network
Copyright © 2018 | 6
DevOps is often misunderstood and taken too literally. Many organizations stay in the existing organizational chart with fixed teams and experts in middleware, databases, network and others. In addition, Business System teams are created vertically next to this setup, as shown in the preceding figure. Such a setup requires many dependencies and leads to “waste” due to which various Business System teams cannot operate completely independent from each other. They need the centralized capabilities as depicted on the left side of the preceding figure. Consider a typical IT organization divided into departments. Each department is aligned with a specialty or profession. You might see
84 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 4 | Organization
separate departments with network specialists, storage specialists, Unix administrators, VMware administrators, DBAs, application managers and others. In such an organization, each application and system depend on all other departments to function correctly. Suppose a new machine or environment is required in the organization, you have to deal with the representative of every department and orchestrate the change requests among them. Each department has its own request forms, workload, and SLAs. Every time a request is handed over to the next department for processing, you have to wait until the request is accepted, planned, and processed. Valuable time is wasted while waiting for the next processing step. It has been found that waiting times can accumulate to 98% of the complete turnaround time of a change.
Organizational Model: The Practical Approach Organizational Model: The Practical Approach
The current preferred way of organizing DevOps teams is to enable The current preferred wayBusiness of organizing DevOps is to enable autonomous autonomous System teamsteams that can “land” their application Business System teams that can “land” their andthat infrastructure code a platform that is and infrastructure codeapplication on a platform is maintained by on a Platform maintained byteam. a Platform team. However, such a self-service platform is a product However, such a self-service platform is a product in itself and in itself and needs maintenance, innovation, and ownership. needs maintenance, innovation, and ownership.
Business System Teams
PaaS IaaS Platform Team
In this model, the infrastructure reuse benefits are largest, without impeding the speed and autonomy of the various Business System teams. It also enables the Business System team to take full responsibility for their service during its complete lifecycle.
Copyright © 2018 | 7
Business System teams manage end users (for example, customer), services, and products (for example, applications). On the other hand, Platform teams manage platform products used by Business System teams. Platform teams do not take over the responsibility of the service offered by Business System teams. They offer platform (or Infrastructure) services through self-service and automation to make the Business System teams work more effectively. This way prevents Business System teams from waiting for their request to be dealt with by the Platform team. However, both teams need to work closely together to ensure the services provided by the Platform team are the correct ones. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
85
Course Book | DASA DevOps Fundamentals
DevOps
entals m
Activity-Focused vs. Product-Focused Activity-Focused vs. Product-Focused ‘Activity-Focused’ (Siloed)
‘Product-Focused’ (Team)
ational Model
omous Teams
rchitecting for DevOps
Governance
Specialty Oriented Functionally Organized Project Focused Work with Individuals Optimized for Resource Utilization
Work Oriented Team Organized Product Focused Work with Teams Optimized for Speed
Features of product-focused approach:
Features of product-focused approach:
Responsibility of product-team extended all the way into production.
Responsibility of product-team extended all the way into
All are responsible and production. accountable for a fully working product.
All are responsible and accountable for a fully working product. The characteristics of an activity-focused organization include: Resources are specialty oriented. In other words, resources perform one specific task in a chain of events at a time, such as updating a database. Resources are functionally organized implying resources are added to specific resource pools reflecting specialisms, for example, a pool of database administrators. Resources work on projects with a beginning and an end, and resources can be assigned to multiple projects at once. The organization works with individuals who are seen to be interchangeable. This type of organization is optimized for resource utilization. The characteristics of a product-focused organization include: Resources are oriented on delivering work requiring multiple skills, such as a developer who creates test fixtures in code also knows how to automate a delivery process. The resources are organized in a team responsible for a product that it delivers and/or maintains. The team is related to the product throughout its entire lifecycle. The applicable ideology being “you build it, you run it”.
86 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 | 8
ps
odel
ams
Module 4 | Organization
This type of organization works with a team who is solely responsible for a running product. This type of organization is optimized for speed/lowering cycletime for a product. Siloed organizations have resources grouped on specialisms and can be optimized from a resource efficiency perspective but not from a process throughput (or flow) perspective. Product-focused (or Agile) teams are multidisciplinary in nature where the entire team has all disciplines to bring a product to production and keep it working in production. By having all disciplines working in one team focused on the end product, expensive handover moments are reduced to a minimum and process throughput is further optimized. Remember, for a DevOps organization, it is all about speed and getting features to the customer as soon as possible. Such a speedy delivery of products helps organizations to use valuable feedback loops to determine DevOps whether Organigram they are on the right track.
DevOps Organigram DevOps teams cannot reach their full potential if the transition towards DevOps DevOps teams cannot reach their full potential if the transition towards organization is not implemented and accepted throughout the entire organization. The DevOps organization is not implemented and accepted throughout following figure shows how a DevOps organization is structured to accept the change at the entire organization. The following figure shows how a DevOps eachorganization level. is structured to accept the change at each level.
The Hierarchy of Importance: Customers on Top and Centering on the Teams that Add Value The Hierarchy of Importance: Customers on Top and Centering on the Teams that Add Value
g for Ops
Customers
ance
Service Team (Business System Team)
Platform Team Finance
HR
PMO
Legal
Management Copyright © 2018 | 9
In a DevOps organization, the introduction of Business System teams (or Product teams) and Platform teams changes the role of other departments, such as Project Management Office (PMO), HR, Legal, Finance, and Purchasing and Contract Management. These functions should provide the constraints, such as financial and legal, within which the teams should operate. The functions also offer services, such as training, licences, and contracted partners and vendors. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
87
Course Book | DASA DevOps Fundamentals
When an IT organization chooses to start using DevOps as its modus operandi, the supporting departments should ensure their products and services change to support the goals of the DevOps IT organization. In other words, their delivery processes should not interfere with the flow of the delivery of value by DevOps teams. The organization can do this by creating new ways of budgeting, new styles of HR management, or new contracts for ensuring DevOps teams can work with contracted suppliers.
A DevOps
mentals m
Autonomous Teams What is autonomy? What is autonomy? Autonomy is independence or freedom. You can relate it to a self-governing community, which is not subjected to control from the outside world. You can relate it to a selfAutonomy is independence or freedom. governing community, which is not subjected to control from the outside world.
izational Model
nomous Teams
Architecting for DevOps Governance
Autonomy leads to self-directed action, and this is what exactly a DevOps team needs. Autonomy is so important that it leads drivestofollowing three action, principles theissixwhat DevOps Autonomy self-directed andofthis exactly a principles: DevOps team needs. Autonomy is so important that it drives following three principles of the six DevOps principles: End-to-End Responsibility
Customer-Centric Action
Cross-Functional Autonomous Teams
Copyright © 2018 | 11
End-to-End Responsibility: Team’s autonomy is necessary for it to be able to take and feel the responsibility for the customer’s services. Customer-Centric Action: Without autonomy, the team’s ability to focus on delivering what customers want will be compromised. Cross-Functional Autonomous Teams: Autonomy is an integral part of cross-functional autonomous teams.
88 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 4 | Organization
evOps
Autonomy of Teams
ntals
Autonomy of Teams
Autonomy of teams is supported by the ability to operate as independent
ional Model
Autonomy teams is supported the ability to operate asupindependent teams. The way to teams. The of way to achieve autonomybythrough DevOps is setting achieve autonomy through DevOps is setting up teams (Business System teams) structured teams (Business System teams) structured around distinct services around distinct andfollowing products, as shown in the following figure. and products, as services shown in the figure.
ous Teams
Business System Teams
hitecting for DevOps
Governance
Self-service to enable autonomy! Monitoring
Deployment
Logging
Service Discovery Security
Control
Auto-Healing
Backup and Recovery
Auto-Update
Platform Services
Platform Team Copyright © 2018 | 12
The adoption of Continuous Delivery across the organization is required to make the teams truly independent (refer Module 6). Some of the characteristics of such independent teams are: The teams have an end-to-end responsibility. There is no handover or transfer of responsibility and accountability. The teams operate (largely) autonomously and work (largely) independently from one another to deliver a continuous stream of (software) change. Interfaces between Business System teams and Platform teams are clearly defined through Application Programming Interfaces (APIs). APIs are used to define the way a computer program communicates with another computer program. The Platform teams offer a rich set of standardized self-service capabilities to Business System teams, such as logging, monitoring, deployment, backup, recovery and many more. Such a set of capabilities/services allows Business System teams to completely manage their (software) products. The Business System teams are the customers of the Platform teams. They use the products of the Platform teams. These products can be used as fully automated self-services. These services are not based on a service ticketing system combined with manual service request fulfillment as such systems usually come with long waiting times. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
89
l
s
r s
e
Course Book | DASA DevOps Fundamentals
The Platform teams are responsible for the qualities of their platform products, such as availability and performance. The Business System teams are responsible for the qualities of their (application) products, such as availability and performance. The system qualities of the platform products can be monitored and managed independently from the availability of the application products, which use the platform products.
Criteria for Autonomous Teams
Criteria for Autonomous Teams In DevOps, a Business System team is responsible for the ‘health’ of
In DevOps, a Business System team is the responsible thesetting ‘health’ of a aservice. a service. One of crucial tasksfor when up such team is toOne of the crucial tasks when settingscope up such a team is of to the scope responsibility of the team, which the responsibility team,the which is crucial for creating the required autonomy of the team. Without the insight on the customer, is crucial for creating the required autonomy of the team. Without the insight on the stack, and required knowledge, it is to difficult to set an customer, technical stack,technical and required knowledge, it is difficult set up an up autonomous autonomous Business System or Platform team. Business System or Platform team. Three Basic Design Criteria for an Autonomous Team
Customer Responsible for the business process People who use the service Technology Stack The team’s area of responsibility
Knowledge The knowledge and skills the team require Copyright © 2018
To determine the autonomy of a team, the organizations should address the following three basic design criteria: Customer: Putting the customers first is providing permanent IT services. It should be clear “Who is the customer of the team?” and “What services are required by the customer?” Such questions require plenary analysis and clear definitions of both the ‘customer’ and the ‘service to be delivered’. Identifying a customer or customer group makes it simpler to determine the value to be delivered.
Technology Stack Technology stack refers to a set of software that provides the infrastructure for a computer.
Technology Stack: Based on the basic characteristics of the customer and the service to be delivered, the technology stack for which the team is responsible can be determined. The ‘deeper’ the technology stack, the more knowledge and skills the team requires. In this regard, the rise of cloud technology
90 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 4 | Organization
with its different variants (IaaS, PaaS, and SaaS) does make it easier to decouple the team from the underlying technology. Knowledge: The next step is to assess the necessary skills and knowledge that need to be present. The distribution of these skills and knowledge might differ from team to team and depends on the customer and the technology stack. However, each team needs to ensure there is enough of each skill and knowledge area to ensure the service is delivered as required by the customers of the service. The estimated units-of-work, such as service requests, customer demand, incidents, changes, and problems, will determine the number of people required within the team. Therefore, at a minimum, the following aspects should be clearly defined for a DevOps team:
DevOps
{{
Who is the customer?
{{
What service is to be delivered?
{{
What are the units-of-work, the team will be processing?
{{
What is the ‘Definition of Done (DoD)’ for the units-of-work?
Decoupling Point: A Key A Consideration for Autonomous Decoupling Point: Key Consideration for Autonomous Teams
entals m
ational Model
Teams
To enable fully autonomous Business System teams that can “land” To enable fully autonomous Business teamsmaintained that can “land” their application and infrastructure codeSystem on a platform by their application and infrastructure code on a platform maintained by a Platform team, you need to establish the a Platform team, you need to establish the decoupling point. Such a decoupling point. Such a point within the technology stack, as shown in the following figure, point within the technology stack, as shown in the following figure, is a is a major factor in establishing autonomy for the Business System teams. major factor in establishing autonomy for the Business System teams.
mous Teams
rchitecting for DevOps
Business System Teams
Governance
Platform Team Copyright © 2018 | 14
You have learnt that DevOps is about autonomous, multi-disciplinary teams providing services to its customer. However, taking this to its extreme causes problems in the form of inefficiencies. Therefore, the currently preferred configuration of DevOps organizations is to have Business System teams and Platform teams. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
91
Course Book | DASA DevOps Fundamentals
The key question is, of course, where does one start and the other end? We need to define what is known as the decoupling point. The decoupling point defines the point within the technology stack where the responsibility is transferred between the Business System team and the Platform team. The Platform team should provide a platform that is a self-service product on its own. This self-service product, however, requires maintenance, innovation, and ownership. In such a model, the Business System team can reuse the infrastructure without impeding their speed and autonomy. The model also enables the Business System team to take full responsibility for their service during its complete lifecycle. Two possible examples of a decoupling point are: The business system teams can request virtual machines from the platform team and configure it to their needs. This gives them a lot of freedom as they can decide on almost anything.
DASA DevOps Fundamentals Premium
Organizational Model
Autonomous Teams
The business system teams deploy their application artifact on a platform supplied by the platform team or cloud provider. They are restricted to the technology that the platform supports.
Solving the Autonomy Problems
Solving the Autonomy Problems Spotifyexample is a great example for autonomous teams. With ofthe Spotify is a great for autonomous teams. With the establishment autonomous establishment onembrace specific the (setDevOps of) teams focused on specific of (setautonomous of) services,teams Spotifyfocused is able to principles ofservices, a truly customer-centric end-to-end responsibility, and the ability to Spotify is able toaction, embrace the DevOps principles of a truly experiment customer-centric and innovate fast.action, As a result, a proper feedback loop end-to-end responsibility, and is theestablished, ability to enabling these teamsexperiment to learn faster better. and and innovate fast. As a result, a proper feedback loop is established, enabling these teams to learn faster and better.
Architecting for DevOps Governance
Tribe
Guild
Chapter
Squad
Source: Spotify logo from Spotify.com
Spotify has organized its employees in teams called Squads, who are free to operate autonomously. Each Squad is grouped together with other Squads that do related tasks, and together they are called a Tribe. Each Tribe has a specific responsibility for a part of the Spotify services, for example, developing the front-end search functionality in the Spotify app or part of the back-end billing structure.
Source: Spotify logo from Spotify.com
Employees within different Tribes doing similar work or using similar skills and knowledge, learn from each other through the use of Chapters and Guilds.
92 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018
Module 4 | Organization
Spotify is still in the process of making continuous improvements to ensure things evolve. By the time you read this section, things might get changed in Spotify. Nonetheless, it still exemplifies the organization aiming the autonomy of teams.
Squad: Defining the Scope of a Team The Business System teams are called Squads at Spotify. Characteristics of Squads Are multidisciplinary and autonomous Are small and self-organizing Sit together and have required skills and tools Use the slogan “Think it, build it, ship it, tweak it” Apply Lean Startup principles, such as MVP and Validated Learning
pe of a Team
Provide support throughout the entire lifecycle of a product, from the idea to production Focus on close collaboration
alled Squads at Spotify.
Apply Lean principles
omous
,
arning
m Spotify services: onmobile mobile devices Spotify services:Used Used on devices and and the Spotify website. Separate parts the Spotify website. Separate parts of the are are the responsibility of separate of theinterface interface the responsibility of Squads. separate Squads.
(2012) Copyright © 2018 | 16
Source: Scaling Agile @ Spotify Henrik Kniberg & Anders Ivarsson (2012)
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
93
ps
Course Book | DASA DevOps Fundamentals
The teams at Spotify are called Squads and these teams are small, multidisciplinary, and autonomous. Each Squad is similar to the Business System team and is designed to feel like a mini-startup. Squads sit together and they focus on specific services. They incorporate every component which is necessary to support a service through its entire lifecycle, from idea to production. Squads decide on their own way of collaborating and are stimulated to apply Lean Startup principles, such as Minimal Viable Product (MVP) and Validated Learning. The slogan used is “Think it, build it, ship it, tweak it”.
Tribe: Organizing Multiple Related Squads Tribe: Organizing Multiple Related Squads
A Tribe is a collection of Squads that work in related areas, as shown
in the that following A Tribe is a collection of Squads workfigure. in related areas, as shown in the following figure.
odel
ams
g for Ops
Music Player
Mobile
Store
Search
Each Tribe: Demos Get together
nce
Subscription
Recommendations
Accounting
Source: https://ucvox.files.wordpress.com/2012/11/113617905-scaling-agile-spotify-11.pdf
Source: Scaling Agile @ Spotify Henrik Kniberg & Anders Ivarsson (2012)
Copyright © 2018 | 17
At Spotify, a Tribe can be the Music Player or backend infrastructure. At a bank, it might be for payments, risk, or clearing. The Tribes function as a sort of a startup company and have a fair degree of freedom and autonomy. A Tribe Lead heads a Tribe who is responsible for providing the best possible habitat for the Squads within that Tribe. The Squads in a Tribe are all physically present in the same office, normally right next to each other, and the lounge areas nearby promote collaboration between the Squads. Tribes hold gatherings on a regular basis. In addition, they have an informal get together to discuss with the rest of the team what are they working on, what have they delivered, and what others can learn from what are they currently doing. Such a get together includes live demos of working software, new tools and techniques, and others.
94 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 4 | Organization
SA DevOps
damentals mium
Chapter: Maintainingand and Developing Standards Chapter: Maintaining Developing Standards The Chapter is all about maintaining, developing, and improving
The Chapterin is a allparticular about maintaining, standards area of developing, expertise. and improving standards in a particular area of expertise.
ganizational Model
utonomous Teams Architecting for DevOps Governance
Source: https://ucvox.files.wordpress.com/2012/11/113617905-scaling-agile-spotify-11.pdf
Source: Scaling Agile @ Spotify Henrik Kniberg & Anders Ivarsson (2012)
Copyright © 2018 | 18
A Chapter is a small family of people having similar skills who work within the same general competency area and the same Tribe. These people come together to ensure their way of working can be discussed and improved, or at least standardized, with colleagues learning best practices from one another. An example of a Chapter can be a group of testers from different Squads coming together on a regular basis to discuss or show different testing frameworks and/or test automation tooling. Another example can be a Chapter of developers, who get together on a regular basis to discuss libraries, frameworks, or development practices.
DevOps
Guild: Community of Interest Guild: Community of Interest
mentals m
Guilds help share knowledge across the organization. These provide
Guilds help share knowledge across the organization. These provide some economies of some economies of scale withoutofsacrificing scale without sacrificing the autonomy Squads. the autonomy of Squads.
zational Model
omous Teams
Architecting for DevOps Governance
Source: https://ucvox.files.wordpress.com/2012/11/113617905-scaling-agile-spotify-11.pdf
Source: Scaling Agile @ Spotify Henrik Kniberg & Anders Ivarsson (2012)
Copyright © 2018 | 19
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
95
Course Book | DASA DevOps Fundamentals
Guilds act as a community of people who share knowledge, tools, code, and practices. Unlike Chapters that are always located in Tribes, Guilds spread across the entire organization. Some of the examples of Guilds are Developer Guild, Tester Guild, and Agile Coach Guild.
vOps
Spotify Model: Measuring all Squads
als
Spotify Model: Measuring all Squads With many (mainly 30 plus size) teams, it becomes really challenging
nal Model
With many (mainlyfor30Spotify plus tosize) teams, it becomes really challenging for Spotify to enhance the skills and performance on a continuous basis. To aidon in this, there are quarterly surveys for all Squads. The the skills and performance a continuous basis. To aid inthe this, there are quarterly image shows the result of one survey for five Squads for all the Squads.following The following image shows thesuch result of one such survey for five grouped together within a Tribe. grouped together within a Tribe.
us Teams
ecting for DevOps
vernance
The circles show the status The current circles show the current status
Click here for the enlarged view.
The arrows The arrows show the trend show the trend
Source: Scaling Agile @ Spotify Henrik Kniberg & Anders Ivarsson (2012)
Source: https://ucvox.files.wordpress.com/2012/11/113617905-scaling-agile-spotify-11.pdf
The aim of the surveys is to help with continuous improvement by focusing on what sort of support is required. The areas within the survey are: Product owner: The Squad has a dedicated Product Owner who prioritizes the work and takes both business value and technical aspects into consideration. Agile coach: The Squad has an Agile Coach who helps the Squad to identify impediments and coaches them to continuously improve their process. Influencing work: Each Squad member can influence his/her work, be an active part in planning, and choose which tasks to work on. Each member can spend 10% of his/her time on hack days. Easy to release: The Squad can (and does!) get stuff live with minimal hassle and synchronization. Process that fits team: The Squad feels ownership of their process and continuously improves it.
96 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 4 | Organization
A mission: The Squad has a mission that everyone knows and cares about, and stories on the backlog are related to the mission. Organization support: The Squad knows where to turn to for problem-solving support, for technical issues as well as ‘soft’ issues.
Activity: Group Discussion Activity Time: 15 mins Imagine you are working in a company, such as Spotify, offering successful Web services to millions of users. You are part of one of the Platform teams that supports multiple Business System teams in developing and supporting all kinds of services. What will be your desired decoupling point?
The AimArchitecting of IT Architecture for DevOps The Aim of IT Architecture
The aim of IT architecture is to support functional requirements with non-functional or quality requirements. The aim of IT architecture is to support functional requirements with
non-functional or quality requirements. Non - Functional Requirements
Autonomy Resiliency
Performance
Scalability
Functional Requirements and Non-functional Requirements A functional requirement describes what a system should do, while non-functional requirements place constraints on how the system will do so.
Maintainability Testability
Security
Reliability
Functional requirements are associated with the tasks a systemCopyright © 2018 | 23 should do. Most people focus on functional requirements as these are related to the expected functioning of the product or service. On the other hand, non-functional requirements refer to the attributes of a service, such as autonomy, resiliency, maintainability, testability, scalability, and reliability. When such attributes are overlooked or not properly implemented, the services fail or do not function as expected. For example, a service might calculate payments in an insecure, slow, or difficult to use way resulting in mistakes due to frustration or misuse. Therefore, proper and timely addressing of non-functional requirements is crucial for the overall quality of services. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
97
Course Book | DASA DevOps Fundamentals
ASA DevOps
Building Qualities
ndamentals emium
Building Qualities
Organizational Model
Building (other) qualities in simultaneously with the functional requirements is a major (other) qualities in simultaneously with the functional enabler of faster and Building better IT services. requirements is a major enabler of faster and better IT services.
Model
eams
ng for evOps
Quality first means adding functionality simultaneously with non-functional requirements and making sure the quality is perfect.
Other Qualities (%)
Architecting for DevOps
ps
Quality First
100
Autonomous Teams
Governance
Addresses implementing non-functionality as an afterthought, only when required and so far as resources are available. Usually done in a separate organizational silo.
Functionality
Source: Practice to perfect, the quality first model, Bertrand Meyer, 1997 Source: Practice to perfect, the quality first model, Bertrand Meyer, 1997
Tips “Quality in a service or product is not what you put into it. It is what the client or customer gets out of it.” - Peter Drucker
Tips
Smaller Services
Traditional Approach: In large traditional programs, development Copyright © 2018 | 24 teams tend to first focus on delivering functional requirements only. However, non-functional requirements, such as stability, reliability, and maintainability, are delivered at a later stage. The approach results in a functional (often unstable) system first, where bugs are taken care of at a later stage. Quality First: When delivering a new value to a product in a continuous manner, or when applying practices related to Continuous Delivery, system quality needs to build indirectly into the system. If this is not done, ‘broken’ features are added to the functional end product on a continuous basis, which will then break the operational system as a whole.
An Enterprise Service Bus (ESB) is fundamentally an architecture which Smaller Services is Using a set of only rules smaller and principles for services Using increases complexity, which in turn, complexity, results in lower only smaller services increases whichquality, in turn, results integrating applications slower, numerous worse, and more expensive IT services. together over a bus-like infrastructure. in lower quality, slower, worse, and more expensive IT services.
From …
To…
Complex Architecture
Smaller Services, More Complexity, Lower Quality
UI
UI
UI
Business Services
nance ESB
BPH
Integrator Services
SAP
Siebel
DWH
Slow, Resource Hungry, Expensive
Slower, Worse, More Expensive
98 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 | 25
Module 4 | Organization
DevOps
For many years, the trend is to reduce the size of IT services. Unfortunately, it does not always lead to a better architecture with fewer issues. In fact, a complex architecture that is transformed to an architecture with smaller services might be harder to manage and lead to lower quality. It is seen that this happens with the use of the Enterprise Service Bus. The idea is to create services that are small but can independently carry out a business function, without impacting other business functions if it does not work.
Relation Between Complexity and Quality
Relation Between Complexity and Quality
entals m
The following graphs shows how complexity increases with rise The following how complexity rise ininnumber of services and in number of graphs servicesshows and how quality fallsincreases with this with increase how quality falls with this increase in complexity. complexity. Any system quality
ational Model
rchitecting for DevOps Governance
Complexity
omous Teams
Complexity
# Services
Ops
s
When the number of services grows, the complexity also grows.
When complexity grows, quality of the system, such as reliability, maintainability, or scalability, falls down.
Conway’s Law and Organizations’ Architecture
Copyright © 2018 | 26
Conway’s Law and Organizations’ Architecture
Use the force, Conway’s law cannot be broken.
l Model
Teams
cting for DevOps
ernance
Organizations influences architecture. Source: agileforall.com Copyright © 2018 │
Source: agileforall.com
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
99
Copyright © 2018 | 27
Ops
ls
al Model
s Teams
cting for DevOps
Course Book | DASA DevOps Fundamentals
Conway’s law states that any piece of software reflects the organizational structure that produced it. The law is difficult to break. It implies that complex organizations are bound to create complex architectures. If an organization wants a simple architecture, it first needs to organize itself in a simple manner. As there is a relation between system complexity and system qualities, you can improve system qualities by reducing complexity. One of the ways to reduce the complexity of the technical solution is to reduce the complexity of the organizational structure.
Microservices Architecture Microservices Architecture The use of Microservice Architecture (MSA) fits today’s overall software engineering goals:fits faster, better, andsoftware cheaper.engineering MSA fits thisgoals: trend as The use of Microservice Architecture (MSA) today’s overall is a fits logical designstructure of a software program faster, better, and cheaper. itMSA thisstructure trend as for it isthe a logical for the design involving of a loosely-coupled modular components known as microservices. software program involving loosely-coupled modular components known as microservices.
From …
To…
Complex Architecture
MSA
UI
UI
UI
Business Services
ernance
ESB
BPH
Integrator Services
SAP
Siebel
DWH
Slow, Resource Hungry, Expensive
Faster, better, cheaper Copyright © 2018 | 28
A few decades ago, delivering a system over a few years time, was a common practice. IT was time and resource hungry. Nowadays, overall software engineering goals are able to deliver better systems, faster with less effort over and over again! If you want to win the competitive app market space, you need to be quick and able to deliver the product with relatively low cost. Many trends in the software development support the goal of creating better software, faster and cheaper, such as: Agile Organizations: Dedicated teams over resourcing, products over projects, prioritization over planning, and outcome over output Continuous Delivery: Cycle time measured in hours or even minutes Autonomous Teams: You build it, you run it, shared nothing is more important than aiming to deliver the best quality
100 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 4 | Organization
Reactive Manifesto: Responsive, resilient, scalable, and loosely-coupled (message-driven) systems that are easy to develop and change Platform as a Service (PaaS): Cheap, easy, and fast runtime environments for apps Examples of Microservices: “Examples include ordering a book in a web shop. This process would start with the selection of a book and end with creating an order. Actually fulfilling the order is a different process that lives in its own service. The fulfillment process might run right after the order process but it doesn’t have to. If the customer orders a PDF version of a book order fulfillment may be completed right away. If the order was for the print version, all the order service can promise is to ask shipping to send the book. Separating these two processes in different services allows us to make choices about the way the process is completed, making sure that a problem or delay in one service has no impact on other services.” Source: http://blog.xebia.com/micro-services-architecture-principle-1-each-microservice-delivers-a-single-complete-business-capability/
MSA Supports Faster, Cheaper, Better Software Development There are many common guidelines for MSA, but on a high level the most important ones are: Autonomous Systems:
ster, Cheaper, Better Software Development Ability to deliver business value, independent {{
services
{{
of other
End-to-end from data storage to user interface functions
guidelines for MSA, but on a high level the most important ones Simplicity:
:
{{
Minimum number of components and interactions
{{ Asother simple as possible, not simpler ness value, independent of services Low functions Coupling/High Cohesion: storage to user interface {{
Low coupling between services
{{ High cohesion within services components and interactions
e, not simpler
ohesion:
n services
services
Business Architecture Technical/Service Architecture Data Architecture
Copyright © 2018 | 29
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 │
101
Course Book | DASA DevOps Fundamentals
MSA: but No Definition, but Certain Common Characteristics MSA: No Definition, Certain Common Characteristics
Instead of using just smaller services and the complexity that comes with services it, MSA should used. It is an approach developing single Instead of using just smaller andbethe complexity that to comes witha it, MSA should b application as a suite of small services, each running in its own used. It is an approach to developing a single application as a suite of small services, each process and communicating with lightweight mechanisms. While running in its own processthere andiscommunicating with mechanisms. While there i no precise definition of lightweight this architectural style, there are precise definition of this architectural there are certain aroun certain commonstyle, characteristics around the common organization,characteristics business capability, automated deployment, intelligence in the endpoints, and organization, business capability, automated deployment, intelligence in the endpoints, an decentralizedand control of languages and data. decentralized control of languages data. Componentization via Services
Organized Around Business Capabilities
Smart Endpoints and Dumb Pipes
Products not Projects
Decentralized Governance
Infrastructure Automation
Decentralized Data Management
Design for Failure Tolerate
Evolutionary Design
Common Characteristics of MSA Componentization via Services: deployable, cloud-ready, and scalable.
C
It
is
independently
Organized Around Business Capabilities: Organizations are organized around business capabilities (MSAs) and use Melvin Conway’s law. Products not Projects: The characteristic appreciates “You build it, you run it.” This involves taking full responsibility. Smart Endpoints and Dumb Pipes: These are simple interfaces having no logic in between, such as the Enterprise Service Bus. Decentralized Governance: There is no standardization on a single technology platform. Decentralized Data Management: Transactions help with consistency but imposes temporal coupling.
102 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 4 | Organization
Infrastructure Automation: It ensures everything works through automated tests and automated deployments. Design for Failure Tolerate: Any service call can fail due to unavailability of the supplier. Therefore, the client has to respond to this as gracefully as possible, detect the failures quickly, and restore the service (auto). Evolutionary Design: The design supports independent replacement and upgradeability.
DevOps
Architecting for Systemic Resilience
entals m
Architecting for Systemic Resilience
ational Model
Resilience means preparing systems, processes, and people for the Resilience means preparing systems, processes, and people for the unexpected events. unexpected events. Therefore, the focus is on preventing errors and Therefore, the focus is on preventing errors and automated recovery actions, as shown in automated recovery actions, as shown in the following figure.
the following figure.
Is the deployment process fully automated and does it prevent configuration errors?
omous Teams
/user Guardian
rchitecting for DevOps Governance
D
A
Prod 1,2, & 3
Do we have a Plan B? MyApp v 1.x v 2.x v 3.x ...
QA 1 & 2
C
B
F
The Reaper
H
Are individual components able to repair themselves or take over tasks in order to contribute to the overall stability?
Dev 1
To what extent does the management and provisioning of the automated environments avoid mistakes?
E
G
Continuous Integration Test Automation Pipeline
To what extent is the testing process automated? Copyright © 2018 | 31
Some portion of your system is likely to go down at some point. Therefore, resilient systems are designed with that expectation. Resilient systems and resilient processes are able to continue operation, though at diminished capacity, in the face of failure. For system resilience and high availability, while keeping the quality requirements for fast and reliable changes, there are no silver bullets. The overall performance comes from multiple areas. Therefore, you should ask a number of questions, such as: How are the non-functional requirements included in the development process so that availability is incorporated into the design? To what extent, we are able to ensure the software architecture with high availability, regardless of the availability of the underlying infrastructural investments?
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
103
Course Book | DASA DevOps Fundamentals
How good is the infrastructure supporting availability and scalability? How good is the software production process in avoiding mistakes? To what extent is the testing process automated? What fallback scenarios are functionality is out of service?
available
when
essential
How do the Software Development teams manage continuity and availability? How fast and reliable is the software delivery process? Is the deployment process fully automated? Does it keep configuration errors out? To what extent are the management and provisioning of environments automated to avoid and prevent mistakes? To what extent are individual components able to repair themselves or take over tasks in order to contribute to the overall stability? Netflix gained world recognition when the company broadcasted details of its Simian Army work in 2010 and 2011. Through the automated efforts of Chaos Monkey, Chaos Gorilla, and a slew of other similar utilities, failure is simulated to develop more resilient processes, tools, and capabilities. John Ciancutti of Netflix writes, “If we aren’t constantly testing our ability to succeed despite failure, then it isn’t likely to work when it matters most — in the event of an unexpected outage”. The Chaos Monkey’s job is to randomly kill instances and services within the architecture. Chaos Monkey is a service which randomly terminates instances. Failures inevitably happen when least desired or expected. If your application cannot tolerate an instance, you will find incidents of failures at odd hours, for example, by being paged at 3 am or while you are having your morning coffee.
Moving from Legacy to Smaller Services Legacy systems, in the context of computing, refers to an outdated (not necessarily old) computer systems, programming languages, or application software that are used instead of available upgraded versions. They usually require high maintenance and can involve intricate patching and modifications.
104 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
systems, in the context of computing, refers to an outdated (not necessari r systems, programming languages, or application software that are used Module 4 | Organization ble upgraded versions. They usually require high maintenance and can in patching and modifications. Iterative Approach Over Big Bang Transformation Strangler Pattern Customers
Existing Monolthic Application
Customers
Customers
Dispatcher
Original Monolthic Application
New Module
Dispatcher Original Monolthic Application
New Module
New Module
New Module
ntinuousdelivery.com/implementing/architecture/ Source: http://continuousdelivery.com/implementing/architecture/
An increasingly apparent and large challenge in IT organizations is how teams can effectively modernize software development and IT operations while still operating and maintaining legacy infrastructure. Often the approach is to merely draw a line in the sand, creating an arbitrary cutoff whereby new implementations make use of the much desired DevOps and Agile methodology. But, what about legacy environments? Many organizations are living in a world with legacy systems. However, the services developed using such systems are distinctly hard to test and deploy. Therefore, it is difficult/impossible to automate these processes/services. Jez Humble, the Continuous Delivery and DevOps Expert, rather than rearchitecting everything, recommend an iterative approach to improving the design of the system. One of the patterns particularly valuable in this context is Strangler Application. In this pattern, the monolithic architecture is iteratively replaced with a more componentized one by ensuring the new work is done following the principles of a service-oriented architecture while accepting that the new architecture well delegates to the system, it is replacing. Over time, more and more functionality will be performed in the new architecture, and the old system being replaced is ‘strangled’.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
105
Course Book | DASA DevOps Fundamentals
The most important reason to consider a Strangler Application over a cut-over rewrite is reduced risk. A Strangler Application can give value steadily, and the frequent releases allow you to monitor its progress more carefully. Using shorter release cycles with Strangler, you can avoid a lot of unnecessary features that cut-over rewrites often generate. Even when designing a new application, it should be designed with the mindset that it will be strangled in the future. The way helps you prepare for the tomorrow’s legacy software today only. ernance is as important as ever. By making it easy to be strangled in the future, you are enabling the graceful fading away of today’s work.
nce
hat the Business receives the right Innovation and Service n time, at agreed quality and at theGovernance right price) by leveraging the l and external IT providers.
When doing DevOps, governance is as important as ever.
Governance… … within teams ✓ … between teams
✓
✓
✓
Source: Weill & Ross, 2004
Governance ensures that the Business receives the right Innovation and Service Delivery as agreed (on time, at agreed quality and at the right price) by leveraging the full potential of internal and external IT providers. Gartner defines IT Governance as: “The processes that ensure the effective and efficient use of IT in enabling an organization to achieve its goals.” Its importance is described by the Information System Audit and Control Association (ISACA), authors of the COBIT 5 framework as: “For organizational investment in ©IT Copyright 2018 to | 34 deliver the full value, it is recognized that IT has to be fully aligned with business strategies and direction, key risks have to be identified and controlled, and legislative and regulatory compliance demonstrated. IT Governance covers this and more, and in light of recent corporate failures, scandals and failure, enjoys a higher profile today than ever before”. The importance of good governance is same for organizations with or without DevOps. To fit governance and DevOps, ensure that doing governance is not a burden in day-to-day operations. In other words, you need to take the waste out of governance, for example, collecting data in a consistent way across the organization.
106 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 4 | Organization
Governance Within Governance WithinTeams Teams Good governance within teams is the responsibility of everyone within the team.
Good governance within teams is the responsibility of everyone within the team.
Copyright © 2018 | 35
As with quality, everyone in the Business System or Platform team is responsible for and involved in its governance. Lean and Agile tools and techniques, such as daily stand-ups, Kanban, visual management, regular planning meetings, and retrospectives, are all forms of governing the delivery of value to customers. Part of the governance within a team is to ensure the mix of skills and knowledge is able to support the work. The three profiles in the DASA Competence framework help ensure this. In DASA, the majority of a DevOps team, as per estimation, fall into the competency area of ‘create and deliver’. The exact balance of skills, particularly Programming vs. Infrastructure Engineering, depends on the total area of responsibility of the team. Although DevOps is definitely a team game, it is still important to clearly define individuals’ roles and responsibilities for success within the team. For example, the team can choose to use a dedicated Product Owner, Scrum Master, Team Manager, Release-to-Operations Expert and others.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
107
Course Book | DASA DevOps Fundamentals
Governance Between Multiple Teams Governance Between Multiple Teams
Governance between multiple teams with multiple customers implies needimplies for Governance between multiple teams with multiplemore customers governance, as depicted inmore the following figure. need for governance, as depicted in the following figure.
el
s
Customers
or s
e
Business System Teams
More autonomy Less governance
Less autonomy More governance Copyright © 2018 | 36
Implementing DevOps across multiple teams brings a new set of challenges, such as: Project dependencies Multiple locations Centralized resource planning Integration of technologies Multiple stakeholders Operational support
DevOps at the enterprise level, or enterprise DevOps, needs additional governance structures not necessarily required in smaller, less complex organizations. The extent to which teams can function (largely) autonomously diminish the need for additional governance. Excessive need for governance, due to complex dependencies between teams, hampers the flow of value to customers.
108 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 4 | Organization
Using Scrum of Scrums with Agile Teams to Coordinate and
DevOps
Using Scrum of Scrums with Agile Teams to Coordinate and Collaborate Collaborate
entals
The Scrum of Scrums meeting, as depicted in the following figure, is an important technique in scaling Scrum to a large project with multiple The Scrum of Scrums meeting, as depicted in the following figure, is an important technique Business System teams.
in scaling Scrum to a large project with multiple Business System teams.
ational Model
Scrum of Scrums of Scrums
mous Teams
chitecting for DevOps
Scrum of Scrums
Governance
Scrum
Scrum
Scrum of Scrums
Scrum
Scrum
The Scrum of Scrums meetings allows a cluster of teams to discuss their work, focusing especially on areas of overlap and integration. Imagine a perfectly balanced project comprising multiple teams. Each team will do their own daily Scrum stand-up meetings. For the coordination between the teams, they will also send one person to attend the Scrum of Scrums meetings. Each team selects a person who will be the best to represent them.
Scrum
Scrum
Copyright © 2018 | 37
Module Summary In this module, you learned that: Organizational Model The organizational model is the business structure or the way a business organizes itself in departments, units, and teams. The operational model is the way a business or an organization operates to get the work done. The current preferred way of organizing DevOps teams is to enable autonomous Business System teams that can “land” their application and infrastructure code on a platform that is maintained by a Platform team. Activity-focused organization are specialty Oriented, Functionally Organized, Project Focused, Work with Individuals, and Optimized for Resource Utilization. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
109
Course Book | DASA DevOps Fundamentals
Product-focused organization Work Oriented, Team Organized, Product Focused, Work with Teams, and Optimized for Speed. Autonomous Teams Autonomy is a self-governing community, which is not subjected to control from the outside world. Set up teams structured around distinct services and products. The three basic design criteria to determine the autonomy of a team are customer, technology stack, and knowledge. The decoupling point defines the point within the technology stack where the responsibility is transferred between the Business System team and the Platform team. Architecting for DevOps The aim of IT architecture is to support functional requirements with non-functional or quality requirements. Building (other) qualities in simultaneously with the functional requirements is a major enabler of faster and better IT services. Complexity increases with the number of services, and quality falls as complexity increases. MSA is a logical structure for the design of a software program involving loosely-coupled modular components known as microservices. Guidelines for MSA include autonomous systems, simplicity, and low coupling/high cohesion. Characteristics of MSA include Componentization via Services, Organized Around Business Capabilities, Products not Projects, Design for Failure Tolerate, and Evolutionary Design. Governance Governance ensures that the Business receives the right Innovation and Service Delivery as agreed (on time, at agreed quality and at the right price). The Scrum of Scrums meeting is an important technique in scaling Scrum to a large project with multiple Business System teams. The Scrum of Scrums meeting is an important technique in scaling Scrum to a large project with multiple Business System teams.
110 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 4 | Organization
Module End Questions Q1. What is the responsibility of a Business System Team? a) The Business System teams are end-to-end responsible for their service during its complete lifecycle. b) The Business System teams are responsible for the correct development of the service. c) The Business System teams are end-to-end responsible for the health of the full technology stack required to run their service. d) The Business System teams are responsible for the satisfaction of the business and other customers. Q2. What is the prime motivation for establishing autonomous teams within DevOps? a) Achieving economy of scale benefits and lowering cost for customers. b) Utilizing the limited capability of IT professionals for effective collaboration. c) Reducing waste and focusing on end user value. d) Fostering the effect of learning from colleagues who do the same work. Q3. Match each term with the corresponding description. Term
Description
a) Agile Organizations
i Focuses on “You build it, you run it, shared nothing” concept
b) Continuous Delivery
ii Provides cheap, easy, and fast runtime environments for apps
c) Autonomous Teams
iii Focuses on responsive, resilient, scalable, and loosely-coupled systems that are easy to develop and change
d) Reactive Manifesto
iv Prefers dedicated teams over resourcing, products over projects, prioritization over planning, and outcome over output
e) Platform as a Service
v Reduces and measures cycle time in hours or minutes
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
111
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
5 Processes Module Objectives Module Objectives Agile and Scrum
Eight types of waste Value Stream Mapping
Relate ITSM to practices in a DevOps culture Minimal Viable Product (MVP)
Story Mapping
Copyright © 2018 | 1
At the end of this module, you will be able to: Explain Agile, Scrum, and Kanban, and how these practices relate to one another. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
113
Course Book | DASA DevOps Fundamentals
Explain the Scrum process at a high level. Explain how Information Technology Service Management (ITSM) processes relate to practices in a DevOps culture. Explain the eight types of waste with examples. Explain how to provide a Value Stream Map for a given process. Identify and remove waste from a process. Explain how to specify and verify using emerging architecture, design, and the concept of Minimal Viable Product (MVP). Explain how Story Mapping works. Relate to the need of harvesting new and innovative ideas.
Module Topics Process Basics DevOps in Relation to ITSM Agile and Scrum Optimizing ProcessesUsing Lean
What is a process?
Business Value Optimization and Business Analysis Using Story Mapping
Process Basics
What is athat process? A process is a sequence of activities build on one another to achieve a result, as shown A process sequence of activities that build one another to n the figure. Understanding the basicsisofa processes is important to on know what aspects can achieve a result, as shown in the figure. Understanding the basics e influenced. Such knowledge helps make the right choices in improving the way of of processes is important to know what aspects can be influenced. working. You should aim to automate optimized processes only. in improving the way of Such knowledge helps make the right choices working. You should aim to automate optimized processes only.
114 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018
vOps
tals
ess Basics
Relation to ITSM
and Scrum
Processes sing Lean
Module 5 | Processes
DevOps in Relation to ITSM ITSM
ITSM Information Technology Infrastructure Library (ITIL) is a best practice
Key Performance Indicators
for IT Service Management (ITSM) that focuses on aligning IT ITIL defines KPI as a metric that services with the needs of the business. ITIL describes processes, is used to help manage an IT Information Technology InfrastructureIndicators Library (ITIL) is a best activities, roles, Key Performance (KPIs), and practice Critical for IT Service Management service, plan, project, or (ITSM) that focuses on aligning IT servicesintegration with the needs the the business. ITILprocess, describes processes, Success Factors (CSFs) for establishing of ITofwith other activity. Key performance activities, roles,strategy Key Performance Indicators (KPIs), Critical Success Factors (CSFs) for organization’s and designing, delivering, andand maintaining the indicators are usedand to measure establishing integration of IT with the organization's strategy and designing, delivering, maintaining service’s value. the achievement of critical the service’s value. ITIL is published as a series of five books that cover the following five success factors. ITIL is lifecycle published as a series of five books that cover the following five ITSM lifecycle stages: ITSM stages: Manage interaction between phases through the continual service improvement approach, which is responsible for measuring and improving the service and the process maturity level.
Optimization ess Analysis ory Mapping
Implement the services to meet the designed requirements.
Define the strategy for the ITSM.
Design the services to support the strategy.
Support the services, managing the operational activities. Copyright © 2018 |
Once all the phases of the lifecycle complete, it is concluded and another one begins. Service Strategy: At the beginning, the IT Service Provider sets the strategy for ITSM by managing the business requirements, translating these into a strategy to deliver service, validating the sustainable costs, and introducing the service in the service portfolio. In addition, IT uses the required resources, such as costs, in consultancy projects at a strategic level. However, IT does not provide a tangible product value to the business which is related to this specific phase. Service Design: When a strategy is complete, IT starts designing the service by setting the service level requirements for the service, analyzing the required availability and capacity, selecting the suppliers who will support the service, defining the adequate service continuity arrangements, validating and designing the security requirements, and introducing the service in the service catalog. Service Transition: During this phase, the service is ready to be implemented in the live environment. The Service Provider Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
115
ps
Course Book | DASA DevOps Fundamentals
defines the transition plan and assesses, approves, implements, and plans the change. After the change implementation, the service is tested in a ‘pre-live’ environment. If the test is successful, the service is documented and its components are introduced in the Asset and Configuration databases. The last activity is to release the service into the live environment and after the ‘go-live’, a post-implementation review is performed. Service Operation: The service starts to be managed and supported in order to reach the agreed service level by managing the users’ support requests, monitoring the service event and alerts, restoring the service after disruptions, avoiding the incident causes and reducing the incident duration, managing the utilization of services in a secure manner, maintaining the software, executing the day-to-day activities, and supporting the infrastructure. Continual Service Improvement: It is involved during all the phases of the service lifecycle. It is responsible to measure the service and the processes, and to document the results in order to improve the service quality and the processes’ maturity. However, such improvements are implemented in the next iteration of the service lifecycle, starting again with Service Strategy.
DevOps and ITSM
DevOps teams are responsible for a wide range of topics described in thefor ITILa publications. Strategy is determined in cooperation with other DevOps teams are responsible wide range of topics described in the ITIL publications. teams. with other teams. Strategy is determined in cooperation
rum
sses ean
Relevant Topics Managed Within the Product (and Platform) Team(s)
Product Owner
sics
n to TSM
DevOps and ITSM
Define Service Strategies
Service Portfolio Financial Management Demand Management Strategy Management for IT Services (Outsourcing, Insourcing, and Co-sourcing)
ation alysis pping
Design
Service Design
Design Coordination Service Catalogue Management Service Level Management Supplier Management Capacity Management Availability Management Service Continuity Management Information Security Management
Implement (projects)
Service Transition Change Management Service Asset and Configuration Management Knowledge Management Release and Deployment Management Service Validation and Testing Change Evaluation Transition, Planning, and Support
Support
Service Operations
Request Fulfillment (Support Reps) Event Management (Monitor) Incident Management (Restore) Problem Management (Duration) Access Management
Quality
Continuous Service Improvement
7-step Improvement Process Service Measurement Service Reporting CSI Approach
Copyright © 2018 | 7
ITIL processes are equally important in a DevOps organization. However, the implementation of the processes might differ from how
116 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 5 | Processes
implementation is done in siloed structured organizations. Some of the reasons why applications of ITIL processes require attention in DevOps organizations are: People are organized differently: Resources are organized as Product teams, resulting in fewer handovers, less complex process description, and minimal documentation. Product teams work as autonomously as possible: Product teams work as autonomously as possible. They are free to do any tasks/activities required to deliver the product (the value) without alignment and approvals of other teams. People are aware that processes might get in the way of the ability to adapt to change: People are aware that they need to adapt to a continuously changing environment. Rigid process descriptions can get in the way of dealing with change. Processes are automated as much as possible: Processes have low variability and are an excellent candidate to automate. Therefore, the major focus is on automating processes to make these faster, more reliable, and repeatable. Design, Transition, Support, and Quality are all dealt within the same team (self-optimization): Service Design, Transition, Support, and Quality are covered within the Product team itself, where team members have the power to decide what is the optimal way to deal with different topics. ITIL provides a complete overview of all of the topics that a DevOps team must address. All teams come together (often product owners and lead engineers) to define strategy: Topics related to service strategy are where all Product teams connect through the use of Product Vision boards. Teams are end-to-end responsible for the design, implementation, and execution of the product: The productcentric approach entails the complete responsibility of product with the team. In a DevOps organization, the ITSM processes are incorporated and executed by Product teams, who take the end-to-end responsibility for a product. As Werner Vogels (CTO and Vice President of Amazon) stated: “You build it, you run it”. As the team is centered around the product, the team itself is responsible for processes as defined in ITSM. Some examples of how some ITIL Service Strategy processes/tasks are mapping in a DevOps organization include: Service Portfolio: Service portfolio is maintained on the Product Vision board shared between the Delivery Product teams. Consequently, the service portfolio is made transparent and is shared with the rest of the organization, making it easier to adapt. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
117
Course Book | DASA DevOps Fundamentals
Some examples of how some ITIL Service Design processes/tasks are mapping in a DevOps organization include:
Service Level Agreement A Service Level Agreement (SLA) is used to document agreements between an IT Service Provider and a customer. It lists the IT Services, defines the service level targets, and identifies the responsibilities of the IT Service Provider and the customer.
Service Level Management: Defining SLAs is the process of recording performance requirements agreed with the customer. It enables the different parties to be aware of the requirements. SLAs also help to ensure when a service will be delivered by multiple parties. A DevOps organization aims at complete autonomy and end-to-end responsibility for delivering a product (value) to the customer and minimizing the dependencies on other organizational parts. Therefore, teams are made end-to-end responsible. As a result, the need for extensive SLAs diminishes though, the need for an agreement with the customer regarding performance remains. Consequently, the customer and team work together to ensure the best possible performance. Hence, you can say that “Collaboration over Contracts” rules the DevOps organizations. Capacity Management: It covers the understanding of business, service, and component capacity. The combination culminates in a plan that describes how the service will develop as a result of its usage. Through a deep understanding of how the customer will use the service and the overall ‘behavior’ of the technology stack, the DevOps team ensures what measures should be taken to maintain the health of the service in the future. Capacity Management also relates to people capacity. In a traditional IT environment, people are assigned one or more projects with a start and an end. Therefore, there is always the challenge of having overcommitted or unavailable people. In a DevOps context, Product teams are stable and fixed and do not require resource-switching between projects.
Food for Thought How does implementation of DevOps maps with or impact the processes (such as Incident Management and Problem Management) of the Service Operation stage?
Some examples of how some ITIL Service Transition processes/tasks are mapping in a DevOps organization include: Change Management: Unmanaged changes can cause instability in software even in a DevOps organization. In a DevOps aligned organization, changes in a system are managed through the same mechanisms used for aligning the business with IT. Changes are managed by the same team as the team is building the system itself. As DevOps teams have not yet achieved full technological autonomy, changes might impact other teams. Therefore, Change Management is required to ensure a smooth flow of changes. However, it requires active management of changes (for example, a weekly Change Advisory Board meeting is too infrequent and causes waste). Release and Deployment Management: In a DevOps organization, deployment of a DevOps team is done completely independent of any other team, even the Platform team. Deployment automation is a one-press-of-a-button process that brings new software live in a matter of minutes. Snapshot
118 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
vOps
als
s Basics
Module 5 | Processes
deployments of software are performed automatically after each code-commit. Such deployments give the team a sense of code deployability and runtime behavior of the code as after deployment one or more remote tests can be conducted.
The Waterfall Approach Agile and Scrum
TheThe Waterfall model is a sequential process to develop software and is introduced in 1970. Waterfall Approach It consists of six stages. The Waterfall model is a sequential process to develop software and is introduced in 1970. It consists of six stages.
elation to ITSM
Requirements Analysis
d Scrum
Design
ocesses ng Lean
ptimization s Analysis y Mapping
Coding Testing
Operations
Activity: Group Discussion Activity Time: 10 mins What are the advantages and disadvantages of developing software applications using the Waterfall approach?
Copyright © 2018 | 9
Reference Readings: http://www.ianswe r4 u .co m/2 0 11 /11 /a d va n ta g e s-a n d disadvantages-of.html#axzz3pBIOYXpZ http://www.optimusinfo.com/blog/traditional-vs-agile-softwaredevelopment/ https://www.scrumalliance.org/community/articles/2013/ january/traditional-and-agile-methods-an-interpretation http://istqbexamcertification.com/what-is-waterfall-modeladvantages-disadvantages-and-when-to-use-it/
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
119
s
Course Book | DASA DevOps Fundamentals
What is Agile? What is Agile? Agile approach is a time-boxed and iterative software delivery. It Agile is a time-boxed and iterative of software delivery.approach It aims toofbuild software to build software incrementally from the start of the project. incrementally from the start ofaims the project.
sics
Effort
n to TSM
rum
sses ean
ation lysis ping
User Stories Add two numbers Subtract two numbers Multiply two numbers Divide two numbers . . . . Factorial of a number
Iteration 1 Iteration 2
Iteration N
1 2
3
4
N
Time (in Days)
Agile focuses on smaller functional units instead of developing the complete software in a go. Copyright © 2018 | 11
User Story A user story is a high-level definition of a requirement from an end-user perspective. It contains just enough information to help the developers produce a reasonable estimate of the effort to implement it.
Tips The Agile way of working strives towards finalizing a complete task first, before starting with a new one.
The Agile way of working breaks a product into functional units according to user stories and prioritizes them to continuously deliver software in short cycles known as iterations. It is often used as a timeboxed, iterative approach to software delivery. It is aimed for building and delivering software incrementally throughout the project instead of trying to deliver the complete product at or near the project end. The Agile way of working largely incorporates feedback loops to help teams to respond to the ever-changing outside world. When features get delivered by an Agile team, these are in a state of ‘done’. The Definition of Done (DoD) describes when a feature is done. In other words, the feature is developed, built, tested, and approved. In addition, it is running in production. Reference Readings: “The Art of Agile Development” by James Shore
120 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
ps
asics
Module 5 | Processes
Traditional vs.Agile Agile Traditional vs. The following figure shows how Agile way of working differs from of shows developing Theof key difference is from the traditional way of The traditional following way figure howsoftware. Agile way working differs delivery of a useful product from the first iteration in the Agile way of developing software. The key difference is the delivery of a useful product from the first working. iteration in the Agile way of working. Traditional
on to TSM
crum
sses Lean
ation alysis pping
First: Completely work out an idea Then: Extremely accurate estimation
Agile
First: Think of an idea - outline Then: Work out the idea, try out and adjust
Production ready Time
Maybe this was already sufficient!!
Always production ready Time Copyright © 2018 | 12
Working in a traditional manner involves large upfront planning (in an ever changing environment) and investment for an idea on which there is no visibility required to test the hypothesis. Investment is made upfront. While, no value is returned to your investor throughout the process of establishing the product. Product development and the underlying investment are based upon hypothesis. One can set a deadline and an idea, but without feedback from the customer on whether the product/idea is wanted and without feedback from the team regarding the velocity of how fast the team can deliver, the risk remains high throughout the project. Agile is about using feedback in your product development cycle. Be it feedback related to the product, team velocity, budget, situation in the market, or even the situation in the world. The feedback allows the team to swiftly react to any of these topics. When working in an Agile manner, the product is ‘shaped’ along the way, using feedback from the end-user. It is similar to sketching a painting. One will start with the first layout of a sketch, evaluate the outline, and adjust the outline, if required, along the way. When contours are to the painter’s (and customer’s) liking, the coloring and further detailing of the painting begins. Throughout the process of painting the picture, the painter, learners, and even customers provide feedback to complete the painting to its perfect end-state. In other words, the product itself is accessible to stakeholders throughout the process. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
121
DASA DevOps Course Book | DASA DevOps Fundamentals Fundamentals Premium
Traditional vs. Agile (Contd.) The following figure shows the two approaches to deliver a project.
The following figure shows the two approaches to deliver a project.
Process Basics
Plan Driven (Traditional)
DevOps in Relation to ITSM
Value Driven (Agile) Resources
Functionality
Agile and Scrum
Time
Fixed
Optimizing Processes Using Lean
Quality
Business Value Optimization and Business Analysis Using Story Mapping
Quality Estimated Resources
Functionality
Time
Source: Agile – Pocketguide voor wendbare organizations
Source: Agile – Pocketguide voor wendbare organizations
Copyright © 2018 | 13
Every project has to work within the boundaries of “requested functionality”, “amount of resources”, “the timeframe to deliver”, and “the level of quality”. When executing a project, one can adjust any of these aspects, such as: Drop the level of quality, so that the deliverables of the project can be delivered faster. Add more resources, so the deliverable of the project can be delivered at an earlier date.
SA DevOps
damentals mium
Add time, so the deliverables of the project can have additional functionality.
Working with Multidisciplinary Feature Teams
Working with Multidisciplinary Feature Teams
‘Activity-Focused’ (siloed): Traditional
‘Product-Focused’ (team): Agile
Process Basics
vOps in Relation to ITSM Agile and Scrum
timizing Processes Using Lean
ess Value Optimization and Business Analysis Using Story Mapping
Specialty Oriented Functionally Organized Project Focused Work with Individuals
Work Oriented Team Organized Product Focused Work with Teams
Features of product-focused approach: Features of product-focused approach: Responsibility of product-team all the way into Responsibility of product-team extends all extends the way into production productionand accountable for a fully working product. All are responsible All are responsible and accountable for a fully working product.
122 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 | 14
Module 5 | Processes
Siloed organizations (depicted on the left hand side of diagram) have its resources grouped on specialties. While this approach might be optimized from a resource-efficiency perspective, but this approach is not optimized from a process-throughput perspective. Some of the process-throughput related issues often seen in siloed organization can be found below: Suboptimal Flow Chain of Events: In a siloed organization, resources tend to optimize individual workloads, without being aware of its implications to the complete chain of activity. From an individual resource perspective, this is hard or even impossible to improve. Low Process Throughput: Handover-moments between silos have a very negative effect on feature/product throughput and in a DevOps organization it should be avoided as much as possible. A biweekly meeting, for instance, in a worse case will introduce a 10 working days delay in throughput for any given activity. Buildup of Work in Progress (WIP): Handover-moments are one of the root causes for building up WIP. A 5-day delay in throughput might build up quite a bulk of work to be performed in later stages, while from a Lean perspective, the goal is to stop accumulation of work. Large, unmanageable amounts of WIP is one of the most common reasons for processes to slow down significantly. Low Responsibility for End-Result: As the different activities related to constructing software is divided over multiple resources, which could be located in different locations, no resource in the process feels responsible for the end-product. Time-Consuming Task-Switching: As siloed resources work on multiple activities related to multiple projects, a lot of timeconsuming task-switching is required. Difficult for Resources to Improve: For a person in one silo, it is very hard to see the implications of his/her actions in another silo, therefore, it becomes difficult for a person to improve actions for the betterment. Constructing software is a team effort which is very hard to achieve when working in different locations with different goals, different priorities, and no visibility on team’s work list. Agile teams (depicted on the right hand side of the diagram) are multidisciplinary in nature which means that the whole team has all disciplines onboard to bring a product to production. By having all disciplines working in one team focused on the end-product, expensive handover-moments are reduced to a minimum and process throughput is further optimized. Remember, that for a DevOps organization, it is all about speed and getting features to the customer as soon as possible, so the organization can use valuable feedback loops to determine whether they are on the right track. There is a wide variety of skills Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
123
Course Book | DASA DevOps Fundamentals
that is embedded in the team and each engineer might incorporate multiple skills. Examples are skills for ‘development’, ‘testing’, ’business analysis’, ‘systems operations’, ‘systems design’, ‘database administration’, etc. As noted, in a DevOps organization, resources typically exhibit multiple skills and actively pursue to enhance skills when needed.
Activity: GroupTask-Switching/Game Activity: Task-Switching/Game Group Activity: Activity Time: 10 Minutes
Activity Time: 10 mins
The multipletasks tasksisistotoperform perform them one a time to avoid Thefastest fastestway wayto toperform perform multiple them one at aattime to avoid task-switching, task-switching, as depicted in the following figure. as depicted in the following figure.
cs
100
to M
90
m
70
es an
on sis ng
%
80 60
Time spent on tasks
50
Loss due to task switching
40 30 20 10 0
1
2
3
4
5
6
No. of Tasks Source: Systems Thinking (Dorset House, 1992) Jerry Weinberg
Source: Systems Thinking (Dorset House, 1992) Jerry Weinberg
Copyright © 2018 | 15
In a siloed organization, resources perform multiple activities related to one or even more projects. Working in this manner requires extensive resource allocation and management skills to stay on track. Although from a resource perspective, resources might seem to be optimally allocated in a siloed organization. However, from a process throughput perspective, individual throughput will slow down as resources require a lot of task-switching to switch between activityrelated contexts. When a lot of task-switching is required from a person, his/her efficiency will degrade. DevOps organizations organize around a product and respect the Agile Scrum framework as a way of working. By doing so, task-switching for resources is kept to an absolute minimum, thus improving the process throughput.
124 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 5 | Processes
12 Principles of the Agile Manifesto 1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. 2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage. 3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale. 4. Business people and developers must work together daily throughout the project. 5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done. 6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. 7. Working software is the primary measure of progress. 8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. 9. Continuous attention to technical excellence and good design enhances agility. 10. Simplicity‒the art of maximizing the amount of work not done‒is essential. 11. The best architectures, requirements, and designs emerge from self-organizing teams. 12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. Source: http://agilemanifesto.org
The Agile Manifesto is underpinned by the preceding 12 principles.
Agile Manifesto The Agile Manifesto was written in February, 2001 by seventeen independent-minded software practitioners.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
125
Course Book | DASA DevOps Fundamentals
The Agile Manifesto
The Agile Manifesto
Agile Manifesto We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals & Interactions over Processes & Tools Working Software over Comprehensive Documentation Customer Collaboration over Contract Negotiation Responding to change over Following a Plan
“While there is value in the items on the right, we value the items on the left more”
Source: http://agilemanifesto.org
Source: http://agilemanifesto.org
The Agile Manifesto is not about choosing one activity or the other. It is about favoring which activity is deemed more valuable towards realizing the end product than the other. For example, it is not the case that no documentation should ever be written. It is about, if you can choose, you should choose a working software over documentation. One of the other examples is to respond to new facts you come across rather than rigorously following a plan, which might have become invalid over the time. One of the most popular frameworks to implement an Agile process is Scrum. It is a framework which defines a process that helps teams to work on complex and adaptive problems, such as software development. In this framework, the Development teams work in short iterations to produce high-quality, value-driven results. Reference Reading: http://agilemanifesto.org/principles.html
126 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyrig
Module 5 | Processes
evOps
ntals
The Scrum Approach in Detail The Scrum Approach in Detail
ess Basics Relation to ITSM
and Scrum
Processes Using Lean
Roles
Optimization ness Analysis tory Mapping
3x
Copyright © 2018 | 18
The Agile framework knows roles, artifacts, states, events, a flow, social objects, and improvement cycles which are explained below: Roles: Team: A multidisciplinary team who is allowed to work on the tasks agreed at the start of a Sprint. Every discipline required to deliver a shippable product (as the output of a Sprint) is contained within the team. Typically, a team consists of members having the skills to define, develop, test, deploy, maintain, and communicate the various aspects of the product. The team members organize themselves and continuously to improve their own process. They take responsibility for the way the various tasks, activities, or the process as a whole are working. A typical Scrum team is about 5-9 members in size. Scrum Master: The person responsible for making sure the team adheres to Scrum behaviors, rules, and guidelines. He/ she is the facilitator who ensures everybody plays by the rules. The Scrum Master not only explains to the team how various tasks/activities are done but also explains it to the external stakeholders. The Scrum Master enables the team to do the tasks that are required to make the product work. Product Owner: The person responsible for maximizing the value of the product. The Product Owner knows the business and the customer. Therefore, he/she defines user stories that matter to the business and the customer, and holds off on other stories. The Product Owner is the only person responsible for Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
127
Course Book | DASA DevOps Fundamentals
The Scrum Approach
maintaining the product backlog. He/she ensures that the user stories adhere to the Definition of Ready (DoR) in terms of how requirements are described, such as the board is properly prioritized in terms of value and the clear and transparent in communication Detail (Contd.) between development and business is achieved.
Artifacts 3x
s
o M
m
s n
n s g
Artifacts:
Copyright © 2018 | 19
Product Backlog: A continuously evolving and ordered list of requirements and topics, required to ensure the optimal product value is achieved. The product backlog is a single source of truth for modifications to the product. All modifications are on a single list to ensure everyone has the same view on what modifications are desired. Sprint Backlog: A set of product backlog items that have been selected for the Sprint. It also contains tasks required for delivering the new feature at the end of a Sprint (for example, activities, such as develop, build, review, and test). The Sprint backlog contains an internal prognosis of the Development team only for the next increment. Potentially Shippable Product: The product increment, which is delivered at the end of each Sprint. If the business requires, this artifact can be shipped to production straight away as it does not have any outstanding tasks.
128 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
vOps
als
Module 5 | Processes
The Scrum Approach in Detail (Contd.)
ss Basics
Relation to ITSM
nd Scrum
Processes sing Lean
States
Optimization ess Analysis ory Mapping
2x
Copyright © 2018 | 20
States: Definition of Ready (DoR): A list of rules (preferably attached next to the Scrum Board) describing standards that must be adhered by user story in order to be accepted by the Development team. Some of the examples of topics of this list are “the user story is on the backlog”, “the Development team understands the problem”, and “the user story is estimated by the Development team”. The DoR ensures requirements are clear from its inception and additional conversations during the Sprint activity are kept to an absolute minimum. It eliminates the need for discussions as much as possible. Definition of Done (DoD): A list of criteria (preferably attached next to the Scrum Board) describing which topics need to be addressed in order for a product to be considered ‘potentially shippable’. It is a simple list containing restraints, such as code, unit plus coverage tested, functionally tested, performance tested, user acceptance tested, reviewed, and documented. It clearly defines a finish mark. The team only delivers that part of the product which adheres to criteria in the list.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
129
Course Book | DASA DevOps Fundamentals
The Scrum Approach in Detail (Contd.)
Events 6x
Copyright © 2018 | 21
Six Repeating Events: The Sprint: A predefined amount of time during which activities on the Sprint backlog are performed. Sprints are often defined per week or every two weeks, but longer duration is also used. Shortening a Sprint results in shorter backlog refinement, poker, and retrospective sessions as the number of topics to discuss will become much lesser as well. The Daily Stand up: Every day, the team comes up to the Scrum Board where each member will explain what he/she did yesterday, where he/she is now, and what he/she will be doing today. Impediments, blocking team members from progressing, are also raised in this standup. A standup should never take more than 15 minutes of time. The Planning Poker Session: At the start of each Sprint (and often during the backlog refinement session), the team plays Planning Poker in order to quantify the amount of work that is required to fulfill a new activity. In this session, a sizing is agreed by the entire team and a ‘common view’ is established on the topics at hand. When performing more poker sessions over time, sizing will become more reliable. Consequently, the team starts to exhibit a specific burn rate, defining how fast the team is performing. The Sprint Review (also known as Product Demo): Each Sprint closes off with a product demo for the team, Product Owner, and the business/customer. The demo is a way to provide and receive feedback from all stakeholders and inject
130 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 5 | Processes
this feedback into the product during the next Sprint. Attending the product demo is essential for improving collaboration, the next product backlog, and of course the product itself. The Team Retrospective: After every Sprint, the team evaluates what went well and what went not-so-well, thus could improve. This is an important aspect of Scrum to continuously improve on the way of working. The Backlog Refinement Session: A session used to anticipate and define what user stories are expected in the next Sprint and communicate uncertainties in case user stories are unclear. The session typically takes place halfway to a Sprint, room for business and Product Owner to improve user The leaving Scrum Approach in Detail (Contd.) stories where required, prior to the start of the next Sprint.
evOps
ntals
The Scrum Flow
ess Basics
Relation to ITSM
and Scrum
Processes Using Lean
Optimization ness Analysis tory Mapping
Copyright © 2018 | 22
Scrum Flow: The Scrum flow, which is started by setting a common Sprint goal, is indicated by the green arrows in the preceding figure. The steps are: 1. Together with Product Owner, the team defines the goal for the next Sprint. 2. Once the goal is determined, the team performs a Planning Poker session to determine the number of stories for the Sprint. 3. The tasks that fit the Sprint and adheres to the rules of DoR, such as tasks are clear enough to be fully processed by the team, are added to the Sprint backlog.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
131
Course Book | DASA DevOps Fundamentals
4. The Sprint starts and engineers will work uninterrupted on agreed tasks. The Product Owner is not allowed to update tasks/user stories in the middle of the Sprint. 5. At the end of the Sprint, the updates are demonstrated to the customer. 6. After the Sprint demonstration, the Sprint review (a retrospective) is performed allowing the process to improve. 7. A product backlog refinement is conducted to help the Product Owner to get user stories to a state where they adhere to DoR. This activity can also be performed in the middle of a Sprint if required. 8. The Product Owner adds updated stories to the product backlog.
The Scrum Approach in Detail (Contd.)
Social Objects 3x
s
o M
m
s n
n s g
Copyright © 2018 | 23
Social Objects: To maintain transparency, social objects are kept in open as much as possible. These objects are managed by the team. Some of these objects include: Scrum Board: A board that lists the various activities for the current Sprint to be completed. One of the examples of such a board is Kanban board, where activities move from left to right over the board from “To do”, “Doing”, “Done” or “Impeded”.
132 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 5 | Processes
Burn Down Chart: During a Planning Poker session, features are assigned velocity points. The total points for all features that are part of a Sprint delivery are added to the Burn Down chart. Whenever a feature is implemented, the “burned” points are deducted from the total. It is the aim for each Sprint to always end with “0” points left. In the beginning, it is quite difficult to end up with “0” points, but over time, team estimations becomes more reliable. The Burn Down chart outlines the burn rate for running the Sprint over times. As a result, a team can steer on making the required progress to burn all points for the Sprint. Impediment Board: This board contains all (external) topics which prevent the team from doing its work. Typically, the Scrum Master ensures impediments are handled. Some of the items of this board can be “not enough desks”, “team divided over multiple locations slows us down”, and “network is down several times a day”.
evOps
The Scrum Approach in Detail (Contd.)
ntals
Improve Product Backlog
cess Basics Relation to ITSM and Scrum
g Processes Using Lean
e Optimization ness Analysis Story Mapping
Copyright © 2018 | 24
Improvement Cycle 1: Improve Product Backlog: The backlog, product, and collaboration are improved over time during each of the product demo sessions.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
133
Course Book | DASA DevOps Fundamentals
A DevOps
amentals um
The Scrum Approach in Detail (Contd.)
Improve Process
Process Basics
Ops in Relation to ITSM
Agile and Scrum
mizing Processes Using Lean
s Value Optimization d Business Analysis Using Story Mapping
Copyright © 2018 | 25
Improvement Cycle 2: Improve Process: The process is improved over time during each retrospective in which topics for improvement are discussed.
Some Advantages of Working Agile
Some Advantages of Working Agile
Visibility
Risk
Business Value
Security Fast fixes in case of vulnerabilities Lower risk-surface-area by developing the right things only
Team coherence and product focus leads to “build quality in”
134 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Agile Development (Continuous Delivery)
Traditional Development
Copyright © 2018 | 26
Module 5 | Processes
The Agile way of working has many advantages, some of which are hard to quantify. Topics, such as team dynamics, customer involvement, and the feeling of product ownership are not easy to measure. However, from a business point of view, some of the topics that can be measured include: Visibility: As Product Owner and business are involved with product development on a regular basis, for instance by attending the Sprintly demo or by launching new shippable features on a regular basis, visibility of what to be delivered is far higher than is the case with traditional development methods. Parts of the product are delivered on a regular basis. Risk: Optimization of product visibility lowers the risk. It becomes clear early in the process whether the team is moving into the right direction and building the right product. It is all about feedback and using this feedback to lower risk.
Tips Delivering a product in a continuous manner also helps from security perspective as defects and vulnerabilities are fixed faster. In addition, building just the functionality that is used by the customer, the “risk-surface-area” (typically the code amounting to functionalities seldom or even never used) will be kept to a minimum.
Business Value: By delivering a shippable product at the end of each Sprint, the product can be used to generate business value throughout the product development cycle. Features are prevented to get ‘stuck’ in the development cycle and are shipped straight away. This approach works in contradiction to the “traditional way of working”, where the product is shipped only near the end of the project (preventing the team to consider the valuable feedback of the end-customer through the software development cycle). From a security perspective, having the ability to deliver a product in a continuous manner have a positive effect on security-related topics as defects and vulnerabilities will be fixed faster compared to traditional quarterly release cycle. In addition, building just the functionality that is actually used by the customer, the “risk-surface-area” (typically the code amounting to functionalities seldom or even never used) will be kept to a minimum. Moreover, the team will work more coherent towards securing quality aspects of the product as they will be focused on the delivery of a product as opposed to performing activities which in itself have no meaning.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
135
Course Book | DASA DevOps Fundamentals
What is Lean?
Optimizing Processes Using Lean What is Lean? Lean is a tested and proven method that uses a collection of tools
LEAN
Resource Waste
Customer Value
Lean is a tested and proven method thatthe uses collection tools toare improve theIt way to improve wayaproducts andofservices produced. is also products and services are produced. It isa also considered a individuals mindset that pushes considered mindset that pushes to think aboutindividuals making to think about making the services betterbetter on a on daily basis. the services a daily basis.
Copyright © 2018 | 28
Lean is a term introduced by the Research team of Toyota to describe its business. The primary idea of Lean is to deliver maximum customer value with minimum waste of resources. Lean is an organized way that considers the following aspects to deliver maximum customer value: Eliminate waste (also known as Muda)
Muri and Mura Muri, a workload which is too high to handle, often results in safety and quality problems. Mura, an irregular production or workload often results in poor planning and/or poor staffing estimates as speed of process becomes impossible to predict.
Eliminate overburden/too high workload (also known as Muri) Eliminate lack of balance in workloads/lack of predictable flow (also known as Mura) Lean should become the philosophy of how individuals work. The goal is to embed Continuous Improvement in organizations with the tagline of “Lean is how we do our work”. Organizations can accomplish Lean in a robust way by forming Kaizen project teams that work towards transforming major improvements to quick wins. These wins are the improvements that can be quickly and easily implemented and are available to users with immediate (visible) benefits.
136 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
he Lean Wastes (“Muda”)
Module 5 | Processes
The Lean Wastes (“Muda”)
chi Ohno, considered the father of Lean, was instrumental in developing the "Seve Tips Taiichi Ohno, considered the father of Lean, was instrumental in stes" model which has become core in many academic approaches. An eighth was developing the “Seven Wastes” model which has become core in An easy aid to memorize the many academic approaches. An eighth waste, non-utilized skills, is n-utilized skills, is commonly overlooked, but it is also a waste. wastes tha eight The wasteseight is TIMWOODS commonly overlooked, but it is also a waste. The eight wastes that (Transportation, Inventory, Motion, identified from a Lean perspective are: can be identified from a Lean perspective are: Waiting, Overprocessing, Over production, Defects, and NonUtilized Skills).
Lean Six Sigma: Wastes
Non-Utilized Skills
Overprocessing
Co
Defects: Rework that is required because an activity was not properly executed in the first instance. This requires one to switch back to the originating activity, stop the progress, analyze the issue, and fix the issue. Overproduction: Producing more than what is actually required, generating WIP. It leads to excessive inventory. Therefore, the next step in the process is to think about where to (temporarily) store/archive the superfluous artifacts and find the artifact when it is eventually required again. Waiting: Wasted/idle time when one has to wait for another task (for instance caused by overproduction) to be finished. A wait is caused under many situations, such as a bottleneck in the endto-end process, irregular meetings, and handover moments. Such delays introduce discontinuity into a process. Non-Utilized Skills (Non-Utilized Talent): Waste caused by centering resources around specialized activities. If one has to perform only one type of task, other possible skills that the resource might exhibit, such as board management, organization, internal team communication, and presenting customer cases, are not used. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
137
Course Book | DASA DevOps Fundamentals
Transportation: Waste caused by the need for transportation, for example, a refrigerator located far away from the sink. In software development, this might be a task-switching as people are assigned more than one project. The fastest way to complete two tasks is to perform them one at a time. Inventory: Waste caused by excessive production taking up space. In a software development context, it is the work (story) that is not completely done as per the DoD. Therefore, you cannot demo or release it. As a result, it becomes a task that can be defined as WIP.
Voice of the Customer The Voice of Customer is represented as the requirements and wishes of the customer. A requirement is a specific property that the product must have; otherwise the customer will not buy it. A wish is a property that excites the customer if the product has this property.
Motion: An example could be one copier machine on the second floor, requiring the user to walk around the building for creating a copy. Another one is the unavailability of meeting rooms, forcing a team to search for a place whenever they need some privacy to discuss. In software development, handover moments can also be considered a waste related to motion. Overprocessing: This waste is caused when a team is unable to understand the Voice of Customer (VoC), or lack of understanding on the product-vision, resulting in gold-plating a product. A Product Owner might play a significant role steadying this type of waste. An easy aid to memorize the eight wastes is TIMWOODS (Transportation, Inventory, Motion, Waiting, Overprocessing, Overproduction, Defects, and Non-Utilized Skills).
About Waste Far more than 50% of functionality in software are rarely or never used. These aren’t just marginally valued features; many are no-value features. Source: The Standish Group, reported in the IEEE conference 2002
On average 45% of projects exceed budget, 7% runs over schedule and 56% delivers less value than expected. Source: McKinsey group with University of Oxford. From research conducted on more than 5400 IT projects with budget over $15 Mil. http://www.mckinsey.com/insights/business_technology/delivering_largescale_it_projects_on_time_on_budget_and_on_value
60%–90% of ideas do not improve the metric they were intended to improve. Source: From Lean Enterprises: data gathered from A/B tests by Ronny Kohavi, who directed Amazon’s Data Mining and Personalization group
Chances are that about two-third of the work you are doing is of either zero or negative value to our customers. This work costs in three ways: Development costs Lost opportunity to do more valuable work
138 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 5 | Processes
DevOps
entals m
Cost of added complexity (a drag on the rate at which you can develop new functionality, and often, reduced operational stability and performance)
Optimizing a Process Using Value Stream Mapping
Optimizing a Process Using Value Stream Mapping Steps
ocess Basics
1
in Relation to ITSM
2
le and Scrum
3
ng Processes Using Lean
4
ue Optimization usiness Analysis g Story Mapping
5 6 7
Customer Objectives and Process Actors Define Activities Define Work in Progress
Identify Rework
Two 4 meter strokes of brown paper Plastic tape to attach the paper to the wall
Post-it: multiples sizes, shapes and colors Assess Activities
Lot’s of sharpies Colored ‘dot' stickers
Determine Proccess Cycle Efficiency Determine Value Add for Each Activity
Value Stream Mapping (VSM) is a tool to gain insight into the workflow of a process and can be used to identify both value adding activities and non-value adding activities in a process stream while providing insight for optimizing the process chain. The results of a VSM can be used in many occasions: from writing out a business case, to defining a prioritized list to optimize processes within your organization, to pinpointing bottlenecks in your existing processes, and/or gain a common understanding of process related issues.
Copyright © 2018 | 31
When creating a VSM of your current software delivery process, you quite possibly be amazed by the amount of waste and may find the room for improvement. It will leave you with a very powerful tool to explain the steps that need to change, as it will leave you with the facts. In many organizations, there is the tendency perform ’ solely ’ local optimizations to steps in the process (for example, per Business Unit), while in reality the largest process optimizations can be gained by optimizing the areas which are in-between the process steps and do not add any value to the customer at all; the non-value adding activities. VSM is a great tool for optimizing the complete process chain, not just the local steps. Always start off to map the ‘as-is’ process to form the baseline. The baseline provides you the required insight into the process-steps that
Baseline A baseline is used to establish an initial data point to decide if improvements are required in services or processes. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
139
Course Book | DASA DevOps Fundamentals
do not add any value to the customer and therefore can be seen as pure waste in your organization. For mapping a VSM, the following tools might come in handy: Long strokes of brown paper Plastic tape to attach the paper to the wall Post-its in multiple sizes, shapes, and colors Many sharpies so all participants in a workshop can participate
Tips The results of a VSM can be used for various purpose, such as writing a business case, defining a prioritized list to optimize processes pinpointing bottlenecks in your existing processes, and/or gain a common understanding of process related issues.
Some dot-voting stickers. It is important to write out the Value Stream as a group process (a workshop), where group-members represent people that are part of the value chain as it is today. This is the only way to spot (hidden) activities and will provide a common understanding of the situation today. Apart from that, failure to execute the Value Stream Mapping activity as a group process will very likely reduce the acceptance rate at the end of the day. It is not a good idea to try to write out a VSM in isolation because important information might be missed.
Step 1: Voice of Customer and ProcessActors Actors VSM Step 1: DefineVSM Voice ofDefine Customer and Process
s
o M
m
s n
on is ng
Copyright © 2018 | 32
1A: Make sure to work with one process at a time and start off with defining customer objectives (the Voice of Customer). A common understanding of the VoC is important because in later stage you will determine with the team, which activities are really adding to this VoC
140 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 5 | Processes
and which steps are not. Quite often these objectives are defined in time, cost, and quality. For the example listed on image, let’s say the customer would like to have a new feature every hour, with a maximum cost of $1000 per feature and with zero defects. First, write down the VoC in the top right corner. 1B: Now, together with the group of people in the workshop, determine all the actors (organizations/persons/teams) that are a part of the current process and glue these actors as orange post-its to the brown paper.
evOps
VSM Step 2: Define VSM Step 2: Activities Define Activities
ntals
ess Basics
Relation to ITSM
and Scrum
Processes Using Lean
Optimization ness Analysis tory Mapping
Copyright © 2018 |
With the group, define the activities that take place within each process actor. Underneath the orange post-its, add green post-its that describe the activities that take place for a given process actor.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
141
Course Book | DASA DevOps Fundamentals
Step 3: Define Work-in-Progress VSM Step 3: DefineVSM Work-in-Progress
cs
to M
m
es an
on sis ng
Copyright © 2018 | 34
3A: Now, add pink post-its describing the number of features/ requirements/objects/activities that are currently in the process in between actors. This is referred as WIP. Whenever there is a high WIP limit in between actors, you have identified a bottleneck causing the process ‘flow’ to stop. 3B: On top of the pink WIP post-its, containing particular high WIP levels, add a small post-it (preferably with a new color) indicating what the group thinks is causing the high WIP. For instance, a document has to be distributed via internal mail, or a wait is introduced for a biweekly meeting, or travel to another location is required. This information can later be used to optimize the process. Note: In the workshop, spend some time in finding WIP within the activities itself (this is not depicted in this example). Spend time on finding information for causes of high WIP and add these causes to the post-it, next to each activity.
142 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 5 | Processes
DevOps
mentals m
VSM Step 4: Identify Rework VSM Step 4: Identify Rework
Process Basics
s in Relation to ITSM
gile and Scrum
ing Processes Using Lean
alue Optimization Business Analysis ng Story Mapping
Copyright © 2018 | 35
Rework is waste. Still, many times you’ll see that a deliverable is returned to a previous step for reprocessing. Together with the group, determine the actors where the rework happens and what are the causes of rework. A nice additional step is to write out First Time Right (FTR) levels.
DevOps
mentals m
VSM Step 5: Assess Activities VSM Step 5: Assess Activities
rocess Basics
s in Relation to ITSM
gile and Scrum
ing Processes Using Lean
alue Optimization Business Analysis ng Story Mapping
Copyright © 2018 | 36
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
143
Course Book | DASA DevOps Fundamentals
Spend some time adding additional comments or assessing activities (green post-its). Some activities might not be optimized, some activities are difficult to handle, or might be considered obsolete from a group perspective. Mention these comments on blue post-its, next to the concerned activity.
ASA DevOps
undamentals emium
VSM Step 6: Determine Time Efficiency VSM Step 6: Determine ProcessProcess CycleCycle Time Efficiency
Process Basics
DevOps in Relation to ITSM Agile and Scrum
Optimizing Processes Using Lean
siness Value Optimization and Business Analysis Using Story Mapping
Copyright © 2018 | 37
Now, as the process is more or less complete, you need to start adding information related to timing, such as process time, wait time, and lead time for determining Process Cycle Efficiency. In this step you would like to determine the following information: Process Time: The real amount of time that is required to perform a task without interruptions Lead Time: The actual time that it takes for the activity to be completed (also known as elapse time) Wait Time: Time when no processing is done at all, for example, attending an event like a biweekly meeting For every activity on the green post-it, write down two numbers vertically aligned on a small post-it (preferably blue color). The top number will reflect the process time (for example 40 hours), while the bottom number will reflects the lead time (for example, 120 hours). You should be able to calculate the wait time and write this information also on blue post-it. In this way, you should be able to calculate the process time and lead time for each process actor by adding process time and lead time for all activities. Write this information on a post-it and stick at the bottom.
144 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 5 | Processes
Note that with these numbers, one can start calculating overall process time, overall lead time, and Process Cycle Efficiency (for example, if overall process time is 40 hours and overall lead time is 120 hours, then Process Cycle Efficiency is 40/120 = 33%).
evOps
VSM Step 7: Determine Value Add and Waste for Eachand Activity VSM Step 7: Determine Value Add Waste for Each Activity
ntals
ess Basics
Relation to ITSM
and Scrum
Processes Using Lean
Optimization ness Analysis tory Mapping
Copyright © 2018 |
This step helps to identify customer value add activities and non-value add activities. Start with categorizing tasks into 2 types: tasks that add value to the customer (Customer Value Add, CVA, also known as Value Add) and tasks that do not add value to the customer (NVA). The NVA can again be split into two categories: tasks that add business value (Business Value Add, BVA, also known as Necessary Non-Value Add or NNVA) and Waste (Non-Value Add). When optimizing a process, wastes are to be eliminated completely as they do not add value to the customer nor the business as a whole. But also for the activities categorized as BVA, you have to ask yourself whether these activities add value to the chain. Business value is the value that helps a company in terms of health and sustainability over a long period of time. Customer value is the value that is perceived by the customer when using a product or service. Mark CVA tasks with a green dot, BVA tasks with a blue dot, and Waste with a red dot. Put the legend on the map for later reference. When identifying CVA, Waste, and the BVA, force yourself to refer back to the VoC determined in a first step and think about who your Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
145
Course Book | DASA DevOps Fundamentals
customer is. In this example, the customer is not the end-user using the system, but the business. And it was the business that wanted faster, cheaper, and better delivery. Now when you start to tag each individual task, give yourself some time in figuring out which tasks actually add to these goals.
DASA DevOps Fundamentals Premium
Tips Process Basics
Minimal Viable Product is often confused with Minimal Marketable DevOps in Relation to ITSM Product (MMP). MMP is the product with the smallestAgilefeature set that and Scrum still addresses the user requirements and can beOptimizing sold and marketed Processes Using Lean successfully. The purpose of creating Optimization an MVP Business is toandValue attain validated Business Analysis Using Story Mapping learning, while MMP is created with the objectives of competitive differentiation, revenue generation, and cost savings.
Business Value Optimization and Business Analysis Using Story Mapping Minimal Viable Product (MVP) Minimal Viable Product (MVP)
The MVP reflects your end-product in a minimal functional form. It is
The MVP reflects your end-product in a minimal functional form. It is used to test new ideas used to test new ideas and verify whether the hypothesis are correct. and verify whether the hypothesis are correct. The following figures help you better The following figures help you better understand the MVP. understand the MVP.
Copyright © 2018 | 40
The MVP is a great tool for verifying hypothesis. The idea is to rapidly build a minimum set of features that is enough to deploy the product and test key assumptions about customers’ interactions with the product. Complex systems are quite often built without the upfront verification whether the system will ever be used. When designing a new system, you should ask yourself whether your anticipated customer is ready for your idea. Another issue with designing new systems is difficulty in anticipating how the end-user will use it. Predicting usage is undoable if you are not incorporating your customer’s feedback. This is where the MVP comes into play. The MVP cycle starts with having an ‘idea’. This idea is subsequently built, resulting in ‘code’. The code is deployed to production and accessed by the end-user, allowing an organization to measure the end-user behavior. These measurements result in valuable ‘data’ from which the organization can ‘learn’. Such a learning helps generate new ideas, and you can continue with the MVP cycle.
Including the MVP in an Agile Process The selection of work from the product backlog must be done in such a way that an MVP is delivered at the end of the sprint. In other words,
146 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Including the MVP in an Agile Process
Module 5 | Processes
The selection of work from the product backlog must be done in such a way that an MVP is value. Feedback delivered atthe theusers end should of the receive sprint. aInusable otherproduct words,that thehas users should receive a usable product on such an MVP helps to ensure the right items are added to the that has value. Feedback on such an MVP helps to ensure the right items are added to the product backlog. product backlog.
s
sics
n to SM
Vision Statement
Product Vision Board
Product Backlog Minimal Viable Product
Change!
Feedback
Users Copyright © 2018 | 41
When a product is designed, you will like to test its hypothesis using an MVP. By working from an MVP perspective, feedback from the customer (or Product Owner in Scrum) can be fed back straight into the product backlog, steering the team in the direction to build a system that the customer will use. Scrum/Agile is perfect for this approach, as it iterates, is transparent, and involves main stakeholders in its execution.
What is Story Mapping? What is Story Mapping?
Mapping an engaging activity whereallall participants are are involved in the process of StoryStory Mapping is anis engaging activity where participants involved in the process of building the product backloginon a wall, as building the product backlog on a wall, as depicted the following figure. depicted in the following figure.
STORY MAPPING
rum
Advantages:
ses ean
Lower risk in development
ation lysis ping
Shorter project startup time
Engaged customer
Fast Return On Investment (ROI)
Copyright © 2018 │
147
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV Copyright © 2018 | 42
Course Book | DASA DevOps Fundamentals
It is different from the less visual and less engaging ‘traditional’ upfront design, taking away all the possibilities to use valuable feedback from your customer with advantages. Some advantages of applying Story Mapping are: A lower risk in development as customer feedback is embedded into the design process. An engaged customer as the system is designed to his or her immediate needs. Shorter project startup time due to removal of endless upfront design sessions. Fast Return On Investment (ROI) as the base product is in the customer’s hands at an early stage.
ow does Story Mapping work?
How does Story Mapping work? Story Mapping is a way to build your MVP in an iterative manner
ry Mapping is a way to buildusing your MVP in an iterative manner using Agile methodologies Agile methodologies to engineer the solution. It helps business, engineer the solution. It helps business, owners, andtheengineers prioritize the product owners, product and engineers to prioritize improvement to of their provement of their product. product. Overall Goal
End-to-end workflow
A
B
C
D
Fully functional
MVP Necessity, Alternatives, Flexibility, Intelligence, Performance, Comfort, Luxury...
Marketable Feature Set
The extra work is inside the features
Fully featured
Copyright © 20
To start with the Story Mapping, a common overall goal is established to determine “what success looks like”. From this overall goal, an endto-end workflow is determined. A workflow, for instance, can be: A: User logs in B: User adds a comment
148 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
s
Module 5 | Processes
C: User searches for other comments D: User logs off Next, a first marketable feature set is defined, which is of interest to your customer. You can consider this set as the MVP. For instance: A1: User logs in B1: User adds a comment D1: User logs off When measuring application usage, many people might be interested to join your system. Therefore, they can add comments, such as a search feature is required and facility to use Facebook login credentials to login the system is required. You can use such information in your next storyline, for example: A2 is a story to add Facebook login to the login functionality C1, C2, and C3 are stories to add several types of search capabilities to your system User Story Mapping helps prioritize your system along the contours of your defined workflow. By working in this manner, the additional work on A (A1 to A2) is all about expanding existing functionality. It works further improving already existing/planned features.
Story Mapping: Slices Story Mapping: Slices
Defining slices ensures each iteration contains a set of features that has slices value for the customer. Later slices include aenhancements to Defining ensures each iteration contains set of features previously created functionality.
that has value for the customer. Later slices include enhancements to previously created functionality.
cs
to SM
Overall Goal
um
Walking Skeleton
es an
slice 2
End-to-end workflow
A
B
C
D
Fully functional
MVP
Marketable Feature Set
ion sis ing
slice 3
slice 4
The extra work is inside the features
Copyright © 2018 | 4
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
149
Course Book | DASA DevOps Fundamentals
Threat modeling steers you towards thinking threats which are more likely to affect your product and are based on a solid understanding of the architecture and implementation of an application. Threats can be addressed with the appropriate countermeasures that fit the user stories in a logical order identical to implementing the story itself. Threat modeling is an activity on its own and not related to Story Mapping, but as can be seen, Story Mapping leaves room for all kinds of initiatives (like Threat modeling) to be included in the product build. As an analogy, consider using the approach of slices towards constructing a painting (in this example, the famous Mona Lisa): The Walking Skeleton (the MVP) is all about sketching the outlines of the overall goal. The second slice will be about correcting the outlines, in case these are incorrect (note the movement of the hand in the painting). The third slice will be about adding color and other details. Throughout the process, the painting will be presented to the customer to get feedback to subsequently incorporate the useful feedback into the painting.
Module Summary In this module, you learned that: Process Basics A process is a sequence of activities that build on one another to achieve a result. DevOps in Relation to ITSM The five ITSM stages that ITIL covers are Service Strategy, Service Design, Service Transition, Service Operation, and Continual Service Improvement. In a DevOps organization, the ITSM processes are incorporated and executed by Product teams, who take the end-to-end responsibility for a product. Agile and Scrum Agile is a time-boxed and iterative approach towards software delivery. Scrum emphasizes empirical feedback, team self management, and performs product increments within short iterations. Traditional plan-driven approach is about following a plan where the amount of functionality is fixed, and resources and time are estimated.
150 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 5 | Processes
A value-driven approach is about working with a fixed amount of resources and time where the amount of functionality that will be delivered is constantly estimated. Resources working in a siloed organization focus on performing activities where resources working in a multidisciplinary team focuses on delivering a product. Switching between tasks, also known as task-switching, is not efficient from the time perspective. When working Agile, the team chooses: {{
Individuals and interactions over processes and tools.
{{
Working software over comprehensive documentation.
{{
Customer collaboration over contract negotiation.
{{
Responding to change over following a plan.
Optimizing Processes Using Lean The eight types of waste can be recalled by thinking TIMWOODS: Transportation, Inventory, Motion, Wait, Overprocessing, Overproduction, Defects, and Non-Utilized Skills. VSM is about identifying (and removing) wastes in a process. Business Value Optimization and Business Analysis Using Story Mapping MVP is a great tool for verifying hypothesis. Story Mapping helps build your MVP in an iterative manner where one can use Agile methodologies to engineer the solution.
Module End Questions Q1. Which four items do the Agile Manifesto consider to uncover better ways of developing products? a) Contract negotiation, processes and tools, responding to change, and customer collaboration b) Comprehensive documentation, working software, processes and tools, and responding to change c) Processes and tools, comprehensive contract negotiation, and following a plan
documentation,
d) Individuals and interactions, working software, customer collaboration, and responding to change
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
151
Course Book | DASA DevOps Fundamentals
Q2. What is correct in a value driven context? a) Resources and time are fixed and functionality is estimated. b) Resources and time are estimated and functionality is fixed. c) Time and functionality are fixed and resources are estimated. d) Quality is fixed and resources, time, and functionality are estimated. Q3. With regard to the Scrum approach, match each term with the corresponding description. Term
Description
a) Product Backlog
i Session that occurs at the closing of a Sprint and generally includes a product demo
b) Potentially Shippable Product
ii Person who ensures that the user stories adhere to the Definition of Ready
c) Scrum Master
iii Product increment delivered at the end of each Sprint
d) Product Owner
iv Person who enables the team to perform the tasks that are required to make the product work
e) Sprint Review
v Session that is used to define what user stories are expected in the next Sprint
f) Backlog Refinement
vi A continuously evolving and ordered list of requirements
152 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
DevOps
entals
6 Automation Module ModuleObjectives Objectives
Application maturity and platform services Importance of cloud
Cloud principles
Concepts, principles, and benefits of continuous delivery
Automated provisioning Impact of automation on software delivery processes Copyright © 2018 | 1
6A Automation Concepts Topics Automation for Delivery of Software Continuous Delivery Core Concepts Continuous Delivery Automation Concepts Continuous Delivery Automation Focus Topics Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
153
Course Book | DASA DevOps Fundamentals
A DevOps
amentals ium
Automation for ivery of Software
6A Automation Concepts: Automation for Delivery of Software
Routine Jobs are Increasingly Automated Routine Jobs are Increasingly Automated
Technology advancements imply that increasingly more routine work
Technology advancements that increasingly more routine work can and will be can and will be imply automated, for example, self-driving cars can replace automated, for taxi example, cangraph replace drivers. Consider the follow drivers. self-driving Consider the cars following that taxi shows the growth of jobsthe in US for theof last two in decades. graph that shows growth jobs US for the last two decades.
ntinuous Delivery Core Concepts
ntinuous Delivery mation Concepts
ontinuous Delivery tion Focus Topics
Source: Federal Reserve Bank of St. Louis, January 2016
Source: Federal Reserve Bank of St. Louis, January 2016
Routine Jobs Examples of routine cognitive jobs include sales and occupations performing administrative tasks. Examples of routine manual jobs include construction, transportation, production, and repair occupations.
The graph shows that the number of routine jobs whether cognitive (requiring mainly the brain) or manual (requiring mainly the body) has not grown. This effect is mainly due to automation and offshoring these type of jobs. Routine tasks are good candidates for automation in IT. Traditionally, IT software delivery has been approached as (one off) projects. A task or work breakdown used to drive the project. From the perspective of a single project, many tasks appear to be non-routine at the start of the project because only a few task iterations are planned. In fact, many project routine activities are executed many times throughout the project (planned or unplanned). Furthermore, after the production handover to operations, many manual routine tasks are required throughout the lifecycle of the software delivered by each project. Continuous Delivery and Data Center/Cloud Automation have a profound impact on the automation of routine tasks in IT. With Continuous Delivery and Data Center/Cloud Automation, many manual tasks, such as installation and deployment activities, are being automated. Combined with Agile and Lean principles, these initiatives identify the routine tasks within the software delivery which can be automated.
154 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 6 | Automation
Automation Changes the Focus Towards Engineering Tasks
SA DevOps
damentals mium
Considering software delivery processes, many tasks to deliver software have a relatively low Towards task variability and aTasks high task Automation Changes the Focus Engineering analyzability. In other words, the software delivery process can be transformed using tasks withmany low tasks variability, depicted the following Considering software delivery processes, to deliveras software have a in relatively low task variability figure.and a high task analyzability. In other words, the software delivery process can be
Continuous Delivery mation Focus Topics
Task Analyzability
Continuous Delivery tomation Concepts
Low (Non-routine Tasks)
Continuous Delivery Core Concepts
High (Routine Tasks)
transformed using tasks with low variability, as depicted in the following figure.
Automation for elivery of Software
Routine RoutineLine Assembly Task
Engineering Software Engineering Design and Coding
Automation Shifts Tasks Craft Craft Construction Task
Low (Few Exceptions)
Non-Routine Non-routine R&D Task
High (Many Exceptions)
Tips Scripted (manual) test execution, (manual) deployment execution, and (manual) code compilation are examples of routine tasks within a software delivery process.
An organization with teams focusing on engineering tasks
Should be based on a non-routine and organic structure
Applying DevOps principles
Task Variability Source: Perrow (1967)
Copyright © 2018 | 6
Source: Perrow (1967)
The figure shows the task classification quadrant by Charles Perrow. The quadrant is based on two dimensions, Tasks Analyzability and Task Variability. Task Analyzability is defined as the extent to which, when an exception is encountered, there are known analytical methods for dealing with it. Task Variability is defined by the number of exceptions to standard procedures encouraged in the application of a given technology. A value stream analysis can help to determine where processes can be improved and what are the tasks with high analyzability and low variability. Once these manual routine tasks are identified, these tasks can be automated with engineering tasks. Scripted (manual) test execution, (manual) deployment execution, and (manual) code compilation are examples of routine tasks within a software delivery process. This results in a shift, where the Software Delivery team members will mainly execute tasks with high variability and high analyzability or engineering tasks. This requires an organization to be based on an organic structure, with largely autonomous, multidisciplinary, self-organized, engineering teams. Organizations need to apply DevOps principles. When processes are highly programmed and automated, they will result in the predicted and standardized output. Automated processes are repeatable. Therefore, cost of process execution is minimized. Joan Woodward defined technical complexity as the extent to which a production process can be programmed so that it can be controlled and made predictable. In other words, automation results in predictable and standardized outcomes. These concepts can be applied to software delivery processes.
Task Qualification Quadrant The four types of tasks depicted in the task classification quadrant are: Routine: Characterized by the lack of exceptions and its depth of comprehension. Traditional manufacturing technologies, such as assembly line task belong to this category. Craft: Characterized by its lack of exceptions and unpredictable outcomes that are difficult to analyze. Construction task that demands the drafting of new designs to resolve building problems is an example of applied craft technology. Engineering: Characterized by many exceptions and its depth of comprehension. Standard and accepted methods are available to provide solutions to problems. Software design and coding task belong to this category. Accountants, most engineers, and laboratory technicians use engineering technologies. Non-routine: Characterized by many exceptions and poor comprehension. Problems appear frequently with no existing solutions. Research and Development tasks belong to this category. Commercial space engineering is an example of a non-routine technology.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
155
s
or re
ery pts
Course Book | DASA DevOps Fundamentals
DevOps Enables Teams to Focus on the Delivery of Value DevOps Enables Teams to Focus on the Delivery of Value
Development of technology has resulted in more andhas more type of can type now of Development of a technology resulted in atasks morewhich and more be automated with ease, such as: tasks which can now be automated with ease, such as: Eliminate Time-Consuming Steps
Automate the Delivery Process
Enforce Platform Standardization
Work with multidisciplinary, self-organizing Agile teams Optimize technical interfaces Eliminate waste (apply Lean concepts)
Automate manual installations and configurations Improve delivery speed, quality, reliability, and repeatability Nail down once and repeat over and over again
Profound standardization results in lower costs Lots of variation is a source of defects and delays Each variation must be maintained and requires specific procedures
ery pts
ery cs
Copyright © 2018 | 7
Automation, combined with other DevOps principles, ensure teams can focus on the delivery of value. This desired focus is first principles in the Agile Manifesto: “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software”. Teams should focus on delivering business value, and everything which hinders that should be eligible for elimination. Automation can be applied to many activities in the software delivery process enabling DevOps teams to focus more on the engineering tasks required to deliver valuable software. In addition, they can utilize a software delivery process which delivers new software features continuously using an optimized flow. Note: Continuous Delivery practices strongly rely on automation to optimize software delivery processes. Continuous Delivery is covered in more detail in the subsequent topics.
Everything as Code Modern software development tools embrace everything as code concepts, Application Programming Interfaces (APIs), Domain Specific Language (DSL), and scripts, such as first class citizens. In the traditional operations space, more and more components can be defined with code, such as software-defined networks. Combining these trends make it possible to define more and more artifacts with code.
156 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
n for ware
Modern software development tools embrace everything as code concepts, Application Programming 6 | Automation Interfaces (APIs), Digital Subscriber Line (DSL), and scripts, such as first class Module citizens. In the traditio operations space, more and more components can be defined with code, such as software-defined networks. Combining these trends make it possible to define more and more artifacts with code.
very epts
very epts
very pics
Copyright ©
Teams who strongly focus on engineering tasks and automation handle most delivery artifacts as code. Not just the software they are writing but also the software configuration, test specifications, documentation and other software delivery tasks. The sources of all these artifacts are managed using a central, consistent source version control system. A version control system ensures end-to-end tracking of any change applied to software, a deployment automation script, or something else. Source files and text files are reasonably easy to manage using such a system compared to large binary files. You can iterate, make the required changes, and wind them back. You can even branch and merge source files relatively easy. For example, rather than a developer workstation setup document (stored as binary) with manual steps to setup a developer’s machine, code can be created and maintained to perform the same task.
6A Automation Concepts: Continuous Delivery Core Concepts “Continuous Delivery is about putting the release schedule in the hands of the business, not in the hands of IT. Implementing Continuous Delivery means making sure your software is always production ready throughout its entire lifecycle – that any build could potentially be released to users at the touch of a button using a fully automated process in a matter of seconds or minutes. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
157
Course Book | DASA DevOps Fundamentals
This in turn relies on comprehensive automation of the build, test and deployment process, and excellent collaboration between everyone involved in delivery – developers, testers, DBAs, systems administrators, users, and the business. In the world of Continuous Delivery, developers aren’t done with a feature when they hand some code over to testers, or when the feature is “QA passed”. They are done when it is working in production. That means no more testing or deployment phases, even within a sprint (if you’re using Scrum).” Source: Jez Humble, August 13th, 2010
The definition of Continuous Delivery by Jez Humble describes key characteristics (highlighted bold) of Continuous Delivery. These characteristics require not only automation but also practices covering processes, people, and technology. Considering the definition, you will find that Continuous Delivery and DevOps are closely associated. Teams which have adopted Continuous Delivery defined by the characteristics listed in the definition by Jez Humble are highly effective. They succeeded in achieving the following characteristics: Increased speed and repetitiveness through automation Agility as there is no WIP Flow in the delivery An autonomous way to operate Right things in the right manner
Continuous Delivery vs. Integration and Deployment Continuous Integration Continuous Integration usually refers to integrating, building, and testing code within the development environment. Martin Fowler
Continuous Deployment Continuous Deployment is subtly different to Continuous Delivery in that release are automatically pushed into production when all tests pass. In Continuous Delivery release is a human decision. Source: Dave Farley
158 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 6 | Automation
Continuous Integration is a part of Continuous Delivery. Within an automated Continuous Delivery process, Continuous Integration covers primarily the build stage.
Continuous Integration Continuous Integration usually refers to integrating, building, and testing code within the development environment. -Martin Fowler
There is a subtle difference between Continuous Delivery and continuous deployment. Continuous Delivery has an explicit (manual) release to production decision. With Continuous Deployment, releases are automatically pushed to production. There is no human release decision. From an implementation perspective, Continuous Delivery and Continuous Deployment are not really different as an automated push to production is not hard to implement when your software is already pushed and validated automatically under all test environments.
DevOps vs. Continuous Delivery DevOps vs. Continuous Delivery Continuous Delivery and DevOps embrace similar principles, concepts, and implementations. Though these two do not have the same origin Continuous Delivery andofDevOps embrace concepts, and each these might putsimilar more principles, emphasis on certain and topics or implementations. Though these two do not have the same origin and each of these might aspects, DevOps and Continuous Delivery should be considered put more emphasis on certain topics or aspects, DevOps and Continuous Delivery should together as the two complement each other. Continuous Delivery be considered together as the two complement each other. Continuous Delivery should not should not be viewed as a part of DevOps or the other way around. be viewed as a part of DevOps or the other way around.
Copyright © 2018 | 12
In the following topics, you will cover the concepts of Continuous Delivery, Continuous Delivery automation, and some aspects of the non-automation focus topics. These non-automation Continuous Delivery focus topics are Agile and Architecture. These two focus topics go beyond automation and a scientific optimization of the software delivery process. The non-automation Continuous Delivery focus topics cover: The Agile focus topics cover aspects of culture, collaboration, sharing knowledge, and a product delivery and support mindset. The Architecture focus topics cover not only the application and infrastructure IT landscape but also the aspects of an organization. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
159
Course Book | DASA DevOps Fundamentals
DASA DevOps Fundamentals Premium
Tips
Benefits of Automating Continuous Delivery Benefits of Automating Continuous Delivery Automation of the software delivery process is one of the core means to deliver faster, Automation of the software delivery process is one of the core means cheaper, and better delivery.
Automation for Automating existing tasks and Delivery of Software procedures is not enough to achieve Continuous Delivery Concepts the characteristics Core of the Continuous Delivery definition. It also requires Continuous Delivery Automation Concepts capabilities, mindset, Agile and Lean principles,Continuous and behavior to Delivery Automation Focus Topics implement Continuous Delivery successfully. These non-automated requirements help drive the actual task automation needs and implementation of task automation.
to deliver faster, cheaper, and better delivery.
Faster
Automate software delivery process
Cheaper
Remove error-prone steps in process
Better
Measure and create a feedback loop
The following pieces of information let you understand how the automation of continuous delivery results in faster, cheaper, and better delivery.
Copyright © 2018 | 13
Faster due to: {{
{{
{{
Automated task execution does not depend on the availability of humans, so the wait time is eliminated. Automated task execution requires no human-to-machine interaction time. Automated task execution reduces the need for (manual) meta tasks (like a review of an installation document).
Cheaper due to: {{
{{
{{
{{
Automated task execution is more reliable than manual task execution. Manual execution errors are expensive and might not be detected immediately. Automated task execution is more repeatable than manual task execution. Automated task execution requires no human effort (which is more expensive than machine time). Automated task execution requires standardization, based on minimal required variations. Every variation requires specific procedures and maintenance.
Better due to: {{
{{ {{
{{
160 │ Copyright © 2018
Automation results in predictable, standardized, reliable, and repeatable outcomes. Automation eliminates manual task execution errors. Automation (with test automation) results in faster feedback loops. As a result, defects are detected early in the process. Automation enables measurement driven evaluation of the delivered software features.
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 6 | Automation
Not Just Automation Automating existing tasks and procedures for project-driven processes and organizations is not enough to achieve the characteristics of the Continuous Delivery definition. Automation is not just a technical implementation activity. Specific team capabilities, mindset, Agile and Lean principles, and behavior are required to implement Continuous Delivery successfully. These non-automated requirements help drive the actual task automation needs and implementation of task automation.
Cycle Time Reduction: The Primary Goal of Continuous Delivery Suppose the Development cost is 10% of revenue for a company, for example, $100 million (Dev cost) for a $1 billion product. In addition, the new product development cycle is every 12 months. To improve the efficiency, you have various options, such as: Option 1: Improve efficiency of development with 10% (↓) in development cost. Impact: $10 million reduction in development cost Option 2: Reduce development cycle with 10% (↓) in product development cycle. Impact: $100 million add to top line revenue (as product starts to sell 1.2 months earlier) No efficiency improvement will outperform cycle time reduction! Cycle time reduction results in a competitive advantage with significant potential financial benefits. In competitive markets, optimized cycle times are crucial for the success of an organization. Therefore, you should apply Continuous Delivery principles to minimize the software delivery cycle time within your organization.
Continuous Delivery Base Principles Continuous Delivery can be adopted with the following three base principles: Base Principle 1: Rigorous Automation (of the software delivery process)
Tips “To improve is to change; to be perfect is to change often.” – Winston Churchill
Base Principle 2: Extreme Feedback (within the software delivery process)
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
161
ps
n for ware
Course Book | DASA DevOps Fundamentals
Base Principle 3: Continuous Change (throughout the software delivery process) Rigorous Automation: Automation of software delivery activities, such as test execution and deployments results, make the software delivery process faster, cheaper, and better. Automated task execution requires no manual (human) machine interaction time as it eliminates wait times as a result of dependencies between manual tasks. In addition, it eliminates the need for manual task execution validation activities. Automation results in a highly reliable, repeatable, standardized process to transform business ideas into working software in production. Extreme Feedback: Extreme feedback helps teams to analyze the effectiveness of their software delivery process and the features they have delivered. It enables teams to experiment with (the implementation of) software features using statistical evaluation methods and real customer feedback. Continuous Change: The first two principles, Rigorous Automation and Extreme Feedback, enable continuous change. The team applies collected feedback, ideas and observations of team members, and best practices from their organization or external best practices to improve their software delivery process and the (software) product.
Continuous Delivery Focus Topics
Continuous Delivery Focus Topics FULLY AUTOMATED SOFTWARE DELIVERY PROCESS
AGILE ORGANIZATION
①
AUTOMATED BUILD
②
AUTOMATED TEST
③
AUTOMATED DEPLOYMENT
④
⑤
Continuous Integration
very epts
T O
very epts
very pics
AUTOMATED PROVISIONING
Deliver fast Deliver often Do the right things
Improve quality Increase predictability
Improve reliability Repeatable Reduce cost Increase speed
ARCHITECTURE
⑥
A
P
Release insight Reduce costs Reduce release time Increase speed Reduce errors Reduce risk Less downtime Cost reduction
Copyright © 2018 | 16
162 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 6 | Automation
Continuous Delivery can be adopted using six focus topics. These topics are: Agile Organization: An Agile organization adopts Agile and Lean principles combined with autonomous, multidisciplinary teams. A DevOps organization with its culture and principles, as already covered in previous modules, represents such an Agile organization and more. Automated Build: Automated build is defined as the automated process to transform code changes (committed by team members) automatically to published deployment artifacts, ready for deployment and validation in (test) environments, in a consistent manner. Automated Test: Automated testing involves automated test execution of test specifications/scripts. Some of the examples of tests are static code quality analysis, unit tests, functional tests, and load tests. Automated Deployment: Automated deployment is defined as the process to automatically deploy published deployment artifacts to application environments. The process includes steps to move deployment artifacts to target (virtual) machines, and steps to configure these (virtual) machines and other servers/components used by the software.
Tips Out of the six focus topics, four of them are automation focus topics and other two are non-automation focus topics. The automation focus topics are Automated Build, Automated Test, Automated Deployment, and Automated Provisioning. The two nonautomation Continuous Delivery focus topics are Agile Organization and Architecture. The Agile focus topics cover aspects of culture, collaboration, sharing knowledge, and a product delivery and support mindset. The Architecture focus topics cover application and infrastructure IT landscape and the aspects of an organization.
Automated Provisioning: Automated provisioning ensures the components of (test) environments, such as network components, server components, and runtime software stacks, can be created on demand using a fully automated (system provisioning) process. Architecture: The architecture(s) defined at all levels, such as enterprise, business, application, and technical, either amplifies or dampens the ability to optimize the software delivery process of teams, for example: {{
{{
Strong coupling between teams results in low autonomy of teams due to cross team dependencies. Software architecture determines the complexity of adopting software code changes.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
163
ps
Course Book | DASA DevOps Fundamentals
6A Automation Concepts: Continuous Delivery Automation Concepts Delivery Implies: Continuous Delivery Continuous Implies: Software has Software to Flowhas to Flow
on for tware
elivery cepts
elivery cepts
elivery Topics
Copyright © 2018 | 18
Continuous Delivery implies software has to flow ensuring a continuous flow from ideas to software features released in production. In development processes, which are not based on the base principles of Continuous Delivery, there is only a limited amount of flow. There are many flow breakers, such as manual handovers, inventories and other types of waste. Some of the characteristics of a non-optimized software delivery process are: The process is based on an organization with siloed teams, where individuals work on (functional) specialty tasks with many handovers between the different specialties. For example, an architect writes a solution architecture document. Once completed, there is a handover to the Information Analyst to specify the functional design. Support activities to build, validate (such as, test), and deploy the software are manual activities based on (newly created or updated) instructions/specifications, every time a support activity has to be executed. The runtime test and production environments are implemented and maintained manually.
164 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 6 | Automation
DevOps
mentals m
Continuous Delivery Implies: Software has to Flow (Contd.)
Automation for ery of Software
nuous Delivery Core Concepts
nuous Delivery ation Concepts
inuous Delivery n Focus Topics
Copyright © 2018 | 19
A DevOps organization based on performing DevOps teams combined with rigorous automation of the software delivery process results in an optimized flow. With an optimized flow, pushed software changes (code) can be released in minutes. A DevOps team ensures high quality software changes are released through a fully automated software delivery pipeline and optimizes the delivery process. In addition, automation ensures fast feedback is available with a high defect removal efficiency rate (fail fast). The characteristics of an optimized flow are: High performing Agile DevOps teams Maximized amount of flow Automated support activities (build, test, and deployment) DASA DevOps Continuous Delivery Enables DevOps Team Performance Automatic provisioning and management of runtime environments
Fundamentals Premium
An optimized software delivery process is vital for the performance of DevOps teams. Su a process incorporates a fast feedback loop and iterative delivery of software features, a Continuous Delivery EnablesAutomation DevOps Team Performance depicted in the following figure. for Delivery of Software
An optimized software deliveryContinuous process Deliveryis vital Concepts for the performance of DevOpsCore teams. Such Continuous Delivery a process incorporates a fast feedback loop Automation Concepts and iterative delivery of software features, as Continuous Delivery Automation Focus Topics depicted in the following figure. Continuous Delivery automation optimizes the software delivery process in such a manner that it enables frequent releases of small chunks of functionality or software
Frequently release small chunks of functionality
Release
Improve Rapidly incorporate feedback, fail fast and learn fast
Get feedback Get regular validation of the product’s value
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
165
Copyrigh
Course Book | DASA DevOps Fundamentals
Tips Adopting Continuous Delivery requires a green field design of the software delivery process. It cannot succeed with local optimizations only. Types of Feedback
features. Frequent releases ensure providing regular feedback both from the delivery process (quality delivered) and the customer (products value). Regular feedback enables the team to rapidly adapt by incorporating the feedback in their software. It also enables them become a fast learning team using fail fast, continuous improvement, and experimentation principles to optimize their output (performance).
Types of Feedback
The following figure shows examples of feedback plotted on top of a software The following figure shows examples of feedback plotted on top of a software delivery process flow delivery process flow defined by the rounded rectangles and the arrows. defined by the rounded rectangles and the arrows.
➊
➋
Feedback on Build and Tests
➌
Feedback on Deployability
Feedback on Runtime Behavior
AGILE
AUTOMATED
AUTOMATED
BUILD
DEPLOY
Agile Team 1
Continuous Integration
Release Management (-very- light)
Agile Team 2
Dynamic Software Library
Agile Team n
AUTOMATED TEST
➊
Test Automation
➍ Feedback from the
Customer
➍
Deployment Automation
➌
➋
A
Infra Team
UTOMATED PROVISIONING
Server 1
Server 2
Server n
Provisioning Copyright © 2018 | 21
Tips Some of the examples of the different types of feedback are: Feedback on Build and Tests: Automated unit test results Automated static code analysis results Feedback on Deployability: Automated deployment execution results Automated application deployment “smoke” tests Automated application health checks
Different sources of feedback can be collected during the delivery and exploitation of software. These different sources can be utilized to optimize the software delivery process. For example, a failed unit test already indicates a software quality problem which can be acted upon. Ignoring this feedback can result in test failures later in the release process or even production incidents. The first two types of feedback provide feedback primarily on the internal quality of the software. However, the other two provide feedback primarily on the external quality of the software.
Fail Fast: Immediate and Visible Failure! In systems design, a fail-fast system is one that immediately reports at its interface any condition that is likely to indicate a failure. Fail-fast systems are usually designed to stop a faulty operation rather than attempt to continue a possibly flawed process.
166 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Ops
s
ation for oftware
Fail Fast: Immediate and Visible Failure! In systems design, a fail-fast system is one that immediately reports at its interface any condition that is likely to indicate a failure. Fail-fast systems are usually designed to stop a faulty operation rather than attempt to continue a possibly flawed process.
Module 6 | Automation
Feedback on Runtime Behavior: Automated user interface functional test results Automated load test results Feedback from the Customer: Revenue/conversion rates
Delivery oncepts
Delivery oncepts
Delivery s Topics
Copyright © 2018 | 22
Implement your automation tasks and software in such a way that it provides feedback as early as possible. In other words, fail fast! For example, stop an automated build and release process when unit tests fail. A fail-fast strategy results in early feedback and improves the transparency (visibility). It allows DevOps teams to anticipate and (structurally) improve their effectiveness.
ery Automation Approach Summarized
The Continuous Delivery Automation Approach Summarized
its of automated
Traditional
hipping smaller often.
a limited set of ases.
detected defects
severe defects in king the release
Experience, evaluate and improve!
Some of the points or the benefits of automated Continuous Delivery include:
continuous Improve time to market by shipping smaller releases to move pain points production more often. ocus topics. Focus by the whole team on a limited set of changes due to smaller releases.
Result in lower number of undetected defects shipped to production. Copyright © 2018 | 23
Eliminate delays caused by severe defects in a limited set of features blocking the release of many solid features. Optimize team delivery with continuous improvements that helps remove pain points in the Continuous Delivery focus topics. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
167
Course Book | DASA DevOps Fundamentals
The Continuous Delivery approach focuses on a vision where teams can concentrate on a limited set of software changes released frequently through a software delivery process, which is optimized with rigorous automation. It does not fit with project-based delivery approaches, such as Waterfall or Rational Unified Process (RUP). Therefore, adopting Continuous Delivery requires a green field design of the software delivery process. It cannot succeed with local optimizations only.
6A Automation Concepts: Continuous Delivery Automation Focus Topics
Overview
Overview already know the six focus topicsautomation. of Continuous Out Delivery You already know about the sixYou focus topics of about Continuous Delivery of automation. Out of these, four are automation focus topics as these, four are automation focus topics as highlighted in the following figure. highlighted in the following figure.
FULLY AUTOMATED SOFTWARE DELIVERY PROCESS
AGILE
ORGANIZATION
①
AUTOMATED BUILD
②
AUTOMATED TEST
③
AUTOMATED
DEPLOYMENT
④
AUTOMATED
PROVISIONING
⑤
Continuous Integration
O
Deliver fast Deliver often Do the right things
Improve quality Increase predictability
Improve reliability Repeatable Reduce cost Increase speed
Release insight Reduce release time Reduce errors Less downtime Cost reduction
T A
P
Reduce costs Increase speed Reduce risk
ARCHITECTURE ⑥
Copyright © 2018 | 25
Over the years, many solutions, tools, frameworks, and best practices have evolved for the Continuous Delivery automation focus topics. These resources can be used when Continuous Delivery is adopted by teams/organizations. The best fit solution depends on the application and infrastructure technology, DevOps maturity, and the problem/ business domain. These focus topics can help you to define and implement Continuous Delivery automation practices. Note: Key concepts of each Continuous Delivery automation focus topics are covered in this topic. This course does not cover details on the implementation of Continuous Delivery automation.
168 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
or e
y s
members automatically to published deployment artifacts, which are ready for deployment and validation in (test) environments. Team Members
Build Process
Environment
˜
Test
Acceptance Production
Pipeline
˜ ˜
y s
Automated Build: Enablescode Automated Software Package Build automation transforms changes committed by team members automatically to Delivery Flow published deployment artifacts, which are ready for deployment and validation in (test) environments. Build automation transforms code changes committed by team
˜
y s
6 | Automation Automated Build: Enables Automated Software PackageModule Delivery Flow
˜
˜ ˜ ˜
Once a team member commits a number of code changes, the changes can be merged automatically, analyzed, compiled, unit tested, and assembled automatically. The automated build process can create a new deployment package and publish it to an artifact repository. In this manner, a continuous flow from code commit to validated deployment package can be implemented.
Artifact Repository Artifact repositories manage Copyright © 2018 | collections of artifacts and metadata in a defined directory structure.
The automated build process is important for a fail-fast strategy. It is the component that provides the first (central) feedback on the quality of committed code changes. A central build system can adopt a failfast strategy. For example, if there are code merge conflicts, the build should fail so the team members can fix the conflicts. An automated build system typically includes: Source Version Control System: For example, Git, Gitlab, GitHub, Atlassian Stash, Subversion Continuous Integration Server: For example, Atlassian Bamboo, Jenkins Code Quality Analysis and Reporting: For example, Sonarcube Artifact Repository: For example, Nexus or Artefactory
Version Control System Version Control System is also known as revision control or source control. It is the management of changes to documents, computer programs, large websites, and other collections of information.
Build Tools: For example, Maven, Gradle, Grunt, NPM, Bower, Ant, Gulp Static Analysis: For example, FindBugs, CheckStyle, PMD, PHPMD Unit Test and Test Runner Frameworks: For example, Junit and Karma Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
169
26
Course Book | DASA DevOps Fundamentals
Automated Test: Optimize Software Validation (Tests) Automated Test: Optimize Software Validation (Tests) The following test pyramid, as developed by Mike Cohn, shows that you should have low-level high-level The following test pyramid, as developed bymany Mikemore Cohn, showsunit thattests youthan should have endmany to-end tests. Therefore, test early!
more low-level unit tests than high-level end-to-end tests. Therefore, test early! End-to-End or Functional Tests: Very slow, inconsequent feedback, costly, late feedback
UI Tests
SLOWER
User Interface (UI) Tests: Slow, tested after deployment, more tedious, late feedback System Tests: Fast, tested after deployment, immediate feedback
System Tests (Software, APIs, Services, Integration Components)
Unit Tests: Quick, tested even before deployment, immediate feedback, “the detail”
Unit Tests
The different types of tests include:
Tips There is a relation between the timings of the tests and the accompanied cost factor associated. As the tests are delayed, the cost factor rises. Suppose an application has 3 layers and each layer has 5 flows. To test all paths through the entire application, it will require: End-to-End Tests: 53 = 125 Unit Tests: 5 x 3 = 15
MORE COSTLY
Functional Tests
Copyright © 2018 | 27
End-to-End or Functional Tests: Tests the flow of the process and ensures functional requirements are met resulting in happy scenarios. End-to-end tests inform you when the code is failing. UI Tests: Tests the product’s UI to ensure it meets its specifications. System Tests: Tests software or hardware on a complete, integrated system to evaluate the system’s compliance with its specified requirements. Unit Tests: Tests the functionality, the edge cases, boundaries, and exceptions. Unit tests inform you where the code is failing. A reverse pyramid (as depicted in the figure) is an anti-pattern. When you automate the higher level tests, it results in a very costly antipattern! Siloed testing is slow, expensive, and in many cases not a viable option. Testers and developers should work together to write, run, automate, and maintain tests, which implies: Whole team testing Test code is production code Tests should be written before code is written. Once written, tests can be used over and over again and help keeping the system stable.
170 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
s
for are
ery pts
ery pts
ery pics
Module 6 | Automation
Automated Test: DevOps Merges Specification and
Automated Test: DevOps Merges Specification and Verification Verification Test automation can be adopted in the software delivery process. Combining Development (BDD) andCombining Test-Driven Test automation can Behavior-Driven be adopted in the software delivery process. BehaviorDevelopment(BDD) (TDD)and principles, testing becomes a first class citizen intesting the Driven Development Test-Driven Development (TDD) principles, becomes a first class citizen in thereplace process, and techniques process, and techniques traditional Waterfallreplace processtraditional artifacts. Waterfall process artifacts. Write a failing feature test
BDD
Write a failing test
TDD Refactor
Make the test pass
n cycles Copyright © 2018 | 28
TDD is an approach to software development process where the developer starts with defining a small test, and then builds the code to make the test succeed. This style of working leads to small iterations and produces many tests that are used to verify the code. The developer can safely refactor the code as it is developed through tests. BDD applies the same principles as TDD and combines it with domaindriven design and object-oriented analysis. The main difference is that BDD focuses on the functional domain opposed to the technical domain of TDD. Given an Agile development process, the development cycle starts with a backlog of user stories. These user stories are refined with clear acceptance criteria. Now, a feature test can be specified using the BDD “Specification By Example” technique. This technique defines functionality using a strict syntax (Gherkin specification). The specified functionality (for example, the feature test scenarios) proves the acceptance criteria of the user story. The feature test specification is also a functional specification of the system. When the feature test is executed, it will fail at this point of time as no code has been written to implement the user story. It is time to specify low level tests, such as unit tests. Specifying these types of test is both a test specification as well as a (detailed) design activity. Writing these type of tests upfront forces you to think about the (planned) software design. When these low level tests are executed immediately, they fail as no code has been written to implement the user story. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
171
Course Book | DASA DevOps Fundamentals
Now, the software for (part of) the user story can be written. The implemented software can be tested with the specified low level tests. The goal is to ensure that all low level tests run successfully. This includes fixing bugs and refactoring the software code and/or test code. Once, all low level tests are successful, the feature tests can be executed. If these tests fail, the process of writing low level tests, adjusting the software code, fixing bugs, and refactoring is repeated until all feature tests and low level tests run successfully.
Automated Deployment: What is application deployment? Application deployment implies: Installing applications Updating applications Configuring resources Configuring middleware components
mated Deployment: What is Starting/stopping application components deployment? Configuring the installed application
ation deployment implies:
Configuration systems like load balancers, routers
alling applications
Verification of components
dating applications
Scaled to the enterprise
nfiguring resources
nfiguring middleware components Tips rting/stopping components The figure depicts the deployment nfiguring installed application of athe versioned software package to multiple environments. The ability nfiguration systems like load to deploy the same versioned ancers, routers package to multiple environments is ificationimportant. of components
aled to the enterprise Environments must be consistent and environment specific configuration settings (like a host name) must be externalized using configuration files/dictionaries. These capabilities ensure consistent behavior across environments and traceability.
Prod 1, 2, and 3
My app v 1.x v 2.x v 3.x
QA 1 and 2
Dev 1
Copyright © 2018 | 29
Providing a new functionality is always a traditional clashing point between development and operations as operations is accountable for continuity. Business units would like to have new functionality live in production as soon as possible, while Service Management is responsible for maintaining applications on a technical perspective and operations is responsible for hosting the application.
172 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
ps
Module 6 | Automation
Application deployment is where the two worlds meet! There is lots of confusion at the boundary of these worlds. Developers are typically concerned with: Understanding requested features Designing components Writing code Writing test cases Compiling code Testing code An Operator is typically concerned with: Installing server hardware and Operating Software (OS) Configuring servers, network, and storage Monitoring servers Responding to outages Appling security measures Maintaining disaster recovery protocols Note: It is essential that, especially on the organization boundaries, everything works as smooth as possible. Repeatability, stability, and consistency are key to smooth functioning.
Benefits Automated Deployment Benefits of of Automated Deployment Full deployments with a press of a button
Cost Effective
Fast
Activities reusable over and over again
n for ware
very epts
very epts
very opics
One standardized central point of access Maintainable
Deployments with full transparency Transparent and and informative Informative
Automated deployments are highly accessible
Reuse of deployment patterns
Auditable
Accessible
Repeatable
Reliable
Scalable
Deployments can be audited easily
Secure
Consistent
Tested and proven deployments
Automation ensures that deployments are fully secure
Subsequent deployments identical over time
Deployment patterns can be applied to many environment over time
Copyright © 2018 | 30
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
173
Course Book | DASA DevOps Fundamentals
Deployment Strategies Apart from being able to govern feature implementations by utilizing an Enterprise Backlog or attending Scrum of Scrums, features can also be governed by applying different techniques, such as: Feature Switches: Feature switches are a software development best practice of gating functionality. These switches are used to turn off or on the deployed functionality using the feature flag. Using these switches, you can manage the entire lifecycle of a feature. Dark Launches: Dark launching is the practice of deploying the first version of a service into its production environment well before release so that you can Soak Test it and find any bugs before you make its functionality available to users. The term was coined by Facebook to describe its technique for proving out its chat service. Soak Test is the practice of testing a service under production load for a period of time to ensure it can handle the load. Blue-Green Deployments: Blue-Green deployment is a way to safely deploy applications serving live traffic by creating two versions of an application (Blue and Green). To deploy a new version of the application, you need to drain all traffic, requests, and pending operations from the current version of the application, switch to the new version, and turn off the old version. Blue-Green deployment eliminates application downtime and allows you quickly rollback to the Blue version of the application, if necessary. Canary Releases, Friends, and Family: The Canary release is a technique to reduce the risk of introducing a new software version in production by slowly rolling out the change to a small subset of users before rolling it out to the entire infrastructure and making it available to everybody. Some examples are: Feature Switch: When doing a deployment, you can disable a new functionality for a specific environment. Disabling features in production while keeping the code merged in the mainline can be beneficial when working on larger features. For example, you run a Web shop and want to update the checkout process to enable it for one test environment. However, the entire environment except that test environment continues to use the old checkout process. In such a scenario, a feature switch can help determine the checkout process to use without making any changes to the code. Dark Launch: Similar to a feature switch, a dark launch can be used to disable or enable functionality. In case of a dark launch, the disabled code executes though it is invisible to the user. An example of a dark launch is Facebook’s chat service. They
174 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
s
Module 6 | Automation
started with the service by letting all their users connect to it while not displaying anything. Blue-Green Deployments: Suppose you have a Web shop application running. In a traditional deployment, you stop the application server, replace the application, and then restart it. However, in a blue-green deployment, a new deployment is added next to the current one. All the traffic is then diverted to the new deployment using a load balancer. Switching back to the old traffic is easy in case any errors occur. You can even remove the old deployment once it stopped receiving traffic. Canary Releases: Canary releases are similar to the bluegreen but rather than switching big bang you start with small steps. For example, you might start with redirecting 5% of the traffic to the new deployment while ensuring everything runs fine. You gradually build this up until 100% of the traffic is routed to the new deployment. At any point, you can decide to switch back to the old version, such as in case of any unexpected errors. Friends and Family: These are similar to canary releases. However, instead of randomly selecting a percentage of users you have a group of beta or alpha testers that always get the latest releases first. These people opt-in to the new features. Therefore, they are more likely to accept any small bugs that might make it to production and can alert you before you release it to customers.
Automated Provisioning Automated Provisioning
very epts
Puppet agent
very epts
very pics
in 10 minutes...
Deploy
n for ware
Provision
in 30 minutes...
Oracle Services
JEE apps
.NET apps
JBoss EAP 6
Tomcat 6
IIS 8.5
Oracle OSB 11g
Oracle Database
Oracle SOA Suite
RHEL 6.4
WIN12 R2
VMWare
Applications and Services
Middleware
OS
Infrastructure
Copyright © 2018 │
175
Copyright © 2018
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Automated Provisioning Goals of automated provisioning: Consistency across application environments Standardization of the platforms and reduction of the number of variations Control, traceability of system changes, no manual changes, and a single, central source of truth New environments are delivered within minutes/ hours instead of weeks/ months, hence, great speed Defect reduction
Rather than provisioning and managing environment components manually, such as an application server, DevOps teams must be able to acquire new environment components on demand, fully automated. No manual setup, installation, configuration, and maintenance should be required. Automated provisioning is defined as the fully automated delivery and maintenance of application environment components. Application environment components are the deployment target containers of the application, such as a database server or an application runtime server. In a DevOps organization, automated provisioning can be the responsibility of DevOps Platform teams. Ideally, these teams deliver platform products, which are used autonomously by DevOps Business System teams. In other words, the Business System teams can use the platform products via fully automated self services. There is a clear separation of responsibilities. Continuous Delivery for Automated Provisioning Automated provisioning or data center automation requires a development perspective rather than an operations perspective. Changes to infrastructure should be provisioned in a controlled manner, including extensive verification of these infrastructure changes. Infrastructure changes must be viewed as code (Infrastructure as Code) rather than change instructions bound to a change ticket. This new perspective implies that the software delivery process based on Continuous Delivery can be applied to manage infrastructure changes (for example, the platform products). This approach requires profound standardization of the platform product characteristics. The preceding figure shows an example of how automated provisioning and automated deployment are related. To be able to deploy your application, provision the infrastructure and the required middleware first. You start off by provisioning the virtual machines with an operating system, and then the middleware. Once the provisioning is done, you can start with deploying your application on top of the middleware.
176 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
ps
Module 6 | Automation
Automated Provisioning Repeat Multiple Times Automated Provisioning Repeat Multiple Times
Deploy
n for ware
ACCEPTANCE
very epts
DEVELOPMENT
very epts
very pics
Provision Note that: Oracle SOA Suite alone consists of 5 hosts. It also requires Oracle Database. Acceptance and Production environments are clustered. It is provisioning 10ths of servers.
TEST
PRODUCTION
INTEGRATION
Imagine the cost and errors, considering: Doing this manually While being in control of rollbacks Multiple times per day!!
Copyright © 2018
The automated provisioning process can be repeated to create platform/software stacks multiple times in a consistent manner. Automated provisioning can be used to provide different environments from a single source on demand using a repeatable process. It will result in consistent and version controlled environments, which are similar. Please note that limited differences between environments are still supported, such as the number of server instances in the environment or environment specific configuration setting values. However, automated provisioning is capable of controlling such (limited) differences. The preceding figure shows how it is easy to scale up and gain more consistent environments using the automated approach. It shows an example of an environment scaled out to multiple environments.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
177
Course Book | DASA DevOps Fundamentals
Containerization (Microservices) Containerization (Microservices) Static Code Analyses Sonar
Product teams Product teams ProductApplication
teams
SCM
Continuous Integration
git
JenMave kins n Jenkins
Application Middleware
Release Automation
DSL
Automated Deployment
Mave Archiva n Repo
DSL
tightly dependent!!
Automated Testing Fitnesse / Selenium, Cucumber, Jmeter, Loadrunner ...
Middleware
Middleware
Middleware
Middleware
Dev infra
Tst infra
Acc infra
Prod infra
layer
ops
layer
layer
layer
Copyright © 2018 | 34
When deploying applications on servers, there is a tight coupling. Therefore, upgrading the application server can cause unexpected changes. For example, you might want to deploy your application on a different version or configuration of the application server. To counter this problem, companies try to keep environments as similar as possible by using automated provisioning. It counters many of the issues that might occur but require coordination to ensure application deployments and infrastructure provisioning happen in sync. In many organizations, these two practices are done by separate teams, which makes the process even more difficult. One possible solution to reduce the complexity of application deployments and infrastructure provisioning is containerization. By bundling the application with the application server, you can create a runnable artifact that does not have dependencies besides the containerization technology. The containerization interface does not change like the libraries bundled with the application server, and therefore, is a more stable interface. This runnable artifact, or a container, can be built and deployed throughout the software delivery pipeline. You can build a container for a test environment and deploy it throughout the Development, Testing, Acceptance, and Production (DTAP) stages. Using the same container ensures the non-environment specific configuration remains the same. As a result, you will not get any unexpected results when deploying.
178 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
ps
Cloud y and iples
vices vOps ation
oning cepts
oduct s and turity
DASA DevOps
Exercise: Defining a Continuous Delivery Backlog
Fundamentals Premium
Module 6 | Automation
List down backlog activities required to automate a software delivery process and categorize these as per Continuous Delivery focus topics. Further, list down prerequisites to perform these activities with the possible impediments.
Automation for
of Software Exercise: DefiningDelivery a Continuous Delivery Backlog Continuous Delivery
Concepts List down backlogCore activities Continuous Delivery required to automate a software Automation Concepts delivery process and categorize Continuous Delivery Automation Focus Topics these as per Continuous Delivery focus topics. Further, list down prerequisites to perform these activities with the possible impediments.
Backlog of activities:
Prerequisites of activities:
Activity 1
Potential impediments of activities:
Activity 2
Activity 3 Activity 4 Activity 5 Activity 6
Activity 7
6B Data Center Automation
Copyright © 2018 | 35
Emergence of Cloud Technology and Principles Cloud Services Concepts in a DevOps Organization Automated Provisioning Concepts Platform Product Characteristics and Application Maturity
6B Data Center Automation: Emergence of Cloud Technology and Principles
Emergence of Cloud Computing Emergence of Cloud Computing
The following timeline figure shows a brief history of cloud computing
Thefrom following timeline figure shows a brief history ofand cloud computing from 1950, and the 1950, and the emergence of cloud computing enablers cloud technology.of cloud computing enablers and cloud technology. emergence Cloud Computing introduced as a term Amazon EC2
Mainframe “time sharing”
Virtual Machines
1969
1950 1950s
2003
1990s 1970s
Docker
Amazon AWS
1997 2002
2006
2017
2009 2013
2020
Hypervisor ARPANET Salesforce.com
Virtualized Private Networks
Estimated Enterprise Spending: $235.1B
Microsoft Cloud Google Cloud Platform
Copyright © 201
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
179
Course Book | DASA DevOps Fundamentals
The following pieces of information discuss the history behind each timeline listed on the figure: 1950s: In the 50s, mainframe computers were huge, occupying entire rooms. Due to the cost of buying and maintaining mainframes, organizations cannot afford to purchase one for each user. The solution was “time sharing” in which multiple users shared access to data and Central Processing Unit (CPU) time. The term “time sharing” is the premise of cloud computing. 1969: J.C.R. Licklider, an American psychologist and Computer Scientist, developed the Advanced Research Projects Agency Network (ARPANET). The network became the basis of the Internet. His vision was for everyone on the globe to be interconnected and accessing programs and data at any site, from anywhere. 1970s: IBM released an operating system called Virtual Machine (VM) that allowed admins to have multiple VMs on a single physical node. The VM operating system took the 50s “time sharing” model to the next level. Most of the basic functions of any virtualization software that you see nowadays can be traced back to this early VM operating system. 1990s: Telecommunications companies started offering virtualized private network connections, which meant it was possible to allow for more users through shared access to the same physical infrastructure. This change enabled traffic to be shifted as necessary to allow for better network balance and more control over bandwidth usage. Meanwhile, virtualization for PC-based systems started in earnest. As the Internet became more accessible, the next logical step was to take virtualization online. 1997: The term “cloud computing” is coined by University of Texas professor Ramnath Chellapella in a talk on “new computing paradigm”. However, the term actually had been used a year earlier in Compaq. 1999: The arrival of Salesforce.com in 1999 pioneered the concept of delivering enterprise applications via simple websites. The services firm covered the way for both specialist and mainstream software firms to deliver applications over the Internet. 2002: Amazon created Amazon Web Services (AWS) providing an advanced system of cloud services from storage to computation. 2003: The first public release of Xen, which creates a Virtual Machine Monitor (VMM) also known as a Hypervisor, a software system that allows the execution of multiple virtual guest operating systems simultaneously on a single machine.
180 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
2009: Google and Microsoft entered the playing field. The Google App Engine brought low-cost computing and storage services. Microsoft followed suit with Windows Azure. 2013: Docker was released. Docker implements a high-level API to provide lightweight containers that run processes in isolation. Docker has become a standard for container technology, which can be used in many public cloud platforms, including Google, Microsoft, and Amazon). 2017: By 2017, enterprise spending on cloud computing will amount to a projected $235.1 billion according to IHS Technology research.
Types of Cloud Services Types of Cloud Services
The Microsoft cloud reference architecture classifies cloud services with a taxonomy based The types, Microsoft architecture classifies cloud services on four ascloud listedreference in the following figure. with a taxonomy based on four types, as listed in the following figure.
vices vOps ation
Infrastructure
Applications
Applications
Data
Data
Runtime
Runtime
Middleware
Middleware
Middleware
O/S
O/S
O/S
Virtualization
Virtualization
Virtualization
Servers
Servers
Storage
Storage
Networking
Networking
Data Runtime
Other Manages
Servers Storage Networking
O/S Virtualization Servers
Other Manages
Applications
Other Manages
Applications
Middleware
(as a Service)
Software
(as a Service)
Runtime
You manage
Platform
(as a Service)
Data
oning cepts
oduct s and turity
On-Premises
You manage
Cloud y and ciples
2006: Amazon introduced the Elastic Compute Cloud (EC2) as a commercial Web service. The EC2 lets smaller companies rent computers on which they can run their own computer applications.
You manage
ps
Module 6 | Automation
Storage Networking
Source: Adapted from https://www.hostingadvice.com/how-to/iaas-vs-paas-vs-saas/
Source: Adapted from http://www.imarda.com/blog/enterprise-cloud-software/
When choosing a cloud service, it is important to realize the more other people manage the components for you, the less choice you have in these components. On-Premise gives you the most freedom but requires you to do everything yourself. SaaS gives you the least freedom but helps reduce the workload of things you have to manage.
Copyright © 2018 | 4
Some examples that can help you choose the right type of cloud service are: On-Premise: If you have strict regulations where your data might be stored or need specific hardware, you might choose to do everything on-premise, for example, banks or government organizations. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
181
Course Book | DASA DevOps Fundamentals
Infrastructure as a Service (IaaS): When you have teams with diverse workloads, you can go with IaaS. It provides the teams the most flexibility when removing the burden of maintaining the physical infrastructure. Platform as a Service (PaaS): If you have more of homogeneous workload, you can go with PaaS. Standardizing the deployment and the target stack application helps teams to focus on the development of their applications. Software as a Service (SaaS): If you want to use a functionality but do not want to develop the software, you can choose SaaS. This approach allows you to remove management of everything except the configuration of the application.
National Institute of Standardization (NIST) Cloud Principles The following points describe the NIST Cloud Principles: Resource Pooling, Abstraction, and Isolation: Resource optimization is a principle that drives efficiency and cost reduction. It is achieved primarily through resource pooling. Abstracting the platform from the physical infrastructure enables optimization of resources through shared use. It enables a higher resource utilization. Isolation for software, running “in the cloud”, ensures software runs independently. This ensures the behavior of one software package does not affect the behavior of another software package. Furthermore, isolation is essential for autonomy. Consider two applications on the same runtime server. They probably cannot be released independently from one another. Elasticity: From the consumer’s perspective, cloud services appear to have infinite capacity. The consumer can use as much or as little of the service as he\she needs. Continuous Service Availability: From the consumer’s perspective, cloud services should always appear available when required. The consumer should never experience an interruption of service, even if failures occur within the cloud environment. Predictability: Predictability is a fundamental cloud principle whether you are a consumer or a provider. From the vantage point of the consumer, cloud services should be consistent having the same quality. Consistency is realized through homogenization of underlying physical servers, network devices, and storage systems. A Service Provider’s Approach: When an organization takes a service provider’s approach for delivering information technology, a key capability is to be able to meter resource utilization and charge users for that usage. IT departments purchase the necessary components and build an infrastructure
182 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 6 | Automation
that is specific to the service requirement, when asked to deliver a specific requirement. This process can result in an increase in time-to-market, higher costs because of duplicate infrastructures, unmet business expectations of agility, and cost reduction. Further compounding the issue, this model is often used when an existing service needs to be expanded or upgraded. IT departments can transform their organization by taking a service provider’s approach. When infrastructure is provided as a service, IT departments can use a shared resource model that ensures economization. They can also combine other private cloud architecture principles and concepts to achieve greater agility for providing services. Multitenancy: Multitenancy implies principle in which an infrastructure can be logically subdivided and provisioned to organizations or organizational units. The most used example is a hosting company that delivers servers to multiple customer organizations. This principle is increasingly used by organizations with a centralized IT department that provides services to multiple business or organizational units and treats each as a customer. Security and Identity: The key characteristics of security and identity are: {{
{{
{{
Protected Infrastructure: It uses security and identity technologies, which enable hosts, information, and applications are secured under all circumstances in the data center, including physical (on premises) and virtual (on premises and in the cloud) situations. Application Access: It enables IT professionals to provide vital application access to internal users, business partners, and cloud users. Network Access: Uses an identity-centric approach to provide users (internal employees or users in remote locations) with more secured access on various devices to help foster and greater productivity. A more secure data center uses common, integrated technology to provide users with easy access with a common identity. A more secure data center also integrates management across physical, virtual, and cloud environments so that a business can leverage IT capabilities without requiring significant financial investments.
Metering: Metering can be defined as real-time monitoring of all (or selected) applications running on the cloud platform. Metered services (also called pay-per-use) is any type of payment structure in which customers have access to potentially unlimited resources but only pay for what they actually use. Metered services are becoming increasingly common in enterprise IT environments. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
183
Course Book | DASA DevOps Fundamentals
DevOps Organizations Adopt Cloud Principles
6B Data Center Automation: Cloud Services Concepts in a DevOps Organization
DevOps Platform teams can adopt cloud principles for their platform product development. DevOps Organizations Adopt Cloud Principles
Business System Teams
(API) Self Service!
Monitoring
PaaS IaaS
Deployment
Logging
Service Discovery Security
Control
Auto-Healing
Backup and Recovery
Auto-Update
Platform qualities!
Platform Team
Copyright © 2018 | A DevOps organization can have DevOps Business System teams and DevOps Platform teams. The Business System teams manage end users (for example, customers) and products (applications), while the Platform teams manage platform products. The Business System teams use the platform products autonomously via self services.
In such an organization, the DevOps Platform teams are forced to adopt cloud principles for their platform product development. These principles ensure teams can work largely autonomously from one another without task dependencies. The cloud principles are a best practice for building platform products within a DevOps organization. These principles imply: A clear separation of responsibilities and accountability across teams. Self-service concepts required for optimized delivery of valuable software to customers (Continuous Delivery). Data center optimization using extensive automation. Standardization and productization of infrastructure components.
184 │ Copyright © 2018
Although the preceding figure shows a two-layered model, it can be stretched to more layers as long as each layer is end-to-end responsible for its product. In other words, you will not get any semifinished products. Each product will be ready to be used by upstream teams. One of the examples can be an IaaS platform team that takes care of virtual machines. A PaaS platform team can build a platform on these virtual machines. On the other hand, business system teams can deploy their application on the platform. Therefore, business
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
43
Module 6 | Automation
system teams can decide whether they want to talk to the IaaS team to get a virtual machine or want to talk to the PaaS team to deploy their application. As a result, the teams stay responsible for their products.
DevOps
Different Conversations
Different Conversations
entals
The following figure highlights an example of communication between the and highlights Operationsan teams of a siloed organization. between the Development and TheDevelopment following figure example of communication
ence of Cloud chnology and Principles
oud Services in a DevOps Organization
d Provisioning Concepts
Operations teams of a siloed organization.
Before we can release your application in the production environment, you have to comply with OUR acceptance criteria.
We can’t test right now, Operations still hasn’t deployed our application to the test environment. We need to rush testing to meet our deadlines!
form Product cteristics and ation Maturity
Development Project Organization
Our application must be released in production NOW though it has known issues. The Business organization demands it and YOU have delayed our project.
An application bug caused the outage. It already took hours to troubleshoot and now we have to go through an emergency patch process to fix it. We need to tighten our acceptance criteria and change management process.
It took Operations 4 hours to figure out that the application was unavailable.
A DevOps
amentals um
Operations
Different Conversations (Contd.)
Copyright © 2018 | 44
The following figure highlights an example of communication in a DevOps organization.
The following figure highlights an example of communication in a DevOps organization.
ergence of Cloud Technology and Principles
Cloud Services epts in a DevOps Organization
Use our platform product self services to deploy your application to any environment. You remain responsible for the exploitation of your application monitoring and maintenance). We ensure that the platform which runs your application is ok.
Our Continuous Delivery pipeline automatically deploys and executes our automated tests in the test environments.
ated Provisioning Concepts
Platform Product haracteristics and pplication Maturity
DevOps Business System Team
We can release our application in production ourselves when we desire. Our process and expertise ensure we deliver high quality software and we can exploit our application. We’ve built it. We run it!
Our application health monitoring notified our complete team with an SMS message less than 30 seconds after our application crashed. We found the root cause within 5 minutes and we already fixed our code to ensure we are not awakened at night.
DevOps Platform Team We can focus on building and running our platform products. The DevOps Business System teams can troubleshoot and patch their application without us involved in the process! Copyright © 2018 | 45
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
185
Course Book | DASA DevOps Fundamentals
DevOps Platform Teams as a “Cloud” Service Provider DevOps Platform Teams as a “Cloud” Service Provider
d d s
s s n
Business System Teams Request Products
g s
ct d y
Use Products (API) Self Service!
PaaS IaaS Platform Team
Platform team can use cloud concepts to define, deliver, and support their platform products. From a customer perspective (or the DevOps Copyright © 2018 | Business System teams), two use cases, request products and use products, must be supported for each product of the DevOps Platform team, as depicted in the above figure.
Tips Public cloud providers support both use cases in a similar manner. For example: Request Products: You can use the Amazon AWS Console to launch a new EC2 instance (a server). Use Products: Once it has been launched and initialized (started), your server is available for use. You can connect to the instance.
The request products use case covers the scenario where the DevOps Business System teams acquire products via self services. It includes the automated delivery of the products (for example, fulfillment). On the other hand, the use products use case covers using the acquired products (by the DevOps Business System team). Suppose Business System teams need a database for their application. The Platform team can create a self-service portal, where the DevOps Business System teams can request a database with a fully automated self service. Once the database is created (automatically), the Business System teams can use the database for persistent storage of data. DevOps Business System teams also require support tools to optimize their delivery process based on Continuous Delivery principles. A Platform team can also provide these tools as products. Acquiring these tools (or products) can also be done using a self-service portal and a fully automated delivery of the (tool) product. Once it becomes available to the Business System teams, they can use the delivery tool(s) to optimize their software delivery process. Note that these delivery tool products can be classified as Software as a Service (SaaS) products. Public cloud providers support both use cases in a similar manner. For example: Request Products: You can use the Amazon AWS Console to launch a new EC2 instance (a server).
186 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
46
DevOps
entals m
Module 6 | Automation
Use Products: Once it has been launched and initialized (started), your server is available for use. You can connect to the instance. It is important to realize that the automated nature of the products requires profound standardization of the products. A standardized platform product is generic and can be used by many teams in the present form. It has no application specific characteristics.
DevOps Business System and Platform Teams Business System Team
Focused on delivering business value through the delivery of software.
They exploit (develop, run and maintain) their application using platform services delivered by the DevOps Platform team.
Responsible for maintenance of their applications in production.
Platform Team
Focused on the delivery and exploitation of platform services for DevOps Business System teams.
Responsible for exploitation of the platform services, not the applications.
All environments used by the DevOps Business System teams represent Production environments for the DevOps Platform team.
Different Types of Clouds to Operate
Different Types of Clouds to Operate In-house Cloud Services Provider
ence of Cloud chnology and Principles
loud Services s in a DevOps Organization
d Provisioning Concepts
tform Product acteristics and ation Maturity
Managed Services Provider
(Public) Cloud Services Provider(s)
There are three extreme options to consider, while deciding if an organization needs DevOps Platforms teams:
Copyright © 2018 | 48
In-house Cloud Services Provider: Build and manage your own complete cloud end-to-end in-house. (Public) Cloud Service Provider(s): No DevOps Platform teams, use (public) cloud services only. Managed Services Provider: Have your cloud build and managed by a (third party) managed service provider. Benefits and drawbacks, given the context and requirements of your organizational needs, are applicable to all these options. Considerations can be cost, legal aspects, required organizational Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
187
Course Book | DASA DevOps Fundamentals
Cloud Brokering “Cloud Services Brokerage (CSB) is an IT role and business model in which a company or other entity adds value to one or more (public or private) cloud services on behalf of one or more consumers of that service via three primary roles including aggregation, integration and customization brokerage. A CSB enabler provides technology to implement CSB, and a CSB provider offers combined technology, people and methodologies to implement and manage CSB-related projects”. Source: Gartner
DASA DevOps Fundamentals Premium
specific services not readily available as a public cloud service and many more. More importantly though, services from the three options can be combined (or mixed). For example, a DevOps Platform team within the organization may utilize a public cloud service for one or more of their products. For example, virtual machines running certain middleware software used by DevOps Business System teams in the organization may not be available as a public cloud service. Consequently, a (in-house) DevOps Platform team can build a platform product for this middleware on top of a public cloud service which delivers a virtual machine including the operating system. In other words, the team builds a platform as a service product in-house based on an Infrastructure as a Service (IaaS) product offered by a public cloud provider. Likewise, a managed service provider may base their products on public cloud services. Note that repackaging and/or extending public cloud services can be considered as cloud brokering.
6B Data Center Automation: Automated Provisioning Concepts Automated provisioning was introduced as part of Continuous Delivery automation. It is a key enabler to optimize the flow of software through the software delivery process. In this topic, we will cover automated provisioning concepts. These concepts can be adopted by Platform teams.
Pets vs. Cattle
Pets vs. Cattle
Emergence of Cloud Technology and Principles Cloud Services Concepts in a DevOps Organization Automated Provisioning Concepts Platform Product Characteristics and Application Maturity
Pets are given names.
Cattle are given numbers.
They are unique.
They are (almost) identical to one another.
They are hand raised and are given proper care. When they get ill, they are nursed back to health.
They are managed as livestock. When they get ill, they are replaced.
Manual system provisioning and system management have many similarities with keeping pets. Servers have distinct names and are maintained manually. This includes activities to update and configure software. Most systems become unique over time as a result of the
Copyright © 2018 | 50
188 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 6 | Automation
DevOps
mentals m
applied manual software and configuration changes. When the system is not performing, its (unique) problems are analyzed and fixed. On the other hand, automated provisioning applies concepts which are similar to managing livestock (cattle). Systems are standardized and managed as a group, and are assigned random numbers. Software and configuration are defined centrally and applied automatically to many systems. Desired State Configuration to Automate Environments
Desired State Configuration to Automate Environments
gence of Cloud Technology and Principles
Push Model
Pull Model
Cloud Services ts in a DevOps Organization
ed Provisioning Concepts
atform Product racteristics and ication Maturity
Desired State
Desired State
Managed Systems
Managed Systems
One Server to Rule Them All
Adopting automated provisioning concepts means automation of environments. Platform teams can use Desired State Configuration (DSC) for this. DSC specifies the desired configuration of systems using a declarative model in a simple standard way that is easy to maintain and understand. Desired state specifies the expected software and configuration for systems. It is applied to systems, such as VMs, and is the (single) source of truth (‘One Server to Rule Them All’) for the managed systems. Desired state (changes) can be applied with a push model or pull model, as shown in the preceding figure. In a push model, the system desired state create/update event is triggered from the system which holds the desired state specification. This system connects to the managed systems and applies the desired state. In a pull model, each managed system (when initialized) polls the system which holds the desired state specification. If the desired state (or desired state change) is applied, the managed system pulls and applies the desired state.
Copyright © 2018 | 51
Tips DSC was designed in consideration to DevOps. Having a single configuration to define an environment means that developers can encode their requirements into a configuration, check that configuration into source control, and operations teams can easily deploy code without the manual processes.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
189
Course Book | DASA DevOps Fundamentals
Automated Provisioning with Mutable Infrastructure Automated Provisioning with Mutable Infrastructure Mutable components implies components can be Mutable components implies provisioning components canprovisioning be automatically changed once automatically changed once these are created, as depicted in the these are created, as depicted in the following figure. following figure.
Specification Version 1
Desired State
Apply (Create)
Managed System
Apply (Update)
Specification Version 2
Configuration Drift
Manual Changes Copyright © 2018 | 52
Tips Docker implements a high-level API to provide lightweight containers that run processes in isolation. Docker has become a standard for container technology.
Automated provisioning of mutable components implies that the components which are provisioning can be changed after these have been created. In other words, provisioned components are maintained. It includes lifecycle management of the software, which is provisioned. For example, operating system security patches are applied to running, already provisioned virtual machines when these software patches are released. Mutable components are not applied manually. The maintenance of each provisioned component is handled individually. You can still treat provisioned mutable components as “cattle”. Desired state, represents the “truth”. A typical setup for automated provisioning, including lifecycle management of mutable components, includes a central source of truth where the generic configuration of the component types is captured and maintained. A (central) Configuration Management Database (CMDB) can be utilized to capture the configuration of the component types. When updates are required, this central system can be updated with these changes. Each provisioned server regularly polls the central system for updates. When updates become applicable, the server automatically retrieves and applies the required updates.
190 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 6 | Automation
A key challenge encountered is the proper implementation of system updates, when software and configurations need to updated rather than simply replacing software and configurations. The latter is not always a viable implementation strategy. Finally, when manual updates still apply to the managed systems, there will be differences between the desired configuration specification and the actual configuration of the managed system. This is called configuration drift. Configuration drift can result in unexpected system behavior and inconsistencies between managed systems.
Automated Provisioning with Immutable Infrastructure Automated Provisioning with Immutable Infrastructure Desired State
ud nd es
es ps on
Specification Version 1
Managed System Apply (Create)
ng ts
ct nd ty
Specification Version 2
Apply (Create)
Automated Provisioning with immutable infrastructure is the practice of always replacing infrastructure instead applying changes, such as upgrading or fixing faulty servers, as depicted in the above figure. Automated provisioning of immutable components implies that the components’ provisioning never changes once these are created. In other words, provisioned components are not maintained. This implies that the provisioned components will become outdated when “must have” changes are available, such as a critical operating system security patch. Therefore, you must destroy and recreate the provisioned components every time you want to adopt changes.
Copyright ©
When you manage systems manually, destroying and recreating components is not a viable scenario. However, when automated it can be viable. The implementation of applications/business systems and their delivery processes determine whether the usage of automated provisioned immutable components is viable. A great advance of immutable components is the lack of maintenance required on running systems. Automation scripts do not have to consider (delta) update scenario’s. Every installation is a full installation and significantly reduces the complexity.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
191
Course Book | DASA DevOps Fundamentals Continuous Delivery for Platform Products
Continuous Delivery for Platform Products Characteristics: Output: product Characteristics: Deliver fast Deliver often Do the right things
Characteristics: Continuous flow of code
Characteristics: Improve quality Compile and construct continuously
AGILE TEAMS
When a Platform Product team builds/ provisions software delivery support tools, they should also use these tools to experience the “customer experience” first hand. In other words, they should “eat their own dog food”.
Characteristics: Release continuously (trust!) Release consistently Reduce costs and defects
AUTOMATED
AUTOMATED
Continuous Integration
Static Tests
Release Management (Light)
BUILD
TEST
DEPLOY
Unit Tests Functional Tests
Deployment Automation
When you adopt the goal of fully automated provisioned components, adopting a software delivery process for your platform products becomes a best practice. In such a process, you can have extensive version control, tracking, (automated) tests, automated rollout of the Copyright © 2018 platform products and more. In other words, it is a best practice to adopt Continuous Delivery for Platform teams. Like Business System teams, these teams can optimize their delivery process using the Continuous Delivery principles.
Automated Provisioning Requires an Engineering Mindset DASA DevOps
Food for Thought Which software development practices can be applied to system management/ Operations, when the Operations is based on rigorous automation?
Characteristics: Improve quality Test continuously Reduce defects and costs
AUTOMATED
Dynamic Software Library
Tips
Characteristics: Continuous flow of code
Automated Provisioning Requires an Engineering Mindset
A high-performing Platform Engineer applies software development A high-performing Platform Engineer applies software development principles to optimize principles to optimize his/her productivity and delivers high quality, his/her productivity and delivers high quality, highly maintainable platform automation code/scripts. highly maintainable platform automation code/scripts. Fundamentals Premium
Emergence of Cloud Technology and Principles
Cloud Services Concepts in a DevOps Organization
Automated Provisioning Concepts Platform Product Characteristics and Application Maturity
Copyright © 2018 | 55
Automated provisioning is based on (lots of) automation code and automation scripts. On the other hand, manual provisioning is based on knowledge of the system administrators, stack overflow and other internet resources, documents, and script snippets. In many cases, system administration is tested by executing a change in a test environment. Work instructions and procedures should ensure the change is done in the same manner on other environments. For each environment, the set of changes executed at once can differ. For example, during the Integration Test phase of a project, multiple changes are applied independently in a test environment. However, in
192 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
| 54
Module 6 | Automation
production, all these changes are applied as a single task. In practice, not all changes are tracked and implemented consistently, which in inconsistent environments. Version management, and the quality and maintainability of platform installation and configuration scripts are important, especially, when you rely on automated provisioning for managing your systems. In software development, it has long been a good practice to automatically track all changes to the software with a software version control system. Software developers publish (commit/push) their (code) changes to a central version control system. New software packages are built from this central system. In addition, software developers can optimize their productivity with (local) tests and code reviews to get fast feedback on their code changes. Writing maintainable code is important for software development. It includes the organization or design of code, such as classes and functions considering the object oriented design, the testability of the code, coding best practices, and technical debt reduction, such as code duplication and code complexity. A performing Software Engineer optimizes his/her productivity and delivers high quality, highly maintainable code. On the other hand, a performing Platform Engineer applies the same development principles to optimize his/her productivity and delivers high quality, highly maintainable platform automation code/scripts. Therefore, DASA considers programming a key capability for DevOps Engineer building platform products.
6B Data Center Automation: Platform Product Characteristics and Application Maturity Services Required by DevOps Business System Teams Capabilities for Running Applications and Tools
Application management self services, such as backup restore, server restart, and changing log levels
Logging self services (aggregated)
Monitoring self services (aggregated)
Billing services
Capabilities for Delivering Changes
Software source configuration/version management self services
Test automation and reporting self services
Continuous Integration and reporting self services
Application deployment self services
Artifact sharing, validation (among other security), and publication self services
Test data syndication (or replication) self services (ensure volatile, anonymized production data is available for certain types of tests)
Release orchestration self services Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
193
Course Book | DASA DevOps Fundamentals
DevOps Business System teams are required to autonomously exploit their application. They build it and they should be able to run it. They can only deliver and run their application, when they have the required tools or platform products in place. The preceding table lists the required platform capabilities that DevOps Business System teams can define for running and delivering their application.
damentals mium
mergence of Cloud Technology and Principles
Product
Product Teams, Cloud Services, and Freedom The freedom of the Product teams depends on the provided type of cloud services, as The freedom of the Product teams depends on the provided type of depicted in the following figure.
Cloud Services ncepts in a DevOps Organization
mated Provisioning Concepts
Platform Product Characteristics and Application Maturity
cloud services, as depicted in the following figure. SaaS
high
Platform Functionalities Offered
SA DevOps
All capabilities are used to exploit the application. Teams should be able to perform root cause analysis, such as executing a post mortem for production incidents. These activities are crucial and the only way to structurally improve the system (incrementally). The capabilities Teams, Cloud andanalysis Freedom should supportServices, these root cause activities.
Fixed, Standardized Application
PaaS Predefined Application Runtimes More Standards
IaaS
IaaS Based System Engineering Amount of Limitations DevOps Business System Teams
high Copyright © 2018 | 58
The platform products control the freedom and restrictions for Business System teams. Using the cloud services classification, IaaS products offer the highest amount of freedom, and SaaS products offer the greatest number of limitations. The flip side of the coin is the amount of functionality provided by the platform. It defines what the Business System teams have to manage themselves. SaaS products offer the maximum number of functionalities. There is not a lot, which has to be managed by the Business System team, while IaaS products provide the least number of functionalities. Any platform product design based on products with standardized characteristics targeting multiple teams/organizations has an embedded compromise between freedom offered to Business System teams and the amount of standardization applied. More standardization results in less variations/complexity in the product portfolio of the Platform teams.
194 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 6 | Automation
evOps
Applications Must be Mature Enough to use Platform Applications Must be Mature Enough to use Platform Services Services
ntals
Advanced platform features and qualities can be used only by
nce of Cloud hnology and Principles
Advanced platform and qualities be usedmanner. only by The applications which are applications which features are implemented in acan compliant implemented a compliant manner. Theare Twelve-Factor App and Reactive Manifesto are Twelve-FactorinApp and Reactive Manifesto examples of application examples of application design guidelines for building mature, cloud optimized applications. design guidelines for building mature, cloud optimized applications.
ud Services n a DevOps Organization
Provisioning Concepts
Reactive Manifesto
orm Product teristics and tion Maturity
Copyright © 2018 | 59
The Twelve-Factor App The twelve factors or best practices of the Twelve-Factor App are: 1. Codebase: One codebase tracked in revision control, many deploys 2. Dependencies: Explicitly declare and isolate dependencies 3. Config: Store config in the environment 4. Backing services: Treat backing services as attached resources 5. Build, release, run: Strictly separate build and run stages 6. Processes: Execute the app as one or more stateless processes 7. Port binding: Export services via port binding 8. Concurrency: Scale out via the process model 9. Disposability: Maximize robustness with fast startup and graceful shutdown 10. Dev/prod parity: Keep development, staging, and production as similar as possible 11. Logs: Treat logs as event streams 12. Admin processes: Run admin/management tasks as one-off processes Source: https://12factor.net/ Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
195
Course Book | DASA DevOps Fundamentals
Reactive Manifesto Reactive systems are more flexible, loosely-coupled, and scalable. Such characteristics make these systems easier to develop and amenable to change. These are significantly more tolerant of failure. When a failure occurs, these systems handle it with elegance rather than a disaster. Reactive systems are highly responsive, giving users effective interactive feedback. Reactive Systems are: Responsive: The system responds in a timely manner if at all possible. Resilient: The system stays responsive in the face of failure. Elastic: The system stays responsive under varying workload. Message Driven: Reactive systems rely on asynchronous message passing to establish a boundary between components that ensures loose coupling, isolation, and location transparency. Note: You will not be tested on the preceding information provided on the Twelve-Factor App and Reactive Systems. Reference Reading: http://www.clearlytech.com/2014/01/04/12-factor-apps-plainenglish/ http://www.reactivemanifesto.org/
Activity: Group Discussion Activity Time: 15 mins List and describe the key considerations for adopting cloud concepts within your organization, such as: Preconditions for adopting cloud principles in your organization
Potential benefits of how cloud adoption will help your organization in moving towards a DevOps organization
Challenges expected from cloud adoption in your organization
Module Summary In 6A Automation Concepts, you learned that: Automation for Delivery of Software Routine jobs are getting automated, and such an automation changes the focus towards engineering tasks. DevOps enables teams to focus on the delivery of value by eliminating time-consuming steps, automating the delivery process, and enforcing platform standardization.
196 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 6 | Automation
Continuous Delivery Core Concepts Continuous Delivery applies practices which cover process, people, and technology. Base principles of Continuous Delivery are rigorous automation, extreme feedback, and continuous change. Six focus topics of Continuous Delivery are Agile Organization, Automated Build, Automated Test, Automated Deployment, Automated Provisioning, and Architecture. Continuous Delivery Automation Concepts Continuous Delivery implies software has to flow ensuring a continuous flow from ideas to software features released in production. Continuous Delivery automation optimizes the software delivery process in such a manner that it enables frequent releases of small chunks of functionality or software features. Feedback on build and test, deployability, runtime behavior, and customers can be collected during the delivery and exploitation of software. Continuous Delivery Automation Focus Topics Test automation can be adopted in the software delivery process using the BDD and TDD principles. Application deployment is the point of confluence between Development and Operations. Automated provisioning refers to fully automated delivery and maintenance of application environment components. In 6B Data Center Automation, you learned that: Emergence of Cloud Technology and Principles The Microsoft cloud reference architecture classifies cloud services with a taxonomy based on four types: On-Premise (or do it yourself), IaaS, PaaS, and SaaS. The various Cloud principles are Resource Pooling, Abstraction, and Isolation; Elasticity; Continuous Service Availability; Predictability; A Service Provider’s Approach; Multitenancy; Security and Identity; and Metering. Cloud Services Concepts in a DevOps Organization The DevOps Platform teams are forced to adopt cloud principles for their platform product development to work autonomously from one another without task dependencies. The three extreme options to consider while deciding if an organization needs DevOps Platform teams are “In-house Cloud Services Provider”, “Managed Services Provider”, and “(Public) Cloud Services Provider(s)”. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
197
Course Book | DASA DevOps Fundamentals
Automated Provisioning Concepts Platform teams can use DSC for automation of environments. Automated provisioning with mutable infrastructure allows components to automatically change once these are created. Automated provisioning with immutable infrastructure is the practice of always replacing infrastructure instead applying changes. Automated provisioning requires an engineering mindset to optimize productivity and deliver high quality products. Platform Product Characteristics and Application Maturity Any platform product design based on products with standardized characteristics targeting multiple teams/organizations has an embedded compromise between freedom offered to Business System teams and the amount of standardization applied. Advanced platform features and qualities can be used only by applications which are implemented in a compliant manner.
Module End Questions Q1. What type of tasks are characterized by high task variability and high tasks analyzability? a) Routine b) Craft c) Engineering d) Non-Routine Q2. What is the concept of logically subdividing and provisioning an infrastructure to organizations called? a) Elasticity b) Metering c) Multitenancy d) Resource Pooling
198 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 6 | Automation
Q3. Implement your automation tasks and software in such a way that it provides feedback as early as possible. In other words, fail fast! Identify the correct approach for “fail fast!” in an automated software delivery process. a) Release software automated to the customer as soon as possible so it can fail fast. This feedback will result in better software quality. b) Stop an automated build and release process when unit tests fail before completing the build or release. c) By implementing automation tasks you can import customer satisfaction into the system. If it fails, you have important data. Q4. Which concept enables the DevOps teams must be able to acquire new environment components on demand, fully automated? a) Agile Organization b) Automated Deployment c) Automated Provisioning d) Automated Test Q5. Match each type of feedback with the appropriate example. Term
Description
a) Feedback on Build and Tests
i Revenue/conversion rates
b) Feedback on Deployability
ii Automated load test results
c) Feedback on Runtime Behavior
iii Automated unit test results
d) Feedback from the Customer
iv Automated application health checks
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
199
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
7 Measure and Improvement
Module Objectives Module Objectives
Importance of Measurement DevOps Metrics Relevance of Monitoring and Logging
At the end of this module, you will be able to:
Copyright © 2018 | 1
Explain the importance of measurement. Relate to DevOps metrics. State the relevance of monitoring and logging. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
201
Course Book | DASA DevOps Fundamentals
Module Topics Importance of Measurement Choosing the Right Metrics Monitoring and Logging
Importance of Measurement Measurement “If you can’t measure, you can’t improve! A successful DevOps implementation will measure everything it can as often as it can… performance metrics, process metrics, and even people metrics.” Source: John Willis, July 16, 2010
Without measurement, it is impossible to achieve and sustain continuous improvement. End-to-End Responsibility and Continuous Improvement are the key principles in DevOps that can be applied in the context of measurement. These two principles have a big impact on what needs to be measured within DevOps. The continuous feedback about the status and performance of the process is fueled by measurements. Therefore, when aiming to strengthen the continuous improvement cycle of a process, it is importantThree to shorten and amplify the feedback loops. Feedback: Ways Model
ps
Importance of
ps
Importance of Feedback: Three Ways Model Importance of Feedback: Three Ways Model
nce of ement
nce of Right ement etrics
gRight and etrics gging
g and gging
First Way: Systems
The First Way focuses on the importance of the performance of the entire
Thinking Thinking system. The practices of the First Wayimportance do not allowof a known defect to passof First Way: Systems The First Way focuses on the the performance through downstream work centers and local optimization to do create global a the entire system. The practices of the First Way The First Way focuses on the importance of the performance ofnot theallow entire First Way: Systems Dev Ops degradation. The practice strives towards achieving single piece of flowlocal and knownThe defect to pass through downstream centers and Thinking system. practices of the First Way do not allowwork a known defect to pass better understanding of the entire system. optimization to create globaland degradation. The topractice strives through downstream work centers local optimization create global Source: Gene Kim, DevOps Guru Dev Ops degradation. The practice strives towards achieving single piece of flow and towards achieving single piece of flow and better understanding of better understanding the entire system. of the entire system. Second Way: Amplify Feedback Loops
Second SecondWay: Way: Amplify Amplify FeedbackLoops Loops Feedback Dev Ops Dev
Ops
Dev
Ops
Third Way: Culture of Continual Experimentation And Learning Third Way: Culture of Continual Experimentation Dev And Learning Ops
The WayGuru is about creating right to left feedback loops. However, the Source:Second Gene Kim, DevOps Source: Gene Kim,downstream DevOps Guru information flows in the process (from left to right). In order to accomplish the right to left approach, feedback should be short so that it is TheSecond Second is about creating right to left loops. feedback loops. The WayWay is about creating right to left feedback However, the received on time to act and make continual corrections. The Second Way information downstream in the process (from left right). In order to However, flows the information flows downstream in tothe process (from results in a deeper understanding and better response. accomplish the right to left approach, feedback should that it is left to right). In order to accomplish the rightbetoshort left so approach, Source: Gene Kim, DevOps Guru received on should time to act make Thetime Second Way feedback beand short so continual that it is corrections. received on to act and results in a deeper understanding and better response. make continual corrections. The Second Way results in a deeper The Third Way is about failing fast, experimenting, learning, developing an Source: Gene Kim, DevOps Guru understanding and better MVP, and getting feedback to response. enable these practices. The Third Way includes allocating time for daily Kaizen events, rewarding teams for taking The Third WayKim, is about failing Source: Gene DevOps Gurufast, experimenting, learning, developing an risks (even when they fail), and introducing faults into the system to increase MVP, and getting feedback to enable these practices. The Third Way resilience. includes allocating time for daily Kaizen events, rewarding teams for taking Source: Gene Kim, DevOps Guru risks (even when they fail), and introducing faults into the system to increase resilience.
Source: Gene Kim, DevOps Guru Source: http://itrevolution.com/the-three-ways-principles-underpinning-devops/
202 │ Copyright © 2018
Copyright © 2018 | 5
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV Source: http://itrevolution.com/the-three-ways-principles-underpinning-devops/ Copyright © 2018 | 5
Metrics Monitoring and Logging
Second Way: Amplify Feedback Loops
Dev
Ops
The Second Way is about creating right to left feedback loops. However, the information flows downstream in the process (from left to right). In order to 7 |be Measure Improvement accomplish the right to left approach, feedbackModule should short and so that it is received on time to act and make continual corrections. The Second Way results in a deeper understanding and better response. Source: Gene Kim, DevOps Guru
Third Way: Culture of Third Way: Culture of Continual Experimentation Continual Experimentation And AndLearning Learning Dev
Ops
The Third Way is about failing fast, experimenting, learning,
The Third Way is about failing fast, experimenting, learning, developing an developing an MVP, and getting feedback to enable these MVP, and getting feedback to enable these practices. The Third Way practices. The Third Way includes allocating time for daily includes allocating time for daily Kaizen events, rewarding teams for Kaizen taking events, forintroducing taking risks (even theytofail), and risks (evenrewarding when theyteams fail), and faults into when the system increase introducing faults into the system to increase resilience. resilience. Source: Gene Kim, DevOps Guru
Source: Gene Kim, DevOps Guru
Source: http://itrevolution.com/the-three-ways-principles-underpinning-devops/
Source: http://itrevolution.com/the-three-ways-principles-underpinning-devops/
Copyright © 2018 | 5
A fundamental part of DevOps is to integrate the feedback into the daily operation. For organizations, this means measuring everything, and sharing those measurements with everyone involved. The importance of feedback is also emphasized in the Three Ways model of Gene Kim, as listed in the preceding table.
With Great Measurement Comes Great Responsibility Only good measurements cannot lead to achievement. What matters is the dare to confront the measurements, draw the right conclusions, and make changes in the required way. This requires a culture where people dare to confront each other and welcome any problem or improvement opportunity they find.
You should dare to establish a balanced view as the measurement of just one aspect does not tell the entire story. You might be focused on improving business, but it can have multiple meanings to DASAthe DevOps Survivorship Bias faster value, many people, such as delivering better value, delivering Fundamentals offering cheaper services, and creating more meaningful work or a Premium better environmental footprint. It sometimes takes courage to look and measure beyond the obvious. The logical error of concentrating on the people or things that "survived" so inadvertently overlooking those that did not because of their lack of visibility Importance of
Leads to overly optimistic beliefs because failures are ignored Measurement Choosing the Right Metrics Choosing the Right Survivorship Bias Metrics
“They don't make 'em like they used to”
Monitoring and The logical error of concentrating on the people or Logging things that “survived” some process and inadvertently overlooking those that did not because of their lack of visibility
Leads to overly optimistic beliefs because failures are ignored “They don’t make ‘em like they used to” Survivorship bias is an example of choosing the right metrics and was used in World War II. The original plan in World war II was to fortify the spots where the bombers had been hit. However, Abraham Wald, the Source: https://en.wikipedia.org/wiki/Survivorship_bias statistician, suggested that the planes who managed to return safely were able to survive the damages. Source: https://en.wikipedia.org/wiki/Survivorship_bias Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
203
Course Book | DASA DevOps Fundamentals
rd. The key questions are:
Therefore, fortifications should be on all the places including those where the bombers were not hit.
anization to continuously improve?
Actions Based on Measurements Measuring performance is relatively straightforward. The key questions are: Are we measuring the right things? Are we using the measurement to help the organization to continuously improve? If you do not take action based on the results of the measurement, there is no point in measuring. Copyright © 2018 | 9
A DevOps
mentals um
Measuring performance is a critical component of DevOps. It is about ensuring decisions are based on facts and figures rather than opinion. Metrics (or KPIs) are used to measure performance. These are vital to understand whether the organization is achieving its goals. Therefore, these should be clearly defined.
Performance Metrics vs. Performance Predictors
Performance Metrics vs. Performance Predictors
Importance of Measurement
oosing the Right Metrics Monitoring and Logging
Performance Metrics
Performance Predictors
Performance metrics are typically output oriented, easy to measure, but hard to improve or influence.
Performance predictors are typically input oriented, hard to measure, and easy to influence.
These metrics are known as lagging indicators. Example: Weight measurement is a performance metric and a lagging indicator.
vs.
These metrics are known as leading indicators. Example: Calories input and calories burned are performance predictors.
Performance predictors for Therefore, predicting these an outcome. Performance predictors are good for predictingare an good outcome. metrics are used to steer teamsthese and metrics organizations in order toteams enhance chance Therefore, are used to steer andthe organizations of a satisfactory outcome. in order to enhance the chance of a satisfactory outcome. Copyright © 2018 | 10
204 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 7 | Measure and Improvement
Measuring Leading Indicators for Culture Cultural Element
Behavior
Strongly Disagree
Continuous Learning and Improvement
Retrospectives, facilitate knowledge sessions, hackathons, make people responsible
Experimenta tion
Provide time, Instant sandbox environment, remove hurdles, applaud ideas, safe to fail
Build Quality in
Test automation, responsibility for endresult, team knows best, root-cause analysis, process automation (deploy, provision), continuous improvement, transparency (monitors)
An Engineering Culture
Hire people with matching ambitions, every repeated task is automated, support experimentation, support automating manual tasks, not only focus on utilization
A Culture of Effectiveness
Begin with end in mind, ask why? people broaden activities, handover moments avoided, move away from rigid processes, achieve results in small batches, Lean
A Culture of Product Thinking
Customer to attend demos, user feedback onto storyboard, allow customers to write about product and respond, organize people around product, team to write blogs about product, you build it, you run it
A Culture of Taking Responsibility
Address derailment openly, let people figure out how to do things, rewards responsibility, reward failure, transparency in what everyone is doing
Disagree
Neutral
Agree
Strongly Agree
Example Leading Indicators for Culture Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
205
Course Book | DASA DevOps Fundamentals
The preceding sample questionnaire for employees can be used within an organization to reflect on cultural topics.
Measuring Leading Indicators for Organizations Cultural Element
Behavior
Product Owner
The Squad has a dedicated Product Owner who prioritizes the work and takes both business value and technical aspects into consideration.
Agile Coach
The Squad has an Agile Coach who helps the Squad to identify impediments and coaches them to continuously improve their process.
Influence Work
Each Squad member can influence his/her work, be an active part in planning, and choose which tasks to work on. Each member can spend 10% of his/her time on hack days.
Easy to Release
The Squad can (and does!) get stuff live with minimal hassle and synchronization.
Process that Fits the Team
The Squad feels ownership of their process and continuously improves it.
A Mission
The Squad has a mission that everyone knows and cares about, and stories on the backlog are related to the mission.
A Culture of Taking Responsibility
The Squad knows where to turn to for problemsolving support, for technical issues as well as ‘soft’ issues.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
Example Leading Indicators for an Organization The preceding sample questionnaire for employees can be used to reflect on organizational topics.
206 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 7 | Measure and Improvement
Measuring Leading Indicators for Process Efficiency Cultural Element
Behavior
Peer Review
Peer reviews are conducted within the teams, not through separate teams.
Version Control
Everything is in version control: code, tests, infrastructure definitions, deployment definitions, configurations, and deployment units.
Continuous Delivery
Everything is automated: Peer Review Process, Build Automation, Test Automation, Deployment Automation, and Systems Provisioning.
Functional Design
Functional designs are provided as test specifications that can be used for test automation.
Process Improvement
10% of time is reserved to continuously improve process and its automation aspects.
Alignment
Business and IT are at same location and in same team. They work together as described as the de facto standard for Scrum.
Engineering Mindset
Manual recurring steps are automated instantly.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
Example Leading Indicators for Process Efficiency The preceding sample questionnaire for employees can be used to reflect on process efficiency.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
207
Ops
s
ance of rement
e Right Metrics
ng and Logging
Ops
s
ance of rement
e Right Metrics
ng and Logging
Course Book | DASA DevOps Fundamentals
Measuring Leading Indicators for Software Development Measuring Leading Indicators for Software Automation Development Automation Level 5 Expert
Continuous Integration
Automated Deployment
Test Automation
Automated Provisioning
Architecture
Agile Continuous Delivery
Automated feature-driven delivery
Deployment automated for advanced “feature go live” scenarios
Teams have adopted businessdriven, featurebased advanced test capabilities
PaaS is innovation accelerator
Teams are free to accelerate and innovate without constraints
Continuous optimization
Limited central Continuous Integration capabilities
No deployment automation
No test automation
No provisioning, (partially) manual process
Technical issues prevent increasing release frequency
Continuous Delivery principles are not applied
Level 4
Advanced
Level 3 Average
Level 2 Basic
Level 1 Beginner
Level 0
Not started
Example Leading Indicators for Automation Copyright © 2018 | 14
The preceding sample model can be used to determine the level of automation for the software delivery process within a company. Using this model, you can plot different types of questions. Note: The indicators listed in the preceding figure are just examples. There is no need to memorize these.
Measuring Leading Indicators for Data Center Automation
Measuring Leading Indicators for Data Center Automation
Level 5 Expert
Security and Compliance
Operational Management
Resiliency
Observability
Provisioning
Data Center Delivery
Product Team Autonomy
Full insight into and control of security and compliance risks
Fully automated insight into capacity, cost, business value, and operational burden
Planned, unplanned, automated resiliency test; predictive autoscaling
Custom metrics, teams configure alerts based on custom events
Infrastructure as code
Continuous delivery of all infrastructure components
Full self-service with graphical UI and APIs
Periodic patch rounds of OS packages
Order new hardware when more capacity requested; hardware is project specific
Backup/restore procedures validated periodically
Logging available via Ops; Basic monitoring for platform parts
Standardized procedures; basic scripts
Standardized procedures; people trained to execute
Standardized request forms picked up whenever Ops get around to it
Level 4
Advanced
Level 3
Average
Level 2 Basic
Level 1
Beginner
Level 0
Not started
Example Leading Indicators for Data Center Automation
208 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2018 | 15
Module 7 | Measure and Improvement
The preceding sample model can be used to determine the level of data center automation within a company. Using this model, you can plot different types of questions. Note: The indicators listed in the preceding figure are just examples. There is no need to memorize these.
Measuring Leading Indicators for Measurements Cultural Element
Behavior
Test Measure ments
Test results of code are available on a constant level to the development team by through monitoring.
Build Measure ments
A build monitor is constantly active, displaying the result of last five builds.
Deployment Measure ments
A deployment dashboard is constantly active, displaying relevant information for the team to decide the next steps.
Release Dashboard
A release dashboard is constantly active, showing package contents and release results. It provides relevant information on contents of deployment.
Systems Monitoring
Systems information on CPU, memory, and disk utilization is constantly fed back to the team.
Systems Usage
The information on application usage, conversion rates, scalability information, and infrastructure sizing is continuously fed back to the team.
Strongly Disagree
Disagree
Neutral
Agree
Strongly Agree
Example Leading Indicators for Measurements The preceding sample questionnaire can be used to determine the level of measurement that is performed within a company.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
209
Course Book | DASA DevOps Fundamentals
DASA DevOps Fundamentals Premium
Importance of Measurement Choosing the Right Metrics Monitoring and Logging
Top Practices (Leading Indicators) Correlated to Top Practices (Leading Indicators) Correlated to Performance Performance Metrics Metrics
Leading Indicators for Deployment Frequency:
Leading Indicators for Lead Time for Changes:
Leading Indicators for MTTR:
Continuous Delivery Version Control Version Control Automated Testing
Version Control Monitoring System and Application Health
Change Fail Rate: The researchers of the State-of-DevOps 2014 report did not find Change Failcorrelating Rate: Thewith researchers 2014 any specific practices this metric. of Butthe theyState-of-DevOps did find that high-performing IT organizations havenot 50%find lower change fail rates than the correlating non-high performers. report did any specific practices with this
metric. But they did find that high-performing IT organizations have 50% lower change fail rates than the non-high performers. Source: 2014 State-of-DevOps Report (PuppetLabs)
Copyright © 2018 | 17
Source: 2014 State-of-DevOps Report (PuppetLabs)
The following pieces of information briefly explain the preceding top practices or the common denominator for high performance organizations: Leading Indicators for Deployment Frequency: {{
{{
Continuous Delivery: Ensures that the software is always production ready throughout its entire lifecycle. In addition, any build can be released to users just at the touch of a button. Version Control: Puts all IT artifacts, such as software, software configuration, test specifications, documentation, deployment scripts, and configuration files, under version control. This activity has a great impact on the throughput due to the increased ability to solve problems, perform testing, and recreate environments.
Leading Indicators for Lead Time for Changes: {{
{{
Version Control: Helps to put stuff in production in a repeatable way with low risks. Delivery, deployment, and provisioning benefit from version control. Automated Testing: Improves the quality of the tests and their usage. Both, automated testing and version control, help to lower the risk (stability) and swiftness (throughput) of changes.
210 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 7 | Measure and Improvement
Leading Indicators for MTTR: {{
{{
Version Control: Enables the quick rollback to the last working state of the affected IT infrastructure when problems occur in the Production environment. In addition, it helps solving problems and searching for any root causes, thus reducing the time to recover. Monitoring System and Application Health: Provides (real time) feedback on the health of IT services through extensive logging and monitoring. In addition, it makes it easy to detect failures, identifies the events that have triggered such a failure, and proactively prevents these from happening at the first place.
Top Five Predictors of IT Performance Top Five Predictors of IT Performance Peer-reviewed change approval process
Version control for all production artifacts
Proactive monitoring
High-trust organizational culture
urce: 2014 State-of-DevOps Report (PuppetLabs)
Win-win relationship between Dev and Ops Copyright © 2018 | 18
Source: 2014 State-of-DevOps Report (PuppetLabs)
The following pieces of information briefly explain the preceding top five predictors: Peer-reviewed change approval process: The 2014 Stateof-DevOps Report concludes that IT performance throughput is lower when approval of changes is done by outside members rather than the team that make the changes, such as the Change Advisory Board (CAB). When the team, responsible for the changes, is made accountable for the quality of the changes through peer review, the performance throughput increases.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
211
Course Book | DASA DevOps Fundamentals
Version control for all production artifacts: Version control is a correlating practice for the three performance metrics: deployment frequency, lead time for changes, and MTTR. Therefore, it is wise to invest in if you want to increase your IT performance. A good version control system provides a Single Source of Truth (SSOT). It is a concept that refers to maintaining exactly one official source of data so that its consumers can get the true current version of the data anytime. Good version control means when incidents occur, it is easier to look for the root causes and rollback to the last good state, reducing the time to recover. Version control also promotes learning and sharing knowledge between teams, especially, when version control is not only limited to application code but also includes system and application configurations and the deployment script. Proactive monitoring: Organizations that work with teams who practice extensive and proactive monitoring of their own products and services are able to solve problems faster and accelerate their continuous improvement. When logging events, such as failures, which is primarily the responsibility of dedicated monitoring teams, such as Operations Bridge, IT performance suffers. High-trust organizational culture: DevOps is all about culture, and the report shows that a high-trusted culture predicts better performance. On the other hand, bureaucratic and fear-based cultures just do the opposite. Win-win relationship between Dev and Ops: Breaking down the barriers in siloed organizations and building strong teams based on DevOps principles enhance IT performance greatly.
IT Performance: Throughput vs. Stability “One of the most exciting outcomes of our research was coming up with a quantitative definition of IT performance. This breakthrough allowed us to show the relationships between DevOps practices, IT performance and organizational performance.
We have debunked the myth that we need to choose between speed (throughput) and reliability (stability). We found that highperforming IT organizations deploy code 30 times more frequently and 200 times faster than their lower-performing peers. They also have 60 percent fewer failures and recover 168 times faster. High performers are able to achieve higher levels of both throughput and stability through the use of DevOps practices — a key reason the movement has gained so much traction”. Jez Humble, DevOps and Continuous Delivery Guru
212 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 7 | Measure and Improvement
In the annual 2014 State-of-DevOps Report by PuppetLabs, IT performance is measured in terms of throughput and stability. The two attributes are essential in achieving high performing IT. Reference Reading: 2015-state-of-devops-report.pdf (https://puppet.com/resources/ whitepaper/2015-state-devops-report)
Effective Transformational Leadership Predictors Effective Transformational Leadership Predictors
TransformationTransformation leaders with significant impact in an organization the following five leaders with significant impact in share an organization common characteristics: share the following five common characteristics: Vision Inspirational Communication Intellectual Stimulation Supportive Leadership
Personal Recognition
ource: 2017 State-of-DevOps Report (PuppetLabs)
Source: 2017 State-of-DevOps Report (PuppetLabs)
Copyright © 2018 | 20
The State of DevOps report 2017 also showed the impact of leadership and organization. Loosely coupled teams and architectures are considered the strongest predictors of improved throughput and quality. Interesting fact from the 2017 report is that the gap between low and high performers narrowed for deployment frequency and lead time. At the same time, it showed that low performers have slower recovery time and higher failure rates. It seems the low performers have not yet mastered the control over quality. Measuring the quality of the products and services is even more important when performing at higher speeds. Reference Reading: 2017-state-of-devops-report.pdf (https://puppet.com/resources/ whitepaper/2017-state-devops-report)
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
213
Course Book | DASA DevOps Fundamentals
SA DevOps
amentals ium
Monitoring and Logging
Continuous Monitoring: The Key to Understand What is Going Continuous Monitoring: The Key to Understand What is Going on
DevOps requires transparency that many IT organizations struggled with. It is imperativ DevOps requires transparency that many IT organizations struggled
PDCA with. It is imperative that everybody (Development, Operations, QA understand that everybody (Development, Operations, QA and other teams) has a true The PDCA cycle was derived by other has a true of what going on insight, in of what is goingand on in theteams) system and the understanding service. Without the iscomplete it becom Importance of Dr. W. Edwards Deming. This Measurement the system and the service. Without the complete insight, it becomes difficult to effectively use the continuous improvement mantra of “Plan, Do, Check, Act” methodology describes the four difficult to effectively use the continuous improvement mantra of “Plan, PDCA. hoosing theessential Right steps (“PLAN, DO, CHECK, Do, Check, Act” or PDCA. Metrics ACT”) that should be carried out systematically to achieve Monitoring and continuous improvement. Logging
Copy
In siloed organizations, the monitoring task is placed with the operations people, possibly within a separate unit. Such a separate unit can have a set of tools to monitor the infrastructure components at different levels of the technology stack, from the application to the network layer. Based on input from others, they configure monitoring and set all kinds of alarms and follow-ups, such as mails or SMS alerts. With the introduction of Agile and DevOps, the siloed way of working might become a bottleneck. The need for faster, better, and integrated feedback and alerts has been growing with the introduction of continuous delivery, automated testing and deployment, continuous integration and others. Without quick and integrated feedback from continuous monitoring, it becomes really difficult to figure out what kind of problems can occur in case of incidents and what potential root causes need to be eliminated. Combine this trend with the end-to-end responsibility of Product teams for the health of their products through the entire lifecycle, and it explains the focus on good monitoring is more than just the basic infrastructure level.
214 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 7 | Measure and Improvement
Scope of Scope Monitoring of Monitoring
Copyright © 2018 | 23
The monitoring strategy is not constrained to just the infrastructure or the application. The scope of monitoring includes: Monitoring Infrastructure: It includes the basic level of infrastructure monitoring, such as servers, storage devices, load balancer, and networking. Monitoring Platform: It includes monitoring the platforms provided and used by the Platform teams (or Business System teams) to run their application. Monitoring Application: A good working infrastructure and platform services do not mean applications on the top will function correctly. Even when no changes are made, applications are still at risk of trouble due to conditions, such as buggy code, thirdparty components, and external systems. Monitoring based on checks, build in application, can be used to detect such issues. This calls for participation of a Monitoring Specialist within the DevOps Team during the design phase. Monitoring Business: Healthy applications can still contribute to the business not meeting their goals. Monitoring should provide feedback to the business at the earliest to take corrective actions. Consider an example of a webshop application of a major airline displaying a price of €1 instead of € 500 for a flight from New York to Amsterdam. Good monitoring will send an alert based on a highly unexpected webshop traffic and sales volume. These kinds of monitoring are usually based on expected transactional data patterns and complement the data more commonly used in Business Intelligence (BI) systems. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
215
Course Book | DASA DevOps Fundamentals
Monitoring the Monitoring: With the growing emphasis on monitoring, the need to monitor the monitoring infrastructure is growing. One of the more common pitfalls with the monitoring of the monitoring is the manual dibbling of monitoring during big releases and forgetting to enable afterwards. A proper automated DevOps Continuous Delivery process should address such issues.
DASA DevOps
Fundamentals Premium
Log Aggregation Monitoring: In Production environments, vast amounts of data is stored, usually distributed over multiple log files. Nowadays, tooling for log aggregation is capable of using all this data to detect issues that might have gone unnoticed.
MonitoringMonitoring Optimized for DevOps Optimized for DevOps
Importance of Measurement Choosing the Right Metrics Monitoring and Logging
Copyright ©
MTTR Mean Time to Recovery (MTTR) is the average time to recover from any failure.
The requirements for all levels of monitoring needs to be a part of the design of a service from the initial stages. During the design phase, the emphasis should be on automation. A good monitoring design and setup further enhances the continuous improvement cycle. Moreover, it helps DevOps teams to speed up their ability to resolve issues (MTTR). Predictive monitoring prevents systems from falling apart. As a result, customer satisfaction will rise and business results will improve. Some of the guidelines to improve the monitoring strategy include: Monitoring tool agents baked into deployments: Monitoring should not be an afterthought, but a part of the design of products and services.
216 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 7 | Measure and Improvement
Alerts/Incidents allocated and delivered directly to the right Product and/or Platform teams: DevOps teams are end-toend responsible and need the direct feedback on the quality of their work. Real-time dashboards for all involved, including customers and business: Visualize everything, including the real-time status of services. A single platform (‘single point of truth’) for how an application is performing: A single source speeds up solving problems and prevents unnecessary discussion. Standardization when possible, niche tools when required: Experimentation, MVP, and courage are stimulated when niche tools are allowed. Incorporate the Service Management System (SMS) or data, historical knowledge, and workflows in case of alerts: Such a system and knowledge provides a great source of insight into the behavior of your users/customers. Incorporate social media (when applicable) data in the monitoring for monitoring the business: Social media is one of the quickest ways to find out what customers are doing with Collecting Feedback from Automated Software Delivery our services/products in the an market.
DevOps
entals m
Pipeline
Collecting Feedback from an Automated Software Delivery Pipeline
mportance of Measurement
Continuous Deployment Pipeline
sing the Right Metrics
onitoring and Logging
Version Control
Build
Unit Tests
Deploy
Auto Test
Deploy to Production
Measure and Validate
Production Feedback
The continuous deployment pipeline breaks down the software delivery process into stages, as shown in the following figure. Each stage is aimed at verifying the quality of new features from a different perspective to validate the new functionality and prevent errors from affecting the users.
Copyright © 2018 | 25
The figure shows the software deployment process as a pipeline flowing from left to right. During all the stages of the software development, the measurements provide feedback to the team. They can use the feedback to visualize the flow to everyone involved and identify bottlenecks and issues that need to be solved. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
217
s
Course Book | DASA DevOps Fundamentals
DASA DevOps Fundamentals Premium
Importance of Measurement
Dashboards to Build the Feedback Culture Dashboards to Build the Feedback Culture
Without feedback, it is impossible to improve in an effective manner. A dashboard displays fact-based Without information (real measurements) that to helps organizations to steer. manner. Some of the feedback, it is impossible improve in an effective A dashboard are:
dashboard displays fact-based information (real measurements) that helps organizations to steer. Some of the dashboard are:
Choosing the Right Metrics Monitoring and Logging
Release Dashboard
Test and Quality Dashboard
Performance Dashboard
Release Dashboard
Build Dashboard
Product Usage Dashboard
Release Dashboard
e of ent
Copyright © 2018 | 26
ght rics
and ging
Example Measurements for Throughput of New Releases Source: http://www.onlinemediamasters.com/google-analytics-custom-dashboardexamples/ Copyright © 2018 | 27
Source: http://www.onlinemediamasters.com/google-analytics-custom-dashboard-examples/
This is an example of a release management dashboard taken from XLRelease.
218 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 7 | Measure and Improvement
DevOps
entals m
Test and Quality Dashboard Test and Quality Dashboard
mportance of Measurement
sing the Right Metrics
onitoring and Logging
Example Measurements to Measure Quality of Software
Test and Quality Dashboard (Contd.)
This is an example of a code quality dashboard taken from SonarQube. Copyright © 2018 | 28
of nt
ht s
d g
Parts of quality can also be measured by collecting customer feedback Source: http://scorebuddy.c9471282.myzen.co.uk/Product/white-papers.html
Source: http://scorebuddy.c9471282.myzen.co.uk/Product/white-papers.html
Copyright © 2018 | 29
This is an example for gathering feedback directly from a customer.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
219
s
e of ent
ight rics
Course Book | DASA DevOps Fundamentals
Build Dashboard Build Dashboard REPORT_INTEGRATION_TESTS 30 builds have failed
#358
14 hours ago
#236
13 hours ago
REPORT_SONAR 7 builds have failed
REPORT_SITE #310
and ging
ps
REPORT_REMOTE_INTEGRATION_TESTS
0 identified problems:
13 hours ago
0 identified problems:
#128
WEL_MASTER_PRD
SHARED_PARENT_BE_NL
Execution aborted 10.1.173-SNAPSHOT
Back in the green!
0 identified problems:
origin/master
1.2.59-BE
13 hours ago
17 hours ago
#5334
21 hours ago
WEL_OPERATIONS 1 builds has failed
WEL_MASTER_PRD_RELEASE 10.1.173-RELEASE
22 hours ago
0 identified problems:
2.0.1.20161116
13 hours ago
WEL_SPRINT_RELEASE 2 builds have failed Refs/heads/RC 0 identified problems:
WEL_SPRINT #910
origin/SPRINT
17 hours ago
#341
17 hours ago
Example Measurements for Measuring Software Build Throughput
This is an example of a software build dashboard taken from a Jenkins Copyright © 2018 | 30 build server.
Example Performance Dashboard Example Performance Dashboard
e of ment
Right trics
and ging
Example Measurements for Measuring Product Performance Source: http://ecmarchitect.com/archives/2014/09/09/3932
Source: http://ecmarchitect.com/archives/2014/09/09/3932
Copyright © 2018 | 31
This is an example of dashboard of gathering metrics about an application load test using JMeter.
220 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Ops
Module 7 | Measure and Improvement
Example Usage Dashboard Example Usage Dashboard
nce of ement
Right Metrics
g and gging
Example Measurements for Product Usage Source: http://www.onlinemediamasters.com/google-analytics-custom-dashboard-examples/
Source: http://www.onlinemediamasters.com/google-analytics-custom-dashboardexamples/
Copyright ©
This is an example of a website usage dashboard taken from Google Analytics.
Logs: A Source of Important Information Logs provide a tracking mechanism that gives insight into the usage of IT services by users and customers. The following table lists some of the examples. Stakeholder
Purpose
Case
DevOps Team
Monitoring
Identifying issues, trends, anomalies and many more
DevOps Team
Troubleshooting
Analyzing information and error messages to understand what is happening in the Production environment
Auditing
Audit trail building
Building a trail of data for auditors to aid them with their audits
Security
Tracking access
Tracking all successful and unsuccessful access attempts. Intrusion detection
Business
Monitoring
Identifying anomalies
Business
Analysis
Collecting data for BI analysis to understand customer’s behavior
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
221
Course Book | DASA DevOps Fundamentals
A logging application is very useful from the security and compliance perspective. Logs can be the source of important data for everything from forensic analysis to audit records. These are so important to DevOps, compliancy, and security that it pays off to implement logging tools that will serve the needs of all stakeholders.
Case Study ‘JC Travel’ JC Travel is an insurance company specialized in insuring travel and leisure activities. The current CIO of JC Travel is Fred. He is responsible for the delivery of the current services (outdated and unreliable IT services) and building the new online platform that is crucial for the future competitiveness of JC Travel. The program has overshot the budget and is running behind the delivery. Fred has been ordered to improve the situation drastically. To do so, he has requested your advice and wants to know if DevOps is something to consider. The delivery of the IT services is the responsibility of the IT department, consisting of several teams specialized in information management, architecture, operations, deployment, and QA. The development of the new online platform is done by the Program team using Scrum. Currently, the Program team is working towards the next bimonthly (every two months) release. However, there are usually major bugs in the releases. Teddy, the Program Lead, was insistent on sticking to release schedule. She pushed the development through the coding process, resulting in bad code. The last four releases were hard to deploy and caused lots of incidents in the Production environment. Within the IT department, the teams have been very busy with bug fixing to resolve all incidents. This has been done mainly through emergency changes that have not been documented very accurately, making new incidents harder to fix. DASA DevOps Fundamentals Premium
Due to the bad performance, the communication between the IT has reached the nadir. Both sides are blaming each other for the current debacle.
Case Study ‘JCand Travel’ (Contd.) department teams the Program teams
Answer based on on thethe case study: Answerthe thefollowing followingquestions questions based case study: Importance of Measurement Choosing the Right Metrics Monitoring and Logging
Q1
What are the key issues you can identify for the case?
Q2
What are the key recommendations for addressing the identified issues?
Q3
What is your overall recommendation (how to approach fixing this) to Fred, the CEO?
222 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Copyright © 2
Module 7 | Measure and Improvement
Module Summary In this module, you learned that: Importance of Measurement Without measurement, it is impossible to achieve and sustain continuous improvement. A fundamental part of DevOps is to integrate the feedback into the daily operation. The Three Ways model of Gene Kim helps achieve such an integration. Only good measurements cannot lead to achievement, therefore, you should dare to establish a balanced view as the measurement of just one aspect does not tell the entire story. Choosing the Right Metrics Performance metrics (or lagging indicators) are typically output oriented, easy to measure, but hard to improve or influence. Performance predictors (or leading indicators) are typically input oriented, hard to measure, and easy to influence. Throughput and stability are the two attributes that are essential in achieving high performing IT. Common denominator for high performance organizations include Deployment Frequency, Lead Time for Changes, and MTTR. The top five predictors of IT performance are peer-reviewed change approval process, version control for all production artifacts, proactive monitoring, high-trust organizational culture, and win-win relationship between Dev and Ops. The five effective transformational leadership predictors are vision, inspirational communication, intellectual stimulation, supportive leadership, and personal recognition. Monitoring and Logging DevOps requires transparency, and without the complete insight, it is difficult to effectively use the continuous improvement mantra of PDCA. The scope of monitoring includes Monitoring Infrastructure, Monitoring Platform, Monitoring Application, Monitoring Business, Monitoring the Monitoring, and Log Aggregation Monitoring. Some of the guidelines to improve the monitoring strategy include monitoring tool agents baked into deployments; alerts/ incidents allocated and delivered directly to the right Product and/or Platform teams; and real-time dashboards for all involved, including customers and business. Logs provide a tracking mechanism that gives insight into the usage of IT services by users and customers. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
223
Course Book | DASA DevOps Fundamentals
Module End Questions Q1. Which of the following is an example of performance predictor metric (or leading indicator)? a) The deployment frequency b) The number of changes approved through peer reviews c) The version control strategy of the platform teams d) The MTTR Q2. Which of the following metric is used to measure throughput (or speed) of IT performance? a) Automated Testing b) Change Fail Rate c) Lead Time for Changes d) MTTR Q3. Which of the following practice correlates highly with a shorter MTTR? a) Automated Testing b) Continuous Delivery c) Change Fail Rate d) Monitoring System and Application Health Q4. What is correct about version control in a DevOps setup? a) Version control has high impact on throughput, but little impact on stability. b) Version control helps to put stuff in production in a repeatable way, but it increases the risk. c) Version control should not be implemented on all IT artifacts, especially that relates to system and application configurations and deployment script. d) Version control helps to find root causes for incidents and reduces the time to recover.
224 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Module 7 | Measure and Improvement
Are you missing something? We have gone through all the modules of the course. It is time to wrap the course now. Do you think we are missing something? Yes, it is DASA DevOps Competence Quickscan™. Would you not like to know where are you now on DevOps competences after going through this foundation level course? Please visit https://scan.devopsagileskills.org/ and complete the scan again to know how this course has helped you to build DevOps competences.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
225
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Exam Preparation Guide
Module Learning Objectives Identify the structure of the exam. Indicate the key components of the exam. Practice the exam.
Topics Covered in This Module 1. Qualification Learning Objectives 2. Learning Level of the Syllabus 3. Certification 3.1 Certification Value 4. Exam Instructions 4.1 Exam Format 4.2 Types of Questions 4.3 Scoring System 5. Tips for Taking Exam
1. Qualification Learning Objectives The DASA DevOps Fundamentals qualification proves knowledge and understanding of DevOps. The qualification holder: Explain the drivers responsible for the emergence of DevOps. Define and discuss the key concepts and principles of DevOps. List and explain the business benefits of DevOps and continuous delivery. Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
227
Course Book | DASA DevOps Fundamentals
Describe the Service Delivery process. Explain the concepts of test automation, infrastructure automation, and build and deployment automation. Describe how DevOps relates to Lean and Agile methodologies. Summarize case studies of IT organizations that are making the transformation to Adaptive IT and DevOps models. List the most common and popular DevOps tools. Discuss the critical success factors for DevOps implementation.
2. Learning Level of the Syllabus The modern version of Bloom’s taxonomy of learning is a widely used classification framework for course syllabi and assessments for certification. The taxonomy classifies learning into six ascending levels. Level 1—the Knowledge Level: Exhibit memory of previously learned materials by recalling facts, terms, basic concepts, and answers. Level 2—the Comprehension level: Demonstrate understanding of facts and ideas by organizing, comparing, translating, interpreting, giving descriptions, and stating main ideas. Level 3—the Application level: Use new knowledge. Solve problems to new situations by applying acquired knowledge, facts, techniques, and rules. Level 4—the Analysis level: Examine and break information into parts by identifying motives or causes. Make inferences and find evidence to support generalizations. Level 5—the Evaluation level: Present and defend opinions by making judgments about information, validity of ideas, or quality of work based on a set of criteria. Level 6—the Creation level: Compile information together by combining elements in a new pattern or proposing alternative solutions The examination questions for the DASA DevOps Fundametal course are based on blooms level 1 and 2 (Knowledge and Comprehension). The DASA DevOps Fundamentals course is expected to provide a foundation level of proficiency for a candidate. The examinations tests this level. The examination format offers/will offer multiple choice questions with a series of corresponding possible answers. Only one answer will be correct.
228 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Exam Preparation Guide
3. Certification DevOps Agile Skills Association (DASA) is the accreditor of this course and intends to accelerate successful adoption of DevOps through training and certification. In line with this, DASA aims to provide the most comprehensive in-depth DevOps training and certification program in the world.
3.1 Certification Value You will receive the required certification from DASA on successful completion of the DASA DevOps Fundamentals exam.
4. Exam Instructions 4.1 Exam Format Prerequisites
There are no prerequisite qualifications for signing up to do the fundamentals exam
Supervised
Live or Web Proctored
Exam Type
Web-Based
Exam Duration
60 minutes (Additional 15 minutes for non-native English speaker)
Pass Score
65% (26 or more correct answers)
Format of Exam (Open book/Closed book)
Closed book
Number of Questions
40
4.2 Types of Questions The foundation level exam is based on multiple-choice questions.
4.3 Scoring System For all questions, the score is based on the correct answer.
5. Tips for Taking Exam In order to successfully take the exam, you are advised to keep the following points in mind: Read the questions carefully. If you are stuck on a question, you should guess the most likely option, mark the question, and come back to it at the end. This way, you will at least have a guess answer if you run out of time.
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
229
Course Book | DASA DevOps Fundamentals
Use theoretical knowledge to answer the questions and select the best option. Eliminate the distracters by using theoretical knowledge and assessment of the information provided. When in doubt, you should guess—there is no negative marking. Time is your biggest enemy. Calculate the time you have per question, make a guess and move on to the next question if you are not sure within the given time. Or start with answering all the questions you know for sure and go back from the start to answer the difficult questions. Where possible convert all questions to true/false statements. If a question looks tricky then it is better to consider them as true/ false statements. You can do this to all objective questions by considering each option separately. Remember: {{ {{
{{
Try to rule out any obviously incorrect options Remember the BEST option is preferred when choosing from multiple correct options Favour look-alike options. Look for repetition of key words from the question in the responses. If words are repeated, the option is worth considering
230 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Mock Exam
DASA DevOpS FunDAmentAlS Mock Exam v1.7 - November 2018
© 2018 - DevOps Agile Skills Association All rights reserved. No part of this publication may be published, reproduced, copied or stored in a data processing system or circulated in any form by print, photo print, microfilm or any other means without written permission by DASA
231 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
exAm DetAilS Exam Duration
60 minutes (Additional 15 minutes for non-native English speaker)
Format of Exam (Open book/Closed book)
Closed book
No. of Questions
40 Note: the mock exam has 35 questions
Pass Percentage
65% (26 or more correct answers)
2
232 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Mock Exam
QueStiOn Set QueStiOn 1 Which DevOps principle focuses on product and service thinking? A. Customer-centric action B. Continuous Improvement C. Create with the end in mind D. Automate everything you can
QueStiOn 2 An organization maintains an independent and autonomous team for each of its services. What is a possible disadvantage of this type of organization structure? A. Quality of delivered features will be low. B. Implementation of changes within a team is slow. C. Reuse of skills within the organization is limited. D. Waiting time for processing the service request is high.
3
233 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
QueStiOn 3 What type of mindset is the core of a DevOps culture? A. Service Mindset B. Skill Mindset C. People Mindset D. Automation Mindset
QueStiOn 4 What is NOT an appropriate predictor of IT performance in a DevOps environment? A. Changes approved outside of the team B. High-trust organizational culture C. Proactive monitoring D. Version control of all artifacts
QueStiOn 5 Erik is working in a Product team (or Business System team) specialized in delivering a specific IT-related product for the Sales department. Which of the following types of activities will most likely be recurring in the agendas of Erik’s team? A. Many handovers moments with other departments. B. Meetings on the utilization of the specialized resources within the organization. C. Monthly release meetings for the bi-monthly release. D. Attending the product demo meeting. 4
234 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Mock Exam
QueStiOn 6 What type of tasks are characterized by low task variability and high task analyzability? A. Routine B. Craft C. Engineering D. Non-Routine
QueStiOn 7 Which of the following software delivery approaches focuses on smaller functional units instead of developing the complete software? A. Agile B. Iterative C. Lean D. Waterfall
5
235 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
QueStiOn 8 What is the lifecycle of Story Mapping? A. 1. Establish a common overall goal 2. Determine the end-to-end workflow 3. Define a first marketable feature set 4. Expand/improve the existing functionality B. 1. Establish a common overall goal 2. Define activities 3. Determine the end-to-end workflow 4. Expand/improve the existing functionality C. 1. Establish a common overall goal 2. Define work in progress 3. Define activities 4. Define a first marketable feature set 5. Expand/improve the existing functionality D. 1. Establish a common overall goal 2. Determine the end-to-end workflow 3. Define activities 4. Define a first marketable feature set 5. Expand/improve the existing functionality
6
236 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Mock Exam
QueStiOn 9 The platform products control the freedom and restrictions for the DevOps Business System teams. Which cloud services classification poses the greatest number of restrictions when the customer aims for flexibility for tailoring the complete platform including hardware and software? A. On-Premise B. IaaS C. PaaS D. SaaS
QueStiOn 10 How do the Service Level Management processes of the ITIL Service Design phase map in a DevOps organization? A. Aims at full autonomy and full responsibility for delivering a product (value) to the customer. B. Brings new software live in a matter of minutes through automation. C. Maintains stable and fixed teams to avoid resourceswitching between projects. D. Manages changes through the same mechanisms used for aligning the business with IT.
7
237 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
QueStiOn 11 What is the difference between Continuous Delivery and Continuous Deployment? A. Continuous Delivery is a manual task, while Continuous Deployment is an automated task. B. Continuous Delivery has a manual release to production decision, while Continuous Deployment has releases automatically pushed to production. C. Continuous Delivery includes all steps of software development life cycle; Continuous Deployment may skip few steps such as validation and testing. D. Continuous Delivery means complete delivery of the application to customer; Continuous Deployment includes only deployment of the application in customer environment.
QueStiOn 12 What is NOT an aspect related to managing work in an organization? A. Attending scrum of scrums B. Using Feature Switches C. Using a scrum board to get members of the team on the same page D. Applying test automation
8
238 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Mock Exam
QueStiOn 13 What are the characteristics of people working in a DevOps based, product-focused organization? A. People are functionally organized. B. People know about business and IT and deliver work, thereby appealing to use any of a person’s skills and/or talents. C. People are specialist-oriented. D. People are assigned to multiple projects at once, for reasons related to resource optimization.
QueStiOn 14 Which phrase fits BEST as a characteristic of a DevOps team, considering that the team is part of an antifragile organization? A. Employee First B. Honor Web-inspired value C. Minimum Viable Bureaucracy D. Self-management
9
239 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
QueStiOn 15 When should you move to the Improve phase of the DMAIC method? A. After collecting the related data and facts about the variables that can influence the problem B. After defining potential solutions to the problem C. After ensuring whether a particular solution works D. After understanding the causes of the problem
QueStiOn 16 In DevOps, Business System teams are autonomous teams, that “land” their application and infrastructure code onto a common platform. This common platform is maintained by a Platform team. What is correct about the responsibilities of these teams? A. The Platform team is always responsible for maintaining the product within its operational environment. B. When a product/service application is “landed” on the platform, the responsibility of the product/service shifts from the Business System teams to the Platform team. C. The Business System team is only responsible for the development phase of new services. D. The Business System teams have an end-to-end responsibility, there is no handover or transfer of responsibility and accountability.
10
240 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Mock Exam
QueStiOn 17 What is the main reason to know exactly who the customer is when setting up a Business Service team? A. To determine what value and functionalities should be delivered by the team B. To establish the required mix of skills and knowledge for the team C. To know what kind of work the team will be handling D. To understand the scope of the technology responsibility of the team
QueStiOn 18 Which DevOps principle appreciates measuring processes, people, and tools? A. Continuous improvement B. Create with the end in mind C. Cross-functional autonomous teams D. People responsibility
QueStiOn 19 Which role should ensure that user stories adhere to the Definition of Ready (DoR)? A. Product Owner B. Scrum Master C. Service Manager D. Scrum Team 11
241 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
QueStiOn 20 John is the Product Owner and is helping his team to understand the product and the customer requirements. By doing so, which type of waste, visible to the customer, is likely to be eliminated? A. Defects B. Non-Utilized Skills C. Transportation D. Motion
QueStiOn 21 What are the appropriate characteristics of Continuous Delivery approach? 1. Complex, but small number of releases 2. A focus on cycle time reduction 3. Resource-based management of the process 4. Self-managed and responsive teams A. 1 and 3 B. 2 and 4 C. 2, 3, and 4 D. 1, 2, 3, and 4
12
242 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Mock Exam
QueStiOn 22 What is the main benefit of automated provisioning? A. Flexible approach to ad-hoc system changes B. Focus on operational perspective to control infrastructure changes C. High speed delivery of new environments D. Variability in application environments
QueStiOn 23 What is correct about implementation of measurements within an organization? A. Defining good measurements are enough for business improvement. B. Differences in measurements can lead to confrontation within organization; so measurements should be avoided. C. Measurement of one aspect can often represent the overall business scenario. D. Organizations should establish a balanced view to confront the measurements and draw the right conclusion.
13
243 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
QueStiOn 24 What is the correct characteristic for performance metrics? A. Performance metrics are output oriented. B. Performance metrics are difficult to measure. C. Performance metrics are easy to improve. D. Performance metrics are also known as leading indicators.
QueStiOn 25 Which model is used by Desired State Configuration (DSC) for specifying the configuration of systems? A. Declarative B. Imperative C. Procedural D. Sequential
14
244 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Mock Exam
QueStiOn 26 What is the correct sequence of the four steps when providing feedback in according to the Feedback model? A. 1. Describe concrete observations. 2. Explain what it does to you. 3. Wait and listen to clarifying questions. 4. Give concrete suggestions OR recognition/incentive. B. 1. Explain what it does to you. 2. Describe concrete observations. 3. Wait and listen to clarifying questions. 4. Give concrete suggestions OR recognition/incentive. C. 1. Wait and listen to clarifying questions. 2. Explain what it does to you. 3. Describe concrete observations. 4. Give concrete suggestions OR recognition/incentive. D. 1. Wait and listen to clarifying questions. 2. Give concrete suggestions OR recognition/incentive. 3. Describe concrete observations. 4. Explain what it does to you.
15
245 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
QueStiOn 27 The development of new software and IT services consist of functional and non-functional requirements. From what point in the development process should the non-functional requirements be addressed, to be able to deliver software and services faster and better? A. From the initiation of the software development B. After the functional acceptance test by the customer representatives C. Simultaneous with the implementation of continuous delivery D. The non-functional requirements are of no concern to the customers
QueStiOn 28 Which statement does NOT define DevOps? A. DevOps is a movement or practice that emphasizes collaboration and communication of both software developers and other Information Technology (IT) professionals. B. DevOps is a framework and job title that focuses on structured processes to organize flow between the Development and Operations teams. C. DevOps is about experiences, ideas, and culture. D. DevOps is an activity of optimizing the developmentto-operations value stream by creating an increasingly smooth, fast flow of application changes from development into operations. 16
246 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Mock Exam
QueStiOn 29 What characteristics should an organization adopt to become a DevOps organization? 1. Automation 2. Product thinking 3. Individual thinking 4. Fail fast 5. Problem avoidance 6. Specialist roles A. 1, 2, and 4 B. 1, 5, and 6 C. 2, 3, and 4 D. 3, 5, and 6
QueStiOn 30 What does ‘resilience’ means with reference to IT architecture? A. Preparing systems for changed requirements B. Preparing systems for security threats C. Preparing systems for upgrades in technology D. Preparing systems for unexpected events
17
247 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
QueStiOn 31 Your DevOps team is a stable team, where team members have been working together for several sprints now. The team is having trouble delivering a new version of the product for use by your customers. You are supposed to be delivering work in sprints of two weeks, but the team is unable to deliver agreed upgrades in time. What is the appropriate approach to meet the timelines in subsequent sprints? A. Extend the sprint to four weeks to give team more time. B. Expect that the team will learn from the mistakes and will fix the problem in the next cycle. C. Shorten the sprint to take small steps and find the problems quickly. D. Focus on only few limited changes that are viable to be delivered in two weeks.
QueStiOn 32 Which component provides the first feedback on the quality of committed application code changes? A. Automated Provisioning B. Automated Build C. Automated Test D. Automated Deployment
18
248 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Mock Exam
QueStiOn 33 In the context of cloud computing, which concept enables dynamic adaptation to different system loads? A. Abstraction B. Elasticity C. Metering D. Multitenancy
QueStiOn 34 Which is the correct sequence of tests when testing new software? A. Functional tests, system tests, unit tests, UI tests B. UI tests, functional tests, system tests, Unit tests C. System tests, unit tests, functional tests, UI tests D. Unit tests, system tests, UI tests, functional tests
QueStiOn 35 Which tool helps lowering risk during development as customer feedback is embedded into the design process? A. Test Automation B. Snapshot Deployment C. Story Mapping D. Value Stream Mapping
19
249 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix A Answers
Module End Questions Module 2: DevOps Introduction Q1.
a) DevOps help us to rapidly respond to changing or emerging customer needs.
iii Customer-Centric Action
b) DevOps teams need to act as “product companies” that explicitly focus on building working products sold to real customers.
v Create with the End in Mind
In a DevOps organization, teams are vertically organized so c) that they can be fully accountable for their services
i End-to-End Responsibility
d) This principle is a dominant concept borrowed from the Lean movement that focuses on adaptability and learning through structured problem-solving
vi Continuous Improvement
These teams are fully empowered and self-sufficient to e) design, build, test, deploy, and run the software. To be able to do this, a team needs team members with T-shaped profile and complementary skills.
Cross-Functional ii Autonomous Teams
The principle focuses on bringing software into production f) through a fully automated process, multiple times per day without problems.
iv Automate Everything You Can
Q2. d) Satisfying the customer Q3. d) 2, 3, and 4
251 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Module 3: Culture Q1. c) An Engineering Culture Q2. c) Leadership and Feedback Q3. a) Integrated multidisciplinary teams Q4. c) Analyze
Module 4: Organization Q1. a) The Business System teams are end-to-end responsible for their service during its complete lifecycle. Q2. c) Reducing waste and focusing on end user value. Q3.
Term
Description
a) Agile Organizations
iv Prefers dedicated teams over resourcing, products over projects, prioritization over planning, and outcome over output
b) Continuous Delivery
v Reduces and measures cycle time in hours or minutes
c) Autonomous Teams
i Focuses on “You build it, you run it, shared nothing” concept
d) Reactive Manifesto
iii Focuses on responsive, resilient, scalable, and looselycoupled systems that are easy to develop and change
e) Platform as a Service
ii Provides cheap, easy, and fast runtime environments for apps
Module 5: Processes Q1. d) Individuals and interactions, working software, customer collaboration, and responding to change Q2. a) R esources and time are fixed and functionality is estimated. Q3.
Term
Description
a) Product Backlog
vi A continuously evolving and ordered list of requirements
b) Potentially Shippable Product
iii Product increment delivered at the end of each Sprint
c) Scrum Master
iv Person who enables the team to perform the tasks that are required to make the product work
d) Product Owner
ii Person who ensures that the user stories adhere to the Definition of Ready
e) Sprint Review
i Session that occurs at the closing of a Sprint and generally includes a product demo
f) Backlog Refinement
v Session that is used to define what user stories are expected in the next Sprint
252 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix A | Answers
Module 6: Automation Q1. c) Engineering Q2. c) Multitenancy Q3. b) Stop an automated build and release process when unit tests fail before completing the build or release. Q4. c) Automated Provisioning Q5.
Term
Description
a) Feedback on Build and Tests
iii Automated unit test results
b) Feedback on Deployability
iv Automated application health checks
c) Feedback on Runtime Behavior
ii Automated load test results
d) Feedback from the Customer
i Revenue/conversion rates
Module 7: Measure and Improvement Q1. b) The number of changes approved through peer reviews Q2. c) Lead Time for Changes Q3. d) Monitoring System and Application Health Q4. d) V ersion control helps to find root causes for incidents and reduces the time to recover.
253 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Mock Exam Answers
AnSwer Key QueStiOn nO.
AnSwer
reFerence mODule
1
C
DevOps Introduction
2
C
Organization
3
A
Culture
4
A
Measure and Improvement
5
D
Organization
6
A
Automation
7
A
DevOps Introduction
8
A
Processes
9
D
Automation
10
A
Processes
11
B
Automation
12
D
Organization
13
B
Processes 20
254 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix A | Answers
14
D
DevOps Introduction
15
D
Culture
16
D
Organization
17
A
Organization
18
A
DevOps Introduction
19
A
Processes
20
A
Processes
21
B
Automation
22
C
Automation
23
D
Measure and Improvement
24
A
Measure and Improvement
25
A
Automation
26
A
Culture
27
A
Organization
28
B
DevOps Introduction
21
255 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
29
A
Culture
30
D
Organization
31
D
Culture
32
B
Automation
33
B
Automation
34
D
Automation
35
C
Processes
22
256 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix B Syllabus
DASA DevOpS FunDAmentAlS DASA DevOpS Syllabus
FunDAmentAlS
Version 1.0.3
November 2018
Syllabus Version 1.0.3
November 2018
257 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
ReleASe
veRSiOn
DAte
Previous
1.0.2
April 2017
Current
1.0.3
November 2018
Next
TBD
TBD
ScOpe AnD puRpOSe OF thiS DOcument The purpose of this document is to inform all parties interested in the DevOps Fundamentals course of the areas covered in the course.
2
258 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix B | Syllabus
the DASA DevOpS cOmpetence mODel The DevOps Agile Skills Association (DASA) competence framework identifies 8 knowledge areas (depicted in redcolored text) and 4 skills (depicted in blue-colored text) that are relevant in DevOps, as shown in the following figure.
Teambuilding DevOps Leadership
3
Continuous Improvement
der
2
Architecture and Design
DASA DevOps Professional Enable and Scale
s Op
ps P
4
S DA
Ow ne r
5
Courage
1
Business Value Optimization
DASA DevOps Professional Specify and Verify
Infrastructure Engineering
DASA DevOps Fundamentals
DASA DevOps Professional Create and Deliver
Business Analysis
Security, Risk, Compliance
Continuous Delivery
Test Specification Programming
chh 1. Novice / 2. Competent / 3. Proficient / 4. Expert / 5. Master
3
259 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Every individual operating in a DevOps team requires to be competent at all 8 knowledge areas and proficient at the 4 skill levels. In order for DevOps teams to be effective, they require all 12 areas to be at the Expert level. Individual team members can specialize in specific areas, in order for teams to achieve these capabilities.
DASA DevOpS FunDAmentAlS Up to 200 times faster software deployment, 30 times increased deployment frequency, and 60 times higher change success rates, organizations such as Netflix, Spotify, and Facebook are revolutionizing the IT game by successfully implementing DevOps principles. The data does not lie. You do not have to be a hot Web company or a monster enterprise to be a DevOps leader. Companies, large or small and young or old, have magnificently made the transition and have the proof of success in their pockets. DevOps training is the starting point for an organization going on the DevOps journey. Improved workflows and faster deployment starts with a core understanding of DevOps fundamental concepts by anyone involved in an Agile and/or DevOps team. DASA develops and evangelizes a vendor neutral DevOps qualification program for professionals, generates interest and awareness for the need for knowledge and skill development, promotes open source certification for DevOps knowledge and skills, and ensures quality of training for the market through a logical and threshold-driven qualification program.
4
260 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix B | Syllabus
Anyone can participate in defining role-based competences, learning paths, and qualification schemes. All existing learning content that maps against the DASA knowledge and skill areas has value. DASA will map content and demonstrate relevance and will maintain an open and logical operating model for training delivery, as shown in the following figure.
LEADERSHIP Lead and Enable
DASA DevOps Leader
DASA DevOps Coach
DASA DevOps Professional Enable and Scale
DASA DevOps Professional Specify and Verify
DASA DevOps Professional Create and Deliver
FOUNDATIONAL Know
DASA DevOps Product Owner
PROFESSIONAL Know and Apply
DASA DevOps Fundamentals provides an extensive introduction to the core Agile DevOps principles covering the essential knowledge and skill competences that have been defined by DASA.
DASA DevOps Fundamentals
The DevOps Fundamentals qualification is designed to provide the core education necessary to build your DevOps vocabulary and to understand its principles and practices. With the help of key DevOps concepts and terminology, reallife case studies, examples and interactive group discussions and extensive exercises in each module you will acquire a fundamental understanding of DevOps. 5
261 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
QuAliFicAtiOn ObjectiveS When you have acquired the required knowledge from this course, you will be able to: • Explain the drivers responsible for the emergence of DevOps. • Define and discuss the key concepts and principles of DevOps. • List and explain the business benefits of DevOps and continuous delivery. • Describe the Service Delivery process. • Explain the concepts of test automation, infrastructure automation, and build and deployment automation. • Describe how DevOps relates to Lean and Agile methodologies. • Summarize case studies of IT organizations that are making the transformation to Adaptive IT and DevOps models. • List the most common and popular DevOps tools. • Discuss the critical success factors for DevOps implementation.
6
262 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix B | Syllabus
hOw DOeS DevOpS FunDAmentAlS Fit intO the DASA cOmpetence FRAmewORk? After completing this course, you will cover the area marked as DevOps Fundamentals in the following figure of the DASA qualification scheme. As a result, you will reach the “Competent” level of the scheme. Teambuilding 5
DevOps Leadership
Courage 4 3
Architecture and Design
2
2
Continuous Improvement
2 1
2
2
DASA DevOps Fundamentals
2
Business Value Optimization
2
Infrastructure Engineering
2 2 2 2
2
Security, Risk, Compliance
Business Analysis
Continuous Delivery
Test Specification Programming
1. Novice / 2. Competent / 3. Proficient / 4. Expert / 5. Master
7
263 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
tARget AuDience The DevOps Fundamentals qualification is primarily aimed at: • Individuals involved in IT development, IT operations, or IT service management • Individuals whose role are touched by DevOps and continuous delivery, such as the following IT roles: ◊ DevOps engineers ◊ Product owners ◊ Integration specialists ◊ Operations managers ◊ Incident and change managers ◊ System administrators ◊ Network administrators ◊ Business managers ◊ Automation architects ◊ Enterprise architects
cOuRSe ReQuiRementS Basic familiarity with Agile, Scrum, Lean, and ITSM principles is beneficial.
8
264 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix B | Syllabus
ceRtiFicAtiOn ReQuiRementS You will receive the required certification from DASA on successful completion of the DASA DevOps Fundamentals exam.
exAm DetAilS The characteristics of the DASA DevOps Fundamentals exam are: Exam Format: y Closed-book format y Web-Based y Participants may bring scratch paper
Questions: y 40 multiple choice questions
Passing Score: y 65%
Exam Duration: y 60 minutes y 15 minutes extra time for non-native English speakers.
9
265 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
leARning OutcOmeS A classification widely used when designing assessments for certification and education is the Bloom’s Taxonomy of Educational Objectives. This classifies learning objectives into six ascending learning levels, each defining a higher degree of competencies and skills. (Bloom et al, 1956, Taxonomy of Educational Objectives). This structured approach helps to ensure: • A clear delineation in learning level content between different qualification levels • Learning outcomes are documented consistently across different areas of the guidance • Exam questions and papers are consistent and are created to a similar level of difficulty.
10
266 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix B | Syllabus
The Fundamentals qualification examines learning outcomes at levels 1 (knowledge) and 2 (comprehension). DASA DevOpS pRODuct OwneR leARning OutcOmeS 1. 2. 3. Knowledge Comprehension Application Generic Definition from Learning Outcomes
Know key Understand key facts, terms concepts from the and concepts manual/guidance from the manual/ guidance
Qualification Know facts, Learning including terms, Outcomes concepts, principles, tools and techniques from the DevOps Fundamentals curriculum
Be able to apply key concepts relating to the syllabus area for a given scenario
4. Analysis Be able analyze and distinguish between appropriate and inappropriate use of the method/ guidance for a given scenario situation
Understand the concepts, principles, and dimensions of DevOps and can explain how these are applied.
SyllAbuS AReAS The following syllabus areas are identified. SyllAbuS AReA cODe
SyllAbuS AReA title
IN
DevOps Introduction
CU
Culture
OR
Organization
PR
Processes
AU
Automation
MI
Measurement & Improvement
11
267 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
SyllAbuS In the following tables, the key aspects of the DevOps Fundamentals Syllabus are described.
intRODuctiOn Syllabus Syllabus Area : Area Code IN Level
Introduction (IN) Topic
Know the historical development of DevOps, the core concepts underlying DevOps and the DevOps Agile Skills Association Specifically to recall: 01
01 01
01
01
02 03
04
•
The relationship between the Digital Transformation and DevOps
•
The high level description of DevOps
•
The history and emergence of DevOps
•
The key elements of the Business Case for DevOps
•
The principal benefits of DevOps
•
DevOps Definitions
•
The Culture of High Performance IT
•
The relationship between DevOps, Agile and Lean IT?
•
DevOps Principles and Aspects of IT
•
The purpose of the DevOps Agile Skills Association (DASA):
•
DevOps Skills Areas, Knowledge Areas, and Competence Framework
•
DASA Qualification Scheme, Mission, and Vision
Understand the following aspects dealt with in the Introduction Specifically to identify: 02
01
Possible problems that can arise due to the wall of confusion between Development and Operations
02
02
The core principles of DevOps
02
03
The 12 competence areas (4 Skill areas, 8 Knowledge areas) of the DASA Competence Framework
02
04
The 3 core profiles of the DASA Competence Framework
12
268 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix B | Syllabus
cultuRe Syllabus Syllabus Area : Area Code CU Level
Culture (CU) Topic
Know the key components of Culture Specifically to recall: 01
01
01
01
01
02
03
04
•
Build the DevOps Organization around teams
•
The Three Horizons Model for Innovation
•
Definition of a DevOps culture
•
Cultural Aspects of a DevOps Team
•
Two key elements of a DevOps Environment: Service Mindset and Quality at the Source
Key Skill Areas of the DevOps Agile Skills Association Competence Framework: •
Team Building
•
Continuous Improvement
•
Courage
•
DevOps Leadership
Skill Area: Team Building •
Definition of a team
•
Three key drivers of motivation: Autonomy, Mastery, Purpose (Pink)
•
Intrinsic motivation as a driver for working in teams
•
Collaboration as a Key Success Factor of a Team
•
Visual Management as a Key Tool of Teambuilding
Skill Area: Continuous Improvement •
Importance of Quality at the Source
•
Cost of Accumulating Technical Debt
•
Role of Solving Problems in Continuous Improvement
•
Structured Problem-Solving
•
The Kaizen Mindset: Tackling the Root Cause of Problems
13
269 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
01
01
01
05
06
07
Skill Area: Courage •
Courage to Act: A Key Behavior of a DevOps Team
•
Courage and Experimentation
•
Psychological Safety as a pre-condition for Courage
•
Relationship Between Experimentation and Complications
•
Experimentation Meetups: A Key Tool of Courage
Skill Area: DevOps Leadership •
Leadership in a DevOps Environment
•
Mission Command philosophy as opposed to Central Command
•
Importance of Leadership to Overcome Five Barriers of Effective Collaboration
•
Role of Leaders in Stimulating the Use of Tools to Develop Effective Habits
•
Feedback: A Key Leadership Tool
Implementation of a DevOps Culture: •
How to build a DevOps culture
•
Impact of Treating Change as a Program
•
Growing Culture: Experimenting, Measuring, and Probing
•
Importance of Tracking the Movement Towards a DevOps Culture
•
Cultural Change: A Collective Movement
Understand the following aspects related to Culture Specifically to identify: 02
01
The key characteristics of a DevOps Culture
02
02
The way to build a DevOps culture
02
03
The challenges moving towards a DevOps Culture
14
270 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix B | Syllabus
ORgAnizAtiOn Syllabus Syllabus Area : Area Code OR Level
Organization (OR) Topic
Know the key aspects of Organization Specifically to recall: 01
01
01
02
Organizational Models: •
Impact of DevOps on the Organization
•
Alignment of Organizational Model with IT Services
•
Traditional Structuring of Teams and Waste
•
Importance of DevOps Hybrid Versions
•
Activity-Focused Versus Product-Focused Approaches
•
DevOps Organogram
Autonomous Teams: •
What is autonomy?
•
Autonomy of Teams
•
Criteria for Autonomous Teams
•
Decoupling Point: A Key Consideration for Autonomous Teams
01
03
Conway’s Law and Organizations’ Architecture
01
04
Solving the Autonomy Problems – A Real-life Example: The Spotify Model
01
05
Architecting for DevOps: •
Aim of the IT Architecture
•
Focus on Building in Quality
•
Move towards smaller services in the IT architecture
•
Relation Between Complexity and Quality
15
271 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
01
01
06
07
•
Micro Services Architecture (MSA) and its Characteristics
•
MSA Supports Faster, Cheaper, Better Software Development
•
Architecting for Systemic Resilience
•
Moving from Legacy to Smaller Services
Governance: •
DevOps Governance
•
Governance Within Teams and Between Multiple Teams
•
Scrum of Scrums with Agile Teams to Coordinate and Collaborate
Understand the following aspects of Organization Specifically to identify: 02
01
02
02
16
272 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix B | Syllabus
pROceSSeS Syllabus Syllabus Area : Area Code PR Level
Processes (PR) Topic
Know the key aspects of Processes Specifically to recall: 01
01
Definition of process and the key components of a process: goal, result, input, throughput, output, customer
01
02
DevOps in Relation to ITSM:
01
01
01
03
04
05
•
ITSM
•
DevOps and ITSM
Agile and Scrum: •
Traditional Versus Agile
•
Role of Multidisciplinary Feature Teams
•
The Agile Manifesto
•
The Scrum Flow
•
Advantages of Working Agile
Optimizing Processes Using Lean: •
What is Lean?
•
The Eight Types of Lean Wastes
•
Optimization of Processes Using Value Stream Mapping
Business Value Optimization and Business Analysis Using Story Mapping: •
Role of Minimal Viable Product in an Agile Process
•
How Story Mapping works?
•
Role of Slices in Story Mapping
Understand the following aspects of Processes Specifically to identify: 02
01
02
02
The advantages and disadvantages of developing software applications using the Waterfall approach
17
273 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
AutOmAtiOn Syllabus Syllabus Area : Area Code AU Level
Automation (AU) Topic
Know the key aspects of Automation Specifically to recall: 01
01
01
01
01
02
03
04
Automation for Delivery of Software: •
Automation of Routine Jobs
•
Automation Changes the Focus Towards Engineering Tasks
•
DevOps Teams and Focus on the Delivery of Value
•
Everything as Code
Continuous Delivery Core Concepts: •
What is continuous delivery?
•
Benefits of Automating Continuous Delivery
•
Cycle Time Reduction: Continuous Delivery Primary Goal
•
Primary Principles of Continuous Delivery
•
Continuous Delivery Versus Integration and Deployment
•
Continuous Delivery Focus Topics
Continuous Delivery Automation Concepts: •
Software has to Flow
•
Impact of Continuous Delivery on a DevOps Team’s Performance
•
Types of Feedback
•
Fail Fast: Immediate and Visible Failure!
•
DevOps Versus Continuous Delivery
Continuous Delivery Automation Focus Topics •
Automation Build and Software Package Delivery Flow
•
Automated Test and Optimized Software Validation (Tests)
•
Automated Test: DevOps Merges Specification and Verification
•
Automated Deployment and its Benefits
•
Deployment Strategies
•
Automated Provisioning
•
Containerization (Microservices)
•
Continuous Delivery Backlog
18
274 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix B | Syllabus
01
01
01
01
05
06
07
08
Emergence of Cloud Technology and Principles: •
Emergence of Cloud Computing
•
Cloud Services, Self Service Infrastructure, Platform, and Software
•
National Institute of Standardization (NIST) Cloud Principles
Cloud Service Concepts in a DevOps Organization: •
Cloud Principles in DevOps Organizations
•
Different Conversations Between Development and Operations in a Traditional Organization
•
Different Interaction Styles Between Development and Operations in a DevOps Organization
•
DevOps Platform Teams as a “Cloud Service Provider”
•
DevOps Business System Product and Platform Product Teams
•
Different Types of Clouds to Operate
Automated Provisioning Concepts: •
Pets Versus Cattle
•
Desired State Configuration to Automate Environments
•
Automated Provisioning with Mutable Infrastructure and Immutable Infrastructure
•
Continuous Delivery for Platform Products
•
Automated Provisioning and Engineering Mindset
Platform Product Characteristics and Application Maturity: •
Services Required by DevOps Business System Teams
•
Product Teams, Cloud Services, and Freedom
•
Use of Platform Services and Maturity of Applications
•
How to apply Cloud concepts to an organization?
Understand the following aspects of Automation Specifically to identify: 02
01
02
02
19
275 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
meASuRe AnD impROvement Syllabus Syllabus Area : Area Code MI Level
Measurement and Improvement (MI) Topic
Know the key aspects of Measurement and Improvement Specifically to recall: 01
01
01
02
Importance of Measurement: •
Need of Measurement and Feedback
•
Importance of Feedback: Three Ways Model
•
Measurements and CALMS
•
Relation Between Measurement and Responsibility
Choosing the Right Metrics •
Survivorship Bias
•
Actions Based on Measurements
•
Performance Metrics Versus Performance Predictors (Leading and Lagging indicators)
•
Measuring Leading Indicators for Culture, Organizations, Process Efficiency, Software Development Automation, Data Center Automation, and Measurements
•
Top Practices Correlated with Deployment Frequency, Lead Time for Changes, and Mean Time to Recover (MTTR)
•
Top Five Predictors of IT Performance
•
IT Performance: Throughput Versus Stability
20
276 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix B | Syllabus
01
03
Monitoring and Logging: •
Continuous Monitoring and its Scope
•
Optimized Monitoring for DevOps
•
Collecting Feedback from an Automated Software Delivery Pipeline
•
Dashboards to Build the Feedback Culture (Release Dashboard, Test and Quality Dashboard, Build Dashboard, Performance Dashboard, and Product Usage Dashboard)
•
Importance of Logging Stakeholders and Usage Examples
Understand the following aspects of Measurement and Improvement Specifically to identify: 02
01
02
02
21
277 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
© 2018 - DevOps Agile Skills Association All rights reserved. No part of this publication may be published, reproduced, copied or stored in a data processing system or circulated in any form by print, photo print, microfilm or any other means without written permission by DASA www.devopsagileskills.org
22
278 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix C Glossary
DASA DevOpS FunDAmentAlS Glossary Version 1.0.0 November 2018
279 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Glossary Terms
Description
Agile
Agile is a time-boxed and iterative approach of software delivery. It aims to build software incrementally from the start of the project.
Agile Benefits
Visibility: As Product Owner and business are involved with product development on a regular basis, for instance by attending the sprintly demo (or by launching new shippable features on a regular basis), visibility of what is delivered is far higher than is the case with traditional development methods. Parts of the product are delivered on a regular basis. Risk: Optimization of product visibility lowers the risk, as it becomes clear early in the process whether the team is moving into the right direction and building the right things. It is all about feedback and using this feedback to lower risk. Business Value: By delivering a shippable product at the end of each sprint, this product can actually be used to generate business value throughout the product development cycle. Features are prevented to get ‘stuck’ in the development cycle and are shipped straight away. This as opposed to the “traditional way of working”, where the product is shipped only near the end of the project (preventing the team to used valuable feedback from your end-customer through the software development cycle).
2
280 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix C | Glossary
Glossary Terms
Description
Automated Provisioning
Automated provisioning is defined as the fully automated delivery and maintenance of application environment components. Application environment components are the deployment target containers of the application. For example, a database server or application runtime server. In a DevOps organization, automated provisioning can be the responsibility of DevOps Platform teams.
Backlog Refinement Session
Scrum Term - This session is used to anticipate and define what User Stories are expected in next sprint and communicate uncertainties for in case User Stories are unclear. The session typically takes place halfway a sprint, leaving room for Business and Product Owner to improve User Stories where needed, prior to the starts of the next sprint.
Build Automation
Build automation transforms code changes, committed by team members, automatically to published deployment artifacts, ready for deployment and validation in (test) environments.
Burn Down Chart
Scrum Term - During Planning Poker, features are assigned so called velocity points. When progressing in time, team estimations will become more reliable. The Burn Down chart outlines the burn rate for the running sprint over times. This way, a team can steer on making the needed progress to burn all points for the sprint.
3
281 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Glossary Terms
Description
CALMS
Key ingredients for DevOps as defined by Damon Edwards and John Willis. Culture, Automation, Lean, Measure and Sharing.
Continuous Delivery
Defined by Jez Humble - “Continuous Delivery is about putting the release schedule in the hands of the business, not in the hands of IT. Implementing Continuous Delivery means making sure your software is always production ready throughout its entire lifecycle – that any build could potentially be released to users at the touch of a button using a fully automated process in a matter of seconds or minutes”.
Continuous Delivery Base Principles
y Rigorous Automation y Extreme Feedback y Continuous Change
Continuous Delivery Benefits
Teams that adopted Continuous Delivery: y Increase speed and repetitiveness through automation. y Are Agile as there is no Work in Progress. y Make sure there is flow in their delivery. y Are able to operate largely autonomously. y Are doing the right things right.
4
282 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix C | Glossary
Glossary Terms
Description
Continuous Deployment
“Continuous Deployment is subtly different to Continuous Delivery in that release are automatically pushed into production when all tests pass. In Continuous Delivery, release is a human decision.” Dave Farley
Continuous Improvement Objectives
y Deliver value faster y Deliver value better y Supply services cheaper y Create more meaning in work y Create a healthier environmental footprint
Continuous Integration (CI)
Continuous Integration (CI) is the practice, in software engineering, of merging all developer working copies to a shared mainline several times a day. (Wikipedia, March 2016) “Continuous Integration usually refers to integrating, building, and testing code within the development environment.” Martin Fowler
Culture
Four elements of a DevOps culture: y Teambuilding y Courage y Continuous Improvement y Leadership
5
283 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Glossary Terms
Description
Daily Stand-up
Scrum Term - Every day, the team comes up to the scrum board where each member will explain what he/she did yesterday, where he/she is now and what he/she will be doing today. Impediments, blocking a team member from progressing, are also raised in this stand-up. A stand-up should never take up more than 15 minutes of time.
DASA Competence Framework
The DASA Competence Framework identifies 8 Knowledge Areas and 4 Skills that are relevant in DevOps.
DASA Knowledge Areas
1. Business Value Optimization 2. Business Analysis 3. Architecture and Design 4. Programming 5. Continuous Delivery 6. Test Specification 7. Infrastructure Engineering 8. Security, Risk and Compliance
DASA Principles
1. Customer Centric Action 2. End to End Responsibility 3. Continuous Improvement 4. Create with the End in Mind 5. Cross Functional Autonomous Teams 6. Automate Everything You can
6
284 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix C | Glossary
Glossary Terms DASA Skills
Description 1. Courage 2. Teambuilding 3. Leadership 4. Continuous Improvement
Defects
Rework that is required because an activity was not properly executed in first instance. This requires one to task-switch back to the originating activity, stopping progress, analyze the issue and fix the issue.
Definition of Done (DoD)
Scrum Term - A list of criteria (preferably attached next to the scrum board) describing what topics need to be addressed in order for a product to be considered ‘potentially shippable’. It’s a simple list containing restraints like these: code, unit and coverage tested, functionally tested, performance tested, user acceptance tested, reviewed, documented. It clearly defines a finish-mark. The team only delivers part of the product that adhere to criteria on the list.
7
285 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Glossary Terms
Description
Definition of Ready (DoR)
Scrum Term - A list of rule (preferably attached next to the scrum board) describing to what standards a user story should adhere in order to be accepted by the Development team. Examples of topics on the list could be: “the user story is on the backlog”, “the development team understands the problem”, “the user story is estimated by the development team”, etc. The DoR is there to make sure requirements are clear from its inception and additional conversations during sprint activity are kept to an absolute minimum. It eliminates the need for discussions as much as possible.
DevOps
DevOps is a cultural and operational model that fosters collaboration to enable high performance IT to achieve business goals.
DMAIC
A problem solving method: Define, Measure, Analyze, Improve, Control.
Engineering Culture
A definition of an Engineering Culture (Palantir): “Engineers build things that solve problems. You don’t have to be a computer scientist or have any particular degree to be an engineer. You just have to speak up when things aren’t right, evaluate ideas on their merits, and build things that fix what’s broken.”
Experimentation
Experimentation means testing a hypothesis, and in practice, it means trying something new based on a need.
8
286 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix C | Glossary
Glossary Terms
Description
Feedback
Four types of feedback can be defined: y Feedback on build an test activities. For example, automated unit test results, automated static code analysis results. y Feedback on deployability. For example, automated deployment execution results, automated application deployment “smoke” tests, automated application health checks. y Feedback on runtime behavior. For example, automated user interface functional test results or automated load test results. y Feedback from the customers! For example, revenue / conversion rates.
Impediment Board
Scrum Term - This board contains topics that keeps the team from doing its work, but which is out of reach for the team itself. Typically, the scrum master makes sure impediments are dealt with. Impediment boards should only contain topics for which the team member itself already tried addressing it. i.e. not ‘everything’ is thrown onto this board. Items might include: “not enough desks”, “team divided over multiple locations slows us down”, “network is down several times a day”.
9
287 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Glossary Terms
Description
Inventory
Waste caused by excess product taking up space. In a software development context, it generally means that the work (story) that is not completely done as per your Definition of Done, and hence you cannot demo or release it, resulting tasks to remain in inprogress state.
ITIL
Information Technology Infrastructure Library (ITIL), is a set of practices for IT Service Management (ITSM) that focuses on aligning IT services with the needs of business.
Kanban
Kanban is Japanese for “visual signal”. The Kanban’s visual nature allows teams to communicate more easily on what work needed attention to make sure flow remains in the process. Kanban is a great tool to visualize work-build up, bottlenecks and helps to reduce waste and maximize value.
Motion
An example could be one copy machine on only the second floor, requiring the user to walk around the building for making a copy. Another one is the unavailability of meeting rooms, forcing a team to search for a spot whenever they need some privacy to discuss. In SW development, handover-moments can also be considered a waste related to motion.
10
288 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix C | Glossary
Glossary Terms
Description
MTTR
Mean Time To Recover
Non-Utilized Talent
Waste caused by centering resources around specialized activities. If one has to perform only one type of task, other possible skills this resource might exhibit (like board management, organization, x-team communication, presenting customer cases) are not used!
Over-processing
Commonly this waste is caused when a team is unable to understand the Voice of Customer (VoC), or lack of understanding the product-vision, resulting in gold-plating a product. A product owner might play a significant role steadying this type of waste.
Overproduction
Producing more than is actually needed, generating WIP (work in progress), requiring the next step in the process to think about where to (temporarily) store/archive the superfluous artifacts and find it once it is needed again.
Planning Poker
Scrum Term - At the start of each sprint (and often during the backlog refinement session), the team plays so-called planning poker in order to size the amount of work that is required to fulfill a new activity. In this session a sizing is agreed by the complete team and a ‘common view’ is established on topics at hand. When performing more poker sessions over time, sizings will become more reliable and the team starts to exhibit a specific burn-rate, defining how fast the team is performing.
11
289 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Glossary Terms
Description
Potentially Shippable Product
Scrum Term - The product increment which is delivered at the end of each sprint. If the business deems required, this artifact can be shipped to production straight away as it does not have any tasks outstanding.
Product Backlog
Scrum Term - A continuously evolving and ordered list of requirements and topics, required to make sure optimal product value is achieved. The Product backlog is the one single source of truth for modifications to the product. One list to rule them all.
Product Demo
Scrum Term - Each sprint closes off with a product demo for team, product owner and the business / customer. The demo is a way to provide and receive feedback from all stakeholder and inject this feedback into the product during a next sprint. Attending the product demo is essential for improving collaboration, the next product backlog and of course the product itself.
Scrum
Scrum is the most commonly used manner of introducing Agility to an organization. Its simplicity and flexibility is appealing to many organizations. Scrum emphasizes empirical feedback, team self management, and performs product increments within short iterations.
12
290 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix C | Glossary
Glossary Terms
Description
Scrum Board
Scrum Term - A visual outline of activities in a Kanban style manner, where activities move from left to right over the board “To do”, “Doing”, “Done”, “Impeded”.
Scrum Master
Role in Scrum - Scrum Master – The person responsible for making sure the team adheres to scrum behaviors, rules and guidelines. It is the facilitator making sure everybody plays by the rules. The scrum master not only explains to the team, but also explains to the external stakeholders the way things are done. The Scrum Master enables the team to do the things that are needed to make things work.
Scrum Product Owner Role in Scrum - Product Owner is responsible for maximizing the value of the product. This means that the Product Owner knows the business and customer and defines user stories that matter to the business and customer and holds off on other stories. The PO is the only person responsible for maintaining the product Backlog. The PO makes sure that User Stories adhere to the Definition of Ready (DoR) in terms of how requirements are described, that the board is properly prioritized in terms of value and that clear and transparent communication between development and business is achieved. Scrum Roles
Team, Scrum Master, Product Owner
13
291 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Glossary Terms
Description
Scrum Team
Role in Scrum - Team – A multidisciplinary team which is allowed to work on the tasks agreed at start of a sprint. Every discipline required to deliver a shippable product (as output of a sprint) is contained within the team. Typically, a team might consist of members with skills to define, develop, test, deploy, maintain and communicate aspects of the product. The team members organize themselves and continuously improve their own process. They take responsibility for the way things are moving. A typical scrum team is about 5 maximum 9 members in size.
Sprint
Scrum Term - A predefined amount of time in which activities on the sprint backlog are performed. Sprints often are defined per week or per every two weeks, but longer is also used. Note that shortening a sprint will also result in shorter backlog refinement, poker and retrospective sessions, as the amount of topics to discuss will become much lesser as well.
Sprint Backlog
Scrum Term - Sprint Backlog – A set of product backlog items that have been selected for the sprint, including required tasks for delivering this new features at the end of a sprint (i.e. activities like develop, build, review, test, etc.). The sprint backlog contains an internal prognoses of the development team itself (no one else), for the coming increment.
14
292 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix C | Glossary
Glossary Terms
Description
Story Mapping
Story mapping is an engaging activity where all participants are involved in the process of building the product backlog on a wall.
Story Mapping Benefits
y A lower risk in development as customer feedback is embedded into the design process. y An engaged customer as the system is designed to his or her immediate needs. y Shorter project startup time due to removal of endless up-front design sessions. y Fast ROI as the base-product is in the customer’s hands at an early stage.
Task Classification Quadrant
Task classification quadrant by Charles Perrow. Method to determine if tasks have a potential for automation. The quadrant is based on two dimensions: tasks analyzability and task variability. 1. Task Variability is defined by the number of exceptions to standard procedures encouraged in the application of a given technology. 2. Task Analyzability is defined as the extent to which, when an exception is encountered, there are known analytical methods for dealing with it.
15
293 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
Glossary Terms
Description
Team Retrospective
Scrum Term - After every sprint, the team evaluates what went well and what went not-so-well, thus could use improvement. This is an important aspect of scrum to continuously improve on the way of working.
Transportation
Waste caused by the need for transportation (i.e. a refrigerator located far away from the sink). In SW development this might be task-switching as people are assigned by more than one project. The fastest way to complete two tasks is to perform them one at a time.
Value Stream Mapping
Lean Term - Value Stream Mapping (VSM) is a tool to gain insight in the workflow of a process and can be used to identify both Value Adding Activities and Non-Value Adding Activities in a process stream while providing handles for optimizing the process chain.
Waiting
Wasted time since one has to wait on another task (for instance caused by overproduction) to be finished. Commonly, a wait is caused by a bottleneck in the end-to-end process or by irregular meetings and handover-moments. Delays introduce discontinuity into a process.
16
294 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix C | Glossary
Glossary Terms
Description
Waste
Lean Term - Lean defines 8 forms of waste: Defects, Overproduction, Waiting, NonUtilized Talent, Transportation, Inventory, Motion, and Overprocessing.
Woodward Technology Typology
When processes are highly programmed and automated they will result in predicted and standardized output. Automated processes are repeatable. Cost of process execution is minimized.
17
295 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Course Book | DASA DevOps Fundamentals
© 2018 - DevOps Agile Skills Association All rights reserved. No part of this publication may be published, reproduced, copied or stored in a data processing system or circulated in any form by print, photo print, microfilm or any other means without written permission by DASA www.devopsagileskills.org
18
296 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix D Release Notes
Release Notes DASA DevOps Fundamentals Release
Version No.
Date
Previous
1.4.0
October 2018
Current
1.4.1
November 2018
Next
TBD
TBD
Course Description
Course Duration: 3 days Number of Modules: 7 Case Study Based: No Associated Certification: DASA DevOps Fundamentals
Components Released
Presentations, Course Book and Instructor Guide (eBook) ●
New Features
●
●
Added DASA DevOps Competence Quick Scan Updated all the graphics related to DASA competence model Made a few minor modifications shared by our respected Subject Matter Experts
Bugs Reported
Action Taken
NA
NA
Known Issues
Expected Fix
NA
NA
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
297
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Appendix E Participant Feedback Form Click here to complete the online feedback form TRAINEE ............................................. COMPANY................................................................ E-MAIL ................................................ COURSE .................................................................. DATE ................................................... INSTRUCTOR ......................................................... Course Content The content presented in this course was at the correct level.
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
The content met the stated objective and expectations.
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
The content was presented in a clear and concise manner.
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Group Exercises Reinforced how I might use the skills taught in the training.
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Directly relate to the content.
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Were engaging for the learners.
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
Strongly Agree
Agree
Neutral
Disagree
Strongly Disagree
About the Instructor Communicated the content effectively.
Addressed learner questions effectively. What did the instructor do well and what can be improved? Did Well:
Could improve:
Copyright © 2018 │
Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
299
Course Book | DASA DevOps Fundamentals
Overall
Communicated the content effectively.
Yes
No
Comments on course effectiveness:
What element of this course can be improved? (This could include any specific module/topic/exercise.)
What are your overall impressions of this course?
Additional Comments:
The stated prerequisite requirements were appropriate and sufficient? Yes No The course materials were relevant and contributed to the achievement of the learning objectives? Yes No The time allotted to the course was appropriate for understanding the learning objectives? Yes No
300 │ Copyright © 2018 Training content licensed to Maria Hincape Gomez, issued on 25-03-2019, edition 2019 by Global K, SA de CV
Published By