This material was presented with an alternative narrative at the CA IT Management Symposium in Johannesburg, South Africa. The slides and narrative to that presentation are available here :
A Systemic Approach to Transformation with DevOps
Stories told around the warm crackling glow of a camp fire.
This sharing activity has gone on for probably as long as man has known how to make fire. Its a bonding experience, developing relationships and strengthening our collective understanding in an environment where different perspectives and world views can be shared passionately as stories.
This blog post uses a campfire tales metaphor to explore the views of a group of three professional individuals (Gene Kim, Charles Betz and Rob England ) by using a fictional scenario where they have gone on a camping trip and are discussing the problems that they have encountered when trying to apply or integrate ITIL and DevOps within their areas of practice.
The perspectives of the individuals are drawn from various publicly accessible blog posts and the strategy also draws on my perspective and experiences in managing Application Engineering teams.
The situation of interest is any large service provider where IT service management is perceived to be operating in a dogmatic way by zealously applying a few reductionist ITIL process silos. It is important to note that ITIL need not be implemented this way (The author has studied the ITIL lifecycle extensively) but unfortunately it often is and for the purposes of exploring how to think strategically to plan improvements this is a stated and dysfunctional starting point.
Some of the consequences of this starting point are siloed operations, a lack of co-ordinated feedback through the service provider, dissatisfied and dysfunctionally politicized staff, limited innovation and disengaged customers using unreliable and incident prone services.
I will be using the blog posts of Gene Kim, Charles Betz and Rob England as input to run through a selection of System Thinking tools in order to strategically explore possible futures that may come about by holistically transforming the dysfunctional starting state.
The future state system models and the dynamics emerging from them are my interpretation of what may happen if the transformations are implemented. So they carry the bias of my perspective but they are a demonstration of a powerful collaborative device – the use of systems thinking and systems practice as a method of inquiry that can lead to deeper understanding between DevOps and IT Service Management practitioners.
Collaboration facilitated by using the models and dynamics as a structured way of discussing the problem situation is more important than the creation of the models themselves. The learning cycles created by these collaboration opportunities may adjust a selected strategy as deeper understanding emerges and a broader variety of perspectives is understood and absorbed.
I will be using the following systems tools to think strategically :
| Systems Tool | Purpose |
| Werner Ulrich’s
Critical Systems Heuristics (CSH)
|
|
| Stafford Beer’s
|
|
| Jay Forrester’s
|
|
The present state ‘problem situation’ IT Service Provider :
The bounded judgements implicit in the present-state system :
Through the lens of Ulrich’s CSH systems can be decomposed and examined to see which ‘boundary judgements’ have been used in the design of the system and one can get an insight into how the current system may be changed to a future state. Below I map a perspective on the ‘boundary judgements’ existing in the ‘IT Service Provider’ system which I will use for the purposes of developing this blog post.
| What’s are the stakes? | Who are the stakeholders? | What are the stakeholding issues? | |
| Sources of Motivation
|
(What is the purpose of the system?) A system to transition services into the production operations environment that are both fit for their designed purpose and fit for their warranted use |
(Who is the beneficiary of the system?) The Service Provider’s customers |
(What is the measure of improvement indicating success in the system?) The tight control of change via a powerful change management authority. A reduction in the number of incidents in the production environment |
| Sources of Control | (What resources that should be under control of the system?) Service Transition and Service Operations staff. Development and Operations staff |
(Who is the decision maker in control of the success of the system?) The Service Provider management |
(What conditions of success are outside the control of the decision maker?) As far as the Service Provider management is concerned all conditions of success will be under its direct control |
| Sources of Knowledge | (What is the expertise required to successfully run the system?) All process managers must be ITIL trained and qualified in their process areas |
(Who is providing the expert knowledge and skills in this system?) The ITIL process managers |
(What are the guarantor assurances that this system is working as the expert intends?) KPIs being met on the ITIL processes will be regarded as assurance of a successful operation of the system |
| Sources of Legitimacy | (How do those affected negatively seek emancipation from the worldview of the system?) The interests of those negatively affected by the system will be represented by the DevOps practicing community |
(Who represents the the interests of those negatively affected by the system? ) The opportunity for those negatively affected to express their desire for freedom from the worldview of the system will be industry speaking events and community dialogue between ITIL and DevOps practitioners |
(What political space is available for those affected by the system and those in control of the system to reconcile their worldviews?) The space to reconcile worldviews at this specific company will be collaboration and dialogue between ITIL and DevOps practitioners working for or on behalf of the company. |
Boundary discourse commences…
Betz, Kim and England stand in front of a roaring campfire looking forward to a night of stimulating conversation. Camping chairs out, stars twinkling above, coffee in hand, they start chatting.
Ulrich’s CSH boundary discourse is used to explore the perspectives of Kim, Betz and England. The setting is a fictional conversation by the practitioners discussing ITIL and DevOps around a campfire in order to develop a shared metaphor that can be used to explore the situation of interest and to critique the boundary judgements that exist within a hypothetical (but typical) system of interest, that is the actual present-state reference system, the dysfunctional IT Service Provider described above by the systems practitioner.
The purpose of the critique is to think strategically about the problem-situation system in order to do strategy making. At the end of the discourse a set of strategic changes will be selected.
The fictional conversation draws on direct quotes from blog posts written by the protaganists and these must be read first in order to properly follow the boundary discourse below.
Kim, (2014) – Trust me: The DevOps Movement fits perfectly with ITSM
Betz, (2015) – ITIL and DevOps: Differences and how they work together
England, (2013) – Kamu: a unified theory of IT management – reconciling DevOps and ITSM/ITIL
Boundary critique of the system’s sources of motivation
Raising his mug of coffee and taking a swig, Kim begins
Kim is saying that there is a developmental opportunity for ITSM practitioners to work with Dev and Ops staff in a different way to create ‘phenomenal organizational outcomes’.
Betz does not seem as sure, he says that ITIL is too plan centric and does not have good facilitation for the management of a flow of work in progress. Although his statement has a much broader context and implies issues related to the flow of work across the entire ITIL lifecycle, it is also relevant in the context of the present-state system because the tight change control regime is making the system very ‘stop/start’ in nature, interrupting flow.
England now raises an interesting point on whether we should concern ourselves with ‘robust’ application services or ‘anti-fragile’ application services. This is known as categorisation and the point is that the different categories imply different approaches. The present-state system is defining its success measure wholly by using a ‘robust’ category approach and the implication is that we can look at ways in which it could be changed by incorporating elements of the ‘anti-fragile’ category approach.
Boundary critique of the system’s sources of control
England throws another log on the campfire as the others start to weigh in. The comments are now taking an interesting direction. They are dealing with the political tensions on the stake of ‘control’ within the system.
Kim is saying that there is a trend for the ITSM community to disregard the inputs of DevOps. In the context of the present-state system this is implying that the decision environment should include elements of the emerging wisdom coming from the DevOps community.
Betz is saying that the ITIL body of knowledge has not had a refresh in a while and may be becoming outdated. In the context of the present-state system, this implies that the decision environment of the system should include new theories and understanding.
England looks a bit concerned and says that those calling for the death of ITIL and the overthrow of ITSM have alienated the ‘Robust’ ITIL camp who perceive DevOps as “a barbarian attempt to pull down the defences that keep IT safe and stable”. Enjoying the metaphor, Betz and Kim chuckle. In the context of the present-state system this implies there needs to be a sense of balance in the control decision environment.
Boundary critique of the system’s sources of knowledge
The fire crackles away, flames rising high as the group starts a substantial discussion around the subject of ‘knowledge’ within the system.
Kim is excited as he talks about the ‘underpinning principles in which all the DevOps patterns can be derived from”’. He goes on to discuss ‘The Three Ways’, ‘Systems Thinking’, ‘Amplify Feedback Loops’ and developing a ‘Culture of Continual Experimentation And Learning’.
Kim then describes four DevOps patterns and describes at length how ITIL process owners and ITSM practitioners can integrate these.
Within the present-state system, the system owners have framed the required expertise as being ITIL process training and this boundary judgement gives this type of expertise more power in the system than other types of expertise. The implication for the present-state system is that additional sources of expertise need to be included in the boundary, namely the DevOps expertise discussed at length by Kim.
Conversation now turns to who the experts should be and what influences any guarantors of success of the system?
Kim is enthusiastic that a combination of DevOps and ITIL practice can be used to make the system work! In the context of the present-state system this implies that both ITIL and DevOps practitioners should be considered collective experts working holistically.
England agrees and is keen to ‘reconcile the world-views of ITSM and DevOps’. He wants the ITSM and DevOps group to ‘sing a common song’ and bursts into a Maori song he has called ‘kamu’.
Betz covers his ears and comments with a smile that it’s a great song, but that next time England should bring someone along who has a better singing voice. Kim laughs.
Betz goes on to make the point that the system must become a ‘socio-technical system’ concerned with ‘execution, feedback and flow’. Kim seems to agree and talks about ‘applying Lean principles to the IT value stream’.
In the context of the present-state system, the implications are clear, the suggestion is that the experts must be a combination of ITIL process managers and DevOps practitioners and the guarantor boundary must go wider than just KPIs on ITIL processes!
It must include lean practice promoting flow and frequent evaluation of systemic feedback occurring within an activity executing sociotechnical system.
Boundary critique of the system’s sources of legitimacy
I as the systems practitioner and industry ITSM and DevOps practitioner reflect on the sources of legitimacy within the present-state system. That is to say, I look at the ability for the people affected by the system to have a platform on which to critically challenge the system and I believe the platform is adequate for DevOps practitioners affected by those involved in the current present-state system (the IT Service Provider organisation).
There is the ability to approach the IT Service Provider management to describe the problem situations they see in the system of interest, and to negotiate strategic accommodations.
This is because the IT service provider in control of the present-state system recognizes a tension in the DevOps practicing community and is curious to hear its view. This speaks to the emancipation boundary container.
Furthermore, it is aware of industry events and blog posts discussing these issues as witness to them, the witnesses boundary container. This very blog post adds to that body of witness.
Although it’s ITSM staff are generally resistant to change the organisation has indicated it will try and understand the DevOps worldview through dialogue between ITIL and DevOps practitioners within its system.
The fire is almost burnt out and and it’s time to hit the sack!
Recommendations
After reflection it is decided that in the first instance Kim’s ‘expertise’ boundary judgement changes within the ‘Sources of Knowledge’ part of the system will be selected as being the strategic change candidates desired to be applied to the present-state system and an interpretation of these will be illustrated / described using Beers VSM.
Only the first two sets of changes will be considered in this blog post.
It is necessary to read the detail on the changes in Kim’s blog post (Kim, 2014), namely
- Extend Development into IT Operations
- Create IT Operations feedback into Development
Before applying Kim’s selected changes (1 and 2), I will based on experience gained as a leader of application engineering teams break down and remove the Dev and Ops silo’s by putting in place a ‘DevOps Practice’ sub system. It will include an ‘Agile Dev Team’ with an embedded ‘Ops’ subsystem in that team. There will be a Continuous Delivery subsystem within ‘DevOps Practice’. I will also remove all the co-ordination system flows in the meta-system to create a clean starting slate.
Strategic change set 0 : Break the Dev and Ops silo
Strategic change set 1 : Extend Development into IT Operations
| System of Interest | VSM System Level | Suggested change and motivation |
| Service Transition | S2 Coordination | Release and Deployment management is collaborating with the Continuous Delivery management to “help that team with our release management readiness checklists, security hardening checklists and so forth, integrating them into the automated build process.” |
| Service Transition | S2 Coordination | Change Management is working with Continuous Delivery management in order to “define pre-authorized changes and deployments, and ensure that production promotions are captured in a trusted system of record that can be reviewed and audited.” |
| Agile Dev Team | S1 Implementation | The developers begin using the continuous delivery system to promote changes down the pipeline into the OPS run environment |
| Continuous Delivery | S2 Coordination | The Continuous Delivery system promotes standard (pre-authorized) changes through the pre-production environments all the way into the OPS run environment |
| OPS Run Env | S2 Coordination | The OPS run environment uses API integration to log standard changes into the SACM database, the trusted system of record. |
Change Management and Continuous Delivery Management collaboration starts a balancing loop, ITIL Change Management Integration, that for a specific application service clearly distinguishes standard from normal change and processes standard via the CD pipeline. The accuracy of the SA&CM CMDB increases, thereby increasing the efficacy of Knowledge Management. Once sufficient standard change is being promoted down the pipeline and being automatically logged via an API into the SA&CM CMDB the collaboration with change management may decrease, hence the balancing nature of the feedback loop.
Collaboration between Release Management and Continuous Delivery Management starts a virtuous reinforcing loop, ITIL Release Management Integration, which increases the requisite variety of release readiness requirements processed and tested via the CD pipeline with the result that the number of successful releases to the operational run environment increases.
| System of Interest | VSM System Level | Suggested change and motivation |
| Agile Dev Team + Continuous Delivery and its sub-systems |
S1 Implementation | ‘Telemetry’ code has been passed into the application and promoted from the Agile Dev Team through all the run environments using Continuous Delivery to instrument the application for monitoring. The data exposed by the telemetry will be used by the ‘Stack State Intelligence’ system. This could for example be NewRelic or AppDynamics APM instrumentation code. |
| Event Management | S1 Implementation | Event Management has created a sub-system, ‘Stack State Intelligence’ to aggregate monitoring events, both infrastructure data and application data. In practical terms, this subsystem would be for example the NewRelic or AppDynamics monitoring dashboards. |
| Event Management | S3* Monitoring | Event Management is skipping a level of management to monitor the continuous delivery system and via proxy all the run environments including DEV,QA,STG and the OPS run Env |
| Event Management + Knowledge Management |
S4 Intelligence | Event Management is scanning ‘Stack State Intelligence’ to detect any incidents in the monitored run environments. Knowledge Management is scanning ‘Stack State Intelligence’ for feedback that it can use in Service Transition, for example CSI feedback that it can take into ‘Release and Deployment’ or ‘Change’ management’. |
| Event Management + Incident Management + Ops and Agile Dev Team |
S2 Coordination | When Event Management detects an incident it is coordinating with Incident Management who are coordinating with the embedded Ops team within the Agile Dev team. The Ops team and Agile Dev team have ready access to ‘Stack State Intelligence’ to quickly resolve incidents reducing MTTR. |
Two reinforcing loops work together to raise the Event Management process efficacy increasing the availability of stack state intelligence and reducing Incident MTTR (Mean Time To Repair).
First blogged in 2015. Republished with minor amendments.
Patrick Hyland
DevOps Associates 