This blog post reflects on a Front Five engagement that involved a process innovation project disruptive to a medium sized organisations value chain activities during a period where their core software product was entering the top of its established S curve life cycle and where the product’s success was driving a consequential rapid expansion of the market.
A major reflection for me has been to consider the appropriate execution timing of a process innovation project (in this case a Continuous Delivery project) that can significantly disrupt the available resources operating the existing value chain activities of the organisations business model.
In this example a client has limited resources that predominantly work on BAU development and support as well as having an overriding focus on ‘market-pull’ to properly understand changing market trends and to feed this into the pre-paradigmatic design stage of an architectural product innovation with the goal of jumping their product S curve and moving through a product discontinuity to compete in an expanding market.
Front Five did an evaluation of the organisations software development and delivery capabilities via the use of an outside-in assessment technology transfer (DORA, 2017). The use of this excellent assessment technology was a mode of enquiry to search for the weakest software development and product delivery capabilities that mattered and that could be targets for innovation. Capability change areas were selected from the assessment outputs and a process innovation project to develop a subset of ‘Continuous Delivery’ (Humble and Farley, 2010) technology commenced.
Our goal was to implement an innovation prototype based on this technology that could be evaluated for readiness before being developed further and then diffused for broad adoption by the organisations software development teams. The project was managed by a Front Five project manager with two consultancy engineers and client staff.
Continuous Delivery is a meta ‘software product delivery’ technology, although applied as a subsystem in a business models value chain activities. It can be thought of as a sociotechnical system comprising a bundle of component, tool and process artefacts as well as the individuals that work with them and the knowledge to apply these in a purposeful systems configuration which creates value. Value is produced when the system is exhibiting emergent properties which are accelerating the pace and reliability of the organisations software product development processes.
The outcomes of the innovation are increased dimensions of throughput and stability when developing and delivering software products (DORA, 2017).
A product roadmap below shows the positioning of Front Five’s process innovation project (the implementation of a subset of continuous delivery technology depicted by the purple bounded area) relative to its environment, the overarching whole of the organisations G2 / G3 product innovation activities.
The project management trajectory
The DORA assessment technology gave strong indicators on which capability areas to select for innovation. As is the case with this qualitative technology, these indicators were abstract with the detail on how to realise them not yet known, so the consultancy project manager commenced an exploratory two week knowledge transfer from the client organisation to consultancy engineers involving interviews and workshops with software staff and managers. This contextualising process created awareness of the complexity of the core product and its current bundle of product delivery capabilities.
Reflecting on the complexity / uncertainty inherent at this stage of an innovation project, an exploratory organic project management approach was chosen to gradually inform the project team on how the DORA selected capability areas would be developed / innovated in the concrete context of this organisations technology landscape and also to get an increasing awareness of the influencing nature of the innovation project’s external environment, the area outside of the purple bounded area on the roadmap.
In “Agile Project Management – Agilism Versus Traditional Approaches” (Fernandez and Fernandez, 2009) differentiate agile management strategies appropriate for different project contexts and describe the initial approach the project manager selected as an ‘Adaptive Strategy’. This was a good selection for the early project context as while the goals (capability areas targeted for innovation by DORA) were known, the ends (specifics of the solution) were uncertain. Just in time planning was used with adaptive feedback to rapidly create a learning system which could then be used to discover what the innovation ends may look like. With environmental constraints acting on this innovation system, the adaptive strategy did have a weakness. The strategy required ‘meaningful customer involvement throughout the project’. Organisational staff needed to become part of the innovation project team and have ongoing involvement alongside consultancy engineers. Unfortunately due to demands in the environment external to the project, involvement of staff resources would prove to be erratic and conflicted.
In ‘Agile Project Management : Steering From The Edges’ (Augustine et al, 2005) describe projects using an agile method as ‘complex adaptive systems’ which can be recomposed dynamically. Considering that the consultancy engineers, the core of the project team would need to have access to different members of the clients staff at different times but knowing that the external environment was placing ongoing demands on these staff the project manager agreed an approach with the project sponsor on how to select resources for periodic team recomposition. Calling this a ‘Transparent Politics Meeting’ it involved meeting senior managers once a week to negotiate for resources and plan an agile sprint accordingly.
It quickly became apparent that this process was ineffective in securing adequate resources as the overriding demands of a portfolio of projects legitimised by a governance structure were taking precedence. It could be argued that there was not yet sufficient legitimation of the innovation project for the negotiation to yield the required staff resources and a further problem is that it was not a line item on the project portfolio. In “Framing of project critical success factors by a systems model” (Fortune and White, 2006) enumerate a number of dynamically connected critical success factors that can be used with a diagnostic Systems model to identify weaknesses in a project’s structure
A set of interactions between the project sponsoring ‘wider system’ and the project implementation ‘system’ includes one that needs to ‘provide resources and legitimates area of operation’. This was a project weakness. When the project manager indicated to the sponsor that he was not getting the resources he needed there initially seemed to be a mismatch in expectations on the level of involvement required by the organisational staff. A suggestion was made that the work should be formulated two to three sprints in advance so that he could reserve the required resources from the other senior managers well ahead of time.
This was a sensible adaptive adjustment by the project manager and sponsor doing a course correction via a ‘performance monitoring subsystem’ in the innovation implementation system when viewed through the lens of Formal Systems with an implied commitment that the project sponsor ‘attempt to influence’ the project’s external environment to secure staff resources for proximate objectives.
In spite of this adjustment, demands in the external environment (R&D on the architectural product innovation, Web Application development, Docker service development, SAP integration, feature and bugfix releases, support) became more acute and I would claim that because the project was not establishing sufficient legitimacy, securing staff resources remained challenging for its duration. In the ‘Connecting to the Core Business’ section of “Engaging with Startups to Enhance Corporate Innovation”, (Weiblen & Chesbrough, 2015) claim that an ‘organisational interface with the core business is a highly critical point in an outside-in program’. An argument can be made that without the sponsor and project manager connecting the process innovation project into the core business product strategy via the technology roadmap so that it could be understood by others in that strategic context, any ‘attempt to influence’ for project resources would be challenging. This aspect of the innovation project could have been improved by integrating the illustrated technology roadmap into the ‘Governance’ meetings and making the case for connecting the process innovation project directly into the product strategy value proposition.
A catalyst for a pivot in the project management approach came when the project sponsor became concerned as to why the team was making specific technology decisions. A choice was made by the consultancy engineers to build the innovation prototype using ‘Jenkins’ technology rather than the existing ‘TeamCity’. Although the consultancy made the case for the new technology platform as being one which was better integrated into an open source ecosystem, unburdened by IPR licence costs and on which experiments could be conducted to reduce the time that the build jobs took to run, goals of the project, the sponsor felt that the justification for the technology platform was not backed by sufficient data and method. It might be argued that more of an experimental innovation culture needed to be developed.
At this stage the project sponsor started assertively asking for a more formulated project plan. The project manager stopped the agile sprints and worked with the core consultancy engineers to switch to a linear rational approach with a view to creating a stage gate development model but with an ‘Iterative Agile Strategy’ (Fernandez and Fernandez, 2009) embedded in the stages. Before stage-gate could be used to formulate the project stages, the consulting team spent a few weeks doing a type of systemic inquiry (Ison, 2009) in order to reflect on learnings to this point, to clarify the purpose of the innovation project, to capture holistically a systems map of a hierarchically integrated technology end state and to formulate innovation project work streams which the team felt would be feasible.
Systems approaches were used to discuss and map the technology hierarchy and to bound an area over which a stage-gate development project would be applied. A strength of the approach was that it eliminated reductionist thinking that had unnecessarily focused on specific applied technologies (such as jenkins) and made explicit what the consultancy was proposing in terms of the overall value proposition of the innovation project.It also allowed others to see how socio technical processes could be innovated and changed.
The bounded technology hierarchy map is shown below with the subsequent stage-gate model showing four work streams that can be managed in parallel.
I would argue that the timing of the project management pivot was good. An uncertain project had started in a complex environment which required significant exploration and an adaptive agile approach was a suitable methodology. Organic learning and experiments over the first two months of the project had led to sufficient learning of both what the project innovation may look like as well as an adequate appreciation of the project environment for a more formulated project management approach to now be used. The project trajectory had moved from complex uncertainty to a more manageable complicated domain (Snowden, 2005). The image below is based on my research of the Cynefin Framework and shows a dynamic illustrated in Snowden’s “Strategy in the context of uncertainty” which I think is well applied to this project management approach trajectory.
In conclusion
This blog post explores the efficacy of realising value on a process innovation project during an organisations period of pre-paradigmatic architectural product innovation. A particular reflection was the timing of the process innovation project, relative to product innovation.
The process innovation technology was a subset of ‘Continuous Delivery’ and the product innovation an architectural innovation to create a ‘G3 SaaS Product Suite’. Product innovation activities were considered as environmental factors influencing the process innovation project and there was a critique of the trajectory of project management approaches used.
My view is that it may have been better to commence the process innovation project once a dominant design was close to emerging on the G3 product innovation and that the project management approach should have been strategically connecting to the value realisation of that product innovation to secure the required project resources for the process innovation project.
The following implementation work was concluded on this project
Work Stream | Stage |
Pre: Value Stream Analysis | The product delivery value stream was mapped end to end and the following baselines taken :A baseline of the TeamCity Build jobs, including all build steps with aggregated min, max and average metrics to analyse for optimisation opportunities in the prototype.
A baseline of automated desktop and web regression tests including socio technical activities in the value stream to analyse for optimisation opportunities A baseline of all QA activities in the run up to a release including automated and manual testing across desktop and web channels to analyse for optimisation opportunities |
Solution Build / Store / Scan | Build Infrastructure : The creation of an experimental platform for the innovative development of ‘Continuous Integration with Continuous Deployment’ using Jenkins technology. The infrastructure is is built as infrastructure-as-code (IaC) using Terraform, Chef and Shell scripts and is provisioned in AWS. The Jenkins master is hosted on an Ubuntu host , whereas the slaves are Windows Server 2012 nodes. By definition, the slave pools are dynamic and can be scaled up and down both horizontally and vertically in order to cater the varying agent application loadsCI Optimisation : The creation of compile agent assemblies connected as agents to the Jenkins master and which are configured to compile all solutions in preparation for compile time optimisation activities. The compile agent is defined as IaC.
Artefact Store : The outside-in transfer of JFrogs Artifactory technology into the pipeline design for evaluation. This included collaboration with JFrog around IPR trial licences and the IaC configuration of a Artifactory MVP on an EC2 hosted stack : Artifactory/Ngnix/MySQL/Route53/LetsEncrypt Artefact Store : The integration of Artifactory into Docker, Nuget and Binary Store build jobs |
Environment Provision / Deploy | Provisioning subsystem : The creation of a capability to use Terraform to provision VMWare hosted pipeline environment VMs from the build infrastructure platform and the bootstrapping of user data customisation in order to prepare an enabling state for the application deployment subsystem as per the pipeline systems diagram. |
Test Development | Lean test pack development : Definition of web and desktop integration test scope including work with client staff on defining migration from Lab management to Team City MSTest |
Post: Documentation | Creation of high quality runbooks for all implemented technology and services. |
Patrick Hyland
- Augustine, S., Payne, B., Sencindiver, F. and Woodcock, S. (2005) ‘Agile project management: steering from the edges’, Communications of the ACM, vol. 48, no. 12, pp. 85–9
- DORA (2017) ’Overview’, DevOps Research & Assessment [Online]. Available at https://devops-research.com/ (Accessed 9th October 2017)
- Fernandez, D.J. and Fernandez, J.D. (2009) , “Agile project management – Agilism versus traditional approaches”,The Journal of Computer Information Systems, vol. 49, no. 2, pp.10-18
- Fortune, J. and White, D. (2006) ‘Framing of project critical success factors by a Formal System Model’, International Journal of Project Management, vol. 24, no. 1, pp. 53–65
- Humble, J. and Farley D (2010) Continuous Delivery: Reliable Software Releases through Build, Test and Deployment Automation, Boston, Pearson Education
- Ison, R (2009) ‘Epistemological Awareness: A Systemic Inquiry’, Cybernetics & Human Knowing, vol. 14, no. 2-3, pp. 197–200(4).
- Snowden, D (2005) ‘Strategy in the context of uncertainty’,Handbook of Business Strategy, vol.6, no. 1, pp. 47-54.
- Weiblen,T. and Chesbrough, H. (2015), ‘Engaging with Startups to Enhance Corporate Innovation’, California Management Review, 57, 2, pp. 66-90