Pipeline Strangler Pattern
The Strangler Pattern made famous by Martin Fowler describes how software evolves in our enterprise -- though this observation can also made with reference to the platforms and pipelines that support our applications.
Evolution, Revolution, and the Past all at once?
One of my favorite animated movies is Kung Fu Panda. To put Po the Panda’s mind at ease, Master Oogway gave the prolific statement: “Yesterday is history, tomorrow is a mystery, but today is a gift.” This means, no need to dwell on the past and the future can not be predicted. I am inspired! Well, one industry goes against that grain is the software industry. It’s true that today is a gift, but with the varying amount of platforms we build and support, we can not just forget about the past. Software moves at a very fast velocity and since software delivery methodologies focus on collaboration, seeing trends and shifts are easier than ever today. Therefore, the future is not as mysterious as Oogway might suggest.Maintaining the past, present, and future architectures drive the need for agility in our software platforms. Imagine the challenge that our Pipelines face by having to support legacy, brownfield, and greenfield patterns.
How a Pipeline Strangler is Born
Martin Fowler goes into detail about the Strangler Pattern. Like a vine growing around a tree or in Kung Fu Panda’s case noodles, the newer parts eventually survive as the vine effectively strangles out the older parts of the tree. In the software world, as more modern architectures take over, the overall architectures will move forward and the legacy architectures will be left to be strangled / die off. The mighty world of software architecture pushes forward -- huzzah! Though for us responsible for making sure that all of the software engineering teams have a fair voice and raising the tide on all ships, letting pieces die would be the easy move not the most prudent move. Pipeline Stranglers can be even more difficult to notice and can be quite detrimental to the innovation. Organizations tend to invest a lot in their pipelines, and once the status quo steps in, it can be hard to change even if the process is rigid or brittle. Pipeline Stranglers are born out of the very nature of the software industry we are in. Innovation moves forward as we reset on new technologies and the technology learning curve.
Funny O'Reilly Animals
Some of my favorite technology books are from O’Reilly. A walk down an engineer’s workspace or even at home, can usually see a few funny animal covers lining the book shelves or today a subscription to the publisher. If we were in charge of ushering in a new technology or just intellectually hungry, a good first place to grasp is O’Reilly. Dissecting the contents of an O’Reilly book, let’s say if a book is 500 pages, the first 30-50 pages would be installation and simple CRUD-like operation and the other 450 or so pages are operational type patterns. Making sure technology has a viable home in your organization is a large part of the adoption and acquisition equation.
A Pipeline Strangler Growing Up
Let’s look at two technologies, distributed metrics provider Prometheus and log analysis provider Splunk. Both are great technologies but can show exactly how a pipeline strangler is born. Let’s say that Acme Corporation has an investment in Splunk and to participate in an application team at Acme, Splunk is the de facto tool. Now a set of engineers have been seeing a few projects that they integrate in the community start to leverage Prometheus on a greenfield project. Acme does not have any operational or pipeline expertise in Prometheus. Though the urge is too great to leverage Prometheus and the engineering team starts to build in the hooks, include the libraries, and ecosystem expansions such as Grafana. Acme’s platform engineering team has an uphill battle to include Prometheus into Acme’s fold. Not wanting to metrics data to just take up storage space on Acme’s SAN’s, they should do something with the metrics. Unfortunately this takes away from the focus of the platform engineering team with the large number of applications still leveraging Splunk. The little straggler starts to appear as a push for distributed metrics vs log analysis starts to occur. Though at Acme, they rely heavily on Splunk and still is a critical part of their infrastructure; certain applications don’t have good instrumentation and rely on Splunk to fill in the gap. Master Oogway, we really need to maintain both.
Strangle as Much as You Want
The beauty of the Harness Platform is that we take away a part of the technology learning curve with convention and configuration. In Acme’s case, the Harness Platform could help integrate both Splunk and Prometheus. Depending on the team or application, adding or subtracting an integration / technology provider is easy in a Continuous Delivery pipeline with Harness. Imagine a world where when demands from application development teams are as simple to bring into the pipeline fold and not tangled like a vine or noodle. You can live in that world today with Harness!