The Importance of Continuous Verification
Continuous Verification: Monitoring your app for abnormalities after a deployment. Learn about the importance of Continuous Verification.
Lately, I’ve been working on a whole lot of competitive content. Basically, pitting Harness against a competitor in a cage match and seeing who comes out as the winner. This has led me to learn a lot, really quickly, about where our competitors are strong and where they are lacking. While there are many areas that could use some improvement amongst the competition, one sticks out majorly: Deployment Verification (or as we call it, Continuous Verification).
If you use Harness, you probably know the term. If not, let’s take a sec to educate.
ELI5: What Is Continuous Verification?
Continuous Verification is the process of monitoring your app for abnormalities after a deployment. What are abnormalities, you ask? Well, it could be a latency issue, for one. Alternatively, it could be a 5xx error that customers get after you deploy. Basically, an abnormality is a result that is less than ideal and could result in reputational damage or revenue loss.
The idea behind CV is that, once you enable it, it learns from your previous deployments (that’s right, it’s machine learning-enabled) and creates a baseline of what a good deploy is so that when an abnormality does present itself, it says, “hey, this isn’t right” and takes some sort of remediative action (like rolling back to the previous stable version). This is done as quickly and seamlessly as possible, ideally before customers notice something’s wrong.
Why Is Continuous Verification Important?
One thing that we’ve learned from customers is that one of the four main CI/CD challenges is quality. Quality includes things like incidents that produce downtime, defects and regressions, and performance. All of which affect customers, which in turn, affects you.
At first glance, reputational damage seems innocuous enough, but let me paint you a picture. Imagine you’re an ecommerce site and you’re deploying some code. A customer happens to be shopping on the site. They try to go to another product, and the site is just spinning. Finally, it crashes, and when it comes back, the person somehow lost their cart. They then angrily tag you on Twitter, calling out that your site crashed and they essentially wasted half an hour putting a cart together. Other customers retweet and say it happened to them too. Not a good look, right?
In my younger days, I used to manage a bunch of ecommerce sites for a health supplement company. I sadly remember getting a few angry calls from customers notifying me that the site went down in the middle of their transaction. Imagine if we’d had CV technology back then - it would have prevented a lot of horror stories, I’m sure.
The other side of the coin is revenue loss. For most large companies, it’s fairly easy to put a price on every minute of downtime - and that amount can easily be in the thousands of dollars. In fact, the link above states that $9,000 is the average price for a single minute of downtime (way back in 2016). Now, imagine you don’t use a product like CV and you have to manually verify deployments, then manually roll back to the last stable version. Also imagine that verifying and rolling back takes, conservatively, an hour (which is generous, since some manual rollbacks take multiple hours - like Iterable at 4 hours, and Entur at 2 hours). The revenue loss is massive and provides more than enough justification to use a tool like CV. The ramifications and losses simply aren’t worth it.
A lesser-thought-of consequence of manually verifying quality is the time spent on deployments by engineers. I unintentionally lumped it in with revenue loss to drive my point home, so I won’t spend too much time on this, except to drill it in. Without CV, verification and rollbacks are manual processes, and those person hours add up! Wouldn’t it be easier to have CV do it all for you?
How Does the Competition Handle CV?
Short answer: not well.
Longer answer: it depends on the vendor, but for the most part, CV is going to be a manual process without Harness.
In my competitive research so far, I’ve only seen CI/CD platforms (or standalone CD vendors) provide native integrations for Prometheus - and it still doesn’t do nearly as much as CV does. The brunt of it is still manual. Harness, however, hooks into Prometheus, Datadog, AppDynamics, New Relic, StackDriver, CloudWatch, and custom monitoring and observability tools - so we’re also more flexible and vendor-agnostic. Your tool of choice is our command. All this on top of handling rollbacks automatically.
What’s Next for Continuous Verification?
Currently, Continuous Verification is only available with Harness CD. That is soon changing: CV is going to be its own module, sometime in the near future. Great news for customers who don’t want to move from their current CI/CD platform, but who want the security of automatic deployment verification and rollback triggers.
It’ll be worthwhile, too. The standalone version of Harness CV will provide:
- 24x7 Service Guard, an always-on service health monitoring, along with deployment markers.
- Deployment verification for both Harness CD FirstGen and NextGen.
- Deployment verification for third-party CD tools, including Jenkins, Spinnaker, and GitLab.
- Infrastructure change verification using Kubernetes events and Terraform audit logs as change sources.
- Verifications initiated from additional Harness change sources, such as Harness Continuous Features and Continuous Efficiency, as well as non-Harness sources, such as AWS CloudTrail and ServiceNow.
- Overall service health monitoring (aggregation of all change sources and 24x7 health monitoring).
- Correlated troubleshooting across change sources to reduce MTTR.
Want to Test Drive CV?
If you’re interested in the security and peace of mind Continuous Verification can provide you, go ahead and get the free trial of Harness. You’ll be able to run deployments and see how CV can save you a ton in time, revenue loss, and reputational damage.