Targeted Rollouts and Release Progression

When you want to try a new Harness feature out, we enable it for you via a feature flag. Our support/account team turns it on for you. Typically, for new features, our release process happens in multiple phases. First, we test internally in lower environments. Second, we test internally in production. Third, we test with a few beta customers in production for a period of time leading to a wider release. Finally, for some features, we release more widely - but still in beta, and often still with some restrictions on who can access them. This is all an example of using targeted rollouts via feature flags, and then progressing releases over time as you learn and respond. We will dig into that further in this blog.

Why Would I Want to Do That?

Feature flags are often thought of as being on the continuum of CI/CD, and there’s good reason for that. Feature flags de-risk your deployments, let you merge and ship faster, and reduce the need for long-running feature branches. In this way, feature flags level off the classic goals and outcomes of CI/CD.

However, feature flags are not only an extension of CI/CD. They are also a separate process that sits alongside CI/CD, enabling new behaviors or removing those behaviors from the CI/CD loop entirely. This simplifies CI/CD and also provides you and your customers with new and better release options. 

When thinking of feature flags this way (as a process that runs alongside CI/CD rather than one that sits on top of it), targeted releases and release progressions are high-value behaviors that immediately emerge.

Targeted releases, specifically, mean turning your change on for a specific subset of your audience initially, and then progressing it to more users if/when you’re ready. Or, alternatively, turning it on for this subset and then turning it off right away if there are issues. This method is especially useful for reducing the amount of project management needed, where you might want to incrementally introduce changes and measure their effects with very low overhead.

Using feature flags like this allows you to stress test changes, have PMs or other folks drive alpha and beta feature feedback, control releases by customer level or region, and work closely with design partners. All without needing to rely on engineering day-to-day for a deploy/rollback cycle. Or to enable/disable things for the targeted users.

How Is it Different Than a Canary Deployment?

We previously wrote about how features flags and canaries should be thought of as working together, not an either/or. But in the context of targeted releases, it’s worth revisiting the differences between feature flags and canaries.

Canaries are a fantastic way to test the code or the artifact for system anomalies, performance issues, scale, and other similar metrics - and roll back as needed. However, feature flags can help you test the change, not just the code.

What does testing the change and not just the code mean, specifically? It means that the code working as intended doesn’t mean the feature is working for users. Does it do what we need for our customers? Can they find it? Can they understand it? What questions do they have? Avoiding a type 2 error---where you mistakenly accept something as fine when it's not---is key.

If we’re in beta of a feature and we’re trying to prioritize the next batch of work in the most useful order, having this kind of feedback guiding what we do next and what has the highest impact is critical. It’s not about “Does the code technically work?” 

This is an example of where we want to think about feature flags not as an extension of CI/CD, where the point is simply releasing the change without risk, but rather as a standalone part of our software development process where we are constantly enabling and disabling things to get feedback, learn, and respond while we are building and deploying every day.

Internal Examples

I’ve already talked a little bit about how we use Harness Feature Flags internally, to turn things on for our customers. That’s one example of how we use targeted rollouts here at Harness. Though, it is one that often makes Slack messages difficult to parse since sometimes we use Harness Feature Flags to turn on a feature inside of Harness Feature Flags for Harness Feature Flag customers (whew!).

There are some other great ways we use targeted releases, though.

• We may turn a feature off for all on-prem customers, but not have to worry about maintaining more complex conditional code or separate builds.
  
• We can turn something on for our Enterprise users without yet having it wired to the account framework that enforces plan limits, which lets us learn more about the feature earlier in the process.
  
• If a feature is early in development, we may give access selectively to our sales engineering team so that they can work with partner customers in a hands-on way to demo the feature, get feedback, or drive conversation around where we are going.

And that’s just how we use targeted rollouts. We’ve heard of plenty of other teams that turn things on and off per region, to test performance or scale. We’ve also heard of customers turning things on and off for specific infrastructure or clusters, to help with migrations or to get data about possible configurations. There’s a lot you can do that isn’t immediately obvious. 

Conclusion

Using feature flags for targeted rollouts is a powerful way to expand how your organization builds software, learns from customers, and de-risks changes along the way.

Most organizations, when starting with feature flags, have the idea in mind that feature flags require the “right” use cases to be useful. They often struggle to know where to start. However, thinking of targeted releases helps explain that the more things are behind flags by default, the more options you’ll have in the future.

You can’t always predict what, when, and where you will want to target. The mindset of targeted releases is about knowing you have the capability to do so, so that as scenarios, questions or needs come up, your organization is able to learn and react.

Want to keep reading about feature flags? Why not check out our pieces on Kill Switches and How To Get Started With Feature Flags?