A successful feature release requires careful planning across five phases: dogfooding, debugging, ramping, scalability, and learning. Each phase mitigates risks, improves decision-making, and ensures long-term success.
The process of launching a new product feature or functionality has undergone significant changes in the last ten years. Back then, it was common for teams to have feature launches tied to code releases in their roadmap with a specific launch date. When the release branch was merged into the main/trunk and pushed to production, new features riding on that branch would be launched to customers.
Feature flags changed this equation by separating code release from feature launch. By putting a feature behind a flag, product managers and engineers got access to a rudimentary switch they could flip on after code ship. In case things went south, they could flip the switch off.
At large scale companies — typically B2C companies — this switch evolved into a ramp. Simply put the flag could be off, on, or on for a randomly selected percentage of users. For example, a product manager or product team could release a feature to 5% of users, or 10% of trial users. This evolution was a game changer: it allowed engineers to test out the scalability of systems supporting the feature and product managers to turn every feature launch into an experiment by tying metrics to feature flags.
However, this capability has also led to confusion. Many teams struggle with the question: how many steps are required in this ramp and how long should we spend on each step?For instance, should a feature launch go from 1% to 5% to 10% to 20% to 50% to 100%? Or should it go 10%-50%-100%? Taking too many steps or taking too long at any step can slow down innovation. Taking big jumps or not spending enough time at each step can lead to suboptimal outcomes.
The experimentation team at LinkedIn has proposed a useful framework for answering this question. As an aside, in the remaining document, I use the terms feature and experiment interchangeably.
In this article we will breakdown a 5 phased launch strategy that should contribute to a successful product launch or feature rollout. This isn’t a template but more of a look at overall phases and the roles each plays.
The first phase of the ramp is dogfooding the feature in production with internal employees. The goal is to detect integration bugs, get design feedback from colleagues, get QA certification in production, or train sales or support teammates on a new feature. At this step, the goal is not to detect performance challenges or to measure the impact of the feature. So, this step can be quick, a couple of days at best. As an aside, this phase is pre-launch was added by me; it is not part of the original LinkedIn framework but can be a good exercise, especially if these stakeholders are of similar personas as your potential users.
This second phase of the ramp is aimed at reducing risk of obvious bugs or bad user experience. If there is a UI component, does it render the right way? Can the system take the load of feature? Specifically, the goal of this phase is not to make a decision on whether the feature improved user experience therefore, there is no need to wait at this phase to gain statistical significance. Ideally, a few quick ramps — to 1%, 5%, or 10% of users — each lasting a day should be sufficient for debugging.
Once we are confident the feature is not risky, the goal shifts to decision making. By decision making, we mean whether the feature is positively impacting the metrics it was designed to improve. The ideal next ramp step is a 50/50 ramp — 50% of users see the feature, 50% do not. From an experimentation or decision making perspective, a 50/50 ramp is the fastest way to gather data on customer impact. You should spend at least a week on this step of the ramp to collect data across high and low traffic days.
The MPR phase tells us whether the feature was successful or not. If it was, we can directly ramp to 100% of users. However, for most non-trivial scale of users, there may be concerns about the ability of your system to handle 100% of users for the feature. To resolve these operational scalability concerns, you can optionally ramp to 75% of users and stay there for one day of peak traffic to be confident your system will not topple over.
The feature or experiment may be successful, but post launch you may want to understand its long-term impact on users. For instance, did it improve retention or reduce churn? If you are dealing with ads, did the new feature lead to long-term ad blindness? This type of “customer feedback” is crucial and should ladder back to KPIs. You can address these ‘learning’ concerns by keeping a hold-out set of 5% of users who are not given the feature for a long period of time, at least a month. This hold-out set can be used to measure long-term impact, which is useful in some cases. The key thing here is to have clear learning objectives, rather than keeping a hold-out set for hold-out’s sake.
Effectively launching a feature requires a series of steps, each with a specific objective. The dogfooding phase is for internal feedback, debugging and scalability phases are meant for risk mitigation, while MPR and learning phase are meant to speed up learning and decision making.
These milestones should help in your feature launch.
But it doesn’t stop there. A strong messaging and marketing plan is needed to engage users with your new feature.
Incorporating the Messaging Strategy: Before launching a feature, it's important to develop a clear and compelling messaging strategy. This strategy should be created in close collaboration with the marketing and product marketing teams. It involves defining the key value propositions of the new feature and how they align with the needs and preferences of the target audience and personas. The messaging strategy should be consistent across all channels, including email, social media, and in-app notifications, to ensure a unified brand experience.
Role of the Marketing and Product Marketing Teams: The marketing and product marketing teams play a pivotal role in crafting and disseminating the feature's messaging. They should work together to create promotional materials, blog posts, and social media content that highlight the feature's benefits and differentiators. Additionally, these teams can leverage customer testimonials and case studies to provide real-world examples of the feature's value. Engaging with the community through webinars, Q&A sessions, and live demos can further amplify the message and encourage adoption.
Social Media Strategy: Social media platforms offer a dynamic environment to promote new features and engage directly with users. The marketing team should develop a comprehensive social media strategy and marketing campaigns that includes teaser posts, launch announcements, and feature spotlights. Interactive content, such as polls, quizzes, and live streams, can drive engagement and foster a sense of community among users.
Integrating Social Media into the Go-to-Market Strategy: Social media should be an integral part of the go-to-market strategy for any feature launch to engage new customers with the feature and relevant info like pricing, pain points, etc. It enables real-time feedback and can significantly increase the reach of the launch messaging. Tailoring content to the specific characteristics and user base of each social platform can optimize engagement and impact. Tracking metrics such as engagement rates, click-through rates, and sentiment analysis can provide valuable insights into the effectiveness of the social media strategy and inform future launches.
These new sections aim to highlight the importance of integrating comprehensive messaging and marketing strategies, including the effective use of social media, into the overall product launch plan. This approach ensures that the feature not only meets technical and operational benchmarks but also resonates with the target audience and achieves maximum market impact.
Feature Management & Experimentation gives you the confidence to move fast without breaking things. Set up feature flags and safely deploy to production, controlling who sees which features and when. Connect every flag to contextual data, so you can know if your features are making things better or worse and act without hesitation. Effortlessly conduct feature experiments like A/B tests without slowing down.