About Twilio

Twilio is a global leader in communication APIs, enabling developers and businesses to embed messaging, voice, and video functionality directly into their software applications. Founded in 2008 and headquartered in San Francisco, Twilio has over 800 employees. As a fast-growing technology company, Twilio constantly seeks new ways to innovate and improve its product offerings. They turned to Split to scale their experimentation efforts and make faster, data-driven product decisions to meet growing customer demands and drive adoption.

Challenge

Knowing the impact of every feature they release

As one of the world’s fastest-growing software companies, Twilio’s innovative engineering team needed a solution to support rich experimentation across multiple use cases. They faced challenges with:

• High-risk releases: Ensuring new features with significant impact (e.g., changes to sign-up flows) were rolled out cautiously and measured for success.
• Assessing feature value: Quantifying the impact of backend code changes aimed at improving metrics like API response times.
• Segmented testing: Running experiments on a range of user segments (e.g., by geography, code language, and customer account level).

Twilio's engineering team built many tools in-house, but as experimentation demands grew, they needed a more robust system that could manage user assignments, record feature treatments, and integrate seamlessly with their data infrastructure.

Solution

Integrating Split for full-stack experimentation

To scale their experimentation efforts, Twilio turned to Split, which provided the necessary features to run sophisticated experiments across multiple teams and environments. The key to success was Split’s flexibility and ability to integrate with Twilio’s existing infrastructure.

Key improvements:

• Fast experimentation setup: With Split, Twilio was able to roll out their experimentation platform six months ahead of schedule, enabling faster testing and quicker product iterations.
• Granular targeting: Split allowed Twilio to perform precise targeting by segmenting users based on customer behaviors and geographical factors, enabling more accurate testing and better insights.
• Seamless integration: Split’s flexibility allowed Twilio to integrate the experimentation platform with their internal employee dashboard, providing a seamless experience for product and engineering teams.
• Data-driven decision-making: Split’s platform empowered teams to run full-stack experiments, providing crucial data to make faster, more informed product decisions.

Laura Schaffer, Product Manager at Twilio, commented, “Split’s full-stack experimentation platform has been built with engineering and product teams in mind. Its robust architecture and rich feature set integrates into our internal platforms and helps power experimentation across our entire engineering organization.”

Results

Accelerated experimentation and improved product decisions

By implementing Split, Twilio transformed its approach to testing and iteration, enabling the company to make data-driven decisions faster and with more confidence:

• Six-month head start: With Split’s feature flagging and experimentation, Twilio’s team accelerated their platform rollout, getting up and running six months ahead of plan.
• Clear data-driven insights: In a key internal debate about adding extra questions to the user sign-up flow, Twilio ran tests using Split. After one month, the experiment revealed that additional questions significantly improved the sign-up experience, validating the sales team’s hypothesis.
• Validated product improvements: By leveraging Split’s capabilities, Twilio was able to test product changes with precision and measure the impact of those changes against key metrics, such as sales opportunities and product adoption.

Business impact

Improved adoption and decision-making efficiency

With Split, Twilio gained several advantages that directly impacted business outcomes:

• Faster product iterations: By speeding up the experimentation process, Twilio was able to quickly assess new features and roll out the most impactful ones.
• Informed decision-making: Split helped Twilio validate key product decisions with data-backed results, reducing uncertainty around product changes and features.
• Efficient use of resources: Engineers and product managers no longer needed to manually track experiments or build custom solutions, allowing them to focus on higher-priority initiatives.

Future 

Expanding their culture of experimentation

Twilio plans to extend the use of Split to more areas of their product and continue scaling experimentation efforts. For example, they are planning an experiment to improve the user verification process through two-factor authentication. The goal is to increase verification success rates, further demonstrating Split’s role in Twilio’s continued success.

Through their work with Split, full-stack experimentation has become a critical part of Twilio’s continuous delivery process, supporting ongoing product innovation and development.