Introducing Switch, Split’s New AI Developer Assistant

We are pleased to announce the launch of Switch, our new AI Developer Assistant. Today, we begin a progressive rollout of this new AI-powered capability to Split customers, starting with our early adopters. 

As the CTO of Split, I keep a pulse on what’s hot in tech and am always thinking about what trends mean for the next horizon of our product. We all see the buzz around GenAI in the market. But, here’s what you may not know about what’s coming next. At Split, we recently conducted a study (to be published next month) of development teams. We found that 72% of respondents have been working on putting AI-powered capabilities into their apps, some for over 2 years. 58% of them plan to release those capabilities in the next 12 months. What that means to me is that the flood of interest in AI is about to transition into a flood of new releases. 

Releasing new AI comes with risk, uncertainty, and pressure to stay ahead of the competition. Teams will need to ensure that they have the proper visibility and insight necessary to make fast release decisions. This is how you roll out new AI with confidence. As a developer myself, I like to learn by doing. So, let me first tell you about Switch, and then I’ll tell you how I used Split to test, iterate and release Switch with speed and safety.

Switch: Creating a Better In-App Developer Experience

‘Switch’ is a GenAI chatbot that utilizes publicly available blogs and tech documents to offer immediate help and support. It will guide users through common issues and provide informed solutions. In fact, it’s a very smart chatbot and can help developers with tasks and questions such as providing instructions for setting up a feature flag, sharing feature flag treatments and best practices, and interpreting metric cards. Switch also offers assistance in various languages, a feature that’s already been appreciated by our multilingual team.

Here are some mappings to the developer experience where Switch can step in.

• Basic: Any time you first sign into a new tool, you’re getting oriented. You have basic how-to questions like “How do I set up a javascript SDK in my codebase.” Then, you normally leave the product to browse the technical documentation and it’s on you to find what you need. Finally, you develop an opinion on how good the tech docs are for that vendor. 
• Intermediate: You’re not brand new. You’re actually an active user and you don’t need basic instructions. What you need is advice on what to do, so you might turn to your team lead, a principal engineer, or some other SME who’s seen it all. “I’m trying to accomplish something. Which approach should I use?” 
• Advanced: You’re already good at this, and you have tough challenges to work on. What you don’t want to be doing is cranking out basic code that burns up your hours and could be done by someone else. “Help me wrap this code in a feature flag so I can decide which of these GenAI models to use.”

Given the following code below in java, please wrap the following method "addSpecialMessageForGPT4(messages)" in a Split feature flag called
"genAI_model" to be executed only for the treatment named "GPT4":

private List<ChatCompletionsMessage> getPromptWithContext(List<ChatCompletionsMessage> history,
List<String> contextDocs, String prompt) {
String context = getContextString(contextDocs);
ArrayList<ChatCompletionsMessage> messages = new ArrayList<>();

messages.add(new ChatCompletionsMessage(
"user",
prompt.replace("{context}", context)
));

addSpecialMessageForGPT4(messages);

return capMessages(messages, history);
}

That’s Switch. Your personalized tech docs curator, SME advisor, and capable delegate.

Using Split @ Split: How Split Was Critical to Bringing Switch to Market Quickly & Safely

Using a Split feature flag to decouple deploy from release, we began a progressive rollout today using our Essential Scheduling tool. This means that we were able to plan the launch days ahead of time and then “set and forget” the release to start off hours. Externally, we began first releasing only to a small group of our customer base who expressed interest via our Early Access Program (split.io/eap), about 5% of our user population, which is critical for limiting the blast radius if something were to go wrong.

Catch Any “Bugs” Before They Bite

Because Split automatically watches every metric for every release, we will be able to quickly detect any performance anomalies instantly with our monitoring and altering capabilities. If a performance degradation is measured in this small group, we will be able to drive instant triage (think, immediate MTTD & MTTR). This is Instant Feature Impact Detection (IFID), which allows us to essentially take all major risks off the table as we release this new capability into our UI. Game-changing!

Rapid Test and Iteration

Before rolling out to our external audience, we rolled out to our internal employees first and then rapidly tested and iterated in production. In fact, we’re been releasing new functionality nearly every day. 

How did we use Split to rapidly test and iterate? There are a trifecta of variables involved in GenAI. They are the GenAI model itself, the parameters, and the prompts. You can find plenty of deep dives on those, so I’ll skip that for today. Let’s call that combination of model/parameter/prompt a configuration, and appreciate that your particular configuration makes for some big swings in the outputs you’ll get. 

I needed to create my playground that would let me test different configurations against each other without having to go through a code push each time that might chew up 30 minutes via conventional release processes. So, I used Split’s Dynamic Configuration feature (read more on Dynamic Configuration here) which is a really good way to map otherwise complex configurations to feature flag treatments. 

Using this feature of Split I could swap in a new configuration of the trifecta in a few seconds. 

• For example, we tried the same question in GPT-3.5 and GPT-4, head-to-head. The former claimed an unsupported feature and gave us a fake code snippet. The latter answered correctly. 
• This week, OpenAI released an upgrade to GPT-4 called GPT-4 Turbo. It took us about 10s to update Switch to the new model so we can test it in production.

What’s Next?

Our work with Switch is not an isolated venture; it's intricately tied into Split's overall vision and strategy. As we continue to expand our AI capabilities, we're enabling our teams—and our users—to focus on higher-value tasks, streamline development workflows, and usher in a new era of intelligent feature management and experimentation.

So, what’s next? The current release of Switch uses absolutely no customer data. One more time, no customer data. In a future iteration, there will be features that benefit from ingestion of customer data, and we will invite customers to explicitly opt-in if they’d like to take advantage of that. No surprises. 

As we navigate this journey, the integration of AI with our feature management platform is not just enhancing our product—it's revolutionizing the way developers and product teams build and manage software. It's an exciting time at Split, and Switch is just the beginning.

Register for the Webinar

Join me on November 15th at 10am PT for the live webinar, Embracing AI in Your Product: CTO's Thoughts and the Launch of Split's AI Chatbot Switch. Register here.

Switch It On With Split

The Split Feature Data Platform™ gives you the confidence to move fast without breaking things. Set up feature flags and safely deploy to production, controlling who sees which features and when. Connect every flag to contextual data, so you can know if your features are making things better or worse and act without hesitation. Effortlessly conduct feature experiments like A/B tests without slowing down. Whether you’re looking to increase your releases, to decrease your MTTR, or to ignite your dev team without burning them out–Split is both a feature management platform and partnership to revolutionize the way the work gets done. Schedule a demo to learn more.

Get Split Certified

Split Arcade includes product explainer videos, clickable product tutorials, manipulatable code examples, and interactive challenges.