Database Versioning - You can Version Those?!

We are going to deploy so many times a day that we are going to be off the charts in Accelerate and be the envy of everyone at DevOps Enterprise Summit. One day an idea comes across that requires an additional column in the database. Easy enough let’s just modify the schema with a DDL and we will be all set.Well not so fast. Some folks right now might be grinding their teeth at the thought of introducing a database change giving up on the agility of continuous delivery. Your data is typically seen as the Achilles’ heel for good reason. There is a rise of versioning tools for databases aimed at adding agility to your weak link in the fence, your relational database.

Why is (Relational) Data hard?

At the end of the day, there has to be something stored somewhere for an application to be worthwhile. Application state might live in an in-memory solution but the persistent data well needs to persist somewhere.If you make a change to the underlying schema, effectively you could lose backward compatibility. Adding a new table or even adding new columns, having to revert could mean trouble.Imagine the number of times executing a script directly on a database (don’t tell your DBA). As much as we strive for configuration control between environments, someone executed a change on a stored procedure, index, schema, etc.Typically you will have two types of state/data. The transactional state/data needed for the user session and the more persistent/static data such as the user’s records. The transactional data is pretty easy. If the user doesn’t have a session, no data. Using an in-memory solution is perfect for that.The challenge is the more persistent records. Remember CRUD [create, read, update, and delete]? Your persistent records are subject to these operations. Most folks will have a DDL for how the data looked at the time of inception. Though as time goes on, imagine how many updates and deletes occur. If you had to programmatically re-create, you are looking at starting back with the last DDL or being slower possibly restoring from a backup and applying changes.With databases, rolling back presents the same challenges. As a good practice, you will take a backup at a point close to the migration to revert. Depending on the changes, if your changes include executing statements could be saved in your RDBMS’sUndo Space. Similar to moving forward, having programmatic ways of doing of rolling back allows for increased automation. Database versioning tools are allowing programmatic migrations.

Changing Tides, hello version control for DBs

The tides are certainly changing in the data world. With the momentum that DevOps movements are pushing on the application stack, the database stack has been forced to follow.A few practices are happening with modern database architecture. New transactional data has moved from relational to NoSQL/unstructured/in-memory solutions with modern clustering and replication mechanisms allowing for changes to propagate in safer manners. Another practice is starting to introduce version control to your database. The same version control that your application stacks have enjoyed for years e.g merges and delta are becoming options with your databases now. Two popular tools are Liquibase and Flyway allowing for programmatic versioning of your database. Though versioning tooling typically requires all teams to adopt the tooling; if one team does not the order/versioning will certainly be thrown off. DBMS Tools has a solid list of database versioning tools. Flyway by Redgate has excellent documentation on getting started for your relational databases. With the newer world of NoSQL databases such as Cassandra, more focus will be on topology changes [e.g adding/subtracting nodes] in the Cassandra Ring for capacity and scaling purposes which can certainly be automated. Harness has your back no matter which level of automation you chose.

Supercharge your Journey with Harness

At Harness, we will not solve your database versioning challenges but will give you all the tools you need to have a world-class pipeline. By having the ability to create pipelines with ease, you / your teams can focus on automating the last frontier e.g your data which has most likely been around longer than you at your organization.As you go through your database versioning journey, the Harness Platform can assist with Workflow Barriers or Approval Steps as great points to apply database changes ahead of a versioning solution. As you continue along programmatic ways for versioning your relational databases, Harness can help orchestrate the APIs needed to version.Cheers!-Ravi