Confidence-building and quality engineering principles are ingrained throughout the SDLC in modern software development teams. The deployable artifact, or build, as one of the main work products that a development team produces typically, has gone through some level of scrutiny before traversing closer to production. Build and artifact-centric quality steps are prudent in a Continuous Integration pipeline.
Quality engineering principles can certainly paint a wide brush on steps and methodologies to be incorporated into a development pipeline. Artifact/build level testing typically focuses on the composition of the artifact e.g source code and dependency/configuration level checks and unit/functional test coverage focusing on new/regression functionality. The integration/system level test coverage usually requires a deployment and more often is handled in a Continuous Delivery pipeline.
A good line in the sand to decide where to run a certain test coverage is that if you can accomplish in your local development environment, these would be good tests to run in your Continuous Integration pipeline. Items that require a deployment or additional application infrastructure are better suited for a Continuous Delivery pipeline. When running tests locally, items that you test are either coordinated with a process that is running a local version of the application or is remotely called; this can be categorized as on or off process testing.
On Process vs Off Process Testing
Usually, you are triggering tests in two fashions. These can be dubbed as on or off process testing. Imagine yourself in front of your favorite integrated development environment [IDE] and you would like to run JUnit or Go tests. A right-click or two would kick-off the test suite/harness and internal to your local machine / IDE process will run the tests on your behalf. This can be considered on process testing.
On the flip side, you might have to submit your codebase or artifact(s) that have been created to some sort of scan or external test. The external scan process typically is another package or platform that is purpose-built. This can be considered off process testing. A good example of off process testing is leveraging a container scanner such as Clair or Twistlock . If you are looking to move into running tests for the first time into your CI pipeline, can certainly tackle the low hanging fruit first.
Moving Your First Tests Into a Pipeline
Once you get your immediate build and packaging steps down into a Continuous Integration pipeline, the next steps are to start to start to layer the confidence-building/testing steps into the pipeline. Similar to running on process testing on your local machine, an approach could be to lift and shift the on process calls into a Continuous Integration pipeline. The good point about on process calls that they are designed to be called by a build script e.g Maven which is low hanging fruit for a CI platform.
Because on process tests are more code coverage focused, it is rare to introduce additional code coverage tests later on in the SDLC; imagine randomly adding JUnit tests in QA. For off process testing like an artifact scan, usually what changes between environments are the rule sets. As you leave your local environment to an integration environment, the rule sets for what is acceptable tend to get slightly harder. Tossing Continuous Delivery in the mix, as you progress environment to environment, the rule sets get more stringent as you march towards production. Leveraging Harness CI / Drone is an excellent way to run and orchestrate tests as part of your Continuous Integration process.
The Harness CI Way
Leveraging Harness CI / Drone is simple to accomplish both your on and off process testing goals. Harness CI / Drone is very convention based. For example if you are creating a GoLang application, calling your suite of Go Tests is as simple as adding another line of YAML in the configuration.
For the off process testing, submitting to Clair for container scanning or SonarQube for code analysis is easily handled by the plug-in ecosystem. Re-ordering is simple also; if you would prefer to run code analysis before or after the Go Tests, for example, can move the steps in the configuration YAML.
Extending the plug-in ecosystem with Harness CI / Drone is also done by convention. If your needs are not meet with existing plug-ins in the ecosystem, creating your own plug-in is not a difficult task and can easily parse information from the configuration YAML within the plug-in. With the advent of Drone joining the Harness Family, the power of CI and CD are now under one roof.
Partner with Harness, CI and CD Together
Modern software development teams are subject to a litany of confidence-building exercises. With the push of shifting left towards the development team, orchestrating automated tests becomes crucial. Certain testing alludes itself better to a CI pipeline vs a CD pipeline; re-iterating the line in the sand if a deployment is needed then you transition over to a CD pipeline. Harness offers best of breed platforms for CI and CD to supercharge your software delivery experience. From idea to production, Harness has you covered. Feel free to take a look at Harness CI / Drone and the Harness Software Delivery Platform, today!