An Introduction To CI/CD

The adoption of CI/CD has changed how developers and testers ship software. What is CI/CD? CI/CD stands for Continuous Integration, Continuous Delivery (or Continuous Deployment), which represents a culture and process around constantly integrating new code. provides developers a solution to the problems associated with constantly integrating new code. This blog is about this transition to CI/CD pipelines that will provide insights into different tools and process changes which can help developers be more successful.

Developer approaches are always changing: long ago, we had Waterfall, then it was Agile, and now it's DevOps. DevOps is how modern developers are building great products. The new methods of Continuous Integration, Continuous Delivery (CI/CD), and Continuous Deployment have come with the rise of DevOps. Conventional software development and delivery methods are rapidly becoming obsolete as deployment frequency increases.

How the CI/CD pipeline flows

Before DevOps, most companies would deploy/ship software in monthly, quarterly, bi-annual, or even annual releases (also know as the Agile days). In the DevOps era, weekly, daily, and even multiple daily deployments are the norm. With SaaS taking over the development world, you can easily update applications on the fly without forcing customers to download new components. If your CI/CD pipeline is working right, they shouldn’t even realize an update has happened. 

Development teams have adapted to the shortened delivery cycles by embracing automation across their software delivery pipeline. Most teams have automated processes to check in code and deploy to new environments. These adaptations have birthed the CI/CD process and have been coupled with a focus on automating the testing process, including unit testing.

What is CI/CD?: The Terms

Continuous integration (CI) focuses on blending the software work products of individual developers together into a repository. This can be done several times a day, with the primary purpose being to enable early detection of integration bugs while also allowing for tighter cohesion and more development collaboration.

The aim of continuous delivery (CD) is to minimize the friction points that are inherent in the deployment or release processes. Typically, a team's implementation involves automating each of the steps for build deployments so that a safe code release can be done at any moment in time.

Continuous deployment (CD) is a higher degree of automation, in which a build/deployment occurs automatically whenever a major change is made to the code.

Each of these stages is part of a deployment (or development) pipeline. In their book Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, Jez Humble and David Farley explain how every change to your software “goes through a complex process on its way to being released. That process involves building the software, followed by the progress of these builds through multiple stages of testing and deployment. This, in turn, requires collaboration between many individuals and possibly several teams."

"The deployment pipeline is a model of this process, and its incarnation in a continuous integration and release management tool is what allows you to see and control the progress of each change as it moves from version control through various sets of tests and deployments to release to users." - Jez Humble and David Farley

Continuous Integration (CI)

When practicing continuous integration, developers frequently integrate their code into a main branch of a common repository. Rather than building features in isolation and submitting each of them at the end of the cycle, a developer is able to contribute software work products to the repository several times on any given day.

The main idea with CI is to reduce integration costs by having developers do it more frequently and sooner than they normally would. In practice, a developer will often discover boundary conflicts between new and existing code at the time of integration. If it’s done early and often, the expectation is that resolving these conflicts will be easier and less costly to perform.

Of course, there are trade offs; namely, this process change does not provide any additional quality assurances. Many organizations find that such an integration can become more costly in terms of time since they rely on manual procedures to ensure that new code doesn’t introduce new bugs and doesn’t break existing code. To reduce friction during integration tasks, continuous integration relies on test suites and an automated test execution. It’s important to realize that automated testing is quite different from continuous testing.

The goal of CI is to refine integration into a simple, easily-repeatable everyday development task that reduces overall build costs and reveals defects early in the cycle. Success with CI will depend on the culture of the development team, specifically if there is incentive for frequent and iterative builds and an eagerness to deal with bugs when they are found frequently. You may have to make necessary cultural changes to the team to ensure these facets are sustainable.

Continuous Delivery (CD)

Continuous delivery is actually an extension of CI, in which the software delivery process is automated further to enable easy and confident deployments into production at any time.

A mature continuous delivery process exhibits a codebase that is always deployable. With CD, software release becomes a routine and no frills event without anxiety or urgency. Teams are able to proceed with daily development tasks with the confidence that they can build a production-grade release ready to be deployed at any time without elaborate orchestration or special late-game testing.

CD depends on a central deployment pipeline in which the team automates the testing and deployment processes. This pipeline is an automated system that executes a progressive set of test suites against the build. CD is highly automatable, and, in some cloud-computing environments, easily configurable.

In each segment of the pipeline, the build may fail a critical test, in which case the pipeline will alert the team. Otherwise, the build continues on to the next test suite, with successive test passes resulting in automatic promotion to the next segment in the pipeline. The last segment in the pipeline will deploy the build to a production-equivalent environment. This is a comprehensive activity, since the build, the deployment, and the environment are all exercised and tested together. The result is a build that is confidently deployable and verifiable in an actual production environment.

A solid exhibit of a modern CI/CD pipeline can be seen with AWS. Amazon is one of the cloud-computing providers that offers an impressive CI/CD pipeline environment, where you can choose from among its many development resources and link your choices together in a pipeline that is readily configurable and easily monitored.

Many consider continuous delivery attractive primarily because it automates all the steps from submitting code into the repository to releasing fully-tested, properly-functional builds that are ready for production. CD is an elaborate automation of the build and testing processes, but decisions about when, how, and what should be released remain a manual process. Continuous deployment can free up time for those discussions by automating all the other steps.

Continuous Deployment

Continuous deployment extends continuous delivery so that the software build will automatically deploy if it passes all tests. In such a process, there is no need for a person to decide when and what goes into production. The last step in a CI/CD system with continuous deployment will automatically deploy whatever build components/packages successfully exit the delivery pipeline. Such automatic deployments can be configured to quickly distribute components, features, and fixes to customers, and provide clarity on precisely what has been pushed to production.

Organizations that employ continuous deployment will benefit immensely from the ability of users to give quick feedback on new deployments. Quick user response on unhelpful or misunderstood features will help the team refocus and avoid devoting more effort into functional areas that are unlikely to produce a good return on their investment. However, as features are being quickly delivered to users, any defects that become evident should be handled promptly or the team risks getting overloaded with trying to fix the latest bugs and release new features.

CI/CD Tools

With the move to DevOps, there has been a surge of new automation tools to help with the CI/CD pipeline. These automation tools typically integrate with various established popular developer tools, including code repository systems like GitHub and bug tracking systems like Jira. As SaaS has become a more popular delivery model, many of these tools are running in the cloud, the same place where many modern developers are running their apps (including the ones at mabl!).

The most popular automation tool is Jenkins (formerly Hudson), which is an open source project supported by hundreds of contributors as well as a commercial company, Cloudbees. Cloudbees even offers several different Jenkins training programs and product add-ons.

Besides open source projects, there are several modern commercial software automation products available including CircleCI, Codeship, and Shippable. Each of these has several different advantages and disadvantages for specific workflows. To really understand which will work for you, I’d encourage trying each of them specifically within your developer workflow to see how they work in your environment (how they work with your tools, your cloud platform, your container system, etc.).

We build mabl on Google Cloud Platform, so we were looking for a product that was compatible and preferably integrated with GCP. We took a look at CircleCI, Codeship, and Shippable to compare their strengths and weaknesses:

What’s next?

Once you’ve deployed a modern CI/CD pipeline, you’ll likely realize that your existing tools and processes in your developer workflow will also need to be modernized. One area that you’re likely to very quickly realize needs a lot of attention is testing. In this blog on Logz.io, John Fahl says that the real key to reaching CD is creating rich testing that gives you confidence in your rapid release schedule. If your team is deploying daily or multiple times a day and your current testing practices take hours to run or are set to run overnight, your testing won’t be able to keep up with the deployments. This is the problem that mabl is aimed at solving.

CI/CD Evolutions in 2020

While CI/CD as a concept, development philosophy, and practice has remained largely the same, the tools that power CI/CD pipelines (which are the drivers behind CI/CD's future) have continued to evolve and branch out in new directions. These evolutions have come as the result of or from anticipating a rise in the types of services teams are using for their pipelines, such as Kubernetes' rising popularity and the increase in container usage driving innovation in tools that connect to containers.

One such tool with a high potential for changing the standard tool set for CI/CD pipelines is Jenkins X: a proposed sub project of Jenkins that focuses on automating CI/CD for the cloud using the core Jenkins engine plus other open source tools like Kubernetes. X is designed to allow "automatic releases with semantic versions, the creation of artifacts, Docker images and Helm charts, automatic promotion of versioned artifacts across your environments via GitOps, and Preview environments created on each pull request" [CloudBees source]. While we haven't heard much about it since its reveal, development is still trucking along, with the team putting out a one year anniversary post reviewing the aims of X.

The continuous delivery arm of the pipeline may also be in for an upgrade, with the tool Harness aimed at strengthening the safety of deployments by tracking them after they've been pushed. Alongside a pipeline builder and workflow wizard to simplify the pipeline construction process, Harness has a continuous verification feature that continuously checks if the deployments teams send out are causing new anomalies or regressions. If any deployments fail, Harness will automate the rollback to the last working version of the app without any need for user interaction. Recently, the team behind Harness has expanded this feature to run 24/7 in case these anomalies surface hours after the deployment or at times when user traffic is higher than average.

CircleCI, an established CI/CD platform that focuses on automating the pipeline from commit to deploy, improved the ability teams have to share their workflows by releasing Orbs in November. Orbs is CircleCI's new package manager and is "designed specifically for configuration of software delivery automation" [CircleCI source]. Using Orbs, CircleCI users are able to easily package the commands, executors, and jobs that make up their CI/CD workflow and then send them around to any teams they want. Those teams are able to use the exact same workflow or tweak them to fit their needs, markedly speeding up the setup of new CI/CD workflows for teams on the same project. CircleCI released standardized orbs for popular pipelines so individual teams can easily get started with CI/CD using popular tools that have plenty of documentation and setup guides.

Want to see how it works?

Mabl is an intelligent test automation solution built for CI/CD. To see how mabl's test automation can integrate into your CI/CD pipeline, start your FREE TRIAL today.