Best Practices For Implementing CI/CD On Kubernetes Using CircleCI

What Is CircleCI?

• CircleCI is Continuous Integration, an engineering process that software developers use to design, test, and deploy applications on multiple platforms with greater ease and speed.
• CircleCI offers products and facilities at an enterprise level, with a startup's versatility. Wherever you live, we're working: Linux, macOS, Ios and Windows-SaaS or behind the firewall. It runs nearly one million jobs a day to support thirty thousand organizations.
• Organizations prefer CircleCI because jobs run quickly and builds can be speed-optimized. It can be configured to run very complex pipelines efficiently with sophisticated caching, docker layer caching, resource classes for running on faster machines, and performance pricing. As a developer using circleci.com, you can SSH into any job to debug your build issues, set up parallelism in your .circleci/config.yml file to run jobs faster, and configure caching with two simple keys to reuse data from previous jobs in your workflow. It offers tracking and insights into your builds as a CircleCI operator, or administrator, installed on your own servers, and uses Nomad Cluster for scheduling - see the CircleCI Operations Guide for full documentation.

How To Manage Kubernetes Using CircleCI

• CircleCI has announced an expansion of partner integrations to better support developer teams’ management, design, testing, and release applications for Kubernetes environments.
• Kubernetes is an open-source container orchestration framework, a software deployment system that allows for much simpler application management and development, at scale, in multiple computing environments. CircleCI uses Kubernetes to release what the company calls "orbs." Orbs are packages containing setup, jobs, commands, and build and operation execution applications.
• CircleCI helps teams to automate their builds, test and release activities to deploy more frequently and with greater confidence, particularly as pressure to produce more and more applications increases. The new partner integrations include orbs for AWS, Azure, VMware, Red Hat, Kublr, and Helm as part of the Technology Partner Program at CircleCI, which launched in November 2018.
• CircleCI and our partners' Kubernetes-based orb integrations allow customers to easily integrate with container registries, effectively manage deployments, and provide on-demand Kubernetes environments. According to CircleCI, orbs have been widely adopted by users with more than 600 orbs published to the registry since the platform's launch. Adding more partners using Kubernetes would open up more cloud environments in which orbs can be exchanged. Using the software makes it possible to publish services and keep the code optimized and coordinated in both AWS and Azure clouds at the same time — or perhaps something marginally easier like maintaining separate regional data centers without too much hassle.

Already working in production with Kubernetes? Facing scaling challenges? Learn how Magalix can help scale your infrastructure, and help your team focus on applications.

LAB: Creating CI/CD Automated Pipeline Using CircleCI

In this Lab we will go through how to create CI/CD pipeline using circleci in the following steps:

Step 1: Setting Up The Project

If you sign in to CircleCI using a Github account, it will ask for permission to read your creations. You can also go to CircleCI and list all your projects, if you give permission to do so. Click the "plan setup" button and proceed.

Helper Utilities

CircleCI needs us to create the configuration file at this path: PROJECT_ROOT/.circleci/config.yml. Ensure you create that file at the root of your project.

If you want to test the CircleCI configuration in local, you can use the following commands:

• circleci config validate: Validates the yaml content of the CircleCI configuration.
• circleci local execute — job linting: Executes the job in local.

Step 2: Create Jobs

Each and every box in the sample pipeline diagram above is a task. It executes the actual collection of steps provided inside a system. The machine that executes each step is called an executor. There are several executors (Docker, Shell, VM), but we'll use the Docker Executor to build an example of "Hello World."

version: 2
jobs:
       build:
               docker:
               - image: busybox:latest
               steps:
               - run:
                       name: Hello World
                       command: echo "Hello World"

Here We’ve Declared:

• We’re going to use CircleCI version 2.
• Created one job called build. If we don’t have workflows (which we’ll talk about shortly) then that default job name should be build.
• We used Docker executor with busybox as the image.
• The list of steps to run inside the Busybox container.

Step 3: Create Workflows

However, this example is not quite sufficient to build a complete pipeline. To create multiple jobs and link them together, we’ll need to create workflows:

version: 2
jobs:
       unit_test:
               docker:
               - image: busybox:latest
               steps:
               - run:
                       name: Unit Test
                       command: echo "Going to run Unit Test"
       linting:
               docker:
               - image: busybox:latest
               steps:
               - run:
                       name: Lint Test
                       command: echo "Going to run Lint Test"
workflows:
       version: 2
       sample_pipeline:
               jobs:
                       - unit_test
                       - linting

We’ve created the workflow named sample_pipeline and assigned it two jobs:

• These two jobs (unit_test and linting) would be running in parallel and one doesn’t depend on the output of the other.
• Also, each job would have its own executor. In our case, both of the jobs use Docker Executor.

Step 4: Pipeline Setup

Here we’ll create multiple jobs with dependencies between them:

version: 2
jobs:
       unit_test:
               docker:
               - image: busybox:latest
               steps:
               - run:
                       name: Unit Test
                       command: echo "Going to run Unit Test"
       linting:
               docker:
               - image: busybox:latest
               steps:
               - run:
                       name: Lint Test
                       command: echo "Going to run Lint Test"
       build_artifact:
               docker:
               - image: busybox:latest
               steps:
               - run:
                       name: Build Artifact
                       command: echo "Going to run the build artifact"
       deploy_to_dev:
               docker:
               - image: busybox:latest
               steps:
               - run:
                       name: Deploy to Dev
                       command: echo "Going to deploy to dev"
       deploy_to_qa:
               docker:
               - image: busybox:latest
               steps:
               - run:
                       name: Deploy to QA
                       command: echo "Going to deploy to QA"
               - run:
                       name: Step no 2
                       command: echo "Going to deploy to QA again"
workflows:
       version: 2
       sample_pipeline:
               jobs:
                       - linting
                       - unit_test                               
                       - build_artifact:
                               requires:
                               - unit_test
                               - linting
                       - deploy_to_dev:
                               requires:
                               - build_artifact
                       - deploy_to_qa:
                               requires:
                               - deploy_to_dev

There Are A Few Points To Note Here:

• Both linting and unit_test jobs would still run parallel.
• Fan In: the build_artifact job would be triggered only when both unit_testandlinting are successful (i.e, both linting & unit_test).
• A job can have multiple steps, as given in deploy_to_qa. These steps are always executed in sequential order and use the same container.

Step 5: Approval Process

Now, the only part that’s missing is the approval process - if we don't have it, then the whole pipeline will run for each and every check-in (and we don’t want that to happen):

workflows:
       version: 2
       sample_pipeline:
               jobs:
                       - linting
                       - unit_test
                       - build_artifact:
                               requires:
                               - unit_test
                               - linting
                       - deploy_to_dev:
                               requires:
                               - build_artifact
                       - hold_for_approval:
                               type: approval
                               requires:
                               - deploy_to_dev
                       - deploy_to_qa:
                               requires:
                               - hold_for_approval

For readability, we’ve pasted only the workflows and left out the jobs part. Again, you can check the entire configuration here. Some points to pay close attention to:

• hold_for_approval is a new job with type marked as approval.
• deploy_to_qa would be triggered only when hold_for_approval is done.

TL:DR

• To sum up, this was a simple tutorial on how to use CircleCI for deployment.
• From here, there are many ways that you can boost your pipeline. The first way would be to construct a single build job for multiple deployments, each deploying to specific clusters or namespaces in Kubernetes.
• If you have specific Git branches for development/staging/production settings, this can be incredibly helpful, ensuring that releases are always segregated.
• You could also create your own image, instead of using buildpack-deps, to use on CircleCI.