GitOps Pipelines: Pull vs. Push Deployments

Background

We’ve been talking about GitOps for a while now ever since Magalix joined Weaveworks in early January. Together, we are working on enabling DevOps teams to accelerate development time frames and eliminate cloud misconfigurations using policy as code. And that’s what we have set on to do. Weaveworks' first product release since the acquisition, Weave GitOps 2022.03, features policy as code capabilities to GItOps pipelines. We call this Trusted Delivery. You can read more on that here.

In this blog, we will take a step back and talk about some of the basic GitOps fundamentals for those just getting started on their GitOps journey. We will briefly introduce GitOps and then explain the pull- and push-based deployment pipelines. 

What is GitOps?

Coined by Weaveworks in 2017, GitOps is an operational model for Kubernetes and other cloud technologies. It takes DevOps best practices - such as version control, collaboration, compliance, and CI/CD tooling and applies them to infrastructure automation. 

GitOps can be summarized as follows:

• It’s an operating model for Kubernetes and cloud-native, providing a set of best practices for continuous deployment.
• A path towards a developer-centric experience for managing applications; it can be used for Op teams and Dev teams. 

Every piece of software pushed to the Git repository triggers a series of automated events, leading to tested, vetted, and approved code deployed into production. Git provides an immutable audit trail, providing visibility on who did what and when to your cluster to ensure compliance and stability.

Principles of GitOps

Git serves as the single source of truth for both infrastructure and applications. GitOps encompasses several guiding principles: 

1- The Entire System Described Declaratively

Like Kubernetes, GitOps is declarative. This means that configuration is guaranteed by a set of facts instead of by a set of instructions. Git is used to “declare’ the desired state for your application, cluster, etc, and GitOps works on maintaining that state. 

2- The Canonical Desired State Versioned in Git

When it comes to GitOps, everything - such as cluster and app configuration and app code - is versioned and controlled in Git, providing a single canonical state of truth. The state is ‘immutable’ in that any change to a new state requires a complete replacement of the previous state, not a change. Every time this happens, the action is recorded automatically (versioning). 

3- Automatically Apply Approved Changes in the System

With GitOps, there’s a software agent that acts as the bridge between a Git repository and the production environment. This agent continuously polls for any changes and automatically ‘pulls’ approved changes into the system.

4- Software Agents to Ensure Correctness and Alert on Divergence

The software agents run in the background and continuously observe the actual system to ensure the desired state is maintained. This ensures that your entire system is self-healing to mitigate the risk of human error. In this scenario, the software agent acts as the operational feedback and control loop. 

Download this Comprehensive ebook for beginners on GitOps here.

Push-Based vs. Pull-Based Deployments

One powerful feature that differentiates GitOps from its predecessor, CI/CD, is the pull-based deployment compared to the traditional push-based approach. Most of the available CI/CD tools use a push-based model where developers push changes to a production system, and pipelines push new deployments to a cluster. 

Push-Based Pipelines

In a push-based model, the pipeline is triggered by a code commit or merge, which then continues its path through a series of encoded scripts to push changes to the Kubernetes cluster. This deployment is implemented by popular CI/CD tools such as Jenkins, Circle CI, or Travis CI. In this approach, the pipeline system has access to the cluster. 

Push-based deployments offer many advantages, including simplicity, ease of use, accessibility, and, most importantly, efficiency. Developers can get started quickly and have a broader pool of collaboration, given the widespread adoption of the push-based model. Onboarding and training new engineers will be easier as existing team members will focus on solving problems instead of understanding multiple technologies.

There are some security concerns with this approach, though. Here, pipelines push deployments directly to the cluster, and thus would need authorization to modify the cluster structure and access all its resources. Pipeline agents handling the credentials are not always secure due to the use of unverified third-party plugins, rare updates, and lack of proper security training.

Pull-Based Pipelines

GitOps uses the pull-based approach where a software agent acts as a bridge between the Git repository and the production environment. This agent sits in your Kubernetes clusters and continually polls the desired state in the environment repo with the actual state in the deployed infrastructure. When it detects a mismatch between the defined state and the running state, the agent automatically ‘pulls’ the defining configuration and merges it into the production environment. 

 The software agent sits inside the Kubernetes cluster and does two things:

1.  It pulls Kubernetes manifests from a git repo and applies them to the cluster.

2.  It monitors your container registry for newer images and updates the Kubernetes resources accordingly.

Using the pull-based method leads to better security “out of the box”. The software agent (Flux) that lives inside your Kubernetes cluster can: 

• ONLY carry out operations permitted by Kubernetes role based access control (RBAC), policy and security. Trust is shared with the cluster and not managed separately.
• Bind natively to all Kubernetes objects and know whether operations have completed or need to be retried.

In contrast, the typical push approach, an actor lives outside the cluster and is responsible for deployment orchestration, by executing commands that loads images. If you’re interested in learning more about the security of the CICD pipeline, take a look at the deeper post by Weaveworks “How Secure is your CICD pipeline?” 

This process is secure as no external client has credentials with admit access to the clusters. The state of the cluster is version-controlled, auditable, and protected from drift. 

To read more about why it’s important to use a pull-based approach, check out this article here. 

Getting Started with GitOps

If you’d like to get started with GitOps right away, explore Weave GitOps Core, a free and open source continuous delivery tool to run apps in any Kubernetes cluster. If you need more support from GitOps pioneers, check out Weave GitOps Enterprise. This continuous operations product can help you easily deploy and manage Kubernetes clusters and applications at scale.

To know more about GitOps, check out Weaveworks’ GitOps blogs and Kubernetes CICD Guide.