A Guide On The Installation Of Spinnaker in Kubernetes Clusters

Spinnaker is an open-source, multi-cloud deployment tool for releasing software changes reliably. You can deploy and automate your application release across multiple cloud environments including Kubernetes, AWS EC2, Google Computer Engine, Google App Engine, Microsoft Azure and many more.

In this post, we will focus on the installation of Spinnaker in a kubernetes cluster using Halyard. Halyard is a command-line administration tool that manages the lifecycle of your Spinnaker deployment, and it's a recommended way to install, configure and update Spinnaker.

At the end of this “How-to” guide, you will have a production-grade Spinnaker which can be used to create your deployment pipelines for one or more Kubernetes clusters.

We will use Google Cloud for demonstration, the process would be the same for other cloud providers with slight modifications in commands and resources.

Prerequisites

One Kubernetes cluster Spinnaker would be set-up in this cluster using Google Kubernetes Engine for the demonstration.
One Virtual Machine (1 vCPU, 3.75 GB, Ubuntu 18.04) for installation of Halyard.
One storage bucket for persistent storage of Spinnaker.
One service account json key for access storage bucket.
An OAuth client ID and access token for Login authentication.

We will create all these resources in subsequent steps. Kindly replace your-project with your actual project in required commands.

Steps For The Installation And Execution

Step 1: Create A Kubernetes Cluster

You can skip this step if you already have one. We will use a “one node” single-zone cluster for demonstration:

gcloud beta container clusters create "kubernetes-cluster-01" \
	--zone "asia-southeast1-a" --machine-type "n1-standard-1" \
	--num-nodes "1" --no-enable-stackdriver-kubernetes \
	--project "your-project"

Step 2: Create One Machine To Install Halyard.

gcloud compute instances create "spinnaker-halyard" \
	--async \
	--boot-disk-size "10GB" \
	--image-family "ubuntu-1804-lts" \
	--image-project "ubuntu-os-cloud" \
	--machine-type "n1-standard-1" \
	--zone "asia-southeast1-a" \
	--project "your-project"

Step 3: Log in To The Machine

gcloud beta compute ssh --zone "asia-southeast1-a" "spinnaker-halyard" --project "your-project"

Step 4: Install Kubectl And Halyard

Install kubectl:

sudo -i
sudo apt-get update && sudo apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl

Install Halyard:

Add spinnaker user and switch to the spinnaker user, then install Halyard:

adduser spinnaker
echo "spinnaker ALL=(ALL:ALL) NOPASSWD:ALL" > spinnaker && chmod 440 spinnaker && mv spinnaker /etc/sudoers.d/
su - spinnaker
curl -O https://raw.githubusercontent.com/spinnaker/halyard/master/install/debian/InstallHalyard.sh
sudo bash InstallHalyard.sh --user spinnaker  (Press Enter and keep default option when it will prompt)
. /home/spinnaker/.bashrc
hal -v

spinnaker@spinnaker-halyard:~$ hal -v
1.31.1-20200207105616

We have successfully installed the Halyard after completion of this step.

Step 5: Create An IAM Service Account And It’s Key

Spinnaker will use this service account to access persistent GCS storage:

gcloud iam service-accounts create spinnaker \
--display-name "spinnaker" \
--project your-project
gcloud iam service-accounts keys create ~/.gcp/key.json \
--iam-account spinnaker@your-project.iam.gserviceaccount.com \
--project "your-project"

Step 6: Create A Storage Bucket

If you want to persist data generated by Spinnaker, you will need to create a storage bucket with the proper access rights. You will need to give the right permissions on this bucket to the Spinnaker service account created in Step 5. Feel free to change BUCKET_NAME as storage buckets are Global resources and must be unique:

BUCKET_NAME=spin-hal-bucket
gsutil mb -l asia-southeast1 -p your-project gs://$BUCKET_NAME/
gsutil iam ch serviceAccount:spinnaker@your-project.iam.gserviceaccount.com:legacyBucketWriter gs://$BUCKET_NAME

Step 7: Connect To Your Kubernetes Cluster On Your Halyard Machine

gcloud beta container clusters get-credentials "kubernetes-cluster-01" --zone "asia-southeast1-a" --project "your-project"

Step 8: Create Kubernetes Service Account

Create spinnaker-service-account and assign ClusterAdmin role to this service account.

CONTEXT=$(kubectl config current-context)
kubectl apply --context $CONTEXT -f https://spinnaker.io/downloads/kubernetes/service-account.yml
TOKEN=$(kubectl get secret --context $CONTEXT $(kubectl get serviceaccount spinnaker-service-account --context $CONTEXT -n spinnaker -o jsonpath='{.secrets[0].name}') -n spinnaker -o jsonpath='{.data.token}' | base64 --decode)
kubectl config set-credentials ${CONTEXT}-token-user --token $TOKEN
kubectl config set-context $CONTEXT --user ${CONTEXT}-token-user

Step 9: Add The Kubernetes Cluster Into Halyard Config

hal config provider kubernetes enable
hal config provider kubernetes account add kubernetes-cluster-01 --provider-version v2 --context $(kubectl config current-context)
hal config features edit --artifacts true

Repeat steps 7 to 9 for each of the clusters in-which you want to manage using Spinnaker or want to install Spinnaker in.

Step 10: Choose The Kubernetes Cluster in Which You Want To Install Spinnaker.

ACCOUNT=kubernetes-cluster-01
hal config deploy edit --type distributed --account-name $ACCOUNT

Step 11: Add Storage Bucket in Halyard Config.

hal config storage gcs edit --bucket $BUCKET_NAME --project your-project --json-path ~/.gcp/key.json
hal config storage edit --type gcs

Step 12: Check Available Spinnaker Versions And Deploy

Now List the available spinnaker versions. Note that this will list the latest available version of Spinnaker and not Halyard:

hal version list

spinnaker@spinnaker-halyard:~$ hal version list
+ Get current deployment
  Success
+ Get Spinnaker version
  Success
+ Get released versions
  Success
+ You are on version "", and the following are available:
 - 1.16.7 (SecretObsession):
   Changelog: https://gist.github.com/spinnaker-release/170d178708b56e83b0289452cb83f347
   Published: Mon Mar 09 19:34:31 UTC 2020
   (Requires Halyard >= 1.17)
 - 1.17.7 (LlamaLlama):
   Changelog: https://gist.github.com/spinnaker-release/eea8c2434c8dcf77de8506ffec705246
   Published: Mon Mar 09 19:37:42 UTC 2020
   (Requires Halyard >= 1.17)
 - 1.18.5 (Longmire):
   Changelog: https://gist.github.com/spinnaker-release/2cfb72a46883b82a657b4e4b8cba6f62
   Published: Mon Mar 09 19:44:00 UTC 2020
   (Requires Halyard >= 1.29)
 - 1.19.0 (Gilmore Girls A Year in the Life):
   Changelog: https://gist.github.com/spinnaker-release/dbc44ac411d5076002b5db7c64b8c63e
   Published: Wed Mar 11 20:22:58 UTC 2020
   (Requires Halyard >= 1.32)

Set VERSION with one of the listed versions:

VERSION=1.16.7
hal config version edit --version $VERSION

Deploy Spinnaker:

hal deploy apply

After this step, Spinnaker would have been deployed in the spinnaker namespace of kubernetes cluster. Note that before hal deploy apply, each configuration was added in hal configuration only and would be applicable on running spinnaker after hal deploy apply command.

We can check all Spinnaker microservices running in spinnaker namespace:

kubectl get all -n spinnaker

spinnaker@spinnaker-halyard:~$ kubectl get all -n spinnaker
NAME                                    READY   STATUS    RESTARTS   AGE
pod/spin-clouddriver-78f5977c8b-6wll2   1/1     Running   0          45m
pod/spin-deck-7d86cd5c56-bnjlt          1/1     Running   0          45m
pod/spin-echo-7d9dcb8894-ggln4          1/1     Running   0          45m
pod/spin-front50-78996dfbf4-g5g2m       1/1     Running   0          45m
pod/spin-gate-5dfd759658-rwxr5          1/1     Running   0          45m
pod/spin-orca-76d64868db-lb7cp          1/1     Running   0          45m
pod/spin-redis-c9b94cd9c-5lz5z          1/1     Running   0          45m
pod/spin-rosco-64544867b8-vzvz8         1/1     Running   0          45m

NAME                       TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/spin-clouddriver   ClusterIP      10.43.254.196           7002/TCP       10d
service/spin-deck          ClusterIP      10.43.240.189           9000/TCP       10d
service/spin-echo          ClusterIP      10.43.252.206           8089/TCP       10d
service/spin-front50       ClusterIP      10.43.250.115           8080/TCP       10d
service/spin-gate          ClusterIP      10.43.248.203           8084/TCP       10d
service/spin-orca          ClusterIP      10.43.247.203           8083/TCP       10d
service/spin-redis         ClusterIP      10.43.242.44            6379/TCP       10d
service/spin-rosco         ClusterIP      10.43.253.57            8087/TCP       10d

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/spin-clouddriver   1/1     1            1           10d
deployment.apps/spin-deck          1/1     1            1           10d
deployment.apps/spin-echo          1/1     1            1           10d
deployment.apps/spin-front50       1/1     1            1           10d
deployment.apps/spin-gate          1/1     1            1           10d
deployment.apps/spin-orca          1/1     1            1           10d
deployment.apps/spin-redis         1/1     1            1           10d
deployment.apps/spin-rosco         1/1     1            1           10d

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/spin-clouddriver-78f5977c8b   1         1         1       10d
replicaset.apps/spin-deck-7d86cd5c56          1         1         1       10d
replicaset.apps/spin-echo-7d9dcb8894          1         1         1       10d
replicaset.apps/spin-front50-78996dfbf4       1         1         1       10d
replicaset.apps/spin-gate-5dfd759658          1         1         1       10d
replicaset.apps/spin-orca-76d64868db          1         1         1       10d
replicaset.apps/spin-redis-c9b94cd9c          1         1         1       10d
replicaset.apps/spin-rosco-64544867b8         1         1         1       10d

Info About Spinnaker Microservices:

Clouddriver: Responsible for integration of cloud providers like AWS, GCP, Azure etc.
Deck: Management UI for Spinnaker.
Gate: Spinnaker API Gateway.
Echo: Spinnaker Eventing Service (sending notifications, acts on incoming webhooks etc).
Front50: Spinnaker Metadata Repository Service to persist the metadata of pipelines, projects etc.
Orca: Orchestration engine for Spinnaker.
Rosco: Produces machine images for Spinnaker.

Step 13: Expose Spinnaker UI Outside Kubernetes Cluster

Deck is the browser-based UI for spinnaker management. We need to expose the deck outside the Kubernetes cluster to access this from our browser. We would achieve this by editing spin-deck service and making it a type of Internal load balancer.

kubectl edit svc spin-deck -n spinnaker

Add the following parameter in the spin-deck yaml file according to kubernetes convention.

In Metadata:

annotations:
  cloud.google.com/load-balancer-type: Internal

In Spec:

- port: 80
type: LoadBalancer

Final yaml For Spin-deck After Changes:

apiVersion: v1
kind: Service
metadata:
  annotations:
    cloud.google.com/load-balancer-type: Internal
  labels:
    app: spin
    cluster: spin-deck
  name: spin-deck
  namespace: spinnaker
spec:
  clusterIP: 10.43.240.189
  externalTrafficPolicy: Cluster
  ports:
  - port: 80
    protocol: TCP
    targetPort: 9000
  selector:
    app: spin
    cluster: spin-deck
  sessionAffinity: None
  type: LoadBalancer

Step 14: Expose Spinnaker Gate

Gate is the API gateway for spinnaker. All api callers communicate with Spinnaker via Gate.

kubectl edit svc spin-gate -n spinnaker

Similar to Step 13 add following in spin-gate yaml

In Metadata:

annotations:
  cloud.google.com/load-balancer-type: Internal

In Spec:

- port: 80
type: LoadBalancer

Final yaml For Spin-gate After Changes:

apiVersion: v1
kind: Service
metadata:
  annotations:
    cloud.google.com/load-balancer-type: Internal
  labels:
    app: spin
    cluster: spin-gate
  name: spin-gate
  namespace: spinnaker
spec:
  clusterIP: 10.43.248.203
  externalTrafficPolicy: Cluster
  ports:
  - port: 80
    protocol: TCP
    targetPort: 8084
  selector:
    app: spin
    cluster: spin-gate
  sessionAffinity: None
  type: LoadBalancer

We can check if both deck and gate have the same type of Internal Load balancer service now.

kubectl get svc -n spinnaker

spinnaker@spinnaker-halyard:~$ kubectl get svc -n spinnaker
NAME                       TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/spin-clouddriver   ClusterIP      10.43.254.196           7002/TCP       10d
service/spin-deck          LoadBalancer   10.43.240.189   10.148.0.49   80:32261/TCP   10d
service/spin-echo          ClusterIP      10.43.252.206           8089/TCP       10d
service/spin-front50       ClusterIP      10.43.250.115           8080/TCP       10d
service/spin-gate          LoadBalancer   10.43.248.203   10.148.0.50   80:31320/TCP   10d
service/spin-orca          ClusterIP      10.43.247.203           8083/TCP       10d
service/spin-redis         ClusterIP      10.43.242.44            6379/TCP       10d
service/spin-rosco         ClusterIP      10.43.253.57            8087/TCP       10d

Step 15: Add Domains in Halyard Config And Deploy Spinnaker

Assign domains on both decks’ and gates’ Load balancer IPs, then add these domains in Halyard config. Let’s use the following domains for the rest of the tutorial, replace these domains with your actual domains in respective commands.

spin-deck -> spinnaker-ui.example.com
spin-gate -> spinnaker-api.example.com

Add these domains in halyard config and deploy them again:

hal config security ui edit --override-base-url http://spinnaker-ui.example.com
hal config security api edit --override-base-url http://spinnaker-api.example.com
hal deploy apply

After this step you would be able to access Spinnaker using http://spinnaker-ui.example.com domain.

Step 16: Add Login Authentication On Spinnaker

We will add Google OAuth for Login into Spinnaker and will allow Login for particular domains only. We would need Google OAuth client Id and secret for this setup. To generate them, follow the steps below:

First navigate to https://console.cloud.google.com/apis/credentials
Click on "CREATE CREDENTIALS" and then on "OAuth client ID"
Select "Web application" in Application type, and enter "Spinnaker" in the Name text box. After that under "Authorized redirect URIs", add http://spinnaker-ui.example.com/login and http://spinnaker-api.example.com/login (Replace these domains with your actual domains created in Step 15). After that click on Create and Copy Client ID and Client Secret.

Step 17: Enable OAuth in Spinnaker

CLIENT_ID=CLIENT_ID_GENERATED_IN_STEP_16
CLIENT_SECRET=CLIENT_SECRET_GENERATED_IN_STEP_16
PROVIDER=google
DOMAIN=example.com
hal config security authn oauth2 edit --client-id $CLIENT_ID --client-secret $CLIENT_SECRET --provider $PROVIDER --user-info-requirements hd=$DOMAIN
hal config security authn oauth2 enable
hal deploy apply

After this step, We have successfully set up Authentication for our Spinnaker UI. We can open our Spinnaker UI Domain at http://spinnaker-ui.example.com and can Login with our example.com Email Id.

TL;DR

Spinnaker is a multi-cloud deployment tool for releasing applications with high velocity and reliability.
We used Halyard (a command-line administration tool) to install Spinnaker in Google Kubernetes Engine.
We exposed Spinnaker UI and API Gateway within the network using Internal Load Balancer and added Login authentication.
We can add more functionality like slack/email notification, Automatic pipeline trigger from jenkins, Role based authorization etc. which we will try to cover in separate threads.

Feel free to get in touch if you encounter any challenge during the installation process, or if you need any help or have helpful suggestions. Happy Reading :)

Vikas Saini is a part of the Magalix Write For Cloud-Native Community Expert Program, and Can be Found On Social Media at Linkedin, Twitter, and Github.

Relevant Topics

DevOps Kubernetes K8s write-for-cloud-native Spinnaker

14-days FREE Trial

A Guide On The Installation Of Spinnaker in Kubernetes Clusters

14-days FREE Trial

Prerequisites

Steps For The Installation And Execution

Step 1: Create A Kubernetes Cluster

Step 2: Create One Machine To Install Halyard.

Step 3: Log in To The Machine

Step 4: Install Kubectl And Halyard

Install kubectl:

Install Halyard:

Step 5: Create An IAM Service Account And It’s Key

Step 6: Create A Storage Bucket

Step 7: Connect To Your Kubernetes Cluster On Your Halyard Machine

Step 8: Create Kubernetes Service Account

Step 9: Add The Kubernetes Cluster Into Halyard Config

Step 10: Choose The Kubernetes Cluster in Which You Want To Install Spinnaker.

Step 11: Add Storage Bucket in Halyard Config.

Step 12: Check Available Spinnaker Versions And Deploy

Deploy Spinnaker:

Info About Spinnaker Microservices:

Step 13: Expose Spinnaker UI Outside Kubernetes Cluster

In Metadata:

In Spec:

Final yaml For Spin-deck After Changes:

Step 14: Expose Spinnaker Gate

In Metadata:

In Spec:

Final yaml For Spin-gate After Changes:

Step 15: Add Domains in Halyard Config And Deploy Spinnaker

Step 16: Add Login Authentication On Spinnaker

Step 17: Enable OAuth in Spinnaker

TL;DR

Relevant Topics

14-days FREE Trial

A Guide On The Installation Of Spinnaker in Kubernetes Clusters

14-days FREE Trial

Prerequisites

Steps For The Installation And Execution

Step 1: Create A Kubernetes Cluster

Step 2: Create One Machine To Install Halyard.

Step 3: Log in To The Machine

Step 4: Install Kubectl And Halyard

Install kubectl:

Install Halyard:

Step 5: Create An IAM Service Account And It’s Key

Step 6: Create A Storage Bucket

Step 7: Connect To Your Kubernetes Cluster On Your Halyard Machine

Step 8: Create Kubernetes Service Account

Step 9: Add The Kubernetes Cluster Into Halyard Config

Step 10: Choose The Kubernetes Cluster in Which You Want To Install Spinnaker.

Step 11: Add Storage Bucket in Halyard Config.

Step 12: Check Available Spinnaker Versions And Deploy

Deploy Spinnaker:

Info About Spinnaker Microservices:

Step 13: Expose Spinnaker UI Outside Kubernetes Cluster

In Metadata:

In Spec:

Final yaml For Spin-deck After Changes:

Step 14: Expose Spinnaker Gate

In Metadata:

In Spec:

Final yaml For Spin-gate After Changes:

Step 15: Add Domains in Halyard Config And Deploy Spinnaker

Step 16: Add Login Authentication On Spinnaker

Step 17: Enable OAuth in Spinnaker

TL;DR

Relevant Topics

Related Articles