This article is an excerpt from The DevOps 2.4 Toolkit: Continuous Deployment To Kubernetes. It assumes that you already have a Kubernetes cluster with nginx Ingress. The article was tested with minikube, minishift, Docker for Mac/Windows, AWS with kops, and GKE. Furthermore, I will assume that you already installed Helm. Finally, I expect you to clone vfarcic/k8s-specs and execute the commands from inside it.

First things first… We need to find out the IP of our cluster or external LB if available. The commands that follow will differ from one cluster type to another.

Feel free to skip the sections that follow if you already know how to get the IP of your cluster’s entry point.

If your cluster is running in AWS and was created with kops, we’ll need to retrieve the hostname from the Ingress Service, and extract the IP from it. Please execute the commands that follow.

LB_HOST=$(kubectl -n kube-ingress \
    get svc ingress-nginx \
    -o jsonpath="{.status.loadBalancer.ingress[0].hostname}")

LB_IP="$(dig +short $LB_HOST \
    | tail -n 1)"

If your cluster is running in Docker For Mac/Windows, the IP is 127.0.0.1 and all you have to do is assign it to the environment variable LB_IP. Please execute the command that follows.

LB_IP="127.0.0.1"

If your cluster is running in minikube, the IP can be retrieved using minikube ip command. Please execute the command that follows.

LB_IP="$(minikube ip)"

If your cluster is running in GKE, the IP can be retrieved from the Ingress Service. Please execute the command that follows.

LB_IP=$(kubectl -n ingress-nginx \
    get svc ingress-nginx \
    -o jsonpath="{.status.loadBalancer.ingress[0].ip}")

Next, we’ll output the retrieved IP to confirm that the commands worked, and generate a sub-domain jenkins.

echo $LB_IP

HOST="jenkins.$LB_IP.nip.io"

echo $HOST

The output of the second echo command should be similar to the one that follows.

jenkins.192.168.99.100.nip.io

nip.io will resolve that address to 192.168.99.100, and we’ll have a unique domain for our Jenkins installation. That way we can stop using different paths to distinguish applications in Ingress config. Domains work much better. Many Helm charts do not even have the option to configure unique request paths and assume that Ingress will be configured with a unique domain.

A note to minishift users

I did not forget about you. You already have a valid domain in the ADDR variable. All we have to do is assign it to the HOST variable. Please execute the commands that follow.

HOST=$ADDR

echo $HOST

The output should be similar to jenkins.192.168.99.100.nip.io.

Now that we have a valid jenkins.* domain, we can try to figure out how to apply all the changes we discussed.

We already learned that we can inspect all the available values using helm inspect command. Let’s take another look.

helm inspect values stable/jenkins

The output, limited to the relevant parts, is as follows.

Master:
  Name: jenkins-master
  Image: "jenkins/jenkins"
  ImageTag: "lts"
  ...
  Cpu: "200m"
  Memory: "256Mi"
  ...
  ServiceType: LoadBalancer
  # Master Service annotations
  ServiceAnnotations: {}
  ...
  # HostName: jenkins.cluster.local
  ...
  InstallPlugins:
    - kubernetes:1.1
    - workflow-aggregator:2.5
    - workflow-job:2.15
    - credentials-binding:1.13
    - git:3.6.4
  ...
  Ingress:
    ApiVersion: extensions/v1beta1
    Annotations:
    ...
...
rbac:
  install: false
  ...

Everything we need to accomplish our new requirements is available through the values. Some of them are already filled with defaults, while others are commented. When we look at all those values, it becomes clear that it would be unpractical to try to re-define them all through --set arguments. We’ll use --values instead. It will allow us to specify the values in a file.

I already prepared a YAML file with the values that will fulfill our requirements, so let’s take a quick look at them.

cat helm/jenkins-values.yml

The output is as follows.

Master:
  ImageTag: "2.116-alpine"
  Cpu: "500m"
  Memory: "500Mi"
  ServiceType: ClusterIP
  ServiceAnnotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
  InstallPlugins:
    - blueocean:1.5.0
    - credentials:2.1.16
    - ec2:1.39
    - git:3.8.0
    - git-client:2.7.1
    - github:1.29.0
    - kubernetes:1.5.2
    - pipeline-utility-steps:2.0.2
    - script-security:1.43
    - slack:2.3
    - thinBackup:1.9
    - workflow-aggregator:2.5
  Ingress:
    Annotations:
      nginx.ingress.kubernetes.io/ssl-redirect: "false"
      nginx.ingress.kubernetes.io/proxy-body-size: 50m
      nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
      ingress.kubernetes.io/ssl-redirect: "false"
      ingress.kubernetes.io/proxy-body-size: 50m
      ingress.kubernetes.io/proxy-request-buffering: "off"
  HostName: jenkins.acme.com
rbac:
  install: true

As you can see, the variables in that file follow the same format as those we output through the helm inspect values command. The only difference is in values, and the fact that helm/jenkins-values.yml contains only those that we are planning to change.

We defined that the ImageTag should be fixed to 2.116-alpine.

We specified that our Jenkins master will need half a CPU and 500 MB RAM. The default values of 0.2 CPU and 256 MB RAM are probably not enough. What we set is also low, but since we’re not going to run any serious load (at least not yet), what we re-defined should be enough.

The service was changed to ClusterIP to better accommodate Ingress resource we’re defining further down.

If you are not using AWS, you can ignore ServiceAnnotations. They’re telling ELB to use HTTP protocol.

Further down, we are defining the plugins we’ll use throughout the book. Their usefulness will become evident in the next chapters.

The values in the Ingress section are defining the annotations that tell Ingress not to redirect HTTP requests to HTTPS (we don’t have SSL certificates), as well as a few other less important options. We set both the old style (ingress.kubernetes.io) and the new style (nginx.ingress.kubernetes.io) of defining NGINX Ingress. That way it’ll work no matter which Ingress version you’re using. The HostName is set to a value that apparently does not exist. I could not know in advance what will be your hostname, so we’ll overwrite it later on.

Finally, we set rbac.install to true so that the Chart knows that it should set the proper permissions.

Having all those variables defined at once might be a bit overwhelming. You might want to go through the Jenkins Chart documentation for more info. In some cases, documentation alone is not enough, and I often end up going through the files that form the chart. You’ll get a grip on them with time. For now, the important thing to observe is that we can re-define any number of variables through a YAML file.

Let’s install the Chart with those variables.

helm install stable/jenkins \
    --name jenkins \
    --namespace jenkins \
    --values helm/jenkins-values.yml \
    --set Master.HostName=$HOST

We used the --values argument to pass the contents of the helm/jenkins-values.yml. Since we had to overwrite the HostName, we used --set. If the same value is defined through --values and --set, the latter always takes precedence.

A note to minishift users

The values define Ingress which does not exist in your cluster. If we’d create a set of values specific to OpenShift, we would not define Ingress. However, since those values are supposed to work in any Kubernetes cluster, we left them intact. Given that Ingress controller does not exist, Ingress resources will have no effect, so it’s safe to leave those values.

Next, we’ll wait for jenkins Deployment to roll out and open its UI in a browser.

kubectl -n jenkins \
    rollout status deployment jenkins

open "http://$HOST"

The fact that we opened Jenkins through a domain defined as Ingress (or Route in case of OpenShift) tells us that the values were indeed used. We can double check those currently defined for the installed Chart with the command that follows.

helm get values jenkins

The output is as follows.

Master:
  Cpu: 500m
  HostName: jenkins.18.220.212.56.nip.io
  ImageTag: 2.116-alpine
  Ingress:
    Annotations:
      ingress.kubernetes.io/proxy-body-size: 50m
      ingress.kubernetes.io/proxy-request-buffering: "off"
      ingress.kubernetes.io/ssl-redirect: "false"
      nginx.ingress.kubernetes.io/proxy-body-size: 50m
      nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
      nginx.ingress.kubernetes.io/ssl-redirect: "false"
  InstallPlugins:
  - blueocean:1.5.0
  - credentials:2.1.16
  - ec2:1.39
  - git:3.8.0
  - git-client:2.7.1
  - github:1.29.0
  - kubernetes:1.5.2
  - pipeline-utility-steps:2.0.2
  - script-security:1.43
  - slack:2.3
  - thinBackup:1.9
  - workflow-aggregator:2.5
  Memory: 500Mi
  ServiceAnnotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
  ServiceType: ClusterIP
rbac:
  install: true

Even though the order is slightly different, we can easily confirm that the values are the same as those we defined in helm/jenkins-values.yml. The exception is the HostName which was overwritten through the --set argument.

The DevOps 2.4 Toolkit: Continuous Deployment To Kubernetes

The article you just read is an extract from The DevOps 2.4 Toolkit: Continuous Deployment To Kubernetes.

This book explores continuous deployment to a Kubernetes cluster. It uses a wide range of Kubernetes platforms and provides instructions on how to develop a pipeline on few of the most commonly used CI/CD tools.

I am assuming that you are already proficient with Deployments, ReplicaSets, Pods, Ingress, Services, PersistentVolumes, PersistentVolumeClaims, Namespaces and a few other things. This book assumes that we do not need to go through the basic stuff. At least, not through all of it. The book assumes a certain level of Kubernetes knowledge and hands-on experience. If that’s not the case, what follows might be too confusing and advanced. Please read The DevOps 2.3 Toolkit: Kubernetes first, or consult the Kubernetes documentation. Come back once you’re done and once you think you can claim that you understand at least basic Kubernetes concepts and resource types.

Give it a try and let me know what you think.