GitOps is nothing new. Or, to be more precise, the principles of GitOps existed long before the term was invented. But hey, that’s the pattern in our industry. It is the fate of all good practices to be misunderstood, so we need to come up with new names to get people back on track. That is not to say that we are in a constant loop. Instead, I tend to think of it as a periodic reset trying to eliminate misinterpretations. GitOps is one of those resets. It fosters the practices and the ideas that existed for a while now and builds on top of them.
The video below is a clip from the "Canary Deployments To Kubernetes Using Istio and Friends" course in Udemy. It provides the introduction to the course we released in December 2019. Additional preview clips are available inside the course. Please use the coupons with discounts provided below.
If you do enrol into the course, please let us know what you think and do NOT forget to rate it.
$9.99 coupon available until December 30, 2019: https://www.udemy.com/course/canary-deployments-to-kubernetes-using-istio-and-friends/?couponCode=66C518BC01D2339CABCB
$13.99 coupon available until January 27, 2020: https://www.udemy.com/course/canary-deployments-to-kubernetes-using-istio-and-friends/?couponCode=7F311AD2C040117054AB
Coupon with whatever is Udemy’s best price: https://www.udemy.com/course/canary-deployments-to-kubernetes-using-istio-and-friends/?referralCode=75549ECDBC41B27D94C4
Welcome to practical guide to canary deployments. Unlike some other work that I did…tutorials, workshops, and so on and so forth, that were very focused on a single tool, this time I will focus more on a process.
We’re going to try to figure out how to do canary deployments inside of Kubernetes. Because Kubernetes is everywhere now, I will assume that you are using Kubernetes. But outside of that, we are going to try to figure out which tools to use, but all serving as the process itself, not for the sake of learning a specific tool. And during that process, we are going to decide which tools to use and why to use them and the end result will be a fully operational canary deployments process that you will be able to plug into any CI/CD tool or any tool that orchestrates the lifecycle of your application.
So we will definitely choose some tools that we will use in a process.
And those tools will be revolving around Istio. I will explain why Istio a bit later.
So we will use Kubernetes and Istio for canary deployments, but the end result will be agnostic to the tool that will orchestrate your processes. We will most likely also have to choose one or two additional tools. Which tools we’ll choose is yet to be discovered.
For now just think of this as being a practical guide to a specific process. And that process today is canary deployments in Kubernetes.
What’s wrong with
jx create cluster and
jx install commands? Why do we need a different way to install, manage, and upgrade Jenkins X? Those are ad-hoc commands that do not follow GitOpts principles. They are not idempotent (you cannot run them multiple times and expect the same result). They are not stored in Git, at least not in a form that the system can interpret and consume in an attempt to converge the desired into the actual state. They are not declarative.
Serverless deployments are gaining traction. Today, we have quite a few choices for converting our applications into serverless inside Kubernetes cluster. One of those, my favorite, is Knative. We’ll explore how we can combine it with Jenkins X to create a fully automated continuous deployment pipeline that deploys serverless applications.
Jenkins X itself is serverless. That helps with many things, with better resource utilization and scalability being only a few of the benefits. Can we do something similar with our applications? Can we scale them to zero when no one is using them? Can we scale them up when the number of concurrent requests increases? Can we make our applications serverless?
If you’d like to follow the examples, I will assume that you already have a cluster with serverless (Tekton-based) Jenkins X up-and-running.
Before we start exploring how to override different components in serverless Jenkins X pipelines, we’ll create a new quickstart project so that we have a sample application to play with.
jx create quickstart \ --language go \ --project-name jx-go-loops \ --batch-mode
Hopefully, this is not the first time you created a quick start project, and you are already familiar with the out-of-the-box pipeline our new application inherited from a build pack. Also, I will assume that you do understand that
buildPack: goinstruction in
jenkins-x.ymlmeans that the pipeline inherits all the steps defined in the corresponding build pack.
Our pipeline is currently building a Linux binary of our application before adding it to a container image. But what if we’d like to distribute the application also as executables for different operating systems? We could provide that same binary, but that would work only for Linux users since that is the architecture it is currently built for. We might want to extend the reach to Windows and MacOS users as well, and that would mean that we’d need to build two additional binaries. How could we do that?
Applying GitOps Principles
Git is the de-facto code repository standard. Hardly anyone argues against that statement today. Where we might disagree is whether Git is the only source of truth, or even what we consider by that.
When I speak with teams and ask them whether Git is their only source of truth, almost everyone always answers yes. However, when I start digging, it usually turns out that’s not true. Can you recreate everything using only the code in Git? By everything, I mean the whole cluster and everything running in it. Is your entire production system described in a single repository? If the answer to that question is yes, you are doing a great job, but we’re not yet done with questioning. Can any change to your system be applied by making a pull request, without pressing any buttons in Jenkins or any other tool? If your answer is still yes, you are most likely already applying GitOps principles.
GitOps is a way to do Continuous Delivery. It assumes that Git is a single source of truth and that both infrastructure and applications are defined using the declarative syntax (e.g., YAML). Changes to infrastructure or applications are made by pushing changes to Git, not by clicking buttons in Jenkins.
Without TLS certificates the applications we install are accessible through a plain HTTP protocol. As I’m sure you’re aware, that is not acceptable. All public-facing applications should be available through HTTPS only, and that means that we need TLS certificates. We could generate them ourselves for each of the applications, but that would be too much work. Instead, we’ll try to figure out how to create and manage the certificates automatically. Fortunately, Jenkins X already solved that and quite a few other Ingress-related challenges. We just need to learn how to tell
jx what exactly we need.
When Jenkins appeared, its pipelines were called FreeStyle jobs. There was no way to describe them in code, and they were not kept in version control. We were creating and maintaining those jobs through Jenkins UI by filling input fields, marking checkboxes, and selecting values from drop-down lists. The results were impossible-to-read XML files stored in the Jenkins home directory. Nevertheless, that approach was so great (compared to what existed at the time) that Jenkins become widely adopted overnight. But, that was many years ago and what was great over a decade ago is not necessarily as good today. As a matter of fact, FreeStyle jobs are the antithesis of the types of jobs we should be writing today. Tools that create code through drag-and-drop methods are extinct. Not having code in version control is a cardinal sin. Not being able to use our favorite IDE or code editor is unacceptable. Hence, the Jenkins community created Jenkins pipelines.