A while ago, we (Viktor Farcic and Darin Pope) thought it would be a good idea to add an out-of-the-box option to use canary deployments in Jenkins X. We should have finished it by now, and yet we did not even start working on it. Instead of just adding it to Jenkins X, we spent considerable time exploring the subject.
Applying GitOps Principles
Git is the de-facto code repository standard. Hardly anyone argues against that statement today. Where we might disagree is whether Git is the only source of truth, or even what we consider by that.
When I speak with teams and ask them whether Git is their only source of truth, almost everyone always answers yes. However, when I start digging, it usually turns out that’s not true. Can you recreate everything using only the code in Git? By everything, I mean the whole cluster and everything running in it. Is your entire production system described in a single repository? If the answer to that question is yes, you are doing a great job, but we’re not yet done with questioning. Can any change to your system be applied by making a pull request, without pressing any buttons in Jenkins or any other tool? If your answer is still yes, you are most likely already applying GitOps principles.
GitOps is a way to do Continuous Delivery. It assumes that Git is a single source of truth and that both infrastructure and applications are defined using the declarative syntax (e.g., YAML). Changes to infrastructure or applications are made by pushing changes to Git, not by clicking buttons in Jenkins.
The serverless flavor of Jenkins X or, as some call it, Jenkins X Next Generation, is an attempt to redefine how we do continuous delivery and GitOps inside Kubernetes clusters. It does that by combining quite a few tools into a single easy-to-use bundle. As a result, most people will not have a need to understand intricacies of how the pieces work independently, nor how they are all integrated. Instead, many will merely push a change to Git and let the system do the rest. But, there are always those who would like to know what’s happening behind the hood. To satisfy those craving for insight, we’ll explore the processes and the components involved in the serverless Jenkins X platform. Understanding the flow of an event initiated by a Git webhook will give us insight into how the solution works and help us later on when we go deeper into each of the new components.
I stand by my claim that “you do not need to understand Kubernetes to use Jenkins X.” To be more precise, those who do not want to know Kubernetes and its ecosystem in detail can benefit from Jenkins X ability to simplify the processes around software development lifecycle. That’s the promise or, at least, one of the driving ideas behind the project. Nevertheless, for that goal to reach as wide of an audience as possible, we need a variety of build packs. The more we have, the more use cases can be covered with a single
jx import or
jx quickstart command. The problem is that there is an infinite number of types of applications and combinations we might have. Not all can be covered with community-based packs. No matter how much effort the community puts into creating build packs, they will always be a fraction of what we might need. That’s where you come in.
To understand intricacies and inner workings of Jenkins X, we need to understand Kubernetes. But, you do not need to understand Kubernetes to use Jenkins X. That is one of the main contributions of the project. Jenkins X allows us to harness the power of Kubernetes without spending eternity learning the ever-growing list of the things it does. Jenkins X helps us by simplifying complex processes into concepts that can be adopted quickly and without spending months in trying to figure out “the right way to do stuff.” It helps by removing and simplifying some of the problems caused by the overall complexity of Kubernetes and its ecosystem. If you are indeed a Kubernetes ninja, you will appreciate all the effort put into Jenkins X. If you’re not, you will be able to jump right in and harness the power of Kubernetes without ripping your hair out of frustration caused by Kubernetes complexity.
Let’s paint a high-level picture of the continuous delivery pipeline. To be more precise, we’ll draw a diagram instead of painting anything. But, before we dive into a continuous delivery diagram, we’ll refresh our memory with the one we used before for describing continuous deployment.
The continuous deployment pipeline contains all the steps from pushing a commit to deploying and testing a release in production.
Continuous delivery removes one of the stages from the continuous deployment pipeline. We do NOT want to deploy a new release automatically. Instead, we want humans to decide whether a release should be upgraded in production. If it should, we need to decide when will that happen. Those (human) decisions are, in our case, happening as Git operations. We’ll comment on them soon. For now, the important note is that the deploy stage is now removed from pipelines residing in application repositories.
The difference between continuous integration, delivery, and deployment is not in processes, but in the level of confidence we have in them.
The continuous deployment process is relatively easy to explain, even though implementation might get tricky. We’ll split our requirements into two groups. We’ll start with a discussion about the overall goals that should be applied to the whole process. To be more precise, we’ll talk about what I consider non-negotiable requirements.
Soon after I started working on The DevOps 2.3 Toolkit: Kubernetes, I realized that a single book could only scratch the surface. Kubernetes is vast, and no single book can envelop even all the core components. If we add community projects, the scope becomes even more extensive. Then we need to include hosting vendors and different ways to set up and manage Kubernetes. That would inevitably lead us to third-party solutions like OpenShift, Rancher, and DockerEE, to name a few. It doesn’t end there. We’d need to explore other types of community and third-party additions like those related to networking and storage. And don’t forget the processes like, for example, continuous delivery and deployment. All those things could not be explored in a single book so The DevOps 2.3 Toolkit: Kubernetes ended up being an introduction to Kubernetes. It can serve as the base for exploring everything else.
The moment I published the last chapter of The DevOps 2.3 Toolkit: Kubernetes, I started working on the next material. A lot of ideas and tryouts came out of it. It took me a while until the subject and the form of the forthcoming book materialized. After a lot of consultation with the readers of the previous book, the decision was made to explore continuous delivery and deployment processes in a Kubernetes cluster. The high-level scope of the book you are reading right now was born.
Everyone wants to implement continuous delivery. After all, the benefits are too big to be ignored. Increase the speed of delivery, increase the quality, decrease the costs, free people to dedicate time on what brings value, and so on and so forth. Those improvements are like music to any decision maker. Especially if that person has a business background. If a tech geek can articulate the benefits continuous delivery brings to the table, when he asks a business representative for a budget, the response is almost always “Yes! Do it.”
A continuous delivery project will start. Tests will be written. Builds will be scripted. Deployments will be automated. Everything will be tied into an automated pipeline and triggered on every commit. Everyone will enter a state of nirvana as soon as all that is done. There will be a huge inauguration party with a vice president having the honor to be the first one to press the button that will deploy the first release to production. Isn’t that a glorious plan everyone should be proud of?