Explaining continuous deployment (CDP) is easy. Implementing it is very hard, and the challenges are often hidden and unexpected. Depending on the maturity of your processes, architecture, and code, you might find out that the real problems do not lie in the code of a continuous deployment pipeline, but everywhere else. As a matter of fact, developing a pipeline is the easiest part.

We will not discuss the changes in your other processes. We will not explore what a good architecture that will support CDP pipelines is. We will not dive into how to code your application to be pipeline-friendly. I assume that you already know all that. I hope that you do understand the basic concepts behind Agile and DevOps movements and that you already started dismantling the silos in your company. I assume that you do know what it means for your software architecture to be cloud-native and that you do implement some if not all of the 12 factors. I guess that you are already practicing Test-Driven Development, Behavior-Driven Development, Acceptance-Driven Development, or any other technique that help you design your applications.

I might be wrong. To be more precise, I’m sure that I’m wrong. Most of you are not there yet. If you are one of those, please get informed. Read more books, do some courses, and convince your managers to give you time, space, and resources you’ll need to modernize your applications. It needs to be done. All those things and many others are what differentiates top performers (e.g., Google, Amazon, Netflix) and the rest of us. Still, none of them is the same. Every high-performing company is different, and yet, they all share some things in common. They all need to ship features fast. They all need to have a high level of quality. And they all acknowledge that highly-available, fault-tolerant, and distributed systems require a very different approach than what most of the rest of us are used to.

We already discussed in this blog and the books I published what a continuous deployment pipeline looks like. In case you’re forgetful (I know I am), here are a few of the rules that represent the short version.

Rule number one: Every commit to the master branch is deployed to production if it passes all the steps of a fully automated pipeline. If you need to involve humans after the commit, it’s not continuous deployment, nor it is continuous delivery. At best, you’re doing continuous integration.

Rule number two: You commit directly to the master branch, or you’re using short-living feature branches. The master branch is the only one that matters. Production releases are made from it. If you do use branches, they are taken from the master branch, since that’s the only one that truly matters. When you do create a feature branch, you are merging back to master soon afterward. You’re not waiting for weeks to do so. If you are, you are not “continuously” validating whether your code integrates with the code of others. If that’s the case, you’re not even doing continuous integration. Unless, you have an elaborate branching strategy, in which case you are only making everyone’s lives more complicated than they should be.

Rule number three: You trust your automation. When a test fails, there is a bug, and you fix it before anything else. I hope that you do not belong to a big group of companies that have flaky tests that sometimes work, and sometimes fail for random reasons. If you do, fix your tests first or remove those that are flaky. It’s pointless to run tests you do not trust. The same can be said for builds, deployments, and just about any other step of the process. If you see yourself in the group of those that do not trust their code, you’ll have to fix it first. Tests are code, just as builds, deployments, and everything else is. When code produces inconsistent results, we fix it, we do not restart it. Unfortunately, I do see a lot of companies that rather re-run a build that failed because of flaky tests than fix the cause of that flakiness. There’s an alarming number of those that solve half of the production problems by restarting applications. Anyways, if you do not trust your automation, you cannot deploy to production automatically. You cannot even say that it is production ready.

Now that we established a set of straightforward ground rules, we can move on and describe the pipeline we should develop. We are going to build something. Since building without running unit and other types of static tests should be declared officially illegal and punishable with public shame, we’ll include those in our build stage. Then we’re going execute the steps of the functional testing stage that will run all sorts of tests that require a live application. Therefore, we’ll need to deploy a test release during this stage. Once we’re confident that our application behaves as expected, we’re going to make a production release, followed with the deploy stage that will not only upgrade the production release but also run another round of tests to validate whether everything works as expected.

You might not agree with the names of the stages. That’s OK. It does not matter much how you name things, nor how you group steps. What matters is that the pipeline has everything we need to feel confident that a release is safely deployed to production. Steps matter, stages are only labels.

The DevOps 2.4 Toolkit: Continuous Deployment To Kubernetes

The article you just read is an extract from The DevOps 2.4 Toolkit: Continuous Deployment To Kubernetes.

This book explores continuous deployment to a Kubernetes cluster. It uses a wide range of Kubernetes platforms and provides instructions on how to develop a pipeline on few of the most commonly used CI/CD tools.

I am assuming that you are already proficient with Deployments, ReplicaSets, Pods, Ingress, Services, PersistentVolumes, PersistentVolumeClaims, Namespaces and a few other things. This book assumes that we do not need to go through the basic stuff. At least, not through all of it. The book assumes a certain level of Kubernetes knowledge and hands-on experience. If that’s not the case, what follows might be too confusing and advanced. Please read The DevOps 2.3 Toolkit: Kubernetes first, or consult the Kubernetes documentation. Come back once you’re done and once you think you can claim that you understand at least basic Kubernetes concepts and resource types.

Give it a try and let me know what you think.