Chat is continuous integration (CI), continuous delivery (CD), continuous deployment (CDP), and continuous testing (CT)? How do they differ from each other? Which one should we use?Continue reading
What do we get if we combine events, workflows, GitOps, progressive delivery, and secrets management? The short answer is that we get automation of everything in Kubernetes in a way that we should be operating in 2021.
We’ll combine Argo Events, Workflows & Pipelines, CD, and Rollouts and sprinkle all that with SealedSecrets, Kaniko, and a few other tools.
Argo Workflows & Pipelines is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. It is a cloud-native solution designed from ground-up for Kubernetes.
Since the increase in popularity of continuous delivery (CD), there was an increase in the number of tools marketed as CD solutions. That’s normal. It is only natural for software vendors to ride the waves. They need to sell, and there’s nothing better to sell than whatever is popular at a given moment. Continuous delivery is one of those "popular" waves.
On the surface, there is nothing wrong with buying tools that solve problems. You have a problem, a vendor has a solution, you buy it, they earn money, and you have a good return on investment. Everybody wins, except when the tool does not do what it’s supposed to do. That’s when we run into issues.
Everyone wants to implement continuous delivery. After all, the benefits are too big to be ignored. Increase the speed of delivery, increase the quality, decrease the costs, free people to dedicate time on what brings value, and so on and so forth. Those improvements are like music to any decision maker. Especially if that person has a business background. If a tech geek can articulate the benefits continuous delivery brings to the table, when he asks a business representative for a budget, the response is almost always “Yes! Do it.”
A continuous delivery project will start. Tests will be written. Builds will be scripted. Deployments will be automated. Everything will be tied into an automated pipeline and triggered on every commit. Everyone will enter a state of nirvana as soon as all that is done. There will be a huge inauguration party with a vice president having the honor to be the first one to press the button that will deploy the first release to production. Isn’t that a glorious plan everyone should be proud of?
At the beginning of 2016, I published The DevOps 2.0 Toolkit. It took me a long time to finish it. Much longer than I imagined.
I started by writing blog posts in TechnologyConversations.com. They become popular and I received a lot of feedback. Through them, I clarified the idea behind the book. The goal was to provide a guide for those who want to implement DevOps practices and tools. At the same time, I did not want to write a material usable to any situation. I wanted to concentrate only on people that truly want to implement the latest and greatest practices. I hoped to make it go beyond the “traditional” DevOps. I wished to show that the DevOps movement matured and evolved over the years and that we needed a new name. A reset from the way DevOps is implemented in some organizations. Hence the name, The DevOps 2.0 Toolkit.
Jenkins (forked from Hudson after a dispute with Oracle) has been around for a long time and established itself as the leading platform for the creation of continuous integration (CI) and continuous delivery/deployment (CD) pipelines. The idea behind it is that we should create jobs that perform certain operations like building, testing, deploying, and so on. Those jobs should be chained together to create a CI/CD pipeline. The success was so big that other products followed its lead and we got Bamboo, Team City, and others. They all used a similar logic of having jobs and chaining them together. Operations, maintenance, monitoring, and the creation of jobs is mostly done through their UIs. However, none of the other products managed to suppress Jenkins due to its strong community support. There are over one thousand plugins and one would have a hard time imagining a task that is not supported by, at least, one of them. The support, flexibility, and extensibility featured by Jenkins allowed it to maintain its reign as the most popular and widely used CI/CD tool throughout all this time. The approach based on heavy usage of UIs can be considered the first generation of CI/CD tools (even though there were others before).
Today is an exciting day for me. I just decided that the book I spent the last eight months writing is ready for general public.
What made me write the book? Certainly not the promise of wealth since, as any author of technical books will confirm, there is no money that can compensate the number of hours involved in writing a technical book. The reasons behind this endeavor are of a different nature. I realized that this blog is a great way for me to explore different subjects and share my experience with the community. However, due to the format, blog posts do not give enough space to explore, in more details, subjects I’m passionate about so, around eight months ago, I decided to start working on The DevOps 2.0 Toolkit: Automating the Continuous Deployment Pipeline with Containerized Microservices book. It treats similar subjects as those I write about in this blog, but with much more details. More importantly, the book allowed me to organize my experience into a much more coherent story.
The book is about different techniques that help us architect software in a better and more efficient way with microservices packed as immutable containers, tested and deployed continuously to servers that are automatically provisioned with configuration management tools. It’s about fast, reliable and continuous deployments with zero-downtime and ability to roll-back. It’s about scaling to any number of servers, designing self-healing systems capable of recuperation from both hardware and software failures and about centralized logging and monitoring of the cluster.
In other words, this book envelops the whole microservices development and deployment lifecycle using some of the latest and greatest practices and tools. We’ll use Docker, Kubernetes, Ansible, Ubuntu, Docker Swarm and Docker Compose, Consul, etcd, Registrator, confd, Jenkins, and so on. We’ll go through many practices and, even more, tools.
At this moment, around 70% is finished and you’ll receive regular updates if you decide to purchase the book. The truth is that my motivation for writing the book is the same as with this blog. I like sharing my experience and this book is one more way to accomplish that. You can set your own price and if you feel that the minimum amount is still too high, please send me a private message and I’ll get back to you with a free copy.
Please give The DevOps 2.0 Toolkit: Automating the Continuous Deployment Pipeline with Containerized Microservices a try and let me know what you think. Any feedback is welcome and appreciated.
In The History of Failed Initiatives I mentioned that microservices as a concept existed for a long time. And yet, most of those who tried organizing their architecture around microservices failed or, to use different words, realized that benefits are shadowed by the increase in complexity (especially in operations). The spark that was born long ago became a star only recently. In order to understand why microservices became a hot topic not long ago, we need to understand better why they failed in the past. The explanation is simple. We neither had the tools nor understood the logic we had to apply. Do we have the tools today? Are we capable of putting in place the right procedures and have the mindset microservices require. I’d say yes, we have and we do. When I say we I mean people with the will and resources to make this happen. As in most other cases, once something new is proven and adopted by some, years need to pass until that something is applied by many. The good news is that I think that we are getting there and, slowly, the more conservative part of the industry (in other words, most of it) is starting to at least consider microservices as an option. Even Gartner had positive words for it. And when Gartner says jump, enterprise jumps.