Tag Archives: Delivery and Deployment

The Short History of CI/CD Tools

Jenkins (forked from Hudson after a dispute with Oracle) has been around for a long time and established itself as the leading platform for the creation of continuous integration (CI) and continuous delivery/deployment (CD) pipelines. The idea behind it is that we should create jobs that perform certain operations like building, testing, deploying, and so on. Those jobs should be chained together to create a CI/CD pipeline. The success was so big that other products followed its lead and we got Bamboo, Team City, and others. They all used a similar logic of having jobs and chaining them together. Operations, maintenance, monitoring, and the creation of jobs is mostly done through their UIs. However, none of the other products managed to suppress Jenkins due to its strong community support. There are over one thousand plugins and one would have a hard time imagining a task that is not supported by, at least, one of them. The support, flexibility, and extensibility featured by Jenkins allowed it to maintain its reign as the most popular and widely used CI/CD tool throughout all this time. The approach based on heavy usage of UIs can be considered the first generation of CI/CD tools (even though there were others before).
Continue reading →

Microservices: When the Stars Aligned

1 Reply

In The History of Failed Initiatives I mentioned that microservices as a concept existed for a long time. And yet, most of those who tried organizing their architecture around microservices failed or, to use different words, realized that benefits are shadowed by the increase in complexity (especially in operations). The spark that was born long ago became a star only recently. In order to understand why microservices became a hot topic not long ago, we need to understand better why they failed in the past. The explanation is simple. We neither had the tools nor understood the logic we had to apply. Do we have the tools today? Are we capable of putting in place the right procedures and have the mindset microservices require. I’d say yes, we have and we do. When I say we I mean people with the will and resources to make this happen. As in most other cases, once something new is proven and adopted by some, years need to pass until that something is applied by many. The good news is that I think that we are getting there and, slowly, the more conservative part of the industry (in other words, most of it) is starting to at least consider microservices as an option. Even Gartner had positive words for it. And when Gartner says jump, enterprise jumps.
Continue reading →

Scaling To Infinity with Docker Swarm, Docker Compose and Consul (Part 4/4) – Scaling Individual Services

7 Replies

This article should be considered deprecated since it speaks about the old (standalone) Swarm. To get more up-to-date information about the new Swarm mode, please read the Docker Swarm Introduction (Tour Around Docker 1.12 Series) article or consider getting the The DevOps 2.1 Toolkit: Docker Swarm book.

This series is split into following articles.

In the previous article we switched from manual to automatic deployment with Jenkins and Ansible. In the quest for zero-downtime we employed Consul to check health of our services and, if one of them fails, initiate deployment through Jenkins.

In this article we’ll explore how to scale individual services.

Continue reading →

Scaling To Infinity with Docker Swarm, Docker Compose and Consul (Part 3/4) – Blue-Green Deployment, Automation and Self-Healing Procedure

3 Replies

This article should be considered deprecated since it speaks about the old (standalone) Swarm. To get more up-to-date information about the new Swarm mode, please read the Docker Swarm Introduction (Tour Around Docker 1.12 Series) article or consider getting the The DevOps 2.1 Toolkit: Docker Swarm book.

This series is split into following articles.

In the previous article we manually deployed the first version of our service together with a separate instance of the Mongo DB container. Both are (probably) running on different servers. Docker Swarm decided where to run our containers and Consul stored information about service IPs and ports as well as other useful information. That data was used to link one service with another as well as to provide information nginx needed to create proxy.

We’ll continue where we left and deploy a second version of our service. Since we’re practicing blue/green deployment, the first version was called blue and the next one will be green. This time there will be some additional complications. Deploying the second time is a bit more complicated since there are additional things to consider, especially since our goal is to have no downtime.

Continue reading →

Scaling To Infinity with Docker Swarm, Docker Compose and Consul (Part 2/4) – Manually Deploying Services

12 Replies

This article should be considered deprecated since it speaks about the old (standalone) Swarm. To get more up-to-date information about the new Swarm mode, please read the Docker Swarm Introduction (Tour Around Docker 1.12 Series) article or consider getting the The DevOps 2.1 Toolkit: Docker Swarm book.

This series is split into following articles.

The previous article showed how scaling across the server farm looks like. We’ll continue where we left and explore details behind the presented implementation. Orchestration has been done through Ansible. Besides details behind tasks in Ansible playbooks, we’ll see how the same result could be accomplished using manual commands in case you might prefer a different orchestration/deployment framework.

Continue reading →

Scaling To Infinity with Docker Swarm, Docker Compose and Consul (Part 1/4) – A Taste of What Is To Come

46 Replies

This article should be considered deprecated since it speaks about the old (standalone) Swarm. To get more up-to-date information about the new Swarm mode, please read the Docker Swarm Introduction (Tour Around Docker 1.12 Series) article or consider getting the The DevOps 2.1 Toolkit: Docker Swarm book.

This series is split into following articles.

Previous articles put a lot a focus on Continuous Delivery and Containers with Docker. In Continuous Integration, Delivery or Deployment with Jenkins, Docker and Ansible I explained how to continuously build, test and deploy micro services packaged into containers and do that across multiple servers, without downtime and with the ability to rollback. We used Ansible, Docker, Jenkins and few other tools to accomplish that goal.

Now it’s time to extend what we did in previous articles and scale services across any number of servers. We’ll treat all servers as one server farm and deploy containers not to predefined locations but to those that have the least number of containers running. Instead of thinking about each server as an individual place where we deploy, we’ll treat all of them as one unit.

Continue reading →

Continuous Integration, Delivery or Deployment with Jenkins, Docker and Ansible

64 Replies

This article tries to provide one possible way to set up the Continuous Integration, Delivery or Deployment pipeline. We’ll use Jenkins, Docker, Ansible and Vagrant to set up two servers. One will be used as a Jenkins server and the other one as an imitation of production servers. First one will checkout, test and build applications while perform deployment and post-deployment tests.
Continue reading →

Continuous Deployment: Implementation with Ansible and Docker

7 Replies

This article is part of the Continuous Integration, Delivery and Deployment series.

The previous article described several ways to implement Continuous Deployment. Specifically, it described, among other things, how to implement it using Docker to deploy applications as containers and nginx for reverse proxy necessary for successful utilization of blue-green deployment technique. All that was running on top of CoreOS, operating system specifically designed for running Docker containers.

In this article we’ll try to do the same process using Ansible (an open-source platform for configuring and managing computers). Instead of CoreOS, we’ll be using Ubuntu.

Source code used in this article can be found in the GitHub repo vfarcic/provisioning (directory ansible).
Continue reading →

Continuous Deployment: Implementation

4 Replies

This article is part of the Continuous Integration, Delivery and Deployment series.

The previous article described several Continuous Deployment strategies. In this one we will attempt to provide one possible solution for reliable, fast and automatic continuous deployment with ability to test new releases before they become available to general users. If something goes wrong we should be able to rollback back easily. On top of that, we’ll try to accomplish zero-downtime. No matter how many times we deploy our applications, there should never be a single moment when they are not operational.
Continue reading →

Continuous Deployment: Strategies

5 Replies

This article is part of the Continuous Integration, Delivery and Deployment series.

Previous article provided introduction to continuous deployment. In this one we’ll continue where we left and explore different strategies to deploy software. The article is in no way an exhaustive list of ways to deploy applications but tries to provide few common ways that are in use today.
Continue reading →