Picture me as a young teenager. After school, we’d go a courtyard and play soccer. That was an exciting sight. A random number of us to be running around the yard without any orchestration. There was no offense and no defense. We’d just run after a ball. Everyone moves forward towards the ball, someone kicks it to the left, and we move in that direction, only to start running back because someone kicked the ball again. The strategy was simple. Run towards the ball, kick it if you can, wherever you can, repeat. To this day I do not understand how did anyone manage to score. It was a complete randomness applied to a bunch of kids. There was no strategy, no plan, and no understanding that winning required coordination. Even a goalkeeper would be in random locations on the field. If he’d catch the ball around the goal he’s guarding, he’d continue running with the ball in front of him. Most of the goals were scored by shooting at an empty goalpost. It was “every man for himself” type of ambition. Each one of us hoped to score and bring glory to his or her name. Fortunately, the main objective was to have fun so winning as a team did not matter that much. If we were a “real” team, we’d need a coach. We’d need someone to tell us what the strategy is, who should do what, and when to go into the offense or fall back to defend the goalpost. We’d need someone to orchestrate us. The field (a cluster) had a random number of people (services) with the common goal (to win). Since everyone could join the game at any time, the number of people (services) was continually changing. Someone would be injured and would have to be replaced or, when there was no replacement, the rest of us would have to take over his tasks (self-healing). Those football games can be easily translated into clusters. Just as we needed someone to tell us what to do (a coach), clusters need something to orchestrate all the services and resources. Both need not only to make up-front decisions, but also to continuously watch the game/cluster, and adapt the strategy/scheduling depending on the internal and external influences. We needed a coach and clusters need a scheduler. They need a framework that will decide where a service should be deployed and make sure that it maintains the desired run-time specification.
A cluster scheduler has quite a few goals. It’s making sure that resources are used efficiently and within constraints. It’s making sure that services are (almost) always running. It provides fault tolerance and high availability. It makes sure that the specified number of replicas are deployed. The list can go on for a while and varies from one solution to another. Still, no matter the exact list of cluster scheduler’s responsibilities, they can be summarized through the primary goal. A scheduler is making sure that the desired state of a service or a node is (almost) always fulfilled. Instead of using imperative methods to achieve our goals, with schedulers we can be declarative. We can tell a scheduler what the desired state is, and it will do its best that our desire is (almost) always fulfilled. For example, instead of executing a deployment process five times hoping that we’ll have five replicas of a service, we can tell a scheduler that our desired state is to have the service running with five replicas.
The difference between imperative and declarative methods might seem subtle but, in fact, is enormous. With a declarative expression of the desired state, a scheduler can monitor a cluster and perform actions whenever the actual state does not match the desired. Compare that to an execution of a deployment script. Both will deploy a service and produce the same initial result. However, the script will not make sure that the result is maintained over time. If an hour later, one of the replicas fail, our system will be compromised. Traditionally, we were solving that problem with a combination of alerts and manual interventions. An operator would receive a notification that a replica failed, he’d login to the server, and restart the process. If the whole server is down, the operator might choose to create a new one, or it might deploy the failed replica to one of the other servers. But, before doing that, he’d need to check which server has enough available memory and CPU. All that, and much more, is done by schedulers without human intervention. Think of schedulers as operators who are continually monitoring the system and fixing discrepancies between the desired and the actual state. The difference is that schedulers are infinitely faster and more accurate. They do not get tired, they do not need to go to a bathroom, and they do not require paychecks. They are machines or, to be more precise, software running on top of them.
That leads us to container schedulers? How do they differ from schedulers in general?
Container schedulers are based on the same principles as schedulers in general. The significant difference is that they are using containers as the deployment units. They are deploying services packaged as container images. They are trying to collocate them depending on desired memory and CPU specifications. They are making sure that the desired number of replicas is (almost) always running. All in all, they do what other schedulers do but with containers are the lowest and the only packaging unit. And that gives them a distinct advantage. They do not care what’s inside. From scheduler’s point of view, all containers are the same.
Containers provide benefits that other deployment mechanisms do not. Services deployed as containers are isolated and immutable. Isolation provides reliability. Isolation helps with networking and volume management. It avoids conflicts. It allows us to deploy anything, anywhere, without worrying whether that something will clash with other processes running on the same server. Schedulers, combined with containers and virtual machines, provide the ultimate cluster management nirvana. That will change in the future but, for now, container schedulers are the top of the engineering accomplishments. They allow us to combine the developer’s necessity for rapid and frequent deployments with sysadmin’s goals for stability and reproducibility. And that leads us to Kubernetes…
The DevOps 2.3 Toolkit: Kubernetes
The article you just read is an extract from The DevOps 2.3 Toolkit: Kubernetes.
The goal of the book is not to convince you to adopt Kubernetes but to provide a detailed overview of its features. I want you to become confident in your Kubernetes knowledge and only then choose whether to embrace it. That is, unless you already made up your mind and stumbled upon this book in search of Kubernetes guidance.
The book is about running containers at scale and not panicking when problems arise. It is about the present and the future of software deployment and monitoring. It’s about embracing the challenges and staying ahead of the curve.
Give it a try and let me know what you think.