Automatically Scaling a System Across Multiple Servers : A Comparison of Docker Swarm and Kubernetes

(1)

Petteri Strömberg

AUTOMATICALLY SCALING A SYSTEM ACROSS MULTIPLE SERVERS

A Comparison of Docker Swarm and Kubernetes

Faculty of Information Technology and

Communication Sciences

Bachelor’s Thesis

April 2021

(2)

ABSTRACT

Petteri Strömberg: Automatically Scaling a System Across Multiple Servers – A Comparison of Docker Swarm and Kubernetes

Bachelor’s Thesis Tampere University

Bachelor's Programme in Science and Engineering April 2021

Running applications in the cloud has many benefits over using physical machines. Cloud computing automates resource allocation and thus enables serving varying loads efficiently. Micro- service applications can harness this functionality particularly well, as they are comprised of small independent services that can be scaled independently. This elasticity sets requirements for the software, such as managing the services across multiple servers and handling their states.

This thesis describes how a target system consisting of stateful containerised services can be distributed and automatically scaled across multiple servers. To achieve this, the system is redesigned to adhere to a microservice architecture. The redesigned system consists of stateless services that can be scaled horizontally with ease. The statelessness of the services is achieved by migrating from disk-based data storage to a streaming solution powered by Apache Kafka.

For distributing the target system, this thesis studies the two most popular container orchestrators: Docker Swarm and Kubernetes. A performance comparison of the orchestrators is conducted based on existing research and the functionality of the tools is compared using available documentation. One of the key features studied is automatic scaling. While Kubernetes supports automatic scaling natively, Docker Swarm services can only be scaled manually. This thesis proposes two solutions for scaling services automatically in Docker Swarm. The solutions use existing third-party software and the application programming interface provided by Docker.

The results of this thesis indicate that while Kubernetes offers a more complete set of features expected when running complex enterprise applications, its complexity causes both performance and management overhead compared to Docker Swarm. Docker Swarm, on the other hand, offers a great alternative where simplicity is prioritised over an extensive set of features. Further- more, this thesis shows that the limited functionality of Docker Swarm can be extended to enable automatic scaling, although a sufficient solution for the target system of this thesis would require maintaining an additional component in-house. In the end, Kubernetes is chosen for orchestrating the target system, as it provides a more complete ready-made solution with built-in automatic scaling.

Keywords: Docker Swarm, Kubernetes, container orchestration, elasticity, automatic scaling, cloud, microservice

The originality of this thesis has been checked using the Turnitin OriginalityCheck service.

(3)

TIIVISTELMÄ

Petteri Strömberg: Järjestelmän automaattinen skaalaus usealle palvelimelle hajautettuna – Ku- berneteksen ja Docker Swarmin vertailu

Kandidaatintutkielma Tampereen yliopisto

Kansainvälinen tekniikan kandidaatin tutkinto-ohjelma Huhtikuu 2021

Sovellusten ajaminen pilvessä tuottaa useita hyötyjä fyysisillä palvelimilla ajamiseen verrattuna.

Pilvilaskenta automatisoi resurssien jakamisen ja täten mahdollistaa muuttuvan kuorman palve- lemisen tehokkaasti. Mikropalvelupohjaiset sovellukset soveltuvat pilveen erityisen hyvin, sillä ne koostuvat pienistä itsenäisistä palveluista, joita voidaan skaalata yksitellen. Tämä elastisuudeksi kutsuttu ominaisuus edellyttää, että usealle palvelimelle hajautettuja palveluita ja niiden tiloja voidaan hallita.

Tässä työssä kuvataan, kuinka tilallisista konteista koostuva kohdejärjestelmä voidaan hajauttaa ajettavaksi usealla palvelimella, ja kuinka järjestelmän skaalaus voidaan automatisoida. Jotta tässä voidaan onnistua, järjestelmä suunnitellaan uudelleen noudattaen mikropalveluarkkitehtuu- ria. Uusi järjestelmä koostuu tilattomista palveluista, joiden horisontaalinen skaalaus on vaiva- tonta. Tilattomuus saadaan aikaan siirtymällä levypohjaisesta datan tallennuksesta tietovirtapoh- jaiseen ratkaisuun Apache Kafkan avulla.

Työssä vertaillaan kahta suosituinta konttien orkestrointityökalua, Docker Swarmia ja Kuber- netesta, joiden avulla järjestelmä voidaan hajauttaa ajettavaksi usealla palvelimella. Työkalujen suorituskykyä vertaillaan olemassa olevan tutkimuksen pohjalta, ja ominaisuuksien vertailussa hyödynnetään työkalujen dokumentaatiota. Työssä keskitytään erityisesti palveluiden automaattiseen skaalaukseen. Kubernetes tarjoaa sisäänrakennetun ratkaisun automaattiseen skaalaukseen, mutta Docker Swarmissa palveluita voi skaalata vain manuaalisesti. Työssä esitellään kaksi vaihtoehtoista ratkaisua palveluiden automaattiseen skaalaukseen Docker Swarmissa. Ratkai- suissa hyödynnetään olemassa olevia ohjelmia ja Dockerin ohjelmointirajapintaa.

Työn tuloksena selviää, että Kubernetes tarjoaa kokonaisvaltaisemman valikoiman ominaisuuksia, joita etenkin vaativimmat yrityssovellukset tarvitsevat toimiakseen. Kuberneteksen mo- nitahoinen rakenne vaikuttaa kuitenkin negatiivisesti sen suorituskykyyn ja tekee siitä Docker Swarmia vaikeamman hallita. Docker Swarm sen sijaan tarjoaa hyvän vaihtoehdon Kubernetek- selle, kun yksinkertaisuus on laajoja ominaisuuksia tärkeämpää. Lisäksi työ osoittaa, että automaattinen skaalaus on mahdollista toteuttaa myös Docker Swarmissa. Kohdejärjestelmälle so- piva skaalausratkaisu vaatisi kuitenkin ylimääräisen komponentin kehittämisen ja ylläpidon. Ku- bernetes valikoituukin kohdejärjestelmän orkestrointityökaluksi, koska se tarjoaa valmiimman or- kestrointiratkaisun ja sisäänrakennetun automaattisen skaalauksen.

Avainsanat: Docker Swarm, Kubernetes, konttien orkestrointi, elastisuus, automaattinen skaalaus, pilvi, mikropalvelu

Tämän julkaisun alkuperäisyys on tarkastettu Turnitin OriginalityCheck –ohjelmalla.

(4)

PREFACE

Three years ago, I was not only waiting for my university admissions results but also waiting to hear back from my first job interview for a software development trainee posi- tion. Now, I am concluding my bachelor’s studies at Tampere University with this thesis conducted in collaboration with the organisation that eventually offered me the job. I am extremely grateful for the opportunities presented to me by both organisations and for the academic knowledge and practical experience I have been able to gain during these past three years.

One of the reasons I loved doing this research was seeing my own development. The target system has been implemented in phases, and I have been there every step of the way. At the beginning of my second year of study, I grew the system from one service to two. During the rest of the year, I designed it such that any number of new services could be added. The original system was designed to work on a single host for a reason: it was created by a second-year Computer Science student, who had very little knowledge of distributed systems. While the development of the system will certainly not end with this thesis, it feels natural to conclude my bachelor’s degree with a work that opens the system up to so many new possibilities.

I would like to take this opportunity to thank my supervisor Kari Systä, who helped me to formulate this work according to the highest academic standards. I would also like to thank my line manager Markus Nenonen and my colleagues, who helped me to align the work with the needs of the organisation and provided me with the resources for conduct- ing this research. Many thanks also belong to my family and friends for their support and their help with proofreading the thesis.

Special thanks to everyone involved in ensuring seamless communication even during these extraordinary times. No people were in physical contact and thus at risk of being exposed to the coronavirus in the making of this research.

Tampere, 23 April 2021

Petteri Strömberg

(5)

LIST OF SYMBOLS AND ABBREVIATIONS

AKS Azure Kubernetes Service

API Application Programming Interface CaaS Containers as a Service

CLI Command Line Interface

CPU Central Processing Unit

DNS Domain Name System

EKS Amazon Elastic Kubernetes Service

GKE Google Kubernetes Engine

HPA Horizontal Pod Autoscaler HTTP Hypertext Transfer Protocol IaaS Infrastructure as a Service

IP Internet Protocol

KaaS Kubernetes as a Service

NFS Network File System

NST Network Simulation Tool, the target software

OS Operating System

PaaS Platform as a Service

RAM Random Access Memory

RTAT Real-Time Analytics Tool, the system tested using NST SaaS Software as a Service

SFTP SSH File Transfer Protocol

SSH Secure Shell

TCP Transmission Control Protocol

UDP User Datagram Protocol

VM Virtual Machine

YAML YAML Ain’t Markup Language, a data-serialisation language

(7)

1. INTRODUCTION

Companies are doing more and more of their computing in the cloud. Instead of acquiring physical computers to run applications, they provision virtual machines from datacentres.

This resource sharing model is called cloud computing. One of its main benefits is the ability to reserve and release resources as needed, which is referred to as elasticity or scalability. To serve a high varying load, applications must often be deployed and scaled across multiple servers.

Applications can accommodate the benefits of cloud computing by adhering to microservice architecture. Instead of working as a single process monolith, in microservice architecture the application is split into multiple independent and loosely coupled services. As such, microservice architecture allows scaling each component of the system separately, leading to more efficient resource usage.

Microservices are run in isolated environments that can be achieved with containers.

Containers are a light-weight alternative for virtual machines and have been popularised by Docker – a company that has become synonymous with the word container. Contain- erised microservice applications can be deployed and managed across multiple servers using container orchestrators. The two most popular container orchestrators are Kuber- netes and Docker Swarm [1], both of which are examined in this thesis.

The target software of this thesis is a containerised system that is used to simulate a network of data sources that send diagnostics data. The system is part of a testing solution for a real-time analytics tool. The current version of the target system runs as a set of Docker containers on a single host machine. This sets limitations to the performance of the system, which in turn limits the performance testing of the real-time analytics tool.

The objective of this thesis is to distribute the target system across multiple servers and develop a solution for scaling the components of the system automatically. While at the moment the system itself is containerised, its current architecture does not allow for distributing and scaling. As a solution, this thesis proposes and implements a microservice architecture for the system. In addition, the thesis studies and compares Docker Swarm and Kubernetes to find out which orchestrator suits the needs of the target system better.

The comparison focuses on the features and performance of the tools as well as the

(8)

amount of maintenance work needed to manage the target system using them. Special focus is given on how automatic scaling can be achieved on each platform.

The organisation developing and using the target system has a lot of experience in working with Kubernetes, but Docker Swarm is relatively unknown within the organisation.

However, Kubernetes is not an obvious choice for the target system, as its shortcomings related to complexity are well-known within the organisation. The aim of this thesis is not only to compare the container orchestration platforms to use with the target system but also to learn more about Docker Swarm and how an organisation mainly working with Kubernetes could benefit from it.

While this thesis focuses on a specific system, it aims to provide a comprehensive and generally applicable comparison of the orchestration tools. The developed microservice architecture is specific to the target system, but the ideas behind it should be applicable more broadly. The implementations in this thesis are intended to work as proofs of con- cept rather than as finalised products. They are used to verify that the presented solu- tions work at a conceptual level and can be used to better understand the implementation requirements for a productised solution.

The thesis is structured as follows. Chapter 2 outlines the theoretical background of this thesis. It starts by introducing the concepts of cloud computing and microservices, followed by the basics of Docker containers. The theory is extended to container orchestration in Chapter 3. The chapter concludes with a comparison between Docker Swarm and Kubernetes based on their documentation and existing research. In Chapter 4, the target system of this thesis is introduced. The purpose and current architecture of the system are presented, and problems related to the current system are contemplated. A microservice architecture is proposed to solve the problems. Finally, the requirements for container orchestration and the orchestrators’ suitability for the target system are discussed. Chapter 5 extends this discussion to include achieving automatic scaling with each orchestrator. The results of this thesis are summarised and discussed in Chapter 6. The final chapter also includes contemplation on the future development of the system and suggestions for future research.

(9)

2. CLOUD SOFTWARE

Cloud technologies have been on the rise for more than a decade. Since the introduction of today’s most popular cloud technologies Docker (2013) and Kubernetes (2014), mi- gration to cloud has become easier and uncertainties related to cloud technologies have reduced significantly. Today, more and more of the world’s computing is done in the cloud. This chapter covers the basics of cloud computing and elaborates how microservice architecture can help in harnessing its benefits. At the end of the chapter, the building blocks of modern cloud applications – Docker containers – are introduced.

2.1 Cloud Computing

Cloud computing is a resource sharing model that is used to give access to computing resources, such as storage and servers. The resources can be accessed easily and on- demand via a network, and they are allocated automatically with minimal interaction with the service provider. Cloud computing services are provided by using three different service models: Infrastructure as a Service (IaaS), in which the physical resources like computing hardware and storage are provided as a service; Platform as a Service (PaaS), which is a platform that provides both the physical infrastructure and the operating systems for developing and deploying applications; and Software as a Service (SaaS), which provides applications as services via a thin client interface or programming interface [2].

One of the key characteristics of cloud computing is rapid elasticity, meaning that the provisioned resources can be scaled up and down quickly to match the demand [2]. This reduces the need to over-allocate resources for varying loads and enables cloud providers to enforce a more constant load by pricing resources based on available supply.

Cloud services can be deployed within an organisation (private cloud) or using an outside service provider (public cloud) [2], such as Google, Amazon, or Microsoft. Whether using private or public cloud – or a combination of them (hybrid cloud) [2] – efficient allocation of resources is a great concern. Meeting the demand exactly, and thus using cloud services efficiently, sets requirements for the applications to be developed. Most im- portantly, the horizontal scaling of the application should happen with ease and automatically. This requirement can be fulfilled with microservice architecture, which is discussed next.

(10)

2.2 Microservices

Microservices are small services that are used as building blocks when creating more complex systems. While in traditional architectures systems consist of parts with different roles, microservice architecture uses more independent and loosely coupled microservices to achieve modularity. The benefits of microservices mainly consist of their independence, which makes them easy to develop, deploy and scale [3].

Systems that use microservices are often more robust than traditional systems. As the components of the system are independent services, failure in one component does not render the other components unusable. Furthermore, as one instance of a service fails, other instances can be created to take over the responsibilities of the failed service. The independence of the services also makes them inherently scalable. Service instances can be added and removed according to need. While applications using traditional architectures can be scaled as well, they usually must be scaled as wholes. When a system is built from microservices, parts of the system can be scaled according to the demand for that specific service [3]. This can make resource use more efficient and reduce both bottlenecks and the over-allocation of resources.

To harness the benefits of both robustness and scalability, the states of the services must be managed. Applications that have some sort of memory of previous events that they use to fulfil their tasks are considered stateful. The information in memory defines the application’s state. Ideally, microservices are stateless [4]. If a service is stateless, it does not matter which instance of it handles which task. This makes scaling and taking tasks over from failed instances easy. Many services still require some state information, which makes scaling and recovering them problematic. As will be seen later, this is the case for the current implementation of the target system of this thesis.

Microservices should be run in isolation to prevent other microservices from affecting their performance. Isolated environments can be achieved through virtualization, but the overhead of traditional virtualization often gets too high when small microservices are used. Containers offer a solution to this, providing isolated execution environments with less overhead and better cost efficiency [3]. The most used container technology is Docker [1], which is discussed next.

2.3 Docker Containers

Docker is an open container platform released in 2013 and developed by Docker Inc.

Docker allows packaging applications with their dependencies and running them in

(11)

loosely isolated environments called containers [5]. Containers are similar to virtual machines (VMs) but strip away more of the functionality not required by the application.

Containers achieve this by sharing host operating system services instead of relying on their own guest operating systems like VMs do. This makes containers more lightweight and faster to start than VMs. The difference between VMs and containers is illustrated in Figure 1.

Figure 1. Virtual machines (on the left) have their own operating systems, while Docker containers share host operating system services via Docker Engine.

Adapted from [6].

Docker is used for building, running, and shipping containerised applications. The oper- ations are powered by Docker Engine. Docker Engine resides on top of the host operating system and operates using its resources. The Engine has a client-server architecture, that consists of a server, a command line client and the API between them. The server includes a daemon process called dockerd, that is responsible for creating and managing Docker objects, such as containers. The daemon is controlled by using the command line interface (CLI) client [7].

Docker packages applications and their dependencies into images. A Docker image contains all the information needed to run a Docker container. Images can be created by building on top of other images or by starting from scratch. The instructions to build an image are defined in a Dockerfile. Docker images can be shared with other users via Docker Hub or private registries. Containers are short-lived instances of Docker images.

Docker uses Linux namespaces to isolate containers from each other and the host. Con- tainers are managed by using either Docker CLI or Docker API. Containers can be connected to networks, and storage can be attached to them with volumes [5]. Volumes are Docker objects used to persist the data that containers generate and use [8].

Docker supports defining and running multi-container applications through a separate tool called Docker Compose. Instead of running services through Docker CLI, Docker

(12)

Compose allows creating a YAML file for defining a set of services and running them with a single command. Volumes and networks for the application can also be defined in the Compose file. Docker Compose enables deploying a microservice application on a single Docker host. It is often used in development environments and continuous inte- gration workflows, but it works in production environments as well [9].

(13)

3. CONTAINER ORCHESTRATION

In microservice architecture, the individual services are relatively simple and easy to develop. On the downside, the architecture of the system becomes more complex as microservices must be managed to form a functioning system. Containers running across multiple servers can be managed, scaled, and maintained by using container orchestrators [10]. Orchestrators can be deployed on IaaS platforms and can be used to offer a new type of service model, Containers as a Service (CaaS) [11]. In orchestration terms, the underlying servers are referred to as nodes and a set of connected nodes as a cluster. The application is deployed on a cluster as a set of containers [12].

Different orchestrators are suited for different needs and offer different features, performance capabilities and levels of complexity. One of the goals of this thesis is to find the most optimal container orchestrator for the target system. The options are limited to the two most popular container orchestrators: Docker Swarm and Kubernetes [1]. The following subsections introduce these tools and compare them in terms of their functional properties and performance.

3.1 Docker Swarm

Docker Swarm is a native container orchestration and cluster management tool for Docker. It started as Docker’s standalone container orchestration project in 2014 [13]. In 2016, Docker started including cluster management and container orchestration features in Docker Engine [14]. The original Docker Swarm project – nowadays referred to as Classic Swarm – has since been archived and the term Docker Swarm is used in general and in this thesis for Docker Engine’s built-in Swarm mode [13, 15]. In a 2019 survey, 21% of technical professional respondents were using Docker Swarm while an additional 22% were planning to bring it into use [1].

One of the main benefits of Docker Swarm over other container orchestrators is its simplicity. Swarm mode is built into Docker releases and no additional software is needed to use it. Docker Engine CLI can be used directly to create a cluster, join it, and deploy applications. Cluster creation is further simplified by handling node specialisation at runtime: both managers and workers can be deployed using the same disk image [15].

A Docker Swarm cluster – or a swarm – consists of one or more manager nodes and one or more worker nodes. The manager nodes maintain a consistent internal state of

(14)

the swarm to ensure fault tolerance [16]. The services deployed to the cluster are run as containers on the worker nodes. Docker Swarm architecture is illustrated in Figure 2.

Figure 2. Docker Swarm architecture. Adapted from [16].

Services can be deployed to a swarm individually or in stacks. Application stacks can be defined with a Docker Compose file, which enables both single Docker Engine and Docker Swarm deployments to use the same deployment definition files [15].

3.2 Kubernetes

Kubernetes is a container orchestration platform developed by Google, who open- sourced it in 2014 [17]. Kubernetes originally used Docker as its container runtime via a separate tool called Dockershim, but the support was recently deprecated in favour of using Docker’s container runtime containerd and other container runtimes directly [18].

Kubernetes is the most popular container orchestration tool with 44% of technical professional respondents using it and an additional 27% planning to use it in the future, according to a 2019 survey [1].

Kubernetes consists of a set of loosely coupled components that are needed for a complete and working cluster. The components can be divided into node components that manage an individual node and control plane components that manage the cluster [19].

Communication within the cluster happens via an API server component, which is part of the control plane. It exposes an HTTP API, that is used to control the cluster from the outside. The cluster is often controlled using the kubectl command line tool, which uses the HTTP API [20]. Kubernetes also provides a GUI for controlling the cluster. A Kuber- netes cluster consists of one or more master nodes and one or more worker nodes. The control plane can run on a single master node or it can be distributed to ensure fault- tolerance and high availability. Worker nodes are managed by kubelet processes running

(15)

on them. Kubelets communicate with the control plane via the API component and ensure that the desired state of the node is achieved [19]. Kubernetes architecture and components are illustrated in Figure 3.

Figure 3. Kubernetes architecture and components. Adapted from [19].

Containers cannot be deployed in Kubernetes directly. The smallest unit of deployment is a Pod, that can contain one or more containers. A Pod represents a service that consists of tightly coupled containers that should be co-located and co-scheduled. Contain- ers in the same Pod share storage and network resources [21].

Every resource in Kubernetes is represented by an object. Objects can be created and deployed by applying YAML configuration files with kubectl [22]. For example, one microservice could consist of a Deployment object that defines two Pod object replicas by using a ReplicaSet object, and a Service object that exposes the microservice. These objects are discussed in more detail later. As the number of objects is quite high even for a simple service, there is a need to package the configuration files to easily deploy applications to other clusters. This need can be solved with HELM, which can be thought of as a package manager for Kubernetes: it enables packaging Kubernetes resource definitions and sharing them with others [23].

3.3 Comparison

Container orchestrators have been previously compared by Truyen et al. [12], Jawarneh et al. [24] and Pan et al. [25], all published in 2019. The study by Truyen et al. presents

“a descriptive feature comparison study of the three most prominent orchestration frame- works”, that is, of Apache Mesos along with Docker Swarm and Kubernetes [12]. The

(16)

research by Jawarneh et al. compares Docker Swarm, Kubernetes, Apache Mesos and Cattle in terms of their functionality and performance [24]. Pan et al. describe “a thorough comparison and performance benchmark of the two most popular container clustering and orchestration tools”, Docker Swarm and Kubernetes [25].

This thesis takes its approach for functional comparison from Jawarneh et al. [24], as they provide a simple yet powerful framework for comparing the orchestrators. As in their work, the comparison is done by splitting the functionality of the orchestrators into three layers: resource management, scheduling, and service management [24]. The comparison is based on the documentation of the most recent versions of the tools to ensure up-to-date results. The most up-to-date versions of Docker, which the Swarm mode is a part of, and Kubernetes at the time of writing this thesis are 20.10.5 [26] and v1.20.0 [27], respectively. The following subsections compare the functionality of Docker Swarm and Kubernetes in these three layers, followed by a performance comparison. The performance comparison consolidates the results of Jawarneh et al. [24] and Pan et al. [25].

After the performance comparison, Docker Swarm and Kubernetes are compared in terms of cluster deployment and management. Results of the comparisons are summarised and compared to existing research at the end of the chapter.

3.3.1 Resource Management

The resource management layer abstracts the underlying resources to ensure high utili- sation levels and low interference between containers. Managed resources include computational resources, volumes, and networking. Computational resources include CPU, memory, and sometimes even graphics processing [24]. Volumes are used to store data created by containers [8]. Volumes grant the containers access to the file system of the host machine or to a remote file system. Networking refers to the TCP and UDP ports and IP addresses that are used within the container network to access the containers [24]. Both Docker Swarm and Kubernetes provide containers with access to computational resources, volumes, and networking resources. The main differences in resource management are in the volumes the orchestrators support and in the way they handle networking.

Docker Swarm supports two types of volumes: data volumes and bind mounts. Data volumes are storage that outlive services and exist independently of them. Volumes are specific to a host, meaning that services running on different nodes cannot access the same volumes. Bind mounts enable the services to access data on the host where the service is deployed. They work by mounting a file system path into the container. Docker advises designing Swarm applications in a way, that bind mounts are not needed. Bind

(17)

mounts require the mounted path to exist on each node, which is why the automatic rescheduling of services on different nodes could lead to unexpected behaviour [28].

Accessing an external file system such as NFS is made possible by using volume drivers [8]. Kubernetes extends the limited functionality of Docker Swarm. In addition to local volumes and bind mounts (hostPath volumes in Kubernetes terms), Kubernetes supports third-party volumes provided by Amazon Web Services, Microsoft Azure and Google Compute Engine. Using open solutions, such as Glusterfs or NFS, is also possible [29].

Docker Swarm supports overlay networks to enable secure network communication between the services in the cluster [30]. Each service is assigned a unique DNS name, which enables querying the service through a DNS server included in Docker Swarm [15]. A service can be made accessible from the outside via a published port. The service can then be accessed on the published port on any of the nodes in the cluster [31].

Kubernetes creates a virtual network for the cluster, which enables connectivity between Pods. Pods are exposed to other Pods via Services¹. Pods can access each other via Services’ IPs and ports, that can be read from environment variables or resolved by using a DNS add-on [32].

3.3.2 Scheduling

The efficient use of resources is enabled by the scheduling layer. The scheduling layer places services in the cluster based on user input. Possible capabilities include configuring the placement of a service within the cluster, scaling the number of service instances, checking the readiness of services to only include services that are ready, restarting failed services, rolling deployment to update the application version without downtime and co-locating services [24]. The scheduling layer is also responsible for automatic scaling, allowing the rapid elasticity expected from cloud applications.

Docker Swarm and Kubernetes both use a declarative service model for scheduling services. This means that the user can define the desired state of services in their application stack, and the orchestrator components try to retain the desired state in the cluster.

The declarative service model ensures the high availability of services. For example, if a cluster node crashes, the services that were running on it are started on the remaining nodes [15]. In Docker Swarm, the desired state is compared to the actual state of the cluster by the cluster manager. If there are differences between the states, the cluster manager tries to reconcile them. In Kubernetes, the state of the cluster is defined by

1 kept in italics to distinguish from other services

(18)

using different types of workloads, such as Deployments and ReplicaSets, and the Ku- bernetes components maintain the state and ensure high availability [33].

Docker Swarm’s state configuration enables the user to define the placement of a service, scale services, and restart failed services [28]. Rolling updates are supported by controlling the delay between deployments to different nodes [15]. There is no built-in capability for readiness checking or the co-location of services [24]. Using Kubernetes workloads enables the user to define the placement of a Pod, scale them, and restart failed Pods [34, 21]. The state of the cluster can be changed at a controlled rate, which enables rolling updates without downtime [17]. Containers can be co-located by using Pods, and Pod readiness checking is supported natively [35].

Docker Swarm is also missing a built-in way to scale services automatically based on resource usage. There are ways to achieve automatic scaling in Docker Swarm, but third-party software and additional configuration is needed [36, 37, 38]. Kubernetes has a component called Horizontal Pod Autoscaler that is used to automatically scale Pods based on CPU utilization level or custom metrics [39]. Achieving automatic scaling in Docker Swarm and Kubernetes is discussed in more detail in Chapter 5.

3.3.3 Service Management

The service management layer is used to deploy complex enterprise applications. It provides functionality for the higher-level management of an application, such as attaching metadata to containers with labels, isolating containers with groups or namespaces, and dividing the incoming load by using load-balancing [24].

The service management features of Docker Swarm are limited to basics. It supports labels for adding metadata to containers [28]. Load-balancing is partially implemented in Docker Swarm. Exposed services can be accessed on each node and the requests are internally balanced among the service instances by using DNS. To ensure that the requests to the cluster are made via a healthy node, an external load balancer is needed to balance the incoming requests between the nodes [15]. It is not possible to isolate containers with groups or namespaces in Docker Swarm [24]. On the other hand, Kuber- netes provides a wide range of service management features. Objects like Pods can be labelled to add metadata to them, and labelled objects can be queried with selectors [40].

Services can be used as load-balancers within the cluster. External load-balancing is done by using an Ingress that exposes routes to the Services from outside the cluster [41]. Pods can be isolated in Kubernetes with namespaces. Namespaces are essentially virtual clusters running on the same physical cluster [42].

(19)

3.3.4 Performance

Container orchestration platforms have two main aspects that affect the performance of a containerised application: container runtime and orchestration overhead. Since both Docker Swarm and Kubernetes are often used with the same container runtime – Docker, or in the case of the latest Kubernetes releases containerd directly – this thesis focuses on the overhead created by the orchestration layer. However, it should be noted that Kubernetes supports a wider variety of container runtimes. This could enable the user to opt for a more efficient container runtime, should one exist. Performance differences of Docker Swarm and Kubernetes have been studied and compared comprehen- sively by Jawarneh et al. [24] and Pan et al. [25], both in 2019. This subsection summarises the results of these studies.

The study by Jawarneh et al. measures the performance of the orchestrators by analys- ing the time they take to provision a cluster, provision applications with different complexities, provision a web application with a high number of replicas and recover from failures. The time it takes to provision an application was measured for images available locally and images retrieved from a Docker registry. The orchestrators were setup for the comparison by using Rancher, which can be used to automate the setup of container orchestrators. Cluster provisioning with Kubernetes took over twice as much time as it took with Docker Swarm. Jawarneh et al. believe that this is due to Kubernetes’ complex architecture. Provisioning times for applications with different complexities (applications consisting of 1 to 4 containers) were similar between the orchestrators independent from how the image was provided. For provisioning a web application with a high number of replicas, Kubernetes was faster when the image was available locally, but slower when it had to be retrieved from the Docker registry. The researchers believe that this is due to the overhead generated by the communication between the Kubernetes agent and the Docker registry. Failover time was measured in two cases: container failure and node failure. Kubernetes responded to container failure faster, but Docker Swarm was over 3 times faster to recover from node failure. Jawarneh et al. indicate that this results from the architectures of the systems: Kubernetes can handle failed containers with a local agent, while its response to node failure is handled by an event-based system that causes a chain effect that in turn induces delay. Docker Swarm handles node failures by using a heartbeat system, which provides a faster recovery [24].

In their research, Pan et al. compare the overheads caused by Docker Swarm and Ku- bernetes. The comparison was done by running 36 different benchmarks that test pro- cessor performance, memory speed and memory bandwidth for running single-container

(20)

applications. In addition, one experiment was done to test the execution of a multi-container application. The experiments were run on systems that use Docker Swarm and Kubernetes as well as on a baseline system using only the underlying container engine.

The orchestrators were then compared to the baseline with the null hypothesis of orchestrators having no performance effect and the alternative hypothesis of orchestrators in- curring performance overhead. According to the results, there were only 2 benchmarks with meaningful overhead associated with Docker Swarm and even in those cases the overheads were low: 1.8% and 3.5%. Meaningful overheads associated with Kubernetes occurred in 1/3 of the benchmarks. The maximum overhead was 12.3%, mean 5.2% and median 4.5%. In the multi-container experiment, Docker Swarm showed no meaningful overhead, while the overhead associated with Kubernetes was 8.3% [25].

3.3.5 Cluster Deployment

The amount of work needed to deploy and manage the system depends on the platform chosen and is thus an important factor to consider when deciding which orchestrator to use. In this section, Docker Swarm and Kubernetes are compared in terms of cluster deployment and management.

A Docker Swarm cluster consists of one or more instances of Docker Engine that can run on a single physical machine or cloud server or that can be distributed across multiple machines [43]. As Docker Swarm’s architecture is simple and cluster deployment easy, the major cloud providers do not offer Docker Swarm as a service. Instead, a Docker Swarm cluster is often created from virtual machines provisioned by using an IaaS platform. Docker Swarm nodes must have Docker installed on them, and they must be able to communicate with each other over a network [44]. A swarm is created by initialising the cluster on one of the manager nodes with a single docker swarm init command, that provides the user with a command that can be copy-pasted and executed on other nodes to join them to the cluster. The init command also outputs a token, which is used to authenticate joining nodes. Once all the nodes have been joined to the swarm, it is ready to accept service deployments [45].

A Kubernetes cluster can be deployed on a local machine or in the cloud. Alternatively, a managed cluster can be acquired as a service [46]. As Kubernetes is a relatively complex orchestrator, many companies choose to buy it as a service. Google Kubernetes Engine (GKE) [47], Amazon Elastic Kubernetes Service (EKS) [48] and Azure Kuber- netes Service (AKS) [49] are examples of Kubernetes as a Service (KaaS) that companies use when they want to outsource cluster deployment and management. A minimum

(21)

viable Kubernetes cluster can be deployed in a private cloud with a toolbox called ku- beadm. To use kubeadm to create a cluster, each node must have kubelet and kubectl installed in addition to kubeadm. Once these tools have been installed and kubelet has been started, creating a cluster is very similar to creating one with Docker Swarm: the cluster is initialised on the master node and the worker nodes are joined to it. The major difference is that Kubernetes Pods communicate with each other via a Pod network, which must be deployed manually. Kubernetes supports Pod networks provided by several separate projects that use Container Network Interface (CNI). A CNI-based network can be deployed to Kubernetes like any other Kubernetes resource: by applying a YAML definition with kubectl [50].

3.3.6 Summary

The differences in Docker Swarm and Kubernetes are explained by their architecture and purpose. As an all-around container orchestrator, Kubernetes has more features with the downside of a more complex architecture required to support them. Docker Swarm is a more streamlined orchestrator intended as an easy and fast solution, but it lacks many of the features of Kubernetes in order to keep the architecture simple. Ad- vanced features such as automatic scaling are left for the user to implement, as is de- scribed later in this thesis. This subsection summarises the differences between Docker Swarm and Kubernetes and compares the results with existing research.

Table 1. Functional differences between Docker Swarm and Kubernetes.

Layer Feature Docker Swarm Kubernetes

Resource Management Memory + +

CPU + +

Local Volumes + +

Remote Volumes + ++

Networking + +

Scheduling Placement + +

Scaling + +

Automatic Scaling +

Readiness Checking +

Restarting + +

Rolling Updates + +

Co-location +

Service Management Labels + +

Groups +

Load Balancing + ++

+ basic functionality, ++ advanced functionality

The functional differences between Docker Swarm and Kubernetes are summarised in Table 1. Advanced functionality is indicated in cases where both orchestrators support the functionality, but one does so to a significantly greater extent. As can be seen, Ku- bernetes’ complex architecture enables it to offer a more comprehensive set of features,

(22)

making it a more complete solution for enterprise applications. While Docker Swarm offers a narrower set of features, it includes enough functionality for many basic applications. The results of the performance comparison presented in Table 2 indicate that the wide variety of features and complex architecture of Kubernetes come with the cost of performance overhead. The simple architecture of Docker Swarm adds little to no overhead compared to running containers on a single Docker host. Kubernetes performs better only when provisioning applications from local images and reacting to container failures, but even these advantages are minor.

Table 2. Performance differences between Docker Swarm and Kubernetes.

Study Measure Docker Swarm Kubernetes

Jawarneh et al. [24]

Cluster provision time ++

Application provision time (local image) +

Application provision time (Docker registry) +

Container failover time +

Node failover time ++

Pan

et al. [25] Single-container application performance + Multi-container application performance + + slight advantage, ++ significant advantage

Kubernetes is also more difficult to deploy. While kubeadm automates many of the steps, installing a Kubernetes cluster requires installing a lot of dependencies, which in turn requires configuring repositories and networks. In addition, the correct versions of the dependencies must be installed to ensure compatibility. Manual network configuration is needed as well. The installation process is thus significantly more complex than creating a Docker Swarm, which only requires installing Docker and running a single command on each networked node. On the other hand, Kubernetes’ complexity can be overcome by acquiring it as a service.

Similar results appear in existing research. Jawarneh et al. conclude that “[i]n terms of functional comparison […] Kubernetes is one of the most complete orchestrators nowadays on the market” but that “its complex architecture introduces, in some cases, a significant overhead that may hinder its performances” [24]. This is in line with the results of this thesis, although their study indicates that Docker Swarm does not support remote file system volumes, rolling updates or load balancing [24], which contradicts Docker Swarm’s up-to-date documentation. This is likely due to updates in Docker Swarm since 2019. Truyen et al. state that while Docker Swarm has less features and “is expected to be used and customized for specific technology segments […]”, “[t]he large number of unique features […] is a strong asset of Kubernetes […]” [12]. This also coincides with the results of this thesis, indicating that Kubernetes offers a more complete set of features, while the functionality of Docker Swarm is intended to be extended by the user.

(23)

4. DESIGNING A SCALABLE SYSTEM

The target system of this thesis is a simulation software that is used to test the performance and stability of a real-time analytics tool. The system consists of containerised stateful services, which makes it runnable in cloud environments but not easily scalable.

In this chapter, the target system’s purpose is elaborated and its current architecture illustrated. This is followed by contemplation of the problems related to the current architecture and a proposal for a new architecture that solves the problems. At the end of the chapter, the requirements for the orchestration of the system are discussed. Further- more, Docker Swarm and Kubernetes are evaluated and compared based on the requirements.

4.1 Purpose of the System

The software that is tested by using the target system will be referred to as Real-Time Analytics Tool (RTAT) in this thesis. To understand the requirements set for the testing software, it is important to understand some concepts related to RTAT. RTAT is a cloud- native system deployed on Kubernetes. It collects data from various data sources via various network protocols, such as TCP and SFTP. Data sources are elements within a network. Events in the network cause the data sources to generate diagnostics records, which RTAT collects and combines. The records from selected data sources are com- bined to form an analytics report for each event. The process of combining records is referred to as correlation. RTAT does the correlation based on the values in the records – correlation keys – and record timestamps. There are several data source types, and the data source instances have hierarchical relationships. A network of data sources can be presented hierarchically as a tree. Figure 4 shows a very simple example of a network of data sources. A real network can consist of thousands of data sources providing diagnostics data to RTAT.

Figure 4. Five data sources presented hierarchically.

Network

Data source type A instance 1

Data source type C instance 1

Data source type C instance 2

Data source type B

instance 1 Data source type C instance 3

(24)

The performance and load testing of RTAT is done by using a suite of components illustrated in Figure 5. The suite of components is used to simulate a network from which RTAT can collect data. The testing process starts by generating the records for each simulated data source by using a tool called Record Generator. The generated records contain the correct correlation keys and a time offset from the beginning of the simulation indicating when the record should be sent. The network can then be simulated by using Network Simulation Tool (NST). NST’s responsibility is to send the records to RTAT with the correct protocols and at the correct time. The split between record generation and network simulation has been made to keep the simulation simple and fast. Generating records beforehand also allows the tester to reproduce issues by repeating the simulation with the same data and timing and to edit datasets manually for example to produce corrupted records. To verify that RTAT works as it should, the analytics reports are ex- ported by using Report Exporter and compared with the original records generated.

Figure 5. Testing software suite used to test RTAT.

The target software of this thesis is NST, which is used to simulate the data sources in a network. The software has been developed for 1.5 years within the organisation and for the most part has been designed and implemented by the writer of this thesis.

(25)

4.2 Architecture of the System

The records generated by Record Generator form a dataset. A dataset includes all the data and configuration needed to run a simulation with NST. The contents of a dataset are organised into subdirectories, which contain records and configuration files for each simulated data source. The configuration files contain for example connection information about where the records should be sent. There is also a Docker Compose deployment definition file docker-compose.yaml inside each dataset. The file is used to deploy the services needed to run a simulation with the dataset. An example dataset can be seen in Figure 6.

Figure 6. An example dataset.

The current version of the testing software works by having a separate service – a sim- ulator – for each data source. There is a separate type of simulator for each data source type. Each data source instance is simulated by a separate instance of the corresponding simulator. Simulators are implemented as Docker containers. As a simulator is deployed, the container is given access to the dataset residing in the host file system using a bind mount. The dataflow in a simulation is presented in Figure 7. The simulator is started by sending an HTTP request containing a start time for the simulation. When the start time is reached, the simulator starts reading the records from the mounted directory. Each record contains a time offset from the start time defining when the record should be sent.

The simulator calculates the time for sending the record based on the start time and the offset. This timestamp is written into the record, and the record is sent at the specified

(26)

time. The simulator also stops automatically when it has sent all the records in the dataset unless an optional repeat parameter is present in the start request. If the repeat parameter is set to true, the simulator continues by starting the reading process from the beginning and by setting the end time of the previous simulation round as the start time of the new one. In this case, the simulator can be stopped with an HTTP request.

Figure 7. Dataflow in a simulation.

To execute a simulation, NST needs a dataset, simulators, and a controller service, all of which reside on a single host machine. The simulators and the controller are deployed by running the docker-compose up command inside the dataset directory. Docker Com- pose uses the deployment definition file inside the directory to start the services with the correct configurations as per their configuration files. A sample file is presented as Pro- gram 1. Each simulator is defined as a service in the deployment file. With some data sources, the data is retrieved by RTAT rather than sent by the simulator. To use these types of simulators, container ports must be mapped to host machine ports to allow incoming connections from RTAT (see simulator-b_instance-1 in Program 1). The simulators access the records on the host machine via bind mounts that are defined using the keyword volume. A bind mount is made for each simulator to access the records and configuration needed for simulation.

The controller is a simple service that redirects the HTTP requests it receives to the simulator containers. The dataset directory is bind mounted inside the controller, giving it access to the Docker Compose file. The controller can use the service names in the file to redirect the requests, as Docker Compose creates a network for the services where each service can be accessed via the name of the service instead of an IP ad- dress.

(27)

Program 1. An example of a Docker Compose file used to deploy NST.

The architecture of the current system can be seen in Figure 8. The architecture is designed for running the simulators on a single Docker Engine and does not allow scaling the system across multiple servers as it is. The problems related to the current implementation and the solutions to them are discussed next.

docker-compose.yaml version: '3.0' services:

simulator-a_instance-1:

image: simulators/type-a volumes:

- ./simulator_a_instance_1:/records

simulator-a_instance-2:

image: simulators/type-a volumes:

- ./simulator_a_instance_2:/records

simulator-b_instance-1:

image: simulators/type-b ports:

- "2222:22" # Ports are mapped if RTAT initiates connections volumes:

- ./simulator_b_instance_1:/records

controller:

image: simulators/controller ports:

- "5000:5000” # This port is used to control the simulators volumes:

- .:/dataset

(28)

Figure 8. Architecture of the current system.

4.3 Problems with the Current Architecture

Until now, the testing of RTAT with NST has been closer to functional testing than to performance testing. It works well with low loads and a low number of simulated data sources but cannot handle the numbers of elements and throughput needed for actual performance or stability testing. This is the main problem to be solved in this thesis. To generate higher loads and simulate a larger number of data sources, the simulation should be distributed across multiple host machines. As the simulators and the controller are Docker containers, they can be managed with a container orchestration tool to achieve this. However, to make the simulation scalable and robust, the architecture must be modified.

As there is a one-to-one correspondence between the simulated data sources and simulator containers, simulating networks of thousands of data sources would require thousands of containers. While it is possible to deploy and manage thousands of containers with a container orchestrator, the overhead can grow to consume a lot of resources. For example, some of the simulators are implemented in languages that require interpreters to run. Deploying thousands of simulator containers would mean running thousands of interpreters, which would in turn generate excess overhead. To fix this issue, the one-to-

(29)

one correspondence must be broken. This means that one simulator container should be able to simulate multiple data sources.

While the simulators adhere to many of the independency requirements for microservices, they are not stateless. To send the correct records at the right time, a simulator needs to know the start time of the simulation and which records it has already sent. As the state currently resides only in the memory of the container, it is very difficult to recover from crashes and take over tasks from other containers when scaling.

There is also a problem with how the data is shared in the system. Currently, the records and configurations are mounted to the simulator containers from the host machine with bind mounts. However, bind mounts can only be used to access the file system of the host on which the container is running. This works well when the simulation is executed on only one host, but simulators running on multiple hosts simply cannot access the data on another host with a bind mount.

The following subsection aims to solve the issues related to the current architecture by proposing a microservice architecture for NST.

4.4 Solving the Problems with Microservice Architecture

The problems of NST can be solved by designing the components according to microservice principles. The simulators should be stateless and share data in a manner that enables scaling. This can be achieved with stream processing. Apache Kafka is the most popular choice for providing streaming functionality in microservice applications [3], and it will be used in this thesis as well.

Apache Kafka – or just Kafka for short – is a publish/subscribe messaging system that works as a distributed streaming platform. In Kafka, messages are organized into topics and within topics into partitions. The messages are read with consumer clients, which can be organised into consumer groups. Members of the same group that consume a topic are assigned different partitions to ensure that each message is only processed once [51]. Kafka thus enables consuming messages with scalable data processors, or in the case of NST, reading records with scalable simulators.

Kafka’s functionality can be leveraged by writing the records for each data source type into separate topics and organising the records into partitions according to data source instance. The simulators can then be made scalable by equipping them with Kafka consumers and by organising them into consumer groups according to data source type. As the simulators start, they subscribe to the Kafka topics that contain the records for the correct data source type. Kafka then balances the partitions between the consumers. As

(30)

the simulators might receive records from multiple partitions, they should be able to start and stop multiple simulation processes accordingly. If the number of simulators of a specific type changes because of scaling or a failure, Kafka automatically rebalances the partitions between the simulators. The process is illustrated in Figure 9.

Figure 9. Enabling simulator scaling with Kafka.

To ensure that each message is sent exactly once, Kafka consumers save their state to Kafka. Each message of a partition has an offset, which the consumer can commit to Kafka to indicate that the message has been processed. Commits can also contain metadata describing the state of the consumer [51]. In the case of NST, each message is committed to Kafka as soon as the simulator has sent the record. As the simulator also needs the start time to calculate the times at which the records are sent, the start time is saved to Kafka as the commit metadata.

Messages are written into Kafka with producers. The records can be written into Kafka by equipping the controller with a producer and allocating partitions for the records manually based on data source instance [51]. The controller can read the datasets from a network storage. Once a simulation starts, the controller reads the records from the network storage and writes them into Kafka. Simulator configurations can be requested from the controller via HTTP. Once a simulator starts consuming a partition, it sends an HTTP request to the controller asking for the configuration for that partition. The controller then reads the configuration from the network storage and sends it to the simulator. The controller can also be requested to remove the records from Kafka when necessary.

(31)

A Kafka setup consists of Kafka servers called brokers, which operate as a part of a Kafka cluster [51]. There are containerised versions of Kafka, which enable running a Kafka cluster on Docker Engine. Furthermore, there are Kafka solutions such as the Confluent Platform that can be deployed with a container orchestrator [52].

With the above modifications, the simulators can be made stateless. The system itself still has a state, which consists of the dataset that is being simulated as well as of the stage of the simulation. Examples of simulation stages are records written into Kafka, simulation started, and simulation stopped. A Kafka topic can be used to handle the state. The state of the system is written into the topic every time it changes and can be accessed by reading the last message in the topic. The state is managed by the controller, which changes it according to user requests. The simulators can subscribe to the state topic and thus get notified for start and stop requests. When a new simulator instance starts, it can read the last message in the topic to know what action to take. The architecture of the distributable and scalable system can be seen in Figure 10.

Figure 10. The architecture of the distributable and scalable system.

The new architecture was tested by implementing new versions of the controller and one of the simulators. Preliminary tests show that the simulation works as expected: when the number of container replicas is changed, the simulator processes in the containers start and stop to keep the number of simulated data sources constant. This way, the

(32)

simulated network remains unchanged from RTAT’s perspective even when NST must reallocate resources. In addition, all records are delivered correctly, although rebalances cause a slight delay in sending the records that should have been sent during the re- balance.

4.5 Evaluation of the Orchestrators

Container orchestrators, such as Docker Swarm and Kubernetes, can be used to deploy and manage the components of containerised systems across multiple servers. The container orchestrator should be chosen based on the application to be orchestrated, con- sidering both functional and non-functional requirements. Functional requirements can be split into the same three categories as in the orchestrator comparison: resource management, scheduling, and service management requirements. Non-functional requirements include performance, usability, maintainability, and complexity concerns. Another important factor to consider is the amount of work needed to migrate from the current solution to the orchestrated one. This section presents the requirements NST sets for a container orchestrator and evaluates the orchestrators based on the requirements.

NST requires specific functionality in the resource management and scheduling layers.

First, in the resource management layer, NST requires access to computational resources, networking, and volumes. For the controller to be able to access the datasets, the orchestrator should allow the connection of a volume to an external filesystem. For the services to be able to communicate with each other, the orchestrator should allow easy networking within the cluster. In addition, the orchestrator should allow easy networking with machines outside the cluster so that the user is able to control the simulation and the simulators are able to connect to RTAT. Secondly, NST should also be stable to ensure the generation of a constant load for testing the stability of RTAT. Thus, as a scheduling layer requirement, the orchestrator should be able to restart containers when necessary. The orchestrator should also allow the user to scale the services in the cluster. As can be seen in Table 1, all these features are supported by both orchestrators.

Kubernetes offers connectivity to a wider variety of external volumes, but both orchestrators can be used with NFS, which is enough to fulfil the requirements of NST. In terms of these features, either one of the orchestrators can be chosen.

The performance of NST is a major concern, but only to a certain extent. While both Kubernetes and Docker Swarm provide sufficient performance for NST, Docker Swarm is more efficient overall. In the case of NST, the ability to recover from failures is important, but neither the container failover time nor the node failover time is emphasised

Automatically Scaling a System Across Multiple Servers : A Comparison of Docker Swarm and Kubernetes

Petteri Strömberg