Automatic Scaling in Docker Swarm

Docker Swarm does not have a built-in functionality for scaling services automatically.

There is also no well-established third-party solution available, which can lead to devel-opers migrating to other orchestrators such as Kubernetes to achieve automatic scaling.

However, there exist some smaller projects and less-known solutions that can be used as a starting point for the solutions presented in this thesis.

Table 3 presents three existing solutions for automatic scaling in Docker Swarm demon-strated by Farcic [36], Kulkarni [37] and Rao [38]. The existing solutions work by collect-ing metrics and sendcollect-ing scalcollect-ing requests to Docker Engine based on the collected met-rics. The metrics are collected using Prometheus, which is an open-source monitoring solution that is used to collect, store, and visualize metrics [54]. Scaling requests are sent to Docker Engine in three different ways: with Jenkins, Orbiter, or a custom program utilising Docker API. Jenkins is an open-source automation server intended for automa-tion tasks [55], and Orbiter is a third-party Golang program that is used to scale services in a swarm via HTTP requests [56]. Jenkins and Orbiter implementations require scaling to be triggered by HTTP requests, while custom programs can monitor Prometheus di-rectly. HTTP triggers are sent with Prometheus’ Alert Manager companion, which is con-figured to monitor Prometheus alerts that are created when the set metric thresholds are reached.

Table 3. The characteristics of the three existing autoscaling solutions for Docker Swarm.

Solution by Farcic [36] Kulkarni [37] Rao [38]

Metric service Prometheus Prometheus Prometheus Metric used Response time # of HTTP requests # of HTTP requests

Trigger Prometheus alert Prometheus alert PromQL API

Scaling logic Up/Down Up/Down Up/Down

Scaling service Jenkins Orbiter Docker API

While the existing solutions certainly work for many use-cases, the functionality they offer does not directly meet the requirements of NST. First, they use HTTP request metrics to scale the services, but the target system should be scaled based on its computing re-source usage. This is only a minor problem since the solutions support other metrics that can be provided through Prometheus. Another problem is that the solutions only offer scaling Up/Down but not calculating the number of replicas needed directly. This can lead to the number of replicas lagging the load when the desired number of replicas differs from the current number of replicas by more than the scaling factor.

The following subsections propose two autoscaling solutions to scale services in a swarm based on CPU utilisation. The first one is based on the solution demonstrated by Kulkarni [37] and the second one uses a custom program similar to the solution by Rao [38].

5.1.1 Prometheus Alerts and Orbiter

To scale a service based on CPU utilisation, container resource usage metrics must be exposed to Prometheus. Container metrics are exposed with cAdvisor. It is a component developed by Google for the exact purpose of giving users easy access to insights on container resource usage. It supports Docker containers natively and is run as a Docker container [57]. Most importantly, cAdvisor supports exposing metrics to Prometheus na-tively [58], which means that Prometheus can be configured to collect – or scrape, in Prometheus’ terms – metrics from cAdvisor services that run on each swarm node. Once the metrics are in Prometheus, they can be retrieved with Prometheus’ own query lan-guage PromQL, which is used to select and aggregate metrics in real-time [59].

The current CPU utilisation level of each container is monitored as the rate of the cAdvi-sor metric container_cpu_usage_seconds_total. The metric is further aggregated with PromQL to represent the average CPU utilisation level of the container instances of a

service. Prometheus is then configured to monitor this metric and create alerts when user-defined thresholds are reached. The alerts are then handled by Alert Manager, which is set to send HTTP requests to Orbiter to scale the services.

An overview of the solution is presented in Figure 11 and the implementation details are described in appendix A.

Figure 11. Autoscaling in Docker Swarm using the existing third-party components.

5.1.2 Custom Autoscaler

To scale the services based on desired replicas and thus solve the issue of replicas potentially lagging the load, a custom autoscaling solution must be implemented. The custom solution consists of four parts: exposing container metrics, collecting them, cal-culating the desired number or replicas, and scaling the services accordingly. The con-tainer metrics are exposed with cAdvisor as in the previous solution.

The custom autoscaling solution requires the implementation of a custom component which is from now on referred to as Autoscaler. Autoscaler essentially combines the functionalities of Alert Manager and Orbiter in the previous solution by monitoring

Pro-metheus directly via the PromQL API and scaling services via the Docker API. This al-lows for the calculation of the number of desired replicas with the following equation, which is also used by the autoscaling solution in Kubernetes [39]:

𝑅_{𝑑𝑒𝑠𝑖𝑟𝑒𝑑} = ⌈𝑅_{𝑐𝑢𝑟𝑟𝑒𝑛𝑡}∙^{𝑀𝑐𝑢𝑟𝑟𝑒𝑛𝑡}

𝑀_{𝑑𝑒𝑠𝑖𝑟𝑒𝑑}⌉, (1)

where Rdesired denotes the desired number of replicas, Rcurrent is the current number of replicas, Mcurrent means current metric value and Mdesired means desired metric value. The current number of replicas can be retrieved via Docker API on the manager node. The current metric value comes from Prometheus. The desired metric value is defined by the user and thus offers a way to control the scaling behaviour.

An overview of the solution is presented in Figure 12. The implementation details are described in appendix B. The approach was tested by implementing a proof-of-concept Autoscaler in Golang by extending the Orbiter project. Preliminary tests show that the solution works as intended: Autoscaler can detect the services that the user wants to scale automatically, retrieve metrics from Prometheus, calculate the number of desired replicas and scale the services accordingly.

Figure 12. Autoscaling in Docker Swarm using a custom Autoscaler.