An Overview of Docker tools
- Build, ship, publish, download, and run docker images.
- When we say “Docker”, we typically mean the Docker engine, the core of the docker platform. It consists of a docker daemon, a Rest API for interacting with the daemon, and a command line interface (CLI) that talks to the daemon through the Rest API.
- Docker Compose:
- Define and run multiple containers linked together on a single host.
- Useful for setting up development and testing workflows.
- Docker Machine:
- Tool for provisioning and managing docker hosts (virtual hosts running docker engine).
- It automatically creates hosts, installs Docker Engine on them, then configures the docker clients.
- You can use Machine to create Docker hosts on your local machine using a virtualization software like VirtualBox or VMWare Fusion.
- Docker machine also supports various cloud providers like AWS, Azure, Digital Ocean, Google Compute Engine, OpenStack, RackSpace etc.
- Docker Swarm:
- A swarm is a group of docker hosts linked together into a cluster.
- The swarm cluster consists of a swarm manager and a set of workers.
- You interact with the cluster by executing commands on the swarm manager.
- With swarm, you can deploy and scale your applications to multiple hosts.
- Swarm helps with managing, scaling, networking, service discovery, and load balancing between the nodes in the cluster.
- Docker Stack:
- Define and run multiple containers on a swarm cluster.
- Just like
docker-composehelps you define and run multi-container applications on a single host,
docker-stackhelps you define and run multi-container applications on a swarm cluster.
Creating docker hosts, initializing a swarm cluster, and deploying a real-world containerized app
Let’s do some hands-on to get acquainted with docker machine, swarm, and docker stack. In this article, we’ll learn how to create docker hosts on our local machine using
docker-machine, initialize a swarm cluster, and deploy a multi-container app on the cluster using
STEP 1: Prepare the Application that will be deployed to swarm cluster
We’ll be deploying a simple Golang App that contains a Rest API to display the “Quote of the day”. It fetches the quote from a public API hosted at
http://quotes.rest/ and caches the result in Redis so that it can return the result immediately from the cache when you hit the API next time.
You can find the application on the following Github repository.
Following is the dockerfile that is used to build the image of the Go app:
# Dockerfile References: https://docs.docker.com/engine/reference/builder/ # Start from golang:1.12-alpine base image FROM golang:1.12-alpine # The latest alpine images don't have some tools like (`git` and `bash`). # Adding git, bash and openssh to the image RUN apk update && apk upgrade && \ apk add --no-cache bash git openssh # Add Maintainer Info LABEL maintainer="Rajeev Singh <firstname.lastname@example.org>" # Set the Current Working Directory inside the container WORKDIR /app # Copy go mod and sum files COPY go.mod go.sum ./ # Download all dependencies. Dependencies will be cached if the go.mod and go.sum files are not changed RUN go mod download # Copy the source from the current directory to the Working Directory inside the container COPY . . # Build the Go app RUN go build -o main . # Expose port 8080 to the outside world EXPOSE 8080 # Run the executable CMD ["./main"]
You can build and publish the image to docker registry using the following commands -
# Build the image $ docker build -t go-docker-swarm . # Tag the image $ docker tag go-docker-swarm callicoder/swarm-demo-app:1.0.0 # Login to docker with your docker id $ docker login Login with your Docker ID to push and pull images from Docker Hub. If you don\'t have a Docker ID, head over to https://hub.docker.com to create one. Username (callicoder): callicoder Password: Login Succeeded # Push the image to docker hub $ docker push callicoder/swarm-demo-app:1.0.0
I’ve already built and published the docker image for our demo app to the docker hub at callicoder/swarm-demo-app:1.0.0
STEP 2: Define application’s services in a yaml file
The application consists of an API service and a Redis cache. We define all the application’s services in a
yaml file called
version: '3.7' # Define services services: # App Service app: # Configuration for building the docker image for the service image: callicoder/swarm-demo-app:1.0.0 ports: - "8080:8080" # Forward the exposed port 8080 on the container to port 8080 on the host machine deploy: replicas: 3 resources: limits: cpus: "0.2" memory: 50M restart_policy: condition: on-failure environment: # Pass environment variables to the service REDIS_URL: redis:6379 depends_on: - redis networks: # Networks to join (Services on the same network can communicate with each other using their name) - webnet # Redis Service redis: image: redis:alpine ports: - "6379:6379" networks: - webnet deploy: replicas: 1 restart_policy: condition: on-failure placement: constraints: [node.role == manager] networks: webnet: driver: overlay attachable: true
STEP 3: Create VMs using docker-machine
Let’s create docker hosts on our local machine using
$ docker-machine create --driver virtualbox swarm-vm1 $ docker-machine create --driver virtualbox swarm-vm2 $ docker-machine create --driver virtualbox swarm-vm3
virtualbox driver for creating the VMs. Make sure that you have virtualbox installed in your system.
docker-machine allows you to create docker VMs on various cloud providers as well like AWS, google cloud etc. For example, Here is how you can create a docker VM on AWS using docker-machine:
$ docker-machine create --driver amazonec2 \ --amazonec2-access-key <ACCESS_KEY> \ --amazonec2-secret-key <ACCESS_SECRET> \ --amazonec2-region "us-east-1" \ --amazonec2-instance-type "t2.micro" \ aws-sandbox
After creating all the VMs, you can list them using the following command -
$ docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS swarm-vm1 - virtualbox Running tcp://192.168.99.100:2376 v18.09.0 swarm-vm2 - virtualbox Running tcp://192.168.99.101:2376 v18.09.0 swarm-vm3 - virtualbox Running tcp://192.168.99.102:2376 v18.09.0
STEP 4: Create a Swarm cluster
Initialize a Swarm cluster
Let’s initialize a swarm cluster by executing
docker swarm init command on the first VM.
$ docker-machine ssh swarm-vm1 "docker swarm init --advertise-addr 192.168.99.100" Swarm initialized: current node (j5obt23bbdcvphmgncd97q5r6) is now a manager. To add a worker to this swarm, run the following command: docker swarm join --token SWMTKN-1-073v8uw449vvkwngz6g0mtgivv50dbbqzt2o9f9jmwvwrkihoj-6glz90svgl4t21z1nnio0oce2 192.168.99.100:2377 To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
The first machine which initializes a swarm becomes the swarm manager. Other VMs can join the cluster using
docker swarm join command.
Add Workers to the Swarm cluster
Let’s add other two VMs as workers to the swarm cluster -
$ docker-machine ssh swarm-vm2 "docker swarm join --token SWMTKN-1-073v8uw449vvkwngz6g0mtgivv50dbbqzt2o9f9jmwvwrkihoj-6glz90svgl4t21z1nnio0oce2 192.168.99.100:2377" This node joined a swarm as a worker. $ docker-machine ssh swarm-vm3 "docker swarm join --token SWMTKN-1-073v8uw449vvkwngz6g0mtgivv50dbbqzt2o9f9jmwvwrkihoj-6glz90svgl4t21z1nnio0oce2 192.168.99.100:2377" This node joined a swarm as a worker.
List all nodes in the swarm cluster
You can now list all the nodes that are part of the cluster by executing
docker node ls command on the swarm manager -
$ docker-machine ssh swarm-vm1 "docker node ls" ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION j5obt23bbdcvphmgncd97q5r6 * swarm-vm1 Ready Active Leader 18.09.0 yb7bab6suhxsfuqlwivi090ir swarm-vm2 Ready Active 18.09.0 tqmisvt9nu8v8o7yn26rh9cox swarm-vm3 Ready Active 18.09.0
Configure the current shell to talk to the docker daemon on the swarm manager
So far, we’ve been using
docker-machine ssh to execute commands on the swarm manager. Instead of doing that every time, you can also configure your current shell to talk to the docker daemon on the swarm manager directly -
$ docker-machine env swarm-vm1 export DOCKER_TLS_VERIFY="1" export DOCKER_HOST="tcp://192.168.99.100:2376" export DOCKER_CERT_PATH="/Users/callicoder/.docker/machine/machines/swarm-vm1" export DOCKER_MACHINE_NAME="swarm-vm1" # Run this command to configure your shell: # eval $(docker-machine env swarm-vm1)
$ eval $(docker-machine env swarm-vm1)
That’s it. Your current shell is now configured to talk to the swarm manager’s docker daemon directly. You can run
docker-machine ls command to verify that
swarm-vm1 is the currently active VM -
$ docker-machine ls NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS swarm-vm1 * virtualbox Running tcp://192.168.99.100:2376 v18.09.0 swarm-vm2 - virtualbox Running tcp://192.168.99.101:2376 v18.09.0 swarm-vm3 - virtualbox Running tcp://192.168.99.102:2376 v18.09.0
The asterisk in the
ACTIVE column indicates that
swarm-vm1 is the active machine. You can also get the currently active machine like this -
$ docker-machine active swarm-vm1
STEP 4: Deploy the app on the Swarm cluster using docker stack
Finally, Let’s deploy the application to the swarm cluster using docker stack -
$ docker stack deploy -c docker-stack.yml swarm-stack Creating network swarm-stack_webnet Creating service swarm-stack_app Creating service swarm-stack_redis
That’s it, the application is now running. You can hit the API by typing the following command -
$ curl http://192.168.99.100:8080 2cf29ce35cee: Welcome! Please hit the `/qod` API to get the quote of the day.
$ curl http://192.168.99.100:8080/qod f689ae553c6b: If I work as hard as I can, I wonder how much I can do in a day?
List all the services in the stack
$ docker service ls ID NAME MODE REPLICAS IMAGE PORTS mtbekecn7dd2 swarm-stack_app replicated 3/3 callicoder/swarm-demo-app:1.0.0 *:8080->8080/tcp 09be7or5dr1o swarm-stack_redis replicated 1/1 redis:alpine *:6379->6379/tcp
Check out service logs
$ docker service logs swarm-stack_app swarm-stack_app.2.yi2ie8euinry@swarm-vm3 | 2019/02/03 06:59:13 Starting Server swarm-stack_app.2.yi2ie8euinry@swarm-vm3 | 2019/02/03 06:59:20 Cache miss for date 2019-02-03 swarm-stack_app.2.yi2ie8euinry@swarm-vm3 | 2019/02/03 06:59:21 Quote API Returned: 200 OK swarm-stack_app.2.yi2ie8euinry@swarm-vm3 | 2019/02/03 07:01:15 Cache Hit for date 2019-02-03 swarm-stack_app.2.yi2ie8euinry@swarm-vm3 | 2019/02/03 07:02:04 Cache Hit for date 2019-02-03 swarm-stack_app.2.yi2ie8euinry@swarm-vm3 | 2019/02/03 07:02:11 Cache Hit for date 2019-02-03
List all the tasks in the stack
Tasks are instances of services running in the stack. Our app service has three replicas, one on every VM. The redis server however is running only on the manager -
$ docker stack ps swarm-stack ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS nccbholn9pu8 swarm-stack_app.1 callicoder/swarm-demo-app:1.0.0 swarm-vm2 Running Running 7 minutes ago qrdt9ba3xrtk swarm-stack_redis.1 redis:alpine swarm-vm1 Running Running 7 minutes ago yi2ie8euinry swarm-stack_app.2 callicoder/swarm-demo-app:1.0.0 swarm-vm3 Running Running 7 minutes ago ovu9ngd15ep4 swarm-stack_app.3 callicoder/swarm-demo-app:1.0.0 swarm-vm1 Running Running 7 minutes ago
Tear down the stack
You can tear down the stack with the following command -
$ docker stack rm swarm-stack Removing service swarm-stack_app Removing service swarm-stack_redis Removing network swarm-stack_webnet
That’s all folks. You can find the complete code for this article on Github:
Check out the following guides on official docker website to learn more: