Sunday, February 28, 2021

How To Remove Docker Images, Containers, and Volumes

 

A Docker Cheat Sheet

Introduction

Docker makes it easy to wrap your applications and services in containers so you can run them anywhere. As you work with Docker, however, it’s also easy to accumulate an excessive number of unused images, containers, and data volumes that clutter the output and consume disk space.

Docker gives you all the tools you need to clean up your system from the command line. This cheat sheet-style guide provides a quick reference to commands that are useful for freeing disk space and keeping your system organized by removing unused Docker images, containers, and volumes.

How to Use This Guide:

  • This guide is in cheat sheet format with self-contained command-line snippets
  • Jump to any section that is relevant to the task you are trying to complete.

The command substitution syntax, command $(command), used in the commands is available in many popular shells such as bash, zsh, and Windows Powershell.

Purging All Unused or Dangling Images, Containers, Volumes, and Networks

Docker provides a single command that will clean up any resources — images, containers, volumes, and networks — that are dangling (not associated with a container):

  • docker system prune
 

To additionally remove any stopped containers and all unused images (not just dangling images), add the -a flag to the command:

  • docker system prune -a
 

Removing Docker Images

Remove one or more specific images

Use the docker images command with the -a flag to locate the ID of the images you want to remove. This will show you every image, including intermediate image layers. When you’ve located the images you want to delete, you can pass their ID or tag to docker rmi:

List:

  • docker images -a
 

Remove:

  • docker rmi Image Image
 

Remove dangling images

Docker images consist of multiple layers. Dangling images are layers that have no relationship to any tagged images. They no longer serve a purpose and consume disk space. They can be located by adding the filter flag, -f with a value of dangling=true to the docker images command. When you’re sure you want to delete them, you can use the docker image prune command:

Note: If you build an image without tagging it, the image will appear on the list of dangling images because it has no association with a tagged image. You can avoid this situation by providing a tag when you build, and you can retroactively tag an images with the docker tag command.

List:

  • docker images -f dangling=true
 

Remove:

  • docker image prune
 

Removing images according to a pattern

You can find all the images that match a pattern using a combination of docker images and grep. Once you’re satisfied, you can delete them by using awk to pass the IDs to docker rmi. Note that these utilities are not supplied by Docker and are not necessarily available on all systems:

List:

  • docker images -a | grep "pattern"
 

Remove:

  • docker images -a | grep "pattern" | awk '{print $3}' | xargs docker rmi
 

Remove all images

All the Docker images on a system can be listed by adding -a to the docker images command. Once you’re sure you want to delete them all, you can add the -q flag to pass the Image ID to docker rmi:

List:

  • docker images -a
 

Remove:

  • docker rmi $(docker images -a -q)
 

Removing Containers

Remove one or more specific containers

Use the docker ps command with the -a flag to locate the name or ID of the containers you want to remove:

List:

  • docker ps -a
 

Remove:

  • docker rm ID_or_Name ID_or_Name
 

Remove a container upon exit

If you know when you’re creating a container that you won’t want to keep it around once you’re done, you can run docker run --rm to automatically delete it when it exits.

Run and Remove:

  • docker run --rm image_name
 

Remove all exited containers

You can locate containers using docker ps -a and filter them by their status: created, restarting, running, paused, or exited. To review the list of exited containers, use the -f flag to filter based on status. When you’ve verified you want to remove those containers, using -q to pass the IDs to the docker rm command.

List:

  • docker ps -a -f status=exited
 

Remove:

  • docker rm $(docker ps -a -f status=exited -q)
 

Remove containers using more than one filter

Docker filters can be combined by repeating the filter flag with an additional value. This results in a list of containers that meet either condition. For example, if you want to delete all containers marked as either Created (a state which can result when you run a container with an invalid command) or Exited, you can use two filters:

List:

  • docker ps -a -f status=exited -f status=created
 

Remove:

  • docker rm $(docker ps -a -f status=exited -f status=created -q)
 

Remove containers according to a pattern

You can find all the containers that match a pattern using a combination of docker ps and grep. When you’re satisfied that you have the list you want to delete, you can use awk and xargs to supply the ID to docker rm. Note that these utilities are not supplied by Docker and not necessarily available on all systems:

List:

  • docker ps -a | grep "pattern
 

Remove:

  • docker ps -a | grep "pattern" | awk '{print $1}' | xargs docker rm
 

Stop and remove all containers

You can review the containers on your system with docker ps. Adding the -a flag will show all containers. When you’re sure you want to delete them, you can add the -q flag to supply the IDs to the docker stop and docker rm commands:

List:

  • docker ps -a
 

Remove:

  • docker stop $(docker ps -a -q)
  • docker rm $(docker ps -a -q)
 

Removing Volumes

Remove one or more specific volumes - Docker 1.9 and later

Use the docker volume ls command to locate the volume name or names you wish to delete. Then you can remove one or more volumes with the docker volume rm command:

List:

  • docker volume ls
 

Remove:

  • docker volume rm volume_name volume_name
 

Remove dangling volumes - Docker 1.9 and later

Since the point of volumes is to exist independent from containers, when a container is removed, a volume is not automatically removed at the same time. When a volume exists and is no longer connected to any containers, it’s called a dangling volume. To locate them to confirm you want to remove them, you can use the docker volume ls command with a filter to limit the results to dangling volumes. When you’re satisfied with the list, you can remove them all with docker volume prune:

List:

  • docker volume ls -f dangling=true
 

Remove:

  • docker volume prune
 

Remove a container and its volume

If you created an unnamed volume, it can be deleted at the same time as the container with the -v flag. Note that this only works with unnamed volumes. When the container is successfully removed, its ID is displayed. Note that no reference is made to the removal of the volume. If it is unnamed, it is silently removed from the system. If it is named, it silently stays present.

Remove:

  • docker rm -v container_name
 

Conclusion

This guide covers some of the common commands used to remove images, containers, and volumes with Docker. There are many other combinations and flags that can be used with each. For a comprehensive guide to what’s available, see the Docker documentation for docker system prunedocker rmidocker rm and docker volume rm. If there are common cleanup tasks you’d like to see in the guide, please ask or make suggestions in the comments.

Saturday, November 7, 2020

Dockerizing a Node.js web app

 

Create the Node.js app

First, create a new directory where all the files would live. In this directory create a package.json file that describes your app and its dependencies:

{
  "name": "nodejs-docker-webapp",
  "version": "1.0.0",
  "description": "Node.js on Docker",
  "author": "First Last <first.last@example.com>",
  "main": "server.js",
  "scripts": {
    "start": "node server.js"
  },
  "dependencies": {
    "express": "^4.16.1"
  }
}

With your new package.json file, run npm install. If you are using npm version 5 or later, this will generate a package-lock.json file which will be copied to your Docker image.

Then, create a server.js file that defines a web app using the Express.js framework:

'use strict';

const express = require('express');

// Constants
const PORT = 8080;
const HOST = '0.0.0.0';

// App
const app = express();
app.get('/', (req, res) => {
  res.send('Hello World');
});

app.listen(PORT, HOST);
console.log(`Running on http://${HOST}:${PORT}`);

In the next steps, we'll look at how you can run this app inside a Docker container using the official Docker image. First, you'll need to build a Docker image of your app.

Creating a Dockerfile

Create an empty file called Dockerfile:

touch Dockerfile

Open the Dockerfile in your favorite text editor

The first thing we need to do is define from what image we want to build from. Here we will use the latest LTS (long term support) version 12 of node available from the Docker Hub:

FROM node:12

Next we create a directory to hold the application code inside the image, this will be the working directory for your application:

# Create app directory
WORKDIR /usr/src/app

This image comes with Node.js and NPM already installed so the next thing we need to do is to install your app dependencies using the npm binary. Please note that if you are using npm version 4 or earlier a package-lock.json file will not be generated.

# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm@5+)
COPY package*.json ./

RUN npm install
# If you are building your code for production
# RUN npm ci --only=production

Note that, rather than copying the entire working directory, we are only copying the package.json file. This allows us to take advantage of cached Docker layers. bitJudo has a good explanation of this here. Furthermore, the npm ci command, specified in the comments, helps provide faster, reliable, reproducible builds for production environments. You can read more about this here.

To bundle your app's source code inside the Docker image, use the COPY instruction:

# Bundle app source
COPY . .

Your app binds to port 8080 so you'll use the EXPOSE instruction to have it mapped by the docker daemon:

EXPOSE 8080

Last but not least, define the command to run your app using CMD which defines your runtime. Here we will use node server.js to start your server:

CMD [ "node", "server.js" ]

Your Dockerfile should now look like this:

FROM node:12

# Create app directory
WORKDIR /usr/src/app

# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm@5+)
COPY package*.json ./

RUN npm install
# If you are building your code for production
# RUN npm ci --only=production

# Bundle app source
COPY . .

EXPOSE 8080
CMD [ "node", "server.js" ]

.dockerignore file

Create a .dockerignore file in the same directory as your Dockerfile with following content:

node_modules
npm-debug.log

This will prevent your local modules and debug logs from being copied onto your Docker image and possibly overwriting modules installed within your image.

Building your image

Go to the directory that has your Dockerfile and run the following command to build the Docker image. The -t flag lets you tag your image so it's easier to find later using the docker images command:

docker build -t nodejs-docker-webapp .

Your image will now be listed by Docker:

$ docker images

# Example
REPOSITORY                      TAG        ID              CREATED
node                            12         1934b0b038d1    5 days ago
<your username>/node-web-app    latest     d64d3505b0d2    1 minute ago

Run the image

Running your image with -d runs the container in detached mode, leaving the container running in the background. The -p flag redirects a public port to a private port inside the container. Run the image you previously built:

docker run -p 49160:8080 -d <your username>/node-web-app

Print the output of your app:

# Get container ID
$ docker ps

# Print app output
$ docker logs <container id>

# Example
Running on http://localhost:8080

If you need to go inside the container you can use the exec command:

# Enter the container
$ docker exec -it <container id> /bin/bash

Test

To test your app, get the port of your app that Docker mapped:

$ docker ps

# Example
ID            IMAGE                                COMMAND    ...   PORTS
ecce33b30ebf  <your username>/node-web-app:latest  npm start  ...   49160->8080

In the example above, Docker mapped the 8080 port inside of the container to the port 49160 on your machine.

Now you can call your app using curl (install if needed via: sudo apt-get install curl):

$ curl -i localhost:49160

HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: text/html; charset=utf-8
Content-Length: 12
ETag: W/"c-M6tWOb/Y57lesdjQuHeB1P/qTV0"
Date: Mon, 13 Nov 2017 20:53:59 GMT
Connection: keep-alive

Hello world

We hope this tutorial helped you get up and running a simple Node.js application on Docker.

Tuesday, November 3, 2020

Quick start: Share images on Docker Hub

 

Set up your Docker Hub account

If you don’t have a Docker ID, follow these steps to create one. A Docker ID allows you to share images on Docker Hub.

  1. Visit the Docker Hub sign up page.

  2. Fill out the form and submit to create your Docker ID.

  3. Verify your email address to complete the registration process.

  4. Click on the Docker icon in your toolbar or system tray, and click Sign in / Create Docker ID.

  5. Fill in your new Docker ID and password. After you have successfully authenticated, your Docker ID appears in the Docker Desktop menu in place of the ‘Sign in’ option you just used.

    You can also sign into Docker Hub from the command line by typing docker login.

Create a Docker Hub repository and push your image

Before creating a repository, ensure you’ve set up your Docker Hub account and have connected it to your Docker Desktop.

Now let’s create your first repository, and push your bulletin board image to Docker Hub.

  1. Click on the Docker icon in your menu bar, and navigate to Repositories > Create. You’ll be redirected to the Create Repository page on Docker Hub.

  2. Type the repository name as bulletinboard and click Create at the bottom of the page. Do not fill any other details for now.

    make a repo

  3. You are now ready to share your image on Docker Hub, however, there’s one thing you must do first: images must be namespaced correctly to share on Docker Hub. Specifically, you must name images like <Your Docker ID>/<Repository Name>:<tag>.

    Make sure you’re in the node-bulletin-board/bulletin-board-app directory in a terminal or PowerShell then and run:

    docker tag bulletinboard:1.0 <Your Docker ID>/bulletinboard:1.0
    
  4. Finally, push your image to Docker Hub:

    docker push <Your Docker ID>/bulletinboard:1.0
    

    Visit your repository in Docker Hub, and you’ll see your new image there. Remember, Docker Hub repositories are public by default.

    Having trouble pushing? Remember, you must be signed into Docker Hub through Docker Desktop or the command line, and you must also name your images correctly, as per the above steps. If the push seemed to work, but you don’t see it in Docker Hub, refresh your browser after a couple of minutes and check again.

Conclusion

Now that your image is available on Docker Hub, you’ll be able to run it anywhere. If you try to use it on a new machine that doesn’t have it yet, Docker will automatically try and download it from Docker Hub. By moving images around in this way, you no longer need to install any dependencies except Docker on the machines you want to run your software on. The dependencies of containerized applications are completely encapsulated and isolated within your images, which you can share using Docker Hub as described above.

Another thing to keep in mind: at the moment, you’ve only pushed your image to Docker Hub; what about your Dockerfile? A crucial best practice is to keep these in version control, perhaps alongside your source code for your application. You can add a link or note in your Docker Hub repository description indicating where these files can be found, preserving the record not only of how your image was built, but how it’s meant to be run as a full application.

Quick start: Build and run your image

Set up

Let us download the node-bulletin-board example project. This is a simple bulletin board application written in Node.js.

git clone https://github.com/dockersamples/node-bulletin-board
cd node-bulletin-board/bulletin-board-app

Define a container with Dockerfile

After downloading the project, take a look at the file called Dockerfile in the bulletin board application. Dockerfiles describe how to assemble a private filesystem for a container, and can also contain some metadata describing how to run a container based on this image.

For more information about the Dockerfile used in the bulletin board application, see Sample Dockerfile.

Build and test your image

Now that you have some source code and a Dockerfile, it’s time to build your first image, and make sure the containers launched from it work as expected.

Make sure you’re in the directory node-bulletin-board/bulletin-board-app in a terminal or PowerShell using the cd command. Run the following command to build your bulletin board image:

docker build --tag bulletinboard:1.0 .

You’ll see Docker step through each instruction in your Dockerfile, building up your image as it goes. If successful, the build process should end with a message Successfully tagged bulletinboard:1.0.

Run your image as a container

  1. Run the following command to start a container based on your new image:

    docker run --publish 8000:8080 --detach --name bb bulletinboard:1.0
    

    There are a couple of common flags here:

    • --publish asks Docker to forward traffic incoming on the host’s port 8000 to the container’s port 8080. Containers have their own private set of ports, so if you want to reach one from the network, you have to forward traffic to it in this way. Otherwise, firewall rules will prevent all network traffic from reaching your container, as a default security posture.
    • --detach asks Docker to run this container in the background.
    • --name specifies a name with which you can refer to your container in subsequent commands, in this case bb.
  2. Visit your application in a browser at localhost:8000. You should see your bulletin board application up and running. At this step, you would normally do everything you could to ensure your container works the way you expected; now would be the time to run unit tests, for example.

  3. Once you’re satisfied that your bulletin board container works correctly, you can delete it:

    docker rm --force bb
    

    The --force option stops a running container, so it can be removed. If you stop the container running with docker stop bb first, then you do not need to use --force to remove it.

Sample Dockerfile

Writing a Dockerfile is the first step to containerizing an application. You can think of these Dockerfile commands as a step-by-step recipe on how to build up your image. The Dockerfile in the bulletin board app looks like this:

# Use the official image as a parent image.
FROM node:current-slim

# Set the working directory.
WORKDIR /usr/src/app

# Copy the file from your host to your current location.
COPY package.json .

# Run the command inside your image filesystem.
RUN npm install

# Add metadata to the image to describe which port the container is listening on at runtime.
EXPOSE 8080

# Run the specified command within the container.
CMD [ "npm", "start" ]

# Copy the rest of your app's source code from your host to your image filesystem.
COPY . .

The dockerfile defined in this example takes the following steps:

  • Start FROM the pre-existing node:current-slim image. This is an official image, built by the node.js vendors and validated by Docker to be a high-quality image containing the Node.js Long Term Support (LTS) interpreter and basic dependencies.
  • Use WORKDIR to specify that all subsequent actions should be taken from the directory /usr/src/app in your image filesystem (never the host’s filesystem).
  • COPY the file package.json from your host to the present location (.) in your image (so in this case, to /usr/src/app/package.json)
  • RUN the command npm install inside your image filesystem (which will read package.json to determine your app’s node dependencies, and install them)
  • COPY in the rest of your app’s source code from your host to your image filesystem.

You can see that these are much the same steps you might have taken to set up and install your app on your host. However, capturing these as a Dockerfile allows you to do the same thing inside a portable, isolated Docker image.

The steps above built up the filesystem of our image, but there are other lines in your Dockerfile.

The CMD directive is the first example of specifying some metadata in your image that describes how to run a container based on this image. In this case, it’s saying that the containerized process that this image is meant to support is npm start.

The EXPOSE 8080 informs Docker that the container is listening on port 8080 at runtime.

What you see above is a good way to organize a simple Dockerfile; always start with a FROM command, follow it with the steps to build up your private filesystem, and conclude with any metadata specifications. There are many more Dockerfile directives than just the few you see above. For a complete list, see the Dockerfile reference.

Manage Docker as a non-root user

  The Docker daemon binds to a Unix socket instead of a TCP port. By default that Unix socket is owned by the user   root   and other users ...