Introduction
We started our docker security journey with Docker basics like Docker images, Dockerfile and Docker Registry, we are now laying the groundwork for attacking the docker containers.
The reconnaissance aka Information gathering is the first step of any security assessment. Once you get a shell; you, as a pentester or red teamer, your aim should be to learn about the victim as much as possible so you can do lateral movement and privilege escalation.
For exactly this reason, you will need to know how to move around in the docker ecosystem for security misconfigurations, hardcoded credentials, and other interesting information.
In this lesson, we are going to cover the reconnaissance techniques which includes
- Starting and stopping of a container.
- Interacting with a container.
- Mounting container file system and volume mount.
- Connecting a container with the host’s network.
- Log storage mechanism.
So without further ado, let’s get started.
Docker Containers
We already know that a running form of an image is a container and container helps in packaging the application and its dependency into a single executable.
Imagine, we compromised a host and we want to do some information gathering. You might not know them now but by the end of this lesson, you will be well versed in the art of Docker information gathering.
Tip: As a pentester, your job is to ensure you are doing a security assessment without taking systems/machines offline. So, it’s important to understand what starts, stops or crashes a docker container.
If you are our paid customer and taking any of our courses like CDE/CDP/CCSE/CCNSE, please use the browser based lab portal to do all the exercises and do not use the below Virtual Machine.
Let’s try to start/create a alpine container using the following command.
$ docker run alpine
But there is no output for the above command. Let’s see what’s going on under the hood (sherlock mode on 😎)
Docker provides a nifty tool called ps to list running containers.
$ docker ps
Interesting! it’s not running anymore. The docker ps command also has -a option to show the stopped containers.
You can see that our alpine container started, the shell executed and then exited.
Note: The docker ps is probably the first command you run once you know the host (ps aux | grep dockerd) is using docker.
This is bad news for us, why?
One, we didn’t get time to analyze the container for security issues, the container was short-lived. Two, there are many alpine containers in the above output. Which one is ours? and how do we ensure, we are targeting the right container (without affecting other containers)?
In short, we have two problems to solve before we can analyze this image.
- Identify the right container.
- Stop a container from exiting.
Identifying the right container
We can use the following techniques to identify the right container for us to play with.
- Identify a container using its creation time.
- Identify a container using container id.
- Identify a container by container name.
Identify a container using its creation time
In this technique, we will run alpine image once again and assume the latest entry in it would be ours.
Scientific technique? probably not as this machine might be a production machine running multiple container instances of the same image. Helps in a pinch? definitely!
Identify a container using container id
By default, the docker container tries to attach to your terminal and shows you the container output in the terminal. Since we are not providing any options to alpine image’s CMD i.e, /bin/sh, it just exits. You can get this container id by using -d option which detaches it from the terminal (STDIN, STDERR, and STDOUT).
$ docker run -d alpine
Identify a container by container name
This technique is not always possible as the application might be already running. But if you are doing an internal assessment then maybe you can stop an existing container and spin up a new one with a new name.
Conveniently docker allows us to name containers using the –name argument.
$ docker run --name myalpine alpine
As you can see the docker container is not running but we know which alpine container is ours i.e, myalpine. It’s also a best practice to name the containers wherever possible.
We have discussed a few of the techniques to find out the right containers to target, let’s move on to analyzing the containers by either pausing/stopping.
Analyzing a container.
- Block the container from exiting by using interactive commands.
- Block the container from exiting by using an infinite loop etc.,
- Pause the container temporarily.
Block the container from exiting by using interactive commands.
The alpine image is created using the following Dockerfile.
FROM scratch ADD alpine-minirootfs-3.10.2-x86_64.tar.gz / CMD ["/bin/sh"]
As you can see line 3, the alpine container doesn’t do much, just runs /bin/sh and exits but we can run some interactive commands inside a container to make it wait for user input (like sh, shell).
You can use -i and -t options to achieve this.
$ docker run -it alpine /bin/sh
-i stands for interactive mode
-t stands for tty (terminal)
Open another terminal/tab and type “docker ps” command, you will see that the container is up and running and we can proceed to analyze it.
Block the container from exiting by using an infinite loop etc.,
Another technique to stop a container from exiting immediately is to run it in an infinite loop using shell commands.
$ docker run -d -it alpine sh -c "while true; do sleep 2; done"
option –detach or -d, means that a Docker container runs in the background of your terminal. It does not receive input or display output.
We can see that our docker container (with id 2cc32e7c0178) is running in the detached mode
Now, if you want to log into a container (for lack of a better word), you can use docker exec command to run any arbitrary commands inside a container like downloading a kernel exploit, installing compilers, security tools etc.,
$ docker exec -it 2cc32e7c0178 sh
Game Over?
Pause the container temporarily.
Sometimes, you just want to stop a container temporarily(pause) to see what happens in the network or on the file system.
Let’s start another alpine container with an infinite loop with date echo and see how it behaves with pausing and unpausing.
$ docker run --name your-alpine -it alpine sh -c "while true; do sleep 3; date; done"
As you can see it outputs the current date every 3 seconds.
Open another terminal and run the following command.
$ docker pause your-alpine
Go back to the first terminal and see the container is no longer showing date. Let’s unpause it using the below command.
$ docker unpause your-alpine
Don’t forget to stop the docker container by either using Ctrl+C or docker stop command
$ docker stop your-alpine
Note: You can usually use container id wherever you can use a name and vice versa.
e.g. docker pause 357de0241336 == docker pause your-alpine
Now that you know how to interact with Docker containers, let’s explore some interesting docker features which are especially relevant to docker security/reconnaissance.
- Docker volumes
- Docker port forwarding
- Docker environment variables
- Docker inspect
- Docker top
- Docker stats
Docker volumes
By default, when you stop and remove a container, all the changes made inside the container are lost. We can use the docker’s volume mounting feature to store data even after container stops or is destroyed.
Pro tip: Unlike VMs, docker volumes are not well guarded and are a good source for sensitive information like passwords, backups, and configurations.
Let’s create an alpine container in the interactive mode and see how docker volume works.
$ docker run -it --rm alpine sh
Here –rm flag tells the daemon to remove/destroy the container once it finishes its execution.
Let’s create a file called top-secret under /tmp directory and see if docker saves this file on the host.
We came out of the container by typing exit in the container shell and unfortunately, we have lost our top-secret file because we haven’t implemented the docker volume feature yet.
Let’s start implementing it by creating a folder in the host-system called secret.
$ mkdir secret
Now, mount it in the docker container using the -v or –volume flag
$ docker run -it --rm -v $(pwd)/secret:/tmp alpine sh
Here the -v flag is used for volume mounting
The left-hand side points to the host volume and the right-hand side points to the container volume.
Let’s repeat the same commands for creating top-secret files under tmp directory and see if we managed to retain this file.
Now go to the secret folder in the host machine and check whether the topsecret file exists even after the termination of our alpine container.
Docker port forwarding.
By default, docker Containers run in an isolated environment, but if you want to expose the services running inside a docker container, we can use a docker feature known as port-forwarding.
Pro tip: If you wish to use TCP bind shell, you would need to use the port forwarding feature to expose that port on the host otherwise it won’t be reachable from outside.
Let’s run a simple Nginx server and use the port-forwarding feature to expose Nginx’s port 80 on the host machine’s port 80.
$ docker run --name my-server -d -p 80:80 nginx
Here -p flag is used for port mapping, the left-hand side indicates the host machine port and the right-side indicates the container port
If you are our paid customer and taking any of our courses like CDE/CDP/CCSE/CCNSE, please use the browser based lab portal to do all the exercises and do not use the below Virtual Machine.
Now, open a web browser and type localhost:80 in the address bar. You will be greeted with Nginx’s default page.
As you can see we have successfully exposed the docker’s isolated Nginx’s port 80 onto the host port 80.
Docker environment variables
Another feature which is notorious for leaking secrets is docker environment variables. If you have access to docker container, you can read all of its environment variables by typing env command.
Let’s see an example to bring this point home.
$ docker run -it --rm -e \ USERNAME=imran -e PASSWORD=imran alpine sh
-e option here exports variables inside the container.
As you can see, anyone with shell access can read sensitive information.
Pro tip: Ensure containers are not passing secrets via environment variables.
Docker inspect
Imagine, you don’t have access to the docker image but you would still want to glean some interesting information then docker inspect command is for you.
It shows the environment variables used, IP addresses, Network IDs, AppArmor settings, image names, the current state of the container, ports, volumes, etc.,
$ docker inspect <container-name>
The above command shows detailed configuration information about the container.
Docker top
Often, you will be interested in knowing the running processes inside a container. Docker provides the top command to glean this information.
Let’s try to start an alpine container and ping google inside the container shell.
$ docker top <container-id>
Task
Stop and remove the running Nginx container and run the below command in the terminal.
$ docker run --name my-server -d -P nginx
-P flag selects a random available port on the host and maps it to the container port.
Use the docker inspect on this container and find out on which port it is running.
Docker stats
Containers allow us to limit the resources they can utilize. The following command restricts the alpine container from using more than 100mb of RAM.
$ docker run --rm -it -m 100mb -d alpine sh
-m flag used to specify the amount of memory that can be allocated to the container.
Once a container runs, you can see this information using docker stats.
$ docker stats container-id
As you can see that only 100mb of memory is allocated to the container.
Pro Tip: You can verify if a particular container is suspectable for a Denial of service attack using this command.
Reference and Further Reading
https://docs.docker.com/storage/storagedriver/select-storage-driver/
https://docs.docker.com/storage/volumes/
Conclusion
In this lesson, we have learned about reconnaissance/footprinting techniques, we first saw how to target the right container, how to analyze the containers and how to stop/pause those containers.
We also saw how to work with docker volumes and observed how they may contain some sensitive information, we then moved on to discussing port forwarding and exposed environmental variables.
In the next lesson, we will see Docker’s attack surface and how we can perform vulnerability assessment on docker ecosystem.
Hi Imran & Team,
Till now everything is easily understandable.
I am not able to import ova image. Tried to download 5times, same issue exists. Could you please let me know what are the softwares installed in the Ova image shared. I will manually install those in Kali / parrot OS.
Regards,
Kalpana
Please check our slack channel #devsecops, we will help you with the issue.
Excellent article, Thanks Imran 🙂
Thanks Mani 🙂
Can you add the links between the different lectures?
Thanks for your comment ML.
Unfortunately, we can’t as these lessons will go away after we figure out the right place to put them. May be on our courses.practical-devsecops.com page.
Also, you can find the appropriate links in your inbox when you sign up for the Docker Security course at https://www.practical-devsecops.com/pricing/
Hi Imran.
This is great course, as your talks. I was able to caught up with Lesson3 and I was looking for the Lesson 4 but couldn’t find. Have published Lesson 4 yet?
Hi Valfredo,
We had to delay for technical reasons. Please follow us on twitter to know about these announcements.
We made the announcement on our social at https://twitter.com/PDevsecops/status/1186640971708162048
Hi Imran,
For the command : “$ docker run -it –rm -e USERNAME=imran -e PASSWORD=imran alpine sh” what username & pwd we can give? Is it mandatory to give those in CMD?
Hi Sharath,
In the given command, using “-e” we are trying to insert some key value pair into environmental variables. These values can be any random strings(example: “-e test=test1”). Imran’s point was to convey that we should not store credentials in this way, if we store we can retrieve using env.
Thanks!
Great lesson Imran.. with detailed steps.
I would like to try it curiosity is about environment readiness with open source tools? If yes then in I can proceed with implementation.