Docker: Moving Dockerd Data Directory to Save System Space

Moving the Docker daemon data directory from /var/lib/docker to a larger partition requires stopping the service, updating the systemd configuration with the --data-root flag, and restarting. This process preserves existing images and containers while redirecting future storage usage to the new location, effectively reclaiming space on the root filesystem without data loss.

Why Move the Docker Data Directory?

By default, Docker stores all its persistent data, including images, containers, volumes, and networks, in the /var/lib/docker directory. On many Linux distributions, this path resides on the root partition (/). As you pull more images or create heavier containers, this directory can grow rapidly, eventually filling up the root filesystem. When the root partition is full, the operating system may become unstable, unable to write logs, update packages, or even boot correctly. Moving this directory to a separate, larger disk or partition is a standard administrative task for production environments. It ensures that container workloads do not impact the stability of the host OS. Note that this operation involves downtime for any running containers, so it should be scheduled during a maintenance window.

Prerequisites and Preparation

Before modifying the Docker configuration, you must ensure you have sufficient space on the target drive. You also need to identify the correct mount point. For example, if you are moving data to a second hard drive mounted at /mnt/docker-data, verify that this directory exists and has appropriate permissions. First, stop all running containers to prevent data corruption. You can do this by stopping the Docker service itself, which gracefully shuts down all managed containers.

sudo systemctl stop docker

Once the service is stopped, no processes should be writing to /var/lib/docker. Verify this by checking if any processes are holding files open in that directory using tools like fuser or lsof, although typically stopping the service is sufficient.

fuser -vm /var/lib/docker

If this command returns no output, the directory is free to be moved. If it shows active processes, investigate them before proceeding.

Step-by-Step Migration Process

The core of this migration involves changing the systemd unit file for Docker. We will use the --data-root configuration option, which is the modern standard for specifying the base directory for storing container data. 1. Create the New Directory Create the destination directory if it does not already exist. Ensure it is owned by the root user or the docker group depending on your security policies, though typically root ownership is required for the daemon to write there.

sudo mkdir -p /mnt/docker-data

2. Copy Existing Data Use rsync to copy the contents of /var/lib/docker to the new location. Using rsync with the -a flag preserves permissions, timestamps, and symbolic links, which is critical for Docker’s internal state management. Do not simply move the folder; copying ensures you have a backup in case something goes wrong.

sudo rsync -avz /var/lib/docker/ /mnt/docker-data/

Verify the copy was successful by comparing sizes or listing contents. Once confirmed, you can safely remove the old directory later, but only after the new setup is verified working.

sudo rm -rf /var/lib/docker/*

3. Update Systemd Configuration You must tell Docker where to find its data on startup. Edit the Docker systemd override file. This method is preferred over editing the main unit file directly, as it survives package updates.

sudo systemctl edit docker.service

In the editor that opens, add the following lines under the [Service] section:

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --data-root /mnt/docker-data

Note the empty ExecStart= line. This clears the original ExecStart directive defined in the main unit file, allowing the next line to define the new command with the additional flag. Replace /mnt/docker-data with your actual target path. Save and exit the editor. 4. Restart and Verify Reload the systemd daemon to pick up the changes and start Docker.

sudo systemctl daemon-reload
sudo systemctl start docker

Check the status to ensure the service started without errors.

sudo systemctl status docker

To confirm the new data root is active, run:

docker info | grep "Docker Root Dir"

The output should display the new path (/mnt/docker-data). If it still shows /var/lib/docker, check the journal logs for syntax errors in the systemd override.

Troubleshooting and Best Practices

If Docker fails to start, check the journal logs for specific error messages related to permissions or missing directories. In some cases, SELinux contexts may need to be updated if you are on an RHEL-based system. Refer to our guide on Understanding Linux Permissions for detailed steps on managing SELinux contexts for custom paths. Additionally, if you are running Docker in a virtualized environment, ensure that the underlying hypervisor properly handles I/O operations on the new disk. Issues with network connectivity or pinging within emulated SSH servers can sometimes mask storage issues, so consult Emulated SSH Server Pinging, Traceroute Fails for Docker on Host VM if you suspect broader infrastructure problems. Always test the migration in a non-production environment first. Verify that your applications can read and write to volumes correctly after the move. Regularly monitor the size of the new data directory to prevent similar space issues in the future.

Frequently Asked Questions

Will moving the data directory delete my existing containers?

No, the migration process preserves all existing images, containers, and volumes. By using rsync to copy the data and updating the --data-root flag, Docker continues to recognize and manage your previous workload exactly as it did before the move.

Can I use a symbolic link instead of changing the config?

While creating a symlink from /var/lib/docker to another location might seem easier, it is not recommended. Symlinks can cause issues with systemd watchdogs, audit logs, and certain storage drivers. The official and supported method is to use the --data-root configuration option in the systemd unit file.

What happens if the new disk becomes full?

If the new disk fills up, Docker will stop accepting new writes, leading to failed deployments and crashed containers. It is crucial to set up monitoring alerts for the new data directory’s disk usage. Consider implementing log rotation and regular cleanup of unused images using docker system prune to maintain healthy disk levels.

Does this affect Docker Compose setups?

No, Docker Compose relies on the Docker daemon’s configuration. Since the daemon now points to the new data root, all Compose projects will automatically store their data in the new location. There is no need to modify individual docker-compose.yml files unless you are using absolute host paths that explicitly reference the old directory.

How do I revert the change if something goes wrong?

To revert, stop the Docker service, edit the systemd override file again, and comment out or remove the --data-root line. Then, restore the original data from your backup (if you kept it) or rely on the fact that the original /var/lib/docker content was copied away. Reload systemd and restart the service to return to the default behavior.