Ansible Playbook: Copy Only Changed or New Files

To copy only new or changed files using Ansible, the most reliable method is leveraging the synchronize module, which acts as a wrapper around rsync. This approach ensures that destination files are updated based on modification time and size differences, significantly reducing transfer overhead compared to standard file copying modules. By configuring the delete parameter appropriately, you can also manage stale files on the target host while maintaining synchronization integrity across your infrastructure.

Understanding the Synchronize Module

The synchronize module is designed specifically for efficient file transfers between local and remote hosts, or between two remote hosts. It uses the rsync utility under the hood, which provides a delta-transfer algorithm. This means it only sends the differences in files rather than transferring entire files every time. For large directory trees, this performance gain is substantial. When managing file permissions alongside synchronization, you might need to adjust settings after the copy operation completes, as discussed in our guide on Managing File Permissions in Ansible Playbooks: chmod for Small Directory Trees vs. Large Directory Trees. The primary advantage of using synchronize over the copy or template modules is its ability to handle incremental updates. The copy module will overwrite the destination file if the source content differs, but it does not inherently check for changes in the same efficient manner as rsync. While copy has a checksum option, it calculates the checksum locally before transfer, which can be slow for large files. In contrast, synchronize performs the comparison on the remote side (or via the rsync protocol), minimizing data movement.

Basic Configuration for Incremental Copies

A basic playbook task utilizing the synchronize module requires defining the source and destination paths. The src parameter specifies the location on the controller (or local host), while dest defines the target path on the managed node. By default, synchronize will skip files that have already been copied and have not changed since the last run. Here is a simple example structure:

- name: Sync files to remote server
  ansible.builtin.synchronize:
    src: /path/to/local/files/
    dest: /path/to/remote/files/
    mode: push

In this configuration, the mode: push parameter explicitly indicates that files are being sent from the control node to the target. If omitted, Ansible defaults to pushing files when connecting to a remote host. The module automatically handles the creation of necessary directories if they do not exist, provided the user has appropriate write permissions. It is crucial to ensure that rsync is installed on both the control node and the target hosts. Without rsync, the synchronize module will fail silently or throw an error indicating the command is missing. If you encounter issues with connection stability during these operations, reviewing strategies for SSH reuse can help maintain consistent performance, as outlined in Can Ansible Reuse SSH Connection for Rsync?.

Handling Deletion of Stale Files

One common requirement in synchronization tasks is ensuring that files deleted from the source are also removed from the destination. By default, the synchronize module does not delete files on the destination that no longer exist in the source. To enable this behavior, you must set the delete parameter to true.

- name: Sync files and delete stale ones
  ansible.builtin.synchronize:
    src: /path/to/local/files/
    dest: /path/to/remote/files/
    delete: true
    mode: push

Using delete: true makes the synchronization bidirectional in terms of existence checks, effectively making the destination a mirror of the source. Exercise caution with this setting, as accidental misconfiguration could lead to unintended data loss on the target server. Always verify the paths and ensure that the source directory accurately reflects the desired state of the destination. For scenarios where you want to preserve certain files or directories from deletion, you can use the exclude parameter to specify patterns that should be ignored during the sync process. This allows for more granular control over which files are considered "stale."

Optimizing Performance with Checksums

While synchronize is generally fast, you may occasionally need to force a re-copy even if files appear unchanged. This can be achieved by using the checksum parameter, which forces the module to compare file checksums rather than relying solely on modification times and sizes. This is useful when files have been modified without updating their timestamps, a situation that can occur with certain backup tools or manual edits.

- name: Force sync based on checksum
  ansible.builtin.synchronize:
    src: /path/to/local/files/
    dest: /path/to/remote/files/
    checksum: true
    mode: push

Setting checksum: true ensures that any change in file content triggers a copy, regardless of timestamp. However, this comes at a performance cost, as calculating checksums for large files takes time. Use this option sparingly, primarily for critical configuration files where content integrity is paramount.

Integrating with Variable Management

When scaling these synchronization tasks across multiple environments, managing variables becomes essential. You can define source and destination paths as variables to make your playbooks more flexible. This approach aligns with best practices for combining common and specific variables, as described in Combining Common and Specific Vars in Ansible. By structuring your variables hierarchically, you can easily switch between development, staging, and production targets without modifying the core logic of the synchronize task. This modularity enhances maintainability and reduces the risk of errors when deploying changes across different infrastructure components.

Frequently Asked Questions

Why is my synchronize task failing with 'rsync not found'?

The synchronize module requires the rsync executable to be present on both the control node and the target hosts. Install rsync using your package manager (e.g., apt install rsync or yum install rsync) on all affected machines.

How do I exclude specific files from being synced?

Use the exclude parameter within the synchronize task to specify patterns of files or directories to ignore. For example, exclude: "*.log" will prevent log files from being transferred to the destination.

Can synchronize work between two remote hosts?

Yes, by setting delegate_to to one of the remote hosts, you can initiate a sync between two remote servers. This requires SSH keys to be configured between those hosts so that the delegated host can access the other.

What is the difference between synchronize and copy regarding performance?

Synchronize uses rsync's delta-transfer algorithm, sending only changed parts of files, whereas copy typically transfers the entire file if the content differs. This makes synchronize significantly faster for large files with minor changes.

Does synchronize update file permissions automatically?

No, the synchronize module does not modify file permissions unless explicitly told to via additional parameters like perms. You should follow up with a separate task using the file module to ensure correct ownership and permissions.