Understanding Rsync Dry Run Behavior
The --dry-run (or -n) option in rsync simulates the transfer process. It performs all the necessary checks—such as comparing timestamps and file sizes—but does not actually copy or modify any files on either the source or destination. While this is excellent for verifying permissions and connectivity, it does not inherently warn you about insufficient storage space on the target drive.
By default, a dry run outputs a list of files that would be transferred. However, standard output includes relative paths and status indicators, but rarely includes the specific byte size of each file. To determine if your destination has enough room, you must parse this output to sum up the individual file sizes.
Constructing the Space Check Command
To accurately calculate the total space required, you need to instruct rsync to print the size of each file during the dry run. The most reliable method uses the --itemize-changes flag combined with a custom --out-format. This allows you to isolate the size field from the verbose output.
Run the following command structure, replacing placeholders with your actual source and destination paths:
rsync -av --dry-run --itemize-changes --out-format='%i %n' /source/path/ /destination/path/ | awk '{sum += $1} END {print sum}'Let us break down why this works correctly where previous attempts failed. The key lies in how awk processes the fields generated by rsync.
Why $NF Fails for Size Extraction
A common mistake is using $NF (the last field) in awk to retrieve the file size. In the itemized output format, the filename is often the last element, especially when paths contain spaces or special characters. Furthermore, the itemized string begins with a code indicating changes (like f.. for file), followed by the size, then the name. If you use $NF, you are likely capturing the filename rather than the numeric size, leading to errors or zero totals because awk cannot add strings to numbers effectively.
Correct Field Positioning
When using --out-format='%i %n', the output consists of two distinct parts separated by a space: the itemization code (which includes the size) and the filename. Actually, a cleaner approach is to use --out-format='%s %n' which explicitly prints just the size and the name. If you stick to the itemize format, the size is typically the second field if you split by whitespace carefully, but explicit formatting is safer.
If you use --out-format='%s %n', the command becomes:
rsync -av --dry-run --out-format='%s %n' /source/ /dest/ | awk '{sum += $1} END {print sum}'Here, $1 reliably refers to the first column, which is the file size in bytes. $2 and beyond represent the filename. Summing $1 gives you the exact byte count required.
Interpreting the Output
Once the command finishes, it will print a single number representing the total bytes needed. You should compare this number against the available free space on your destination drive. You can check available space using the df -h /destination/path command.
If the calculated size exceeds the available space, rsync will fail midway through a real transfer, potentially leaving partial files or corrupting the backup state. By running this check first, you avoid such catastrophic failures. For large datasets, consider adding --info=progress2 to see progress during the actual run, but keep the dry run separate for space verification.
Remember that directory metadata and hidden files are included in this calculation. If you have exclusion rules, ensure they are applied in the dry run command as well. This ensures the size estimate matches what the final backup will actually consume. For more complex exclusion logic, you might want to review how include options work alongside exclusions.
Can't Get Rsync to Exclude Specific Directories? Here's the Include Option to Try provides detailed guidance on managing which files contribute to that size total. Properly configuring these filters prevents unnecessary data from inflating your space requirements.Handling Large Files and Network Transfers
If you are transferring over SSH or network mounts, the local filesystem check on the destination might not reflect remote quotas immediately. In such cases, the dry run size is still valuable, but you must account for network latency and potential overhead. Additionally, some remote filesystems report block sizes differently than the actual file size. Always err on the side of caution and ensure at least 10-15% free space beyond the calculated total.
For extremely large backups, consider splitting the operation into smaller batches. This allows you to monitor space usage incrementally. If you frequently manage large directories, tools like rclone offer built-in bandwidth and space checks that might simplify your workflow.
How to Set Up Rclone Backup and Encrypt Directory on External Disk discusses alternative tools that handle encryption and syncing, though rsync remains the standard for precise control over file attributes and permissions on Unix-like systems.Frequently Asked Questions
Does rsync dry run delete files?
No, a dry run never deletes or modifies any files. It only reads the source and checks the destination state. However, if you use --delete along with --dry-run, it will show you which files *would* be deleted, helping you audit potential data loss before committing to the change.
How do I check free space on Linux?
Use the df -h /path/to/destination command. The -h flag displays human-readable sizes (GB, MB). Look at the "Avail" column to see how much space is free. Compare this value against the total bytes printed by your rsync dry run command.
What if my filenames have spaces?
Using --out-format='%s %n' handles spaces safely because awk splits by whitespace, assigning the size to $1 regardless of spaces in the filename. The filename starts at $2 and continues to the end of the line. As long as you sum $1, spaces in names do not affect the size calculation.
Can I use this method for Windows?
Standard rsync is not native to Windows, but it is available via WSL (Windows Subsystem for Linux) or Cygwin. The commands described here work identically within those environments. Ensure your path separators match the OS conventions used by your rsync implementation.
Is rsync idempotent?
Yes, rsync is idempotent. Running it multiple times with the same parameters will result in the same final state. If no files have changed, it transfers nothing. This property makes dry runs safe to repeat until you are confident in the space requirements and filter settings.