Use rsync -a --link-dest=DIR to clone hard linked folders while preserving existing links. This command creates a new backup directory where unchanged files remain as hard links to the source, saving space and time. It is ideal for incremental backups that maintain a consistent snapshot of your data structure.
Understanding Hard Links and Rsync
Hard links are filesystem entries that point to the same inode (data block) on disk. Unlike symbolic links, which are pointers to a path, hard links are indistinguishable from regular files. When you delete one hard link, the file persists as long as at least one other link exists. This property is crucial for efficient storage management, especially in backup scenarios where identical files should not consume additional disk space. Rsync is a versatile utility for synchronizing files and directories. By default, it copies files byte-by-byte, which can be slow and wasteful if many files have not changed. The--link-dest option changes this behavior. It tells rsync to compare files against a reference directory. If a file in the source matches a file in the reference directory, rsync creates a hard link to the reference file instead of copying it. This approach significantly reduces I/O operations and storage usage for repeated backups.
For more details on how rsync handles periodic backups with hard links, see Transferring Periodic Backups of /mnt/tosh Using Rsync with Hard Link Handling.
Prerequisites and Setup
Before running the cloning command, ensure you have the necessary permissions and tools. You need access to both the source drive and the destination drive. The destination must support hard links; most Linux filesystems like ext4, XFS, and Btrfs do, but network filesystems or Windows drives may behave differently. First, create the destination directory structure. If you are performing an incremental backup, you likely already have previous snapshots. Identify the most recent snapshot directory. This will serve as your--link-dest target. If this is your first backup, you might use the source directory itself as the reference, though this is less common for true incremental strategies.
Ensure that the rsync version installed supports --link-dest. Most modern distributions include a version that does. You can check by running rsync --version. If the output mentions "link dest" or similar capabilities, you are ready to proceed.
Executing the Clone Command
The core command structure involves specifying the archive mode, the link destination, and the source and target paths. Archive mode (-a) preserves permissions, timestamps, and ownership, which is essential for maintaining the integrity of your system files.
Run the following command in your terminal:
rsync -a --link-dest=/path/to/previous/snapshot /source/directory/ /destination/new/snapshot//path/to/previous/snapshot with the absolute path to the last successful backup. Replace /source/directory/ with the folder you want to clone. Ensure the trailing slash is included on the source to copy the contents rather than the directory itself. Finally, specify the new destination directory.
If you encounter issues with large directories, consider adding the --progress flag to monitor the transfer rate. For very large datasets, the --delete option can be used to remove files in the destination that no longer exist in the source, keeping the clone in sync. However, use this cautiously to avoid accidental data loss.
Related reading: Using rsync delete-excluded without implied delete for backup explains how to manage deletions safely during synchronization.
Verifying the Result
After the command completes, verify that hard links were created correctly. Use thels -i command to list files along with their inode numbers. Compare the inode numbers of files in the new snapshot with those in the previous snapshot. If they match, the files are hard linked.
You can also check the disk usage. Run du -sh /destination/new/snapshot/ and compare it with the size of the previous snapshot. If the content has not changed significantly, the new snapshot should occupy very little additional space, confirming that hard linking worked as intended.
Be aware that hard links only work within the same filesystem. If your source and destination are on different physical disks or partitions, rsync will fall back to copying files. In such cases, you cannot preserve hard links across the boundary.
For insights on handling sparse files during such transfers, refer to How to Shrink Sparse Files in Linux After Copying Them Without the -S Operator.
Frequently Asked Questions
Can I use --link-dest with remote destinations?
No, --link-dest requires the reference directory to be accessible locally on the machine running rsync. It compares files using local inode information, which cannot be done across network boundaries directly. You must ensure the reference snapshot is stored on the local disk before initiating the transfer.
What happens if I delete a file in the source after cloning?
Deleting a file in the source does not affect the cloned hard links in the destination. Since the destination holds its own hard link to the original inode, the data remains intact in the backup. This makes hard-linked backups safe for retention even if the source data is modified or removed.
Does --link-dest work with symlinks?
Rsync handles symbolic links separately from hard links. The -a flag preserves symlinks as symlinks. If a symlink points to a file that is hard-linked via --link-dest, the symlink in the destination will still point to the same relative path. It does not convert symlinks into hard links.
Why is my new backup taking up full disk space?
This usually occurs if the source and destination are on different filesystems, preventing hard link creation. It can also happen if the --link-dest path is incorrect or if the files have different permissions or timestamps that cause rsync to treat them as changed. Check the rsync output for any warnings about unsupported features.
Is there a risk of corrupting data with hard links?
There is no inherent risk of corruption from using hard links. However, if you accidentally modify a file in the "reference" snapshot thinking it is independent, you will overwrite the data for all clones pointing to that inode. Always treat reference snapshots as read-only to prevent unintended data loss across multiple backup versions.