Can't Get Rsync to Exclude Specific Directories? Here's the Include Option to Try

Rsync excludes directories by processing rules in order, so you must define inclusion patterns before exclusion rules. By placing --include='*/' and specific file filters before a final --exclude='*', you ensure that only the desired directory structures and contents are copied while ignoring everything else.

Understanding Rsync Rule Evaluation Order

The most common reason rsync fails to exclude specific directories is a misunderstanding of how it evaluates filter rules. Unlike many other tools that apply all exclusions simultaneously, rsync processes every single rule sequentially from top to bottom. The first rule that matches a file or directory determines its fate; subsequent rules for that same item are ignored completely. This linear evaluation process means that the position of your include and exclude flags is critical to the success of your backup strategy. If you place an exclusion rule like --exclude='Documents/*' at the very beginning of your command line, rsync will encounter it immediately. It will see any path containing "Documents" and skip it forever, regardless of whether you have included that directory later in the command. To successfully copy only specific folders, you must invert this logic: explicitly include what you want to keep before telling rsync to ignore the rest.

The Importance of Directory Traversal

A subtle but vital aspect of this ordering involves directory traversal itself. If you want to copy files inside a specific subdirectory, you must also allow rsync to enter that directory. In rsync syntax, a trailing slash in a pattern (like foo/) refers to the contents of the directory, while no trailing slash (like foo) refers to the directory entry itself. If you exclude a directory without including it first, rsync will not even look inside it to find the files you might want. Therefore, you need a broad include rule for all directories to ensure the tree structure is accessible.

Constructing the Correct Filter Chain

To implement a selective sync where you copy only specific directories and ignore the rest, you should construct your command using a specific sequence of flags. The general pattern starts with allowing all directories to be traversed, then includes the specific content you need, and finally excludes everything else. Here is the standard approach using the example parameters mentioned in your query:

1. Allow Directory Traversal: Start with --include='*/'. This tells rsync to always descend into every directory it encounters. Without this, rsync might skip entire branches of your file system because it does not know they exist.
2. Include Specific Content: Add your specific targets, such as --include='Documents/*'. This ensures that the contents of the Documents folder are matched against the inclusion criteria.
3. Exclude Everything Else: End with --exclude='*'. This acts as a catch-all rule. Since it is at the end, it will only affect items that did not match any previous include rules.

When combined, the command looks like this:

rsync -av --include='*/' --include='Documents/*' --exclude='*' /source/ /destination/

In this scenario, rsync enters all directories (*/), finds the Documents folder, copies its contents, and then ignores all other files and folders that do not fit the inclusion criteria.

Refining Your Selection

You can expand this pattern to include multiple distinct directories. For example, if you also want to back up a folder named "Photos," you simply add another include statement before the final exclude:

rsync -av --include='*/' --include='Documents/*' --include='Photos/*' --exclude='*' /source/ /destination/

This flexibility allows you to curate your backups precisely. However, remember that each additional include rule adds complexity. If you find yourself managing dozens of specific paths, consider using a filter file instead of a long command line. You can create a text file containing these rules one per line and pass it to rsync using the --filter option, which makes maintaining complex exclusion lists much easier.

Common Pitfalls and Debugging

Even with the correct order, users often face issues due to hidden characters or case sensitivity. Always verify that your paths are exact. For instance, documents and Documents are treated as different directories on Linux systems. Additionally, relative versus absolute paths can cause confusion. If you are syncing from a source path that includes leading directories, ensure your include patterns account for that prefix. Another frequent error is forgetting that -a (archive mode) implies -r (recursive). While recursion is necessary for directory trees, it interacts with the filter rules. If you use --dry-run before executing the actual sync, you can see exactly which files are being accepted or rejected based on your current rule set. This is an essential diagnostic step. You can also check available space on your destination drive to ensure the operation won't fail midway, similar to checking space before a full system backup as discussed in our guide on Determining If Enough Space is Available in Destination Drive for rsync Dry Run. Furthermore, preserving timestamps is crucial for incremental backups. Using the -t option ensures that file modification times are maintained, which helps rsync determine what needs updating in future runs. Understanding the -t option and its impact on sync efficiency is detailed in our article on Understanding the '-t' Option in rsync.

Verifying Your Configuration

Before committing to a large data transfer, always test your configuration. Use the --dry-run flag alongside -v (verbose) to simulate the process. Review the output carefully. Look for lines prefixed with "d" (directories) and "f" (files). Ensure that only the intended directories appear in the list. If you see unexpected files being copied, re-evaluate your include/exclude order. Remember, the last matching rule wins. By mastering the sequential nature of rsync's filter rules, you gain precise control over your data synchronization. Whether you are backing up personal documents or synchronizing project assets across servers, this method provides a robust and reliable solution for selective copying.

Frequently Asked Questions

Why does rsync still copy excluded files?

This usually happens because an earlier include rule matched the file before the exclude rule was reached. Since rsync stops evaluating rules after the first match, you must ensure your exclusion rules are placed after your inclusion rules in the command line.

How do I exclude hidden files like .git?

You can add a specific exclude rule for dot-files by adding --exclude='.*' to your command. Place this after your main include rules but before the final catch-all exclude to ensure it is processed correctly.

Can I use regular expressions in rsync excludes?

Yes, rsync supports basic regular expressions in filter rules. You can use patterns like --exclude='\.log$' to exclude all files ending in .log. However, simple wildcard patterns are often sufficient and easier to debug for most users.

What is the difference between --exclude and --exclude-from?

The --exclude flag takes a single pattern directly in the command line, while --exclude-from reads a list of patterns from a specified text file. The latter is useful when you have a large number of exclusion rules that would make the command line unwieldy.

Does rsync preserve permissions when excluding directories?

Rsync preserves permissions for the files and directories it actually copies. Excluded directories are simply skipped entirely, meaning their permissions and ownership are irrelevant to the destination since nothing is transferred from them.