Bash arrays and path expansions (globbing) are distinct mechanisms for handling lists of items in shell scripts. Arrays store data in indexed variables, while path expansions generate file paths from patterns before commands execute. Confusing them leads to errors like treating a single string as multiple files or failing to iterate correctly.
Understanding Bash Arrays
Bash arrays allow you to store multiple values in a single variable. Unlike simple scalar variables that hold one value, arrays can hold an ordered collection of strings. There are two types: indexed arrays, which use integer indices starting at 0, and associative arrays, which use arbitrary strings as keys. Indexed arrays are the most common form used in general scripting tasks.
To declare an indexed array, you can assign values directly using parentheses. For example,
my_array=(item1 item2 item3) creates an array with three elements. You can access individual elements by their index, such as
${my_array[0]} for the first item. To retrieve all elements, use
${my_array[@]}. This syntax expands to each element as a separate word, which is crucial for correct iteration.
Iterating over an array is straightforward using a
for loop. The command
for item in "${my_array[@]}"; do echo "$item"; done safely handles spaces within elements because the double quotes preserve each element as a single unit during expansion. Without quotes, words containing spaces would be split incorrectly.
Distinguishing Path Expansions
Path expansion, also known as globbing, is performed by the shell before any command runs. It replaces wildcard patterns with matching file names. Common wildcards include
* for any sequence of characters,
? for a single character, and
[...] for character classes. When the shell encounters a pattern like
*.txt, it searches the current directory for files ending in .txt and substitutes the pattern with the list of matching filenames.
This process happens automatically for unquoted arguments. If you type
ls *.txt, the shell expands
*.txt into a list of files, and
ls receives those filenames as separate arguments. However, if you quote the pattern, such as
ls "*.txt", the shell does not expand it. Instead,
ls receives the literal string
*.txt, which usually results in an error unless a file named literally
*.txt exists.
A key difference between arrays and path expansions is timing. Array assignment stores the values as provided. Path expansion occurs at execution time based on the filesystem state. An array contains exactly what you put in it, regardless of whether files exist. Path expansion only yields results if matching files are present. If no files match a glob pattern, the pattern remains unchanged unless the
nullglob option is set.
Common Pitfalls and Best Practices
One frequent mistake is confusing how arrays handle whitespace versus how globbing handles it. In arrays, spaces inside quoted elements are preserved. In globbing, the resulting list of filenames is separated by spaces when passed to commands. Always quote your array expansions to prevent word splitting issues, especially when filenames contain spaces or newlines.
Another pitfall involves mixing arrays and globs in loops. Using
for f in *.txt relies on globbing, which may fail silently if no files match. Using
for f in "${my_array[@]}" relies on array data, which is predictable. For robust scripts, prefer arrays when the list of items is known in advance or generated programmatically. Use globbing when you need to dynamically discover files in a directory.
Consider the scenario where you need to process a dynamic list of log files. Storing them in an array allows you to manipulate the list before processing. For instance, you can filter, sort, or modify the list. Globbing provides a static view of the directory at the moment of expansion. If files change between expansion and processing, the array reflects the original state, while a subsequent glob would reflect the new state.
For more insights on handling complex script arguments, see
Troubleshooting Bash Script with Suspected Issues of Spaces, Dashes, Filenames and Quoting. Understanding quoting rules is essential when working with both arrays and expanded paths.
When debugging scripts, echoing the exact command used can reveal how expansions were interpreted. Check
Echo the exact command used to start a Bash script for techniques to trace these expansions effectively.
Frequently Asked Questions
Can I mix arrays and globs in the same variable?
No, a variable cannot simultaneously be an array and a glob pattern. A variable holds either a scalar string, an array, or it is unset. You can assign the result of a glob expansion to an array, but the array itself just stores the resulting filenames as strings.
What happens if a glob pattern matches no files?
By default, the shell leaves the pattern unchanged. For example,
*.nonexistent will be passed literally to the command. To make it expand to nothing, enable the
nullglob shell option using
shopt -s nullglob.
How do I iterate over an associative array?
Use the
@ symbol with curly braces, like
${assoc_array[@]}. This expands to all values. To get keys, use
${!assoc_array[@]}. Ensure you declare the array as associative with
declare -A before assigning values.
Why does my array loop break on spaces?
If you forget to quote the array expansion, like
for i in $my_array, the shell performs word splitting on the result. This splits elements containing spaces into multiple items. Always use
"${my_array[@]}" to preserve element boundaries.
Is path expansion faster than using an array?
Globbing is generally very fast as it is handled internally by the shell. However, performance differences are negligible for small lists. Arrays offer more flexibility for manipulation, making them preferable for logic rather than raw speed.