Using the
-f concat demuxer with a command line pipe often fails because FFmpeg expects a physical file path or stdin input that is fully buffered, rather than a stream generated by shell commands. The most reliable workaround is to write the concatenated file list to a temporary text file and pass that file’s path to the
-f concat option, ensuring the demuxer can read the structure correctly before processing begins.
Understanding the Concat Demuxer Requirements
The
-f concat flag in FFmpeg invokes a specific demuxer designed to join multiple media files sequentially. Unlike standard input streams that can be piped directly from one process to another, this demuxer requires a structured text file where each line defines an input file. The syntax is strict: it must begin with a
file directive followed by the absolute or relative path to the media asset. When you attempt to generate this list dynamically using shell pipes (such as
echo or
find), FFmpeg may not receive the data in the expected format or timing, leading to errors like "Unable to open" or "Invalid argument."
This behavior stems from how FFmpeg initializes its input context. It attempts to parse the entire file list into memory or establish file handles before starting the decoding pipeline. A pipe, which is a unidirectional byte stream, does not provide the seekable or static nature that the demuxer anticipates for its configuration phase. Consequently, even if the content is correct, the mechanism of delivery via a pipe often breaks the initialization sequence.
Why Pipes Fail with Concat Lists
Shell pipes are powerful for streaming data, but they lack the random access capabilities required by many FFmpeg components during setup. When you use a command like:
echo "file 'video1.mp4'" | ffmpeg -f concat -i - -c copy output.mp4
FFmpeg receives a stream on standard input. However, the concat demuxer specifically looks for a file descriptor that supports reading the list structure reliably. In many environments, especially Windows or certain Linux configurations, the demuxer cannot properly interpret a pipe as a valid input source for the list itself. It may hang waiting for EOF (End Of File) or fail to parse the
file directives because the stream is being consumed by the shell interpreter before FFmpeg gets a stable handle.
Additionally, quoting issues often arise when generating lists on the fly. Paths with spaces require careful escaping, which becomes difficult to manage when constructing the string inside a pipe command. This leads to malformed entries in the virtual list, causing the demuxer to reject the input entirely.
The Temporary File Solution
The robust solution involves creating a temporary file to hold the file list. This approach decouples the generation of the list from the execution of the FFmpeg command, allowing each step to complete independently. First, you generate the list of files into a text file. Then, you instruct FFmpeg to read that file using the
-f concat option.
Here is the general workflow for Bash environments:
- Create a temporary file using
mktemp or a fixed name.
- Populate the file with the required
file directives.
- Run FFmpeg pointing to this temporary file.
- Clean up the temporary file after completion.
For example:
# Create the list file
list_file=$(mktemp)
echo "file '/path/to/video1.mp4'" > "$list_file"
echo "file '/path/to/video2.mp4'" >> "$list_file"
# Run FFmpeg
ffmpeg -f concat -safe 0 -i "$list_file" -c copy output.mp4
# Clean up
rm "$list_file"
This method ensures that the concat demuxer receives a standard file input, which it handles natively and without error. The
-safe 0 flag is often necessary if your paths contain special characters or are not strictly local, though it depends on your FFmpeg version and security settings.
Handling Dynamic File Lists
In scenarios where you need to concatenate hundreds of files, manually writing the list is impractical. You can automate the list generation using shell tools like
find or
ls. For instance, to list all MP4 files in a directory:
find . -name "*.mp4" -type f | sed 's/^/file '\''/' | sed 's/$/'\''/' > list.txt
This command finds all MP4 files, prepends
file ' to each line, appends
', and redirects the result to
list.txt. You then pass
list.txt to FFmpeg. This technique is far more reliable than trying to pipe the output of
find directly into FFmpeg's stdin for the concat demuxer.
If you are working with large datasets, consider using
mktemp to avoid naming collisions and ensure cleanup. Always verify the contents of the generated list file before running FFmpeg to catch any path errors early.
Alternative Approaches
If you do not want to create a temporary file, you might consider using the
-f image2pipe or other demuxers depending on your source type, but these are less common for simple video concatenation. Another alternative is using FFmpeg's filter graph with
concat filter, which processes streams in memory. However, this method is significantly more CPU-intensive because it requires re-encoding the audio and video streams unless you use complex stream mapping tricks that are prone to instability.
For most users, the temporary file method remains the gold standard for speed and reliability. It allows FFmpeg to operate in
-c copy mode, which is nearly instantaneous compared to re-encoding. If you encounter issues with file permissions or disk space when creating temporary files, ensure your system has write access to the
/tmp directory or specify a different location using environment variables.
Debugging Common Errors
When troubleshooting, always check the exit code of FFmpeg. A non-zero exit code indicates failure. Use the
-v debug flag to get verbose output, which will show exactly where the demuxer fails to parse the input. Often, the error message will point to a missing quote or an invalid path in the list file. Ensure that all paths in the list are accessible from the user account running FFmpeg. Relative paths can be tricky; using absolute paths in the list file eliminates ambiguity.
How to Capture Time Parameter in ffmpeg Output on EndeavourOS discusses related debugging techniques for capturing metadata, which can help verify that your input files are being read correctly. Additionally,
How to Create Animated GIF Sequence Image Files using FFmpeg covers similar input handling concepts that apply when dealing with sequential media assets.
Frequently Asked Questions
Can I use stdin instead of a file for the concat list?
No, the concat demuxer generally does not support reading the file list from standard input pipes due to buffering and initialization constraints. It requires a seekable file descriptor to parse the list structure reliably before starting the media pipeline.
What does the -safe 0 flag do?
The
-safe 0 flag disables the safety checks that prevent FFmpeg from accessing files outside the current directory or those with unusual names. It is often required when your file list contains paths with spaces, special characters, or references to parent directories.
Why is my concatenated video glitching or having audio sync issues?
Glitches usually occur when you try to concatenate files with different codecs, resolutions, or frame rates without re-encoding. The
-c copy option only works seamlessly between identical streams. If formats differ, you must remove
-c copy to trigger re-encoding, which resolves sync and quality issues at the cost of time.
How do I handle filenames with spaces in the list file?
Each filename in the list file must be enclosed in single quotes immediately after the
file keyword, like
file 'my video.mp4'. If you generate the list programmatically, ensure your script adds these quotes around every path to prevent FFmpeg from misinterpreting spaces as delimiters.
Is there a way to avoid creating temporary files on slow disks?
While avoiding temp files is ideal, it is technically challenging with the concat demuxer. You could use a RAM disk (like
/dev/shm on Linux) to store the temporary list file, which provides near-instantaneous read/write speeds and avoids physical disk wear. This is a practical compromise for high-performance workflows.