Understanding Stream Indexing and Input Mapping
When working with multiple inputs in FFmpeg, each file is assigned an index starting from zero. The first input is-i input1.mp4, the second is -i image.png, and so on. Understanding this indexing is critical when referencing streams within the filter graph. For instance, if you provide two images and one video, the video might be at index 2 while the images are at indices 0 and 1.
You must explicitly map these streams to outputs using the -map flag. Without mapping, FFmpeg may automatically select only one video and one audio stream based on its default selection logic, which often ignores your carefully constructed filter graph. By using -map 0:v:0 or similar syntax, you force FFmpeg to include specific streams in the final container. This explicit control prevents missing frames or silent tracks that frequently plague complex multi-input workflows.
The Loop Filter for Static Images
A common challenge arises when combining a short video with long-running background images. The video might last five seconds, but you want the image to fill the entire timeline. Theloop filter solves this by repeating the input stream until it matches the duration of another stream. However, the parameter naming can be confusing. In modern FFmpeg versions, the correct parameter is loop, not tbs. Using incorrect parameters will result in a parsing error.
The syntax typically looks like [0:v]loop=loop=-1:size=1[img_loop]. Here, loop=-1 indicates infinite looping, meaning the image repeats indefinitely. The size=1 specifies that the looped segment contains one frame. If your image sequence has multiple frames, adjust the size accordingly. This filtered output [img_loop] can then be used in subsequent operations, such as overlaying or concatenation, ensuring it persists for the required duration regardless of the source length.
For more details on managing stream durations and timing flags, see our guide on Understanding FFmpeg's ss, -t, -to, Streamcopy, and Keyframe Differences. This knowledge helps you avoid manual duration calculations by letting FFmpeg handle temporal alignment internally.
Concatenating Video and Image Segments
Once your streams are prepared, you likely need to join them into a single coherent output. Theconcat filter is the standard tool for this. It requires all segments to have identical format properties, including resolution, pixel format, and frame rate. If your video is 1920x1080 at 30fps and your image is 640x480, you must first scale and pad the image to match the video specifications.
Use the scale and pad filters before concatenation. For example, [1:v]scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2:black[v1] ensures the image fills the screen without distortion. After scaling, both streams should be compatible for concatenation. The concat=n=2:v=1:a=0 filter tells FFmpeg to concatenate two video segments with no audio. This method preserves the visual continuity between the static background and the moving video content.
If you are dealing with device inputs or capturing specific frames, refer to Capture One Frame from FFmpeg Video Stream with Device B for insights on handling raw device data before filtering.
Advanced Overlay and Transparency Handling
Beyond simple concatenation, overlays allow you to place images on top of video clips. Theoverlay filter supports transparency via alpha channels. If your PNG image has an alpha channel, FFmpeg can composite it directly over the video stream. Use [0:v][1:v]overlay=x=0:y=0[outv] to place the image at the top-left corner. You can also animate the position by replacing static coordinates with expressions like x='if(gte(t,2),200,x)' to move the overlay after two seconds.
Ensure your output codec supports alpha. MP4 containers generally do not support alpha transparency in H.264. Instead, use a format like WebM with VP9 or ProRes 4444 for intermediate processing. When exporting to MP4, you may need to flatten the alpha channel by compositing the transparent image onto a solid background color first. This step involves creating a solid color video and overlaying the transparent image onto it, effectively baking the transparency into the RGB pixels.
For troubleshooting stream detection issues, check How to detect with ffmpeg if the stream has video or audio or both to ensure your inputs contain the expected media types before applying complex filters.
Performance Optimization and Error Handling
Complex filter graphs can be CPU-intensive. Use hardware acceleration options like-hwaccel if your GPU supports the codecs involved. Additionally, monitor FFmpeg’s log output for warnings about dropped frames or buffer overflows. These errors often indicate mismatched frame rates or insufficient buffer sizes in the filter chain. Adjusting the setpts filter can help resynchronize timestamps if drift occurs during long concatenations.
Always test with a small subset of your media first. Create a five-second clip and a single image to verify the filter graph works as intended before running full-length processing. This iterative approach saves time and resources, allowing you to refine the command structure incrementally.
Frequently Asked Questions
Why does my concatenated video have black gaps?
This usually happens when the durations of the input streams do not align perfectly. Ensure you use the loop filter on images to extend their duration to match the video exactly. Mismatched lengths cause FFmpeg to pad shorter streams with silence or black frames.
Can I use different resolutions for concat filter inputs?
No, the concat filter requires all segments to have identical width, height, and pixel format. You must pre-process each input using scale and pad filters to normalize dimensions before concatenation.
How do I remove audio from a video before filtering?
You can ignore audio streams by specifying only video maps with -map 0:v. Alternatively, use the anull filter in your filter graph if you need to process audio streams but output nothing, though mapping is simpler for exclusion.
What is the difference between loop=-1 and loop=0?
A value of -1 means infinite looping, repeating the stream indefinitely until stopped. A value of 0 plays the stream once without repetition. Use -1 for background images that must persist longer than other video segments.
Why is my alpha channel not visible in the final MP4?
Standard H.264 encoding in MP4 does not support transparency. You must either export to a format like WebM or ProRes 4444, or composite the transparent layer onto a solid background color using the overlay filter before final encoding.