Bad log sizes in Linux usually stem from unrotated application logs or Btrfs compression overhead, not filesystem corruption. You can reclaim space by identifying large files with find, rotating logs via logrotate, and triggering Btrfs defragmentation or balance operations to return unused blocks to the free pool.
Identifying the Culprit Files
The first step in resolving a bad size of log issue is to locate which specific files are consuming the majority of your disk space. Often, developers assume the entire partition is full due to system overhead, but a single runaway process writing debug output can fill terabytes of storage in hours. To pinpoint these files, you must look beyond standard directory listings which often hide deep nesting or hidden directories. Use thedu command with the --max-depth=1 flag to see a summary of space usage per directory. This helps you narrow down whether the issue lies in /var/log, user home directories, or application-specific data folders. For a more granular view, especially when dealing with thousands of small files that collectively take up massive space, use find / -type f -size +100M -exec ls -lh {} \;. This command searches the entire filesystem for files larger than 100 megabytes and lists them with human-readable sizes.
In Btrfs environments, it is crucial to distinguish between actual data consumption and apparent usage. Btrfs uses Copy-on-Write (CoW), meaning deleted files may still occupy physical blocks until a balance operation occurs. Therefore, seeing high usage does not always mean the data is actively written; it might just be fragmented or unreclaimed. Always verify if the file system reports high usage via btrfs filesystem usage compared to what df -h shows, as discrepancies here indicate pending background tasks.
Managing Application Logs Properly
Once you have identified large log files, the immediate fix is often to rotate or truncate them. However, truncating a file that is currently being written to by a running service can cause errors or lost data. The correct approach involves usinglogrotate, which handles rotation safely by renaming the current log file and creating a new empty one, then signaling the application to reopen its log descriptor.
Create or edit the configuration file for your specific application in /etc/logrotate.d/. A typical configuration should specify rotation frequency, such as daily or weekly, and retention policies, like keeping the last 10 rotated copies. Include the compress directive to save space on older logs and delaycompress to ensure the most recent rotated file remains uncompressed for easier debugging. If an application does not support standard log rotation signals, you may need to configure it to send a HUP signal to the process after rotation.
For services managed by systemd, check if they have their own journaling limits configured in /etc/systemd/journald.conf. Setting SystemMaxUse=500M ensures that the journal does not grow indefinitely. Restarting the systemd-journald service applies these changes immediately. This prevents future accumulation of massive log files that contribute to the bad size of log problem.
Btrfs Space Reclamation Strategies
Btrfs handles space differently than ext4 or XFS. When you delete files, the space is marked as free within the subvolume, but the underlying block devices do not immediately reflect this availability for other uses unless a balance operation runs. This behavior often leads to confusion where users delete gigabytes of data but see no change in available disk space. To reclaim this space, runbtrfs balance start -dusage=1 /mountpoint. This command scans the filesystem for chunks with low usage and redistributes the data to consolidate free space into larger, manageable chunks. It is a background process that can take significant time depending on the amount of data involved. Monitor progress with btrfs balance status /mountpoint.
If fragmentation is contributing to the perceived bad size of log, consider running a defragmentation command. Use btrfs filesystem defrag -r /path/to/directory to recursively defragment files and directories. Defragmentation improves read performance and can slightly reduce space usage by allowing better compression ratios if compression is enabled. Note that defragmentation is write-intensive and should be performed during low-load periods to avoid impacting system performance.
Preventing Future Issues
Prevention is key to maintaining a healthy Btrfs filesystem. Regularly schedulebtrfs balance operations using cron jobs, especially after large deletions. This ensures that free space is consistently returned to the global pool. Additionally, monitor disk usage trends over time using tools like btrfs filesystem usage integrated into monitoring scripts.
Ensure that your log rotation configurations are tested periodically. A misconfigured logrotate script can fail silently, leading to rapid disk filling. Set up alerts for disk usage thresholds at 80% and 90% capacity to catch issues before they become critical. For Btrfs specific health, enable scrubbing by scheduling btrfs scrub start /mountpoint regularly to detect and repair silent data corruption.
Detecting Btrfs Finished Reclaiming Free Space Background provides deeper insights into how the filesystem manages internal chunk allocation during these operations. Understanding these mechanisms helps administrators predict when balance operations will complete and how much space will actually be freed.
BTRFS Partition Completely Full: How to Make it Workable Again offers emergency procedures if your filesystem becomes entirely unusable due to space exhaustion, complementing the preventive measures discussed here.
Frequently Asked Questions
Why does df show full disk after deleting large logs?
Btrfs uses Copy-on-Write, so deleted files remain in physical blocks until a balance operation reclaims them. Run btrfs balance start to redistribute data and return the space to the free pool. Until this happens, the filesystem cannot allocate those blocks to new files.
Can I compress existing log files to save space?
Yes, if your Btrfs subvolume has compression enabled (e.g., zstd or zlib), newly written data will be compressed automatically. Existing files will only be compressed if you run a defragmentation or copy operation that triggers CoW. Check your mount options for compress=zstd to ensure active compression.
What causes log files to grow uncontrollably?
Uncontrolled growth is typically caused by missing or misconfigured log rotation rules. Applications may also enter error loops, continuously writing stack traces or debug information. Verify that logrotate is active and that the application sends proper signals upon log rotation.
Is btrfs scrub necessary for fixing bad log sizes?
No, scrub checks for data integrity and repairs bit rot, but it does not reclaim disk space. Use scrub to detect corruption, but use balance and defrag to manage space usage. Confusing these two commands is common but they serve distinct purposes in filesystem maintenance.
How do I find which process is writing to a log file?
Use the lsof +L1 command to list open files with link counts less than one, indicating deleted but still open files. Alternatively, use fuser /path/to/logfile to identify the PID of the process holding the file open. This helps you restart the service to release the file handle.