Crontab tasks often fail silently because the cron environment lacks the user’s interactive shell configuration, such as environment variables and working directories. This guide provides a systematic approach to diagnosing why a script that works manually does not execute via cron, focusing on path resolution, permissions, and logging.
Creating a Diagnostic Test Script
Before troubleshooting complex jobs, isolate the issue with a simple test script. Create a file named
test_cron.sh in your home directory. This script should explicitly log its execution context to a known location, allowing you to verify if the cron daemon is even attempting to run it.
#!/bin/bash
# test_cron.sh
echo "=== Cron Job Execution ===" >> /tmp/cron_debug.log
date >> /tmp/cron_debug.log
whoami >> /tmp/cron_debug.log
env >> /tmp/cron_debug.log
/usr/bin/echo "Script completed successfully" >> /tmp/cron_debug.log
Make this script executable by running
chmod +x ~/test_cron.sh. The use of absolute paths for commands like
/usr/bin/echo and
/usr/bin/date is critical here, as cron does not source your
.bashrc or
.profile, meaning standard command aliases or shortcuts may not resolve. Add a line to your crontab using
crontab -e to run this script every minute for testing purposes:
* * * * * /home/yourusername/test_cron.sh
Replace
yourusername with your actual Linux username. After waiting for one minute, check the output file with
cat /tmp/cron_debug.log. If the file exists and contains data, the cron daemon is functioning correctly, and the issue lies within your specific job’s logic or environment. If the file is missing or empty, the problem is at the system level, likely involving permissions or the cron service itself.
Common Pitfalls: Paths and Environment Variables
The most frequent cause of cron failures is the assumption that the environment matches your interactive terminal. When you log in via SSH or a GUI, your shell loads variables like
$PATH,
$HOME, and
$LANG. Cron starts with a minimal environment, typically only including
$HOME,
$LOGNAME,
$MAIL,
$PATH,
$SHELL, and
$USER.
If your script calls another script or binary without an absolute path, cron will fail to find it. For example, running
python script.py might work in your terminal but fail in cron if
python is not in the default cron
$PATH. Always use absolute paths for both the interpreter and any scripts called by your main job. You can find the absolute path to your interpreter by running
which python3 or
which bash in your terminal.
Additionally, relative paths in your script are resolved against the user’s home directory, which is usually safe, but explicit absolute paths are more robust. If your script relies on reading files from a specific directory, ensure those paths are absolute. For instance, instead of
./data.csv, use
/home/yourusername/data.csv. This eliminates ambiguity regarding the current working directory when the job runs.
For more insights on how script execution differs across environments, you might find
Determining Script Execution on Windows, Linux, and MacOS useful, as it highlights environmental discrepancies that often trip up cross-platform developers.
Debugging Permissions and Ownership
Even if the script is present and executable, permission issues can prevent execution. Ensure the script has the execute bit set for the user running the cron job. Use
ls -l ~/test_cron.sh to verify permissions. The output should show
x (execute) rights for the owner. If the script resides in a shared directory, ensure the cron user has read and execute permissions on the file and all parent directories.
Another common oversight is the ownership of the log files. If your script attempts to write to a directory where the cron user (often
root if editing root’s crontab, or your user if editing your own) lacks write access, the job will fail silently. In the diagnostic script above, we wrote to
/tmp, which is world-writable, avoiding this issue. However, if you log to a custom directory, ensure it is writable by the executing user.
Check the system logs for clues. On systemd-based systems, view cron-related errors with
journalctl -u cron.service or
journalctl -u crond.service depending on your distribution. These logs often reveal permission denied errors or syntax issues in the crontab file itself.
Syntax Errors in Crontab
Crontab syntax is strict. A single extra space or incorrect field count can disable the entire entry. The format is
minute hour day-of-month month day-of-week command. Each field must contain a valid value or an asterisk (
*) for "every". Remember that days of the week are 0-7, where both 0 and 7 represent Sunday.
Ensure there are no trailing spaces after the command. Also, avoid using tabs if your editor converts them inconsistently; spaces are safer. If you recently edited your crontab, verify the changes were saved correctly. Some editors may leave backup files open or save in a different encoding. It is good practice to restart the cron service after making significant changes, although many distributions apply changes immediately upon saving.
If you need to understand how environment variables like
CURL behave differently in non-interactive shells, reviewing
Understanding Window Specific URL Curl Output with Bash can provide context on why certain network tools might behave unexpectedly without proper initialization.
Handling Output and Errors
By default, cron emails stdout and stderr to the user who owns the crontab. If email is not configured, this output is lost. To capture errors, redirect both standard output and standard error to a log file in your crontab entry:
* * * * * /path/to/script.sh >> /var/log/my_script.log 2>&1
This redirection ensures that any error messages generated during execution are preserved for analysis. Without this, debugging becomes significantly harder as you lose visibility into why a command failed.
Frequently Asked Questions
Why does my script work manually but fail in cron?
The primary reason is the difference in environment variables between an interactive login shell and the minimal cron environment. Cron does not load your
.bashrc or
.profile, so paths to executables and custom variables are often missing. Always use absolute paths for all commands and scripts.
How can I see detailed error messages from a cron job?
Redirect both standard output and standard error to a log file in your crontab entry using
> /path/to/log 2>&1. Alternatively, check the system journal with
journalctl -u cron.service to see if the cron daemon itself reports errors during execution.
Does cron respect the PATH variable from my .bash_profile?
No, cron uses a predefined, minimal
$PATH that typically includes directories like
/usr/bin and
/bin. It does not inherit your user’s customized
$PATH. You must either use absolute paths in your scripts or explicitly define the
$PATH at the top of your crontab file before defining jobs.
What should I do if my cron job runs but produces no output?
First, verify that the script is executable with
chmod +x. Second, check if the script is failing silently due to unhandled exceptions. Add logging statements inside the script to confirm it reaches specific points. Finally, ensure the script writes to a directory where the cron user has write permissions.
Can I use relative paths in my cron job command?
While you can use relative paths for the script itself (resolved against the home directory), it is highly recommended to use absolute paths for reliability. Relative paths for internal operations within the script should also be avoided or converted to absolute paths to prevent issues if the working directory changes unexpectedly.