How to find out exactly which process was killed by OOM killer?

Have you ever encountered the term "OOM killer" while troubleshooting your computer and wondered what it means? OOM stands for "Out of Memory," and the OOM killer is a feature in the Linux kernel that helps prevent system crashes when memory is running low. When the system is low on memory, the OOM killer selects and terminates processes to free up memory and keep the system running smoothly.

However, when the OOM killer kills a process, it can be challenging to determine exactly which process was terminated. In this article, we will explore various methods to find out which process was killed by the OOM killer, helping you diagnose and resolve any memory-related issues.

Method 1: Checking System Logs

The first method involves checking the system logs, which contain valuable information about the OOM killer's actions. Follow the steps below to access the system logs:

1. Open a terminal window.
2. Type the following command to view the system logs:
   sudo tail -f /var/log/syslog
3. Press Enter.

By running the above command, you will see the system logs in real-time, including any OOM killer events. Look for lines that mention the OOM killer, such as "Out of memory" or "Killed process." The logs usually include the process ID (PID) of the terminated process, which can help you identify it.

Method 2: Using the dmesg Command

If you prefer a command-line approach, you can use the dmesg command to retrieve information about the OOM killer's actions. Follow the steps below:

1. Open a terminal window.
2. Type the following command:
   dmesg | grep -i "Out of memory"
3. Press Enter.

This command filters the system logs for any lines containing the phrase "Out of memory." The output will display information about the OOM killer events, including the PID of the terminated process.

Method 3: Analyzing Memory Dump Files

If you have enabled memory dump files on your system, you can analyze them to determine which process was killed by the OOM killer. Follow the steps below:

1. Open a terminal window.
2. Type the following command to locate the memory dump file:
   sudo find /var/crash -name "*.dump"
3. Press Enter.
4. Once you locate the memory dump file, use a tool like crash or gdb to analyze it. These tools require some technical expertise, so it is recommended to seek assistance from a knowledgeable person if you are unfamiliar with them.

By analyzing the memory dump file, you can extract information about the processes running at the time of the OOM event and identify the terminated process.

Method 4: Using Process Monitoring Tools

If you want to monitor your system in real-time to identify processes terminated by the OOM killer, you can use process monitoring tools like top or htop. These tools provide detailed information about system processes, including their memory usage.

Follow the steps below to use the htop tool:

1. Open a terminal window.
2. Type the following command to install htop if it is not already installed:
   sudo apt-get install htop
3. Press Enter.
4. Type the following command to run htop:
   htop
5. Press Enter.
6. Observe the list of processes displayed by htop. The processes using the most memory will be listed at the top. Look for any processes that are consuming an unusually high amount of memory.

If a process is using an excessive amount of memory, it may be a candidate for termination by the OOM killer.

Conclusion

By following the methods outlined in this article, you can determine which process was killed by the OOM killer. Understanding which process was terminated can help you identify memory-hungry applications or potential memory leaks, allowing you to optimize your system's performance and prevent future OOM events.

References