Why does a Python script only use 6 out of 12 cores?

A Python script typically uses only half the available cores because of the Global Interpreter Lock (GIL), which prevents multiple native threads from executing Python bytecodes simultaneously in CPython. To utilize all twelve cores, you must switch from multi-threading to multi-processing or use libraries that release the GIL, such as NumPy or Cython.

The Root Cause: The Global Interpreter Lock

The primary reason your Python application does not scale across all CPU cores is the Global Interpreter Lock (GIL). In standard CPython, the GIL is a mutex that protects access to Python objects, ensuring that only one thread executes Python bytecode at a time. This design simplifies memory management and makes the interpreter single-threaded safe, but it effectively serializes CPU-bound tasks running in separate threads. When you create multiple threads using threading.Thread, the operating system schedules them on different cores. However, the GIL forces these threads to take turns accessing the interpreter. If your task is CPU-intensive, such as heavy mathematical calculations or data processing, the threads spend most of their time waiting for the lock rather than performing useful work. Consequently, you observe high CPU usage on only one or a few cores, while the rest remain idle, regardless of how many threads you spawn. This limitation is specific to CPython. Other implementations like Jython or IronPython do not have a GIL, but they are rarely used in modern production environments. For the vast majority of Python developers, the GIL is a fundamental constraint of the standard library's threading model. Understanding this mechanism is crucial before attempting to optimize performance, as adding more threads will not solve the bottleneck for CPU-bound workloads.

Solution 1: Multi-Processing with the multiprocessing Module

The most robust solution for CPU-bound tasks is to bypass the GIL entirely by using processes instead of threads. Each Python process has its own independent interpreter instance and its own GIL. Therefore, spawning multiple processes allows true parallel execution across all available CPU cores. You can achieve this using the built-in multiprocessing module. Instead of creating threads, you create processes. The overhead of inter-process communication is higher than thread communication, but for CPU-heavy tasks, the benefit of parallel execution outweighs this cost. Here is a basic example of how to structure your code:

import multiprocessing
import os

def worker():
    # Perform CPU-intensive work here
    result = sum(i * i for i in range(1000000))
    print(f"Worker {os.getpid()} finished")

if __name__ == "__main__":
    # Create a pool of workers equal to the number of cores
    num_cores = multiprocessing.cpu_count()
    with multiprocessing.Pool(processes=num_cores) as pool:
        results = pool.map(worker, range(num_cores))

In this example, multiprocessing.cpu_count() detects the number of logical processors, which should return 12 for your system. The Pool object manages a fixed number of worker processes. When you call pool.map, it distributes the workload evenly among these processes. Each process runs independently, allowing all 12 cores to be utilized simultaneously. For more complex scenarios where you need fine-grained control over process creation and communication, you can use Process objects directly along with Queue or Pipe for data exchange. However, for most bulk processing tasks, the Pool interface provides the best balance of simplicity and performance.

Solution 2: Libraries That Release the GIL

Not all Python code is affected by the GIL. Many popular scientific computing libraries, such as NumPy, Pandas, and SciPy, are implemented in C or Fortran. These libraries release the GIL during long-running computational operations. This means that if your Python script primarily delegates work to these libraries, you may see better utilization of multiple cores even without explicit multi-processing. However, this behavior is not guaranteed for all functions within these libraries. You must check the documentation for specific functions to see if they release the GIL. For custom C extensions, you can explicitly release the GIL using the Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros in C code. If you are writing performance-critical code, consider using tools like Cython or Numba. Cython allows you to write C-like code in Python and compile it, giving you precise control over the GIL. Numba can JIT-compile Python functions to machine code, often releasing the GIL automatically for numerical loops. For general-purpose parallelism beyond simple function mapping, consider using distributed computing frameworks like Ray or Dask. These tools provide higher-level abstractions for parallel computing and can handle larger-than-memory datasets efficiently. They manage process distribution and data serialization automatically, reducing the boilerplate code required for manual multi-processing.

Diagnostics and Verification

Before implementing changes, verify that your system actually has 12 accessible cores. Use nproc on Linux or PowerShell’s (Get-CimInstance Win32_Processor).NumberOfLogicalProcessors on Windows. Sometimes, virtualization settings or power management profiles limit the number of active cores. Use monitoring tools like htop on Linux or Task Manager on Windows to observe CPU usage per core. If you see spikes on only 6 cores, it confirms the GIL bottleneck. After switching to multi-processing, monitor again to ensure all 12 cores show activity during execution. How to Run Any GGUF Model Using Transformers or Any Other Library discusses similar optimization challenges in AI workloads, where efficient resource utilization is critical.

Frequently Asked Questions

Does the GIL affect I/O-bound tasks?

No, the GIL does not significantly impact I/O-bound tasks like reading files or making network requests. During I/O operations, threads release the GIL, allowing other threads to run. Therefore, multi-threading is still effective for applications that spend most of their time waiting for external resources.

Can I remove the GIL from Python?

You cannot remove the GIL from standard CPython without rebuilding the interpreter with experimental patches. While there are ongoing projects to make the GIL optional or remove it entirely, no stable release exists yet. For now, using multi-processing is the standard workaround.

Is multi-processing slower than multi-threading?

Multi-processing has higher overhead due to process creation and inter-process communication costs. However, for CPU-bound tasks, the speedup from parallel execution far outweighs this overhead. For I/O-bound tasks, multi-threading remains faster due to lower context-switching costs.

How do I share data between processes?

You can share data using multiprocessing.Queue, multiprocessing.Pipe, or shared memory objects like Array and Value. Avoid sharing mutable state directly; instead, pass data through queues or use read-only shared memory for performance.

What happens if I use both threads and processes?

You can combine both approaches, but it adds complexity. Typically, you use processes for CPU-bound parallelism and threads for concurrent I/O within each process. Ensure that any threading within a process does not contend heavily on the GIL for CPU work.