Read Variables from .env File using Pydantic in Tech Support Site

Use Pydantic's BaseSettings class to automatically load and validate environment variables from a .env file. By defining a settings class with typed fields, Pydantic handles parsing, type conversion, and default values seamlessly, ensuring your application configuration is robust and error-free during startup.

Understanding BaseSettings and Environment Loading

The core of this solution relies on the pydantic.BaseSettings class, which extends standard Pydantic models to prioritize environment variables over explicit arguments or defaults. When you instantiate this class, it automatically scans the operating system's environment for variables matching the field names defined in your model. This mechanism eliminates the need for manual parsing logic using libraries like os.environ or third-party tools like python-dotenv, as Pydantic integrates these capabilities directly into its initialization process.

To make this work, you must ensure that the pydantic-settings package is installed alongside Pydantic itself. The BaseSettings class looks for a .env file in the current working directory by default. If found, it loads key-value pairs from this file before checking the actual system environment. This allows developers to maintain local configurations separately from production secrets, providing a clean separation of concerns without complex setup scripts.

Installation Requirements

Before implementing the settings class, verify that your environment includes the necessary dependencies. Run the following command to install both Pydantic and its settings extension:

pip install pydantic pydantic-settings

This installation ensures that the BaseSettings class is available for import. Without pydantic-settings, the base Pydantic library does not include the automatic environment variable loading functionality required for this pattern. Once installed, you can proceed to define your configuration structure.

Defining the Settings Class

Create a Python module dedicated to application configuration. Define a class that inherits from BaseSettings. Each attribute within this class represents an environment variable. You should specify the expected data type for each field, such as str, int, bool, or float. Pydantic will attempt to cast the string value from the environment into the specified type.

Consider the following example where we define settings for a database connection and an API key:

from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    db_host: str = "localhost"
    db_port: int = 5432
    api_key: str
    debug_mode: bool = False

    class Config:
        env_file = ".env"

In this snippet, db_host has a default value of "localhost". If the environment variable DB_HOST is not set, Pydantic uses this default. The api_key field has no default, making it a required variable; if it is missing from the environment and the .env file, Pydantic will raise a validation error upon instantiation. The nested Config class explicitly points to the .env file, though this is often the default behavior. Explicitly setting it improves code readability and intent.

Handling Variable Naming Conventions

Pydantic follows a specific convention for mapping class attributes to environment variables. By default, it converts the attribute name to uppercase and replaces underscores with double underscores if necessary, but typically, it expects the environment variable name to match the attribute name exactly in uppercase. For example, the attribute db_host maps to the environment variable DB_HOST.

If your .env file contains variables with different naming conventions, you can customize the mapping. However, sticking to the standard UPPER_SNAKE_CASE format is recommended for consistency across teams. Ensure that your .env file reflects these exact names:

DB_HOST=localhost
DB_PORT=5432
API_KEY=your_secret_key_here
DEBUG_MODE=true

Note that boolean values are case-insensitive. Setting DEBUG_MODE=true, True, or 1 will all evaluate to True in the Python context. Similarly, integers and floats are parsed according to standard numeric formats. This automatic parsing reduces boilerplate code significantly compared to manual string-to-type conversions.

Instantiating and Using Configuration

Once the class is defined, instantiate it at the entry point of your application. This single step triggers the loading of variables from the .env file and the system environment. You can then pass this settings object to other parts of your application, such as database clients or HTTP sessions.

# main.py
from my_settings import Settings

settings = Settings()

print(f"Connecting to {settings.db_host}:{settings.db_port}")
print(f"API Key loaded: {bool(settings.api_key)}")

If any required variable is missing or invalid, Pydantic raises a ValidationError. This fail-fast approach prevents your application from running with incomplete or malformed configuration, which is critical for maintaining stability in production environments. For more complex data structures, such as lists or dictionaries, Pydantic supports advanced types that can parse JSON strings from environment variables, offering flexibility beyond simple scalar types.

Common Pitfalls and Best Practices

Avoid hardcoding sensitive information directly in your source code. Always rely on the environment or .env files for secrets. Additionally, do not commit your .env file to version control systems. Instead, use a .env.example file to document the required variables for other developers. This practice maintains security while facilitating onboarding.

Another common issue arises when dealing with nested configurations. While BaseSettings supports nested models, deeply nested structures can become difficult to manage via flat environment variables. In such cases, consider using JSON-encoded strings for complex objects. For related configuration challenges, you might find it useful to explore how to handle data extraction errors or numeric type issues in Python, as seen in guides about extracting error data or resolving data type mismatches in DataFrames.

Frequently Asked Questions

Does Pydantic automatically reload .env changes?

No, Pydantic loads the environment variables only once when the Settings instance is created. It does not monitor the .env file for changes during runtime. To apply new configuration, you must restart the application or manually re-instantiate the settings object if your framework supports dynamic reloading.

How do I handle optional variables without defaults?

If a variable is optional but lacks a default value, Pydantic treats it as required unless you provide a default. To make a field truly optional, assign a default value of None and annotate it as Optional[str] or similar. This allows the variable to be absent from the environment without raising a validation error.

Can I use Pydantic with FastAPI?

Yes, Pydantic is the default configuration tool for FastAPI applications. FastAPI projects frequently use BaseSettings to manage app configuration, leveraging the same patterns described here. This integration provides a seamless experience for building scalable web services with validated configuration.

What happens if a variable has the wrong type?

Pydantic attempts to coerce the string value from the environment into the defined type. If the conversion fails, such as trying to parse "not_a_number" as an int, Pydantic raises a ValidationError. This stops the application from starting, alerting you to the misconfiguration immediately rather than causing cryptic errors later.

Is python-dotenv still needed?

Generally, no. pydantic-settings includes built-in support for loading .env files, making python-dotenv redundant for basic usage. However, you might still use python-dotenv if you need to load multiple .env files with specific precedence rules or perform custom loading logic outside of Pydantic's scope.