Config module

In larger projects, where the Python code spans over several files, it’s common practice to extract all tunable parameters and constants into a dedicated config module. Not only does this approach bundle all declarations together, but the direct support of Python code inside the module offers the highest flexibility. For instance one can implement default values that can be readily overridden. In advanced setups, there’s even the possibility for more complicated secret management using GPG or querying some parameters directly from an API.

Sample project structure

├── README.markdown
├── cli.py*
└── pyproject/
    ├── __init__.py
    ├── analysis.py
    └── config/
        ├── __init__.py
        ├── overrides.py
        └── secrets.py.example

Where the executable cli.py provides command-line access to running the code

#!/usr/bin/env python
"""Pyproject command-line interface."""
from pyproject.analysis import run_data_analysis


if __name__ == "__main__":
    run_data_analysis()

The pyproject/__init__py can be empty and serves to define the package namespace. The analysis.py loads config parameters and defines how to perform the data analysis.

"""Data analysis algorithms."""
from pyproject.config import UNIVERSAL_RESULT, SCALE_FACTOR


def run_data_analysis():
    """Wrapper to run full data analysis."""
    print(f"Scale factor: {SCALE_FACTOR}")
    print(f"Scaled result: {UNIVERSAL_RESULT * SCALE_FACTOR}")

Last, but not least, the config/__init__.py module encapsulates the declaration of all parameters.

"""Config parameters for the py-project."""

UNIVERSAL_RESULT = 42

"""
Some default values below can be overridden by
adding definitions to the optional `overrides.py` file.
"""
try:
    from pyproject.config.overrides import SCALE_FACTOR
except ImportError:
    SCALE_FACTOR = 1.0

"""
Loading sensitive data from `secrets.py`.
"""
try:
    from pyproject.config.secrets import SECRETS
    API_CREDENTIALS = SECRETS("API_TOKEN")
except ImportError:
    print("Warning: running without definition of config/secrets.py")

All sensitive data is contained inside a dedicated secrets.py file, which is excluded from version control. However, a secrets.py.example is included, to serve as template and show what the file should contain.

"""
Secret parameters with sensitive data.

Copy this secrets.py.example file to secrets.py
and adapt the credentials below.
"""

SECRETS = {
    "API_TOKEN": "1234567890"
}