venv

venv#

Python includes (since version 3.3 released in 2012) the venv module to provide lightweight support for virtual environments, without the need for any third-party tools.

Usage#

When inside the folder of your Python project, the following command creates a new virtual environment

$ python -m venv .venv

where .venv is the conventional name for the virtual environment and the prepended dot makes the associated folder hidden on Unix systems.

Additional options allow to control some aspects during venv creation. See python -m venv --help for details. In particular you can grant your venv access to the system-wide packages. This can be useful on HPC clusters, where the number-crunching libraries have been optimized for the platform and should not simply be re-installed with default options inside the virtual environment.

$ python -m venv --system-site-packages .venv

Tip

When using a version control system like git, always make sure to ignore the .venv folder by adding it to your project’s .gitignore file. The virtual environment can quickly grow to over 100MB and is platform-dependent. It must therefore be excluded from the git history.

Once created, the virtual environment needs to be activated.

$ source .venv/bin/activate

C:\> .\venv\Scripts\Activate.ps1

Afterwards the pip and python commands should refer to the binaries inside the virtual environment.

$ which pip
/path/to/your/project/.venv/bin/pip

In particular pip will install packages locally into the virtual environment.

When done, you can either close your shell or deactivate the virtual environment

$ deactivate

Behind the scenes#

Let’s take a closer look at what is happening behind the scenes when creating and activating a virtual environment.

Contents of .venv#

We use the tree command to list the (first three levels of) contents of the .venv/ directory.

$ tree -L 3 .venv/
.venv/
├── bin/                    # directory with binaries and executable scripts
│   ├── Activate.ps1        # scripts to activate venv for different shells
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── pip*                # local pip wrapper scripts
│   ├── pip3*
│   ├── pip3.11*
│   ├── python -> python3.11*
│   ├── python3 -> python3.11*
│   └── python3.11 -> /path/to/your/python3.11*  # symlink to your Python interpreter
├── include/                # directory to contain C header files
├── lib/
│   └── python3.11/
│       └── site-packages/  # directory to contain local Python packages
└── pyvenv.cfg              # config file

Activation of .venv#

Feel free to inspect the code of .venv/bin/activate to see what exactly is happening when activating a virtual environment. We highlight the main points:

  1. the VIRTUAL_ENV shell environment variable is defined with the absolute path to the .venv directory. This variable can be used to check if one is running inside a virtual environment and display its name in the shell prompt.

  2. the .venv/bin folder is prepended to your PATH so that all executables inside take precedence. This ensures that running a pip command actually invokes the pip from inside the virtual environment, without having to give the full path .venv/bin/pip. Similarly for the python interpreter itself and any executables installed by the Python packages inside the virtual environment.

Note that, given the activation is merely changing your shell’s PATH, you don’t really need to first activate a virtual environment. Alternatively you may call the interpreter or scripts directly using their full .venv/bin/ path.

Conversely, when calling deactivate all changes are reverted by resetting the shell environment variables to their previous state.

Contents