For everyone using or writing Python programs, there eventually comes a time when additional libraries have to be installed. The Python ecosystem makes getting hold of them very easy, but that also means it’s easy to end up with a tangled mess of system libraries, user libraries, libraries which override system libraries, and so on. Let’s make some sense of the different installation methods.
System packages
This is perhaps the easiest and least tangled installation method, but it does limit you to the libraries (and versions) packaged by your Linux distribution. Different distributions have different packaging policies and available libraries, so the one you need might not be available. Debian-based distributions generally name their packages python-* (for example, python-pymongo for the MongoDB drivers) whereas Red Hat systems are less consistent, and sometimes – but not always – match the Python module name (in the same example, pymongo.x86_64).
So to install PyMongo for a Debian system, it’s:
# apt-get install python-pymongo
and for a Red Hat system:
# yum install pymongo
The PIP tool
When libraries aren’t packaged by the distribution at all, or a different version is needed, pip(and its Python 3 cousin, pip3) can download and install them locally outside of the distribution’s package manager. If invoked as a normal user, pip install downloads module to a user-specific location so that others aren’t affected – they will continue to use the system library, if available. When invoked as the root user, the library is installed system-wide and overrides the distribution’s version:
$ pip install pymongo
Collecting pymongo
Downloading https://files.pythonhosted.org/packages/04/9c/18d54ce25a3219b13cf92616de87cac7a8e2eab26eb2947606010669d41c/pymongo-3.7.2-cp27-cp27mu-manylinux1_x86_64.whl (411kB)
100% |████████████████████████████████| 419kB 609kB/s
Installing collected packages: pymongo
Successfully installed pymongo-3.7.2
$ python
Python 2.7.13 (default, Sep 26 2018, 18:42:22)
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymongo
>>> print pymongo.__version__
3.7.2
At this point, you’re taking responsibility for keeping pymongo up-to-date with any relevant security fixes. The local installation overrides the distribution’s packages even when they are a lower version number, so you need to monitor the upstream project to ensure you’re not left exposed.
Virtual Environments
To avoid polluting the system environment with lots of overridden modules, a completely self-contained environment can be created with the virtualenvtool (available through your distribution’s package manager or a local pip installation). Once created, the environment is activated and then Python commands including pip apply only to the virtual environment, not the system:
$ virtualenv myenv Running virtualenv with interpreter /usr/bin/python2 New python executable in /home/user/myenv/bin/python2 Also creating executable in /home/user/myenv/bin/python Installing setuptools, pkg_resources, pip, wheel...done. $ source myenv/bin/activate (myenv) $ which python /home/user/myenv/bin/python (myenv) $ pip install pymongo Collecting pymongo Using cached https://files.pythonhosted.org/packages/04/9c/18d54ce25a3219b13cf92616de87cac7a8e2eab26eb2947606010669d41c/pymongo-3.7.2-cp27-cp27mu-manylinux1_x86_64.whl Installing collected packages: pymongo Successfully installed pymongo-3.7.2 (myenv) $ pip freeze pkg-resources==0.0.0 pymongo==3.7.2
The virtualenv command takes the directory to be used for the environment, and creates an skeleton environment there. It is activated with the source shell built-in, and the environment name is prepended onto the prompt so that it’s clear we’re in a virtual environment. From now until the terminal is closed, pip, python and installed modules will act only within the virtual environment – as pip freeze shows, only the pymongo module is available because it’s the only module we’ve installed.
This technique ensures a pristine and controlled environment for each application which needs to use it, but at the same cost as local system installations – it’s up to you to keep it up-to-date now. Modules installed in the environment do not override the system environment as a matter of course – the environment must be activated first, so that allows for much more control.
virtualenv is quite a low-level tool – the Python project recommends the more friendly pipenv for managing environments.
Photo by David Clode on Unsplash