Blog: PHP, Python, Linux, Web services & Continuous delivery

Experimenting with virtualenv

virtualenv is a great tool that you can use to create isolated python environments that contain their own installation directories, libraries and package managers. This isolation enables you to control package dependencies at the application level rather than the system level and helps avoid the situation where upgrading a system library for the benefit of one application would cause a different application to break. However, its important to note that it does still share the base system's python installation with the virtual environment.

Its stable enough for production and provides the opportunity to ship your application with all dependencies bundled in. This is how I've been using it for development, all examples are in the context of installation on Debian Linux.

Create a new virtualenv

I generally use pip to install packages so I include the '--distibute' option. There are also other options (i.e. --extra-search-dir) to share some system locations with the virtualenv but I prefer to keep them as separate as possible.

You can store config options in a configuration file to save typing them every time, by default virtualenv will have a look for the config file virtualenv.ini in the directory $HOME/.virtualenv/.

# create a new virtual environment with distribute rather than setuptools
python virtualenv --distribute MyVirtualEnv

A new directory, MyVirtualEnv, will be placed within the current working directory. Here is a quick look at what is created:

 

MyVirtualEnv/

   bin/

   include/

   lib/

   Python2.7/

     abc.py -> /usr/lib/python2.7/abc.py

     codecs.py -> /usr/lib/python2.7/codecs.py

     etc.....

   local/

 

  • The bin directory contains the python interpreter that the virtual environment will use. The pip and easy_install package management scripts and several virtualenv activation scripts that we use (see below) to step into the new virtualenv are also found here.

  • The include directory contains a symlink to the system location /usr/include/pythonX.X. /usr/include/ is where all of the system's general-use include files for the C and C++ programming languages are placed.

  • The lib directory contains symlinks to the base system's python library for the version of python that you create the environment in.

  • The local directory contains just 3 symlinks for bin, include and lib which point to the three directories listed above.

Now the environment is created we can step into it. To do this we use the activate script in our virtualenv bin directory

source MyVirtualEnv/bin/activate

Your command line should now be prefixed with '(MyVirtualEnv)' which means that any python, pip and easy_install commands will be executed in the context of your virtualenv. For example:

which pip
# outputs /path/to/your/virtualenv/MyVirtualEnv/bin/pip
which python
# outputs /path/to/your/virtualenv/MyVirtualEnv/bin/python

Now I'll install some packages that I'm experimenting with and do some development...

pip install requests
pip install scrapy

When your finished and want to throw everything away all you need to do is just delete the MyVirtualEnv directory.

Virtualenv with different versions of Python

As shown above, the environment created with virtualenv uses the base system's python installation. If you have multiple versions of Python installed then you can tell virtualenv which one to use when you initially create the environment using the '--python' option. For example, If i had Python version 2.7 and Python version 3.0 installed on my OS and I want to create a virtualenv environment using version 3.0 of Python I would run the following:

python virtualenv --distribute -p virtualenv --python=/usr/bin/python3.0 MyVirtualEnv 

If you examine the MyVirtualEnv/lib directory then you should see a Python3.0 sub directory that contains symlinks to your OS's python 3 libraries.

comments powered by Disqus