Cookie Consent by Cookie Consent

Virtualenvwrapper Installation and Usage

Development and Programming 2019-03-08 FriRead Time: 7 min.

Introduction

In a previous post we saw how we can create isolated python environments so that we have different python packages, python versions, or both, for specific projects, without messing with each other python environment.

Here we will take a look at a very useful tool, Doug Hellmann's virtualenvwrapper, which allows us to create, delete and overall manage our python virtual environments in a seamless way. Like its name suggests, it is essentially a wrapper for Ian Bicking’s virtualenv tool but we will see that it can make our python life much easier.

Installation

A simple way of obtaining virtualenvwrapper is to install it via pip. Depending on your Linux distribution, the virtualenvwrapper.sh script you will need to source may be located in somewhat unexpected locations, even through pip, because of the default location of the python site-packages, i.e:

  • Ubuntu 18.04:
root@ubu-ws ~# dpkg -S virtualenvwrapper | grep virtualenvwrapper.sh
virtualenvwrapper: /usr/share/virtualenvwrapper/virtualenvwrapper.sh
root@ubu-ws ~# apt remove -y virtualenvwrapper
root@ubu-ws ~# pip install virtualenvwrapper
root@ubu-ws ~# whereis virtualenvwrapper.sh
virtualenvwrapper: /usr/local/bin/virtualenvwrapper.sh
root@ubu-ws ~#
  • Debian 9
root@deb-ws ~# dpkg -S virtualenvwrapper | grep virtualenvwrapper.sh
virtualenvwrapper: /usr/share/virtualenvwrapper/virtualenvwrapper.sh
root@deb-ws ~# apt remove -y virtualenvwrapper
root@deb-ws ~# pip install virtualenvwrapper
root@deb-ws ~# whereis virtualenvwrapper.sh
virtualenvwrapper: /usr/local/bin/virtualenvwrapper.sh
root@deb-ws ~#
  • CentOS 7
[root@cen-ws ~]# repoquery -l python-virtualenvwrapper | grep virtualenvwrapper.sh
/etc/profile.d/virtualenvwrapper.sh
/usr/bin/virtualenvwrapper.sh
[root@cen-ws ~]# pip install virtualenvwrapper
[root@cen-ws ~]# whereis virtualenvwrapper.sh
virtualenvwrapper: /usr/bin/virtualenvwrapper.sh
[root@cen-ws ~]#

Keep in mind that it should be installed into the same global site-packages location where virtualenv is installed [1], and so you should install virtualenvwrapper the same way you installed virtualenv package, or just install virtualenvwrapper if you have not installed virtualenv previously.

~# pip install virtualenvwrapper
~# whereis virtualenvwrapper.sh
virtualenvwrapper: /usr/local/bin/virtualenvwrapper.sh
~#

Enable at Shell Startup

Note that on CentOS systems we get an extra script, /etc/profile.d/virtualenvwrapper.sh. For those of you not familiar with the /etc/profile file and the /etc/profile.d directory (take a look at the end of /etc/profile), these locations correspond to system-wide startup files used by the Bourne shell (aka Bash) and Bourne compatible shells (ksh, ash, ...), and are sourced by your shell when you:

  • login via TTY
  • login via SSH
  • launch a new shell

So, we should configure loading of the virtualenvwrapper in either

  • /etc/profile
  • /etc/profile.d/virtualenvwrapper.sh
  • ~/.bashrc (user specific)
  • ~/.profile (user specific)

and its content should be [1]:

1
2
3
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/Devel
source /usr/local/bin/virtualenvwrapper.sh

where WORKON_HOME is where your virtual environments will be stored, and PROJECT_HOME is the location of your development project directories (see [3]). The last line just loads all of the virtualenvwrapper methods.

After following the installation instructions above, you have to load the virtualenvwrapper methods, by launching a new shell, or if you already have a shell open you can source your startup file manually, i.e:

  ~$ . .profile

Usage

After loading the virtualenvwrapper.sh, you will have the following commands available:

Command Reference [2]
CommandDescription
mkvirtualenvCreate a new named (specified) environment, in the WORKON_HOME directory
mktmpenvCreate a new named (automatically generated) environment in the WORKON_HOME directory
lsvirtualenvList all of the environments
showvirtualenvShow the details for a single virtualenv
rmvirtualenvRemove an environment, in the WORKON_HOME
cpvirtualenvDuplicate an existing virtualenv environment. The source can be an environment managed by virtualenvwrapper or an external environment created elsewhere.
allvirtualenvRun a command in all virtualenvs under WORKON_HOME
workonList or change working virtual environments
deactivateSwitch from a virtual environment to the system-installed version of Python
cdvirtualenvChange the current working directory to $VIRTUAL_ENV
cdsitepackagesChange the current working directory to the site-packages for $VIRTUAL_ENV
lssitepackagesShows the content of the site-packages directory of the currently-active virtualenv
add2virtualenvAdds the specified directories to the Python path for the currently-active virtualenv
toggleglobalsitepackagesControls whether the active virtualenv will access the packages in the global Python site-packages directory
mkprojectCreate a new virtualenv in the WORKON_HOME and project directory in PROJECT_HOME
setvirtualenvprojectBind an existing virtualenv to an existing project
cdprojectChange the current working directory to the one specified as the project directory for the active virtualenv
wipeenvRemove all of the installed third-party packages in the current virtualenv
virtualenvwrapperPrint a list of commands and their descriptions as basic help output

In this post we will not go through every command, but we will go through a process similiar to the previous post to get you started. For more info about each command please see the official documentation

Managing your Virtual Environments

First let us see the command you will probably use the most workon that, when you do not pass any arguments, it shows a list of all the available virtual environments (located at WORKON_HOME)

~$ workon
~$

Well, we have not created any yet, so let us create a new one.

~$ mkvirtualenv AdS
Running virtualenv with interpreter /usr/bin/python2
New python executable in /home/t0rrant/.virtualenvs/AdS/bin/python2
Also creating executable in /home/t0rrant/.virtualenvs/AdS/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
(AdS) ~$ workon
AdS
(AdS) ~$

Looks good so let us install the sympy package for this environment

(AdS) ~$ pip install sympy
Collecting sympy
  Downloading https://files.pythonhosted.org/packages/dd/f6/ed485ff22efdd7b371d0dbbf6d77ad61c3b3b7e0815a83c89cbb38ce35de/sympy-1.3.tar.gz (5.9MB)
    100% |████████████████████████████████| 5.9MB 3.5MB/s
Collecting mpmath>=0.19 (from sympy)
  Downloading https://files.pythonhosted.org/packages/ca/63/3384ebb3b51af9610086b23ea976e6d27d6d97bf140a76a365bd77a3eb32/mpmath-1.1.0.tar.gz (512kB)
    100% |████████████████████████████████| 522kB 4.9MB/s
Building wheels for collected packages: sympy, mpmath
  Building wheel for sympy (setup.py) ... done
  Stored in directory: /home/t0rrant/.cache/pip/wheels/6c/59/86/478e3c0f298368c119095cc5985dedac57c0e35a85c737f823
  Building wheel for mpmath (setup.py) ... done
  Stored in directory: /home/t0rrant/.cache/pip/wheels/63/9d/8e/37c3f6506ed3f152733a699e92d8e0c9f5e5f01dea262f80ad
Successfully built sympy mpmath
Installing collected packages: mpmath, sympy
Successfully installed mpmath-1.1.0 sympy-1.3
(AdS) ~$

We can confirm that the sympy package is only available to this environment:

(AdS) ~$ python -m sympy.abc
(AdS) ~$ deactivate
~$ python -m sympy.abc
/usr/bin/python: No module named sympy
~$ workon AdS
(AdS) ~$

Assuming that we have a project called AdS, located in our PROJECT_HOME directory, we can associate the newly created environment to that project, via the setvirtualenvproject command [3]:

(AdS) ~$ cdvirtualenv
(AdS) ~/.virtualenvs/AdS$ setvirtualenvproject $PWD ~/Devel/AdS
Setting project for AdS to /home/t0rrant/Devel/AdS
(AdS) ~/.virtualenvs/AdS$ cdproject
(AdS) ~/Devel/AdS$ ls
AdS_sympy_test.py

As you can see we can now call cdproject which will take us to the project associated to the currently active virtual environment. Neat =)

Note: At this time (virtualenvwrapper v4.8.4) you can only have one project associated to one virtual environment (project foo should work with environment A), or one project associated with any number of projects (project bar should work with environments A, B, ..., Z), you cannot have one virtual environment associated with multiple projects. You can of course activate your desired virtual environment and use it within any directory, you will just not be able to use cdproject to change to the root directory of your project.

Just to check if the sympy is working in our environment:

1
2
3
4
5
# AdS_sympy_test.py
#!/usr/bin/env python
from sympy import test

test("_basic")
(AdS) ~/Devel/AdS$ python AdS_sympy_test.py
============================= test process starts ==============================
executable:         /home/t0rrant/.virtualenvs/AdS/bin/python  (2.7.15-candidate-1) [CPython]
architecture:       64-bit
cache:              yes
ground types:       python
numpy:              None
random seed:        21534663
hash randomization: on (PYTHONHASHSEED=417663367)

sympy/core/tests/test_basic.py[16] ................                         [OK]

================== tests finished: 16 passed, in 0.06 seconds ==================
(AdS) ~/Devel/AdS$

Now we already did everything with this virtual environment so we exit it:

(AdS) ~/Devel/AdS$ deactivate
~/Devel/AdS$

and we can delete the virtual environment as our project development is now finished:

~/Devel/AdS$ rmvirtualenv AdS
Removing AdS...
~/Devel/AdS$ workon
~/Devel/AdS$

And we are done!

Conclusion

In this post we saw how to install and use virtualenvwrapper to manage our Python virtual environments. We followed a development flow similar to a previous post about Python virtual environment, ending with a similar environment. The virtualenvwrapper tool is very versatile, and it is possible to extend it via plugins, which was not covered here. These extensions can allow you to, i.e: create new projects based on predefined templates, or interact with external tools, for more information take a look at the extensions page.

Like always, feel free to post any comments or questions in the comment box below.

References

[1](1, 2) --, Installation - virtualenvwrapper 4.8.4.dev1 Documentation. [link]
[2]--, Command Reference - virtualenvwrapper 4.8.4.dev1 Documentation. [link]
[3](1, 2) --, Project Management - virtualenvwrapper 4.8.4.dev1 Documentation. [link]

Tags: linux slurm python scheduler virtualenv sympy


avatar
Manuel Torrinha is an information systems engineer, with more than 10 years of experience in managing GNU/Linux environments. Has an MSc in Information Systems and Computer Engineering. Work interests include High Performance Computing, Data Analysis, and IT management and Administration. Knows diverse programming, scripting and markup languages. Speaks Portuguese and English.

Related Content


comments powered by Disqus