Virtualenvwrapper Installation and Usage
Introduction
In a previous post we saw how we can create isolated python environments so that we have different python packages, python versions, or both, for specific projects, without messing with each other python environment.
Here we will take a look at a very useful tool, Doug Hellmann's virtualenvwrapper, which allows us to create, delete and overall manage our python virtual environments in a seamless way. Like its name suggests, it is essentially a wrapper for Ian Bicking’s virtualenv tool but we will see that it can make our python life much easier.
Installation
A simple way of obtaining virtualenvwrapper is to install it via pip. Depending on your Linux distribution, the virtualenvwrapper.sh script you will need to source may be located in somewhat unexpected locations, even through pip, because of the default location of the python site-packages, i.e:
- Ubuntu 18.04:
root@ubu-ws ~# dpkg -S virtualenvwrapper | grep virtualenvwrapper.sh
virtualenvwrapper: /usr/share/virtualenvwrapper/virtualenvwrapper.sh
root@ubu-ws ~# apt remove -y virtualenvwrapper
root@ubu-ws ~# pip install virtualenvwrapper
root@ubu-ws ~# whereis virtualenvwrapper.sh
virtualenvwrapper: /usr/local/bin/virtualenvwrapper.sh
root@ubu-ws ~#
- Debian 9
root@deb-ws ~# dpkg -S virtualenvwrapper | grep virtualenvwrapper.sh
virtualenvwrapper: /usr/share/virtualenvwrapper/virtualenvwrapper.sh
root@deb-ws ~# apt remove -y virtualenvwrapper
root@deb-ws ~# pip install virtualenvwrapper
root@deb-ws ~# whereis virtualenvwrapper.sh
virtualenvwrapper: /usr/local/bin/virtualenvwrapper.sh
root@deb-ws ~#
- CentOS 7
[root@cen-ws ~]# repoquery -l python-virtualenvwrapper | grep virtualenvwrapper.sh
/etc/profile.d/virtualenvwrapper.sh
/usr/bin/virtualenvwrapper.sh
[root@cen-ws ~]# pip install virtualenvwrapper
[root@cen-ws ~]# whereis virtualenvwrapper.sh
virtualenvwrapper: /usr/bin/virtualenvwrapper.sh
[root@cen-ws ~]#
Keep in mind that it should be installed into the same global site-packages location where virtualenv is installed [1], and so you should install virtualenvwrapper the same way you installed virtualenv package, or just install virtualenvwrapper if you have not installed virtualenv previously.
~# pip install virtualenvwrapper
~# whereis virtualenvwrapper.sh
virtualenvwrapper: /usr/local/bin/virtualenvwrapper.sh
~#
Enable at Shell Startup
Note that on CentOS systems we get an extra script, /etc/profile.d/virtualenvwrapper.sh. For those of you not familiar with the /etc/profile file and the /etc/profile.d directory (take a look at the end of /etc/profile), these locations correspond to system-wide startup files used by the Bourne shell (aka Bash) and Bourne compatible shells (ksh, ash, ...), and are sourced by your shell when you:
- login via TTY
- login via SSH
- launch a new shell
So, we should configure loading of the virtualenvwrapper in either
- /etc/profile
- /etc/profile.d/virtualenvwrapper.sh
- ~/.bashrc (user specific)
- ~/.profile (user specific)
and its content should be [1]:
|
|
where WORKON_HOME is where your virtual environments will be stored, and PROJECT_HOME is the location of your development project directories (see [3]). The last line just loads all of the virtualenvwrapper methods.
After following the installation instructions above, you have to load the virtualenvwrapper methods, by launching a new shell, or if you already have a shell open you can source your startup file manually, i.e:
~$ . .profile
Usage
After loading the virtualenvwrapper.sh, you will have the following commands available:
Command | Description |
---|---|
mkvirtualenv | Create a new named (specified) environment, in the WORKON_HOME directory |
mktmpenv | Create a new named (automatically generated) environment in the WORKON_HOME directory |
lsvirtualenv | List all of the environments |
showvirtualenv | Show the details for a single virtualenv |
rmvirtualenv | Remove an environment, in the WORKON_HOME |
cpvirtualenv | Duplicate an existing virtualenv environment. The source can be an environment managed by virtualenvwrapper or an external environment created elsewhere. |
allvirtualenv | Run a command in all virtualenvs under WORKON_HOME |
workon | List or change working virtual environments |
deactivate | Switch from a virtual environment to the system-installed version of Python |
cdvirtualenv | Change the current working directory to $VIRTUAL_ENV |
cdsitepackages | Change the current working directory to the site-packages for $VIRTUAL_ENV |
lssitepackages | Shows the content of the site-packages directory of the currently-active virtualenv |
add2virtualenv | Adds the specified directories to the Python path for the currently-active virtualenv |
toggleglobalsitepackages | Controls whether the active virtualenv will access the packages in the global Python site-packages directory |
mkproject | Create a new virtualenv in the WORKON_HOME and project directory in PROJECT_HOME |
setvirtualenvproject | Bind an existing virtualenv to an existing project |
cdproject | Change the current working directory to the one specified as the project directory for the active virtualenv |
wipeenv | Remove all of the installed third-party packages in the current virtualenv |
virtualenvwrapper | Print a list of commands and their descriptions as basic help output |
In this post we will not go through every command, but we will go through a process similiar to the previous post to get you started. For more info about each command please see the official documentation
Managing your Virtual Environments
First let us see the command you will probably use the most workon that, when you do not pass any arguments, it shows a list of all the available virtual environments (located at WORKON_HOME)
~$ workon
~$
Well, we have not created any yet, so let us create a new one.
~$ mkvirtualenv AdS
Running virtualenv with interpreter /usr/bin/python2
New python executable in /home/t0rrant/.virtualenvs/AdS/bin/python2
Also creating executable in /home/t0rrant/.virtualenvs/AdS/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
(AdS) ~$ workon
AdS
(AdS) ~$
Looks good so let us install the sympy package for this environment
(AdS) ~$ pip install sympy
Collecting sympy
Downloading https://files.pythonhosted.org/packages/dd/f6/ed485ff22efdd7b371d0dbbf6d77ad61c3b3b7e0815a83c89cbb38ce35de/sympy-1.3.tar.gz (5.9MB)
100% |████████████████████████████████| 5.9MB 3.5MB/s
Collecting mpmath>=0.19 (from sympy)
Downloading https://files.pythonhosted.org/packages/ca/63/3384ebb3b51af9610086b23ea976e6d27d6d97bf140a76a365bd77a3eb32/mpmath-1.1.0.tar.gz (512kB)
100% |████████████████████████████████| 522kB 4.9MB/s
Building wheels for collected packages: sympy, mpmath
Building wheel for sympy (setup.py) ... done
Stored in directory: /home/t0rrant/.cache/pip/wheels/6c/59/86/478e3c0f298368c119095cc5985dedac57c0e35a85c737f823
Building wheel for mpmath (setup.py) ... done
Stored in directory: /home/t0rrant/.cache/pip/wheels/63/9d/8e/37c3f6506ed3f152733a699e92d8e0c9f5e5f01dea262f80ad
Successfully built sympy mpmath
Installing collected packages: mpmath, sympy
Successfully installed mpmath-1.1.0 sympy-1.3
(AdS) ~$
We can confirm that the sympy package is only available to this environment:
(AdS) ~$ python -m sympy.abc
(AdS) ~$ deactivate
~$ python -m sympy.abc
/usr/bin/python: No module named sympy
~$ workon AdS
(AdS) ~$
Assuming that we have a project called AdS, located in our PROJECT_HOME directory, we can associate the newly created environment to that project, via the setvirtualenvproject command [3]:
(AdS) ~$ cdvirtualenv
(AdS) ~/.virtualenvs/AdS$ setvirtualenvproject $PWD ~/Devel/AdS
Setting project for AdS to /home/t0rrant/Devel/AdS
(AdS) ~/.virtualenvs/AdS$ cdproject
(AdS) ~/Devel/AdS$ ls
AdS_sympy_test.py
As you can see we can now call cdproject which will take us to the project associated to the currently active virtual environment. Neat =)
Note: At this time (virtualenvwrapper v4.8.4) you can only have one project associated to one virtual
environment (project foo should work with environment A), or one project associated with any number of
projects (project bar should work with environments A, B, ..., Z), you cannot have one virtual
environment associated with multiple projects. You can of course activate your desired virtual environment and use it
within any directory, you will just not be able to use cdproject
to change to the root directory of your
project.
Just to check if the sympy is working in our environment:
|
|
(AdS) ~/Devel/AdS$ python AdS_sympy_test.py
============================= test process starts ==============================
executable: /home/t0rrant/.virtualenvs/AdS/bin/python (2.7.15-candidate-1) [CPython]
architecture: 64-bit
cache: yes
ground types: python
numpy: None
random seed: 21534663
hash randomization: on (PYTHONHASHSEED=417663367)
sympy/core/tests/test_basic.py[16] ................ [OK]
================== tests finished: 16 passed, in 0.06 seconds ==================
(AdS) ~/Devel/AdS$
Now we already did everything with this virtual environment so we exit it:
(AdS) ~/Devel/AdS$ deactivate
~/Devel/AdS$
and we can delete the virtual environment as our project development is now finished:
~/Devel/AdS$ rmvirtualenv AdS
Removing AdS...
~/Devel/AdS$ workon
~/Devel/AdS$
And we are done!
Conclusion
In this post we saw how to install and use virtualenvwrapper to manage our Python virtual environments. We followed a development flow similar to a previous post about Python virtual environment, ending with a similar environment. The virtualenvwrapper tool is very versatile, and it is possible to extend it via plugins, which was not covered here. These extensions can allow you to, i.e: create new projects based on predefined templates, or interact with external tools, for more information take a look at the extensions page.
Like always, feel free to post any comments or questions in the comment box below.
Tags: linux slurm python scheduler virtualenv sympy
Related Content
- Creating Custom Resources for Your Cookbooks
- Hitchhiker's Guide to Chef
- An Advanced Guide to Salt
- A Comprehensive Introduction to Salt
- Slurm in Ubuntu Clusters - Part 2