Adding Python modules

It is strongly recommended to use and maintain your own python virtual environments, you can add any python module you want within it. See the instructions in the next section.

If you want to use a Python module on Ponyland globally that is not yet available, but packaged in Ubuntu, then please contact admin. Be warned, though, that it might take a few days before everything will be configured correctly.

A quick introduction to virtualenv

Virtualenv is a program that allows you to create 'virtual Python environments'. Using virtualenv has two main advantages:

• You no longer have to worry about version differences and dependencies. That is, if your script uses an older version of module x, and module x is updated system-wide, your script might break. If you build and use your script inside a virtualenv, virtualenv makes sure the right version of the right library is always available.
• You no longer have to worry about permissions. Inside your virtualenv you're free to install any module you want.

To create an environment, simply type:

 $ virtualenv env

This will create a folder named env (you can call it whatever you want) under the current directory, which will include a special python interpreter and a place to store your modules. Alternatively, you can also do:

 $ python3 -m venv env

To use your environment, type:

$ source env/bin/activate

Within your environment you now now you pip install as usual for whatever you want to install.

To stop using it, type:

$ deactivate

For more info see this page.

Quota, cache, and disk-space issues

On ponyland, your home directory is severely limited in size (there is a quota on it). We therefore do NOT recommend building any virtual environment there. Instead, use your own /vol/tensusers/ directory.

Even if you don't use your home directory, pip might still use it to cache file in ~/.cache/pip, which may still lead to errors like not enough space on disk. You can safely delete this directory to clear up space. You can also explicitly set another cache directory when invoking pip to install huge packages, for example: pip install torch --cache-dir /vol/tensusers/$USER/pip_cache/.

Rebuilding environments and using requirements.txt

It is recommended to keep a requirements.txt file that lists all installed modules, one on each line, according to the following syntax. You can automatically output such a file using pip freeze.

This allows you to reinstall everything later with a single command: pip install -r requirements.txt. Recreating your virtual environments and reinstalling all packages in it is necessary whenever the system has a large upgrade, including a major upgrade of the Python interpreter (this happens about once every two years). Keeping a requirements.txt makes this a painless process.

Conda environments

Conda environments are an alternative to classical virtual environments. They use Anaconda, a Python and R distribution for scientific computing. However, we do NOT recommend this on ponyland as it has little to no added value in most cases.