Skip to content

Installation

Before deep diving into the wonderful world of EasyBuild and getting your hands dirty with the hands on exercises coming up in this tutorial, you will need to install EasyBuild.

In this section we outline a couple of different ways of doing this, and also the things you should pay attention to. By the end, you will have a functional EasyBuild installation that you can use for the remainder of this tutorial.

Summary

Requirements

Linux

The main target platform for EasyBuild is Linux, since it is heavily focused on installing software on HPC system where Linux is dominant operating system (to the point where 100% of the current Top500 list of supercomputers are running Linux).

EasyBuild is also compatible with macOS, but the included easyconfig files are heavily focused on Linux so most software installations supported by EasyBuild won't work out-of-the-box on macOS. You can still use the EasyBuild command line interface on macOS for other tasks though, like contributing back to the project.

Python

EasyBuild is implemented in Python, and is compatible with both Python 2.7 and Python 3.5+ (that is, Python 3.5 or a newer version of Python 3).

To check which Python version you have, use:

python -V

No additional Python packages are required by EasyBuild, the ones that come with the standard Python distribution are sufficient. Some additional Python packages can be leveraged for specific features. More on that later.

Environment modules tool

An environment modules tool is required for using EasyBuild.

We strongly recommend using Lmod, a Lua-based modern environment modules implementation and the most commonly used modules tool in the EasyBuild community. Other implementations, like the original Tcl-based one, are also supported.

To check if you have a modules tool installed, use:

module --version

If this produces output that starts with something like "Modules based on Lua: Version 8.2.5" you have Lmod installed, which is the default modules tool used by EasyBuild, and you are all set for installing and using EasyBuild. Any sufficiently recent Lmod version (8.x or even 7.x) should be fine.

If you see output that starts with a line like "VERSION=3.2.10" or "Modules Release 4.5.0", you have the original Tcl-based environment modules tool installed and EasyBuild will need to be configured to use it after installation.

If the module function is not defined either you do not have a modules tool installed or your environment is not properly set up to use it. In this case, please refer to the EasyBuild documentation here for more information.

EasyBuild as a Python package

EasyBuild consists of a number of interdependent Python packages, and is available via both GitHub at https://github.com/easybuilders, as well as via the standard Python Package Index (PyPI) at https://pypi.org/project/easybuild.

As you may be aware the Python packaging ecosystem is bit convoluted, which is reflected in the many different ways in which you can install a Python package. In addition, EasyBuild is packaged in 3 components (framework, easyblocks, easyconfigs) which slightly complicates the installation.

Nevertheless, you don't need to be a rocket scientist to install EasyBuild (and even if you are, that's OK too), so don't worry.

You can install EasyBuild just like you can install any other Python software that is released via the standard Python Package Index (PyPI), through one of the standard Python installation tools (like pip, virtualenv, pipenv, ...). And since EasyBuild is a software installation tool in its own right, we actually have a couple of additional tricks up our sleeve!

Python 2 or Python 3?

For EasyBuild it does not matter whether you install it on top of Python 2 or Python 3. The functionality provided is identical. However, since Python 2 is end-of-life, we strongly recommend using Python 3 if you have the choice.

By default EasyBuild will use the python command to run, but you can control this if needed. For more information, see the EasyBuild documentation.

Installing EasyBuild

We present two methods for installing EasyBuild. It is up to you which one you prefer, both result a fully functional EasyBuild installation.

Time to get your hands dirty!

Since EasyBuild is released as a Python package on PyPI you can install it using pip, the most commonly used tool for installing Python packages.

You may need to take additional steps after the installation, depending on the exact installation command.

Note

There are various other ways of installing Python packages, which we won't cover here. If you are familiar with other tools like virtualenv or pipenv, feel free to use those instead to install EasyBuild.

Running pip install

Installing EasyBuild with pip is as simple as running the following command:

pip install easybuild

However, you may need to slightly change this command depending on the context and your personal preferences:

  • To install EasyBuild system-wide, you can use sudo (if you have admin privileges): 

    sudo pip install easybuild

  • To install EasyBuild in your personal home directory, you can use the --user option: 

    pip install --user easybuild
    This will result in an EasyBuild installation in $HOME/.local/.

  • To install EasyBuild in a specific directory you can use the --prefix option: 

    pip install --prefix _PREFIX_ easybuild
    In this command, you should replace '_PREFIX_' with the location where you want to have EasyBuild installed (for example, $HOME/tools or /tmp/$USER).

pip vs pip3

On systems where both Python 2 and Python 3 are installed you may also have different pip commands available. Or maybe pip is not available at all, and only "versioned" pip commands like pip3 are available.

If you (only) have pip3 available, you can replace pip with pip3 in any of the pip install commands above.

Updating your environment

If you used the --user or --prefix option in the pip install command, you will need to update your environment to make EasyBuild ready for use. This is not required if you did a system-wide installation in a standard location.

Note

Keep in mind that you will have to make these environment changes again if you start a new shell session. To avoid this, you can update one of the shell startup scripts in your home directory (.bashrc for example).

$PATH

Update the $PATH environment variable to make sure the eb command is available: 

export PATH=_PREFIX_/bin:$PATH
Replace '_PREFIX_' in this command with the directory path where EasyBuild was installed into (use $HOME/.local if you used pip install --user).

$PYTHONPATH

If you installed EasyBuild to a non-standard location using pip install --prefix, you also need to update the Python search path environment variable $PYTHONPATH to instruct Python where it can find the EasyBuild Python packages. This is not required if you used the --user option, since Python will automatically consider $HOME/.local when searching for installed Python packages.

Update $PYTHONPATH by running a command like:

export PYTHONPATH=_PREFIX_/lib/pythonX.Y/site-packages:$PYTHONPATH

Here, you need to replace the X and Y with the major and minor version of your Python installation, which you can determine by running python -V. For example, if you are using Python 2.7, make sure you are using 'python2.7' in the command to update $PYTHONPATH.

And of course, you again need to replace '_PREFIX_' with the installation prefix where EasyBuild was installed into.

For example:

# update $PYTHONPATH if EasyBuild was installed in $HOME/tools with Python 3.6
export PYTHONPATH=$HOME/tools/lib/python3.6/site-packages:$PYTHONPATH

$EB_PYTHON and $EB_VERBOSE

If you want to control which Python version is used to run EasyBuild, you can specify the name or the full path to the python command that should be used by the eb command via the $EB_PYTHON environment variable.

For example, to ensure that eb uses python3:

export EB_PYTHON=python3

To get a better view on which python commands are being considered by the eb command, you can (temporarily) define the $EB_VERBOSE environment variable. For example:

$ EB_VERBOSE=1 eb --version
>> Considering 'python3'...
>> 'python3' version: 3.6.8, which matches Python 3 version requirement (>= 3.5)
>> Selected Python command: python3 (/usr/bin/python3)
>> python3 -m easybuild.main --version
This is EasyBuild 4.2.1 (framework: 4.2.1, easyblocks: 4.2.1) on host example

Method 2: Bootstrapping EasyBuild

Note

This section covers an alternative installation method.

If you already have EasyBuild installed, you can skip ahead to the next section.

If pip is not available or if the installation with pip is not working out for some reason, you can resort to using the bootstrapping procedure for installing EasyBuild.

In essence, the bootstrap script installs EasyBuild into a temporary location and then uses this temporary EasyBuild installation to install EasyBuild into the specified directory and provide a module for it.

Step 1: Downloading the bootstrap script

First, download the latest version of the EasyBuild bootstrap script from GitHub.

A common way to do this is by running this curl command:

curl -O https://raw.githubusercontent.com/easybuilders/easybuild-framework/develop/easybuild/scripts/bootstrap_eb.py

Step 2: Running the bootstrap script

To install EasyBuild using the bootstrap script simply run it using the python command and specify the installation prefix as an argument:

python bootstrap_eb.py _PREFIX_

Replace '_PREFIX_' with the location where you want to have EasyBuild installed (for example, $HOME/tools or /tmp/$USER).

Step 3: Loading the EasyBuild module

Once the bootstrap procedure completed, you should be able to load the module that was generated alongside the EasyBuild installation. You will need to do this every time you start a new shell session.

First, make the module available by running the following command (which will update the module search path environment variable $MODULEPATH):

module use _PREFIX_/modules/all

Replace '_PREFIX_' in the same way as you did when running the bootstrap script.

Then, load the EasyBuild module to update your environment and make EasyBuild available for use:

module load EasyBuild

Note

No output will be generated by either of these module commands. That is expected behaviour and completely normal.

Verifying the installation

Regardless of how EasyBuild was installed, you can now run a couple of basic commands to verify the installation:

Checking the version

To check which EasyBuild version you have installed, run:

eb --version

The output should match with the latest EasyBuild version.

Consulting the help output

You can consult the help output of the eb command, which produces a long list of available options along with a short informative message.

eb --help

Showing the default EasyBuild configuration

To inspect the current EasyBuild configuration, you can use this command:

eb --show-config

This should tell you that EasyBuild (ab)uses $HOME/.local/easybuild as a default location. More on configuring EasyBuild in the next part of the tutorial.

System information

You ask EasyBuild to collect and print some information about the system you are using it on (OS, CPU, Python, etc.) using this command:

eb --show-system-info

Updating EasyBuild

Before we wrap up here, a brief word about updating EasyBuild.

Once you have EasyBuild installed, the easiest way to update to a newer version is by instructing EasyBuild to install the latest available version as a module:

eb --install-latest-eb-release

This will result in a new EasyBuild installation, which is entirely separate from the EasyBuild installation you are currently using (so it is not an in-place update).

The location where this new EasyBuild version will be installed is determined by the active EasyBuild configuration.

Exercise

Install EasyBuild in your home directory

Make sure that the EasyBuild installation uses the python3 command to run, rather than the standard python command.

Choose your own adventure (or try both installation methods)!

  • perform a bootstrap installation into $HOME/easybuild
  • install EasyBuild with pip (or another very similar command...) using either the --user or --prefix option

Check that the installation works by running the verification commands outlined above.

(click to show solution using bootstrapping)

To perform a bootstrap installation, it suffices to download the bootstrap script and run it using the python3 command.

$ curl -O https://raw.githubusercontent.com/easybuilders/easybuild-framework/develop/easybuild/scripts/bootstrap_eb.py
...
$ python3 bootstrap_eb.py $HOME/easybuild
[[INFO]] EasyBuild bootstrap script (version 20200203.01, MD5: fcb6314d4e0747db9c28a71f8bb2870c)
[[INFO]] Found Python 3.6.8 (default, Apr  2 2020, 13:34:55) ; [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]

[[INFO]] Installation prefix /home/easybuild/easybuild
...

Afterwards we can just load the generated module, which already takes care of correctly setting $EB_PYTHON to ensure the right python command is used:

$ module use $HOME/easybuild/modules/all
$ module load EasyBuild
$ echo $EB_PYTHON
/usr/bin/python3
$ eb --version
This is EasyBuild 4.2.1 (framework: 4.2.1, easyblocks: 4.2.1) on host example.
(click to show solution using pip)

To ensure that EasyBuild is installed with Python 3, we need to use the pip3 command rather than just pip.

We can install EasyBuild using pip3 install --user, and update $PATH and define $EB_PYTHON to ensure the right python command is used (the order in which we do this doesn't matter):

export PATH=$HOME/.local/bin:$PATH
export EB_PYTHON=python3
pip3 install --user easybuild

Or we can use pip3 install --prefix, but then we need to update both $PATH and $PYTHONPATH (after checking the Python version), and also define $EB_PTYHON:

$ python -V
Python 3.6.8

export PATH=$HOME/bin:$PATH
export PYTHONPATH=$HOME/lib/python3.6/site-packages:$PATH
export EB_PYTHON=python3
pip3 install --prefix $HOME easybuild

Make sure you have a working EasyBuild installation before proceeding with the rest of the tutorial!

Last update: April 21, 2022