Application modules old
The system of modules used in Metacentrum provides user environment necessary to run requested applications; it allows to configure the user environments independently on the server's platform or files' locations. To run a certain software, the user must first load a module for this software.
In simple words: module = application = software.
|
Note:When used on frontends, the modules may behave a little differently than on computational nodes. The reason is that frontends are intended for interactive user work (job preparation, manipulation of data etc.) and may have therefore different system settings than computational nodes. If you need to run environment-sensitive operation (typically highly optimized compilations), do not test it on frontend, but request an interactive job and run your test compilations on the same node on which your final compilation will be done.
|
Usage
The basic command to work with modules is module.
Get a list of all available modules (applications):
$ module available
Abbreviated forms of this command:
$ module avail
$ module av
Get e.g. list of all modules containing "Matlab":
$ module avail matlab # list all modules starting with "matlab"
$ module avail {M,m}atlab # list all modules starting with "matlab" or "Matlab"
Use grep to search through list of modules:
$ module avail 2>&1 | grep -i atlab # list all modules containing string "atlab" in their name
$ module avail 2>&1 | grep -i 'atlab-9\.' # list all modules containing string "atlab-9." in their name
Load a specified module:
$ module add matlab-9.9 # load Matlab ver. 9.9, for example
Limit the module validity to a bash subshell
Within a single session you can use modules separately by enclosing them to parentheses. Parentheses, in general, create a subshell in bash, i.e. all modules loaded within a certain subshell will be unloaded automatically after the subshell exits.
Example: normally the following two modules will conflict if loaded within the same session:
(BUSTER)melounova@skirit:~$ module add python/3.8.0-gcc This is the user wrapper module using spack module python/3.8.0-gcc-rab6t. (BUSTER)melounova@skirit:~$ module add python/3.8.0-gcc-rab6t python/3.8.0-gcc-rab6t(49):ERROR:150: Module 'python/3.8.0-gcc-rab6t' conflicts with the currently loaded module(s) 'python/3.8.0-gcc' python/3.8.0-gcc-rab6t(49):ERROR:102: Tcl command execution failed: conflict python
However you can load them separately in subshells:
(BUSTER)melounova@skirit:~$ (module add python/3.8.0-gcc; python) This is the user wrapper module using spack module python/3.8.0-gcc-rab6t. ... >>> quit() (BUSTER)melounova@skirit:~$ (module add python/3.8.0-gcc-rab6t; python) ... >>> quit()
List currently loaded modules:
$ module list
Currently Loaded Modulefiles:
1) metabase 2) matlab-9.9
Unload a module:
$ module unload matlab-9.9 # unload matlab-9.9, for example
$ module del matlab-9.9 # equivalent to the above
Clear all currently loaded modules:
$ module purge # get rid of everything!
Display information about the module:
$ module display tensorflow-2.0.0-gpu-python3
-------------------------------------------------------------------
/packages/run/modules-2.0/modulefiles/tensorflow-2.0.0-gpu-python3:
module add python36-modules-gcc # these modules will be loaded, too
module add cudnn-7.6.4-cuda10.1
prepend-path PATH /software/tensorflow/2.0.0/python36-gpu/bin # path to where the software is installed
prepend-path PYTHONPATH /software/tensorflow/2.0.0/python36-gpu/lib/python3.6/site-packages # another path to some python libraries
-------------------------------------------------------------------
Module vs modulefile
"A module" consists of 2 parts: the binary (executable), which is the software itself, and a modulefile, which is a text file defining paths to the binary, paths to external libraries and possible dependencies on other modules.
In simple words: module = binary + modulefile.
The modulefiles reside in some of the directories included in variable MODULEPATH (try echo $MODULEPATH).
For many applications, there are several modules available. This means there are several copies of the software available, differing in version, compiler and other features. For example,
(BUSTER)melounova@skirit:/packages/run/modules-2.0/modulefiles$ ls tensorflow* tensorflow-1.13.1-gpu-python3 tensorflow-1.3.0-gpu tensorflow-1.5.0-cpu-python3 tensorflow-1.7.1-gpu-python3 tensorflow-1.3.0-cpu tensorflow-1.3.0-gpu-python3 tensorflow-1.5.0-gpu tensorflow-2.0.0-gpu-python3 tensorflow-1.3.0-cpu-python3 tensorflow-1.4.1-gpu tensorflow-1.7.1-cpu-python3
says there are 11 versions of TensorFlow software differing by version, Python version (2 or 3) and whether it can be used for GPU computing (gpu) or not (cpu).
Modulefile example
#%Module1.0
#!
#! Title: gsl-1.16
#! Platforms: amd64_linux32
#! Version: 1.16
#! Description: GNU Scientific Library tools collection
#!
#! Author: Petr Hanousek, #35289
#!
#!
proc ModulesHelp {} {
global ModulesCurrentModulefile
puts stdout "modulehelp $ModulesCurrentModulefile"
}
set basedir /software/gsl/1.16/gcc
prepend-path PATH ${basedir}/bin
prepend-path PKG_CONFIG_PATH ${basedir}/lib/pkgconfig
prepend-path LD_LIBRARY_PATH ${basedir}/lib
prepend-path MANPATH ${basedir}/share/man
- the number after the author's name you can see in some modulefiles is the number of RT ticket which requested the installation
- the only necessary path is
PATH, where the executable is - a modulefile can add other modules (
module add ...) if there is a dependency basediris path to the install directory - it can be pretty anywhere; in case of non-standard installation, e.g. tohomedirectory, don't forget to set variableMODULEPATHaccordingly (see How to install an application)
Generic module
In some cases (typically when the only difference is in version) there is a generic module without the version name, which is nothing more than an alias of a newest version.
For example, module matlab is (currently) a shortcut for matlab-9.9.
$ module display matlab ------------------------------------------------------------------- /packages/run/modules-2.0/modulefiles/matlab: prepend-path PATH /software/matlab-9.9/bin:/software/matlab-meta_ext/bin prepend-path MATLABPATH /software/matlab-meta_ext/ -------------------------------------------------------------------
Documentation
- official Environment Modules Project site
- module manpages (
man module) - modulefile manpages (
man modulefile)