CVMFS

CVMFS (CernVM filesystem) is a filesystem developed in Cern to allow fast, scalable and reliable deployment of software on the worldwide-distributed computing infrastructure. CVMFS is a read-only filesystem. Files and their metadata are transferred to user on demand with the use of aggressive memory caching. For file distribution, standard HTTP is used, allowing usage of various website caching. CVMFS verifies integrity of transferred data by cryptographic hashes. CVMFS software consists of client-side software for access to CVMFS repositories (similar to AFS volumes) and server-side tools for creating new repositories of CVMFS type.

Access to CVMFS repositories

Repositories can be accessed from computational nodes and frontends under the /cvmfs/ directory.

By default repositories are not mounted. Mounting must be initialized by user's attempt to access the directory (e.g. ls /cvmfs/software.metacentrum.cz/). This means the user must know in advance what he/she looks for and where to find it. The information about repository content is usually available on the repository administrator's website.

Metacentrum, apart from it's own repository, provides access to CERN, Elixir, EGI and OpenScienceGrid repositories.

In case you are interested in some particular external repository not included, let us know as support@metacentrum.cz.

Metacentrum

The Metacentrum repository contains ready-to-use software modules. It is accessible in /cvmfs/software.metacentrum.cz/. Users usually access the software through module add SOFTWARE command.

Galaxy

Galaxy repository:

• /cvmfs/singularity.galaxyproject.org/
• ...

CERN

CERN repository:

• /cvmfs/atlas.cern.ch/
• /cvmfs/grid.cern.ch/
• ...

OpenScienceGrid

Singularity images are available at:

• /cvmfs/singularity.opensciencegrid.org/

EGI

EGI CVMFS repositories.

• /cvmfs/biomed.egi.eu/
• ...

Installation

If you wish to access CVMFS repositories from your machine, follow the general steps at https://cvmfs.readthedocs.io/en/stable/cpt-quickstart.html

Metacentrum CVMFS repository configuration

The configuration files for Metacentrum CVMFS repository should be as follows:

• /etc/cvmfs/keys/metacentrum.cz/software.metacentrum.cz.pub:

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwQ1TEVWELY7kAX9bBkiN
GGIQ9JDsAc1wcdORdiMaNjEVTJGs/H8kHAGxdicnFFtVDZjKFoPCEJS3RiTMa2eS
uKvMIu9RlZeW5kpkDrSftZL3m9aUhRCcVQR4LDuI9NC0QvYZQySEQn8CQzePHkcD
irFG45KMb0x3XRXa26gD4BqisyVuAsTHhVltRErnrsS9gzwTCSBezfMvzNmfCpJn
lok8/SqztHHuwZDQhPkGcpYznecgbImo3/LM3y91iUGWRki2wLzBYsLnJ12Af98j
fOJC+q+UIsZE1KQk1qcw8Xn0FiwZohYNnh7xe6RM/66j+Y/1RWZhzVGedL/39U4S
HwIDAQAB
-----END PUBLIC KEY-----

• /etc/cvmfs/domain.d/metacentrum.cz.conf:

CVMFS_SERVER_URL="http://mick.grid.cesnet.cz/cvmfs/@fqrn@;http://ronnie.meta.zcu.cz/cvmfs/@fqrn@"
CVMFS_KEYS_DIR=/etc/cvmfs/keys/metacentrum.cz
CVMFS_USE_GEOAPI=yes

• /etc/cvmfs/default.local:

CVMFS_DEFAULT_DOMAIN=metacentrum.cz
CVMFS_REPOSITORIES="software.metacentrum.cz"
CVMFS_HTTP_PROXY=DIRECT

The reload the configuration

cvmfs_config reload

Test the functionality by

cvmfs_config probe
ls /cvmfs/software.metacentrum.cz/

Metacentrum CVMFS proxy

Metacentrum is also running CVMFS proxy for CERN, EGI and OpenScienceGrid repositories.

Simple add /etc/cvmfs/domain.d/<domain>.local file for specific domain with following content and reload configuration.

• /etc/cvmfs/domain.d/egi.eu.local

CVMFS_HTTP_PROXY="http://angus.grid.cesnet.cz:3128"

Usage examples

CVMFS repositories contain a lot of software immediately available on servers with CVMFS client. A lot of software is for the most abundant EGI grid OS (currently CentOS7). For example sft.cern.ch contains over 450 external packages as well as HEP specific tools and generators. An example how to use the latest stable GCC 9 on a CentOS 7 machine in a Bash shell:

 source /cvmfs/sft.cern.ch/lcg/contrib/gcc/9/x86_64-centos7/setup.sh

There are also unpacked singularity images which enable to start CentOS7 on debian systems:

luna.fzu.cz$ cat /etc/debian_version
9.12
luna.fzu.cz$ singularity shell /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7/
Singularity> cat /etc/redhat-release
CentOS Linux release 7.7.1908 (Core)

To be able to use /cvmfs from the singularity container, use bind:

luna.fzu.cz$ singularity shell  /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7/
Singularity> ls -l /cvmfs/sft.cern.ch
ls: cannot access /cvmfs/sft.cern.ch: No such file or directory
Singularity> exit
exit
luna.fzu.cz$ singularity shell --bind /cvmfs /cvmfs/atlas.cern.ch/repo/containers/fs/singularity/x86_64-centos7/
Singularity> ls -l /cvmfs/sft.cern.ch
total 15
drwxr-xr-x 16 117 124  4096 May 27 14:34 lcg
-rw-r--r--  1 117 124 10452 Aug 20  2019 README.md