PyTorch

From MetaCentrum
Jump to navigation Jump to search

Description

PyTorch is a python package that provides two high-level features:

  • Tensor computation (like Numpy) with strong GPU acceleration
  • Deep Neural Networks built on a tape-based autograd system

Usually one uses PyTorch either as:

  • A replacement for numpy to use the power of GPUs.
  • a deep learning research platform that provides maximum flexibility and speed

Usage

Upcoming modulesystem change alert!

Due to large number of applications and their versions it is not practical to keep them explicitly listed at our wiki pages. Therefore an upgrade of modulefiles is underway. A feature of this upgrade will be the existence of default module for every application. This default choice does not need version number and it will load some (usually latest) version.

You can test the new version now by adding a line

source /cvmfs/software.metacentrum.cz/modulefiles/5.1.0/loadmodules

to your script before loading a module. Then, you can list all versions of pytorch and load default version of pytorch as

module avail pytorch/ # list available modules
module load pytorch   # load (default) module


If you wish to keep up to the current system, it is still possible. Simply list all modules by

module avail pytorch

and choose explicit version you want to use.

PyTorch as a Singularity container

Newer versions of PyTorch are available solely as Singularity images optimized for usage with NVidia GPUs (NVidia GPU Cloud, NGC). The NGC packages are placed in the directory /cvmfs/singularity.metacentrum.cz; you have to list the directory first to see its contents:

ls /cvmfs/singularity.metacentrum.cz

To use a selected version of PyTorch image, run the image within interactive job as:

qsub -I -l select=1:mem=16gb:scratch_local=10gb:ngpus=1:gpu_cap=cuda60:cuda_version=11.0 -q gpu -l walltime=4:00:00
singularity shell --nv /cvmfs/singularity.metacentrum.cz/NGC/PyTorch\:21.03-py3.SIF

More about Nvidia GPU cloud usage can be found at NVidia deep learning frameworks wiki page.

PyTorch as a modulefile

Older versions of PyTorch can be obtained also as modulefiles. Please keep in mind that the usage of PyTorch as modulefile is deprecated and newer versions will not be installed.

All currently installed versions of PyTorch support following methods of computations:

  • CPU-only computations
  • GPU-accelerated computations based on CUDA 8.0 and CuDNN 7.0 libraries

PyTorch 1.1.0:

PyTorch 0.3.0:

  • pytorch-0.3.0_python-3.6.2_cuda-8.0 (January 2018); version 0.3.0.post4 - with Python 3.6.2, CUDA 8.0 and CuDNN 7.0
  • pytorch-0.3.0_python-2.7.6_cuda-8.0 (January 2018); version 0.3.0.post4 - with Python 2.7.6, CUDA 8.0 and CuDNN 7.0

Usage:

(JESSIE)leontovyc_roman@tarkil:~$ module add pytorch-1.1.0_python-3.6.2_cuda-10.1
(JESSIE)leontovyc_roman@tarkil:~$ python -c 'import torch; print(torch.__version__);'
1.1.0

Documentation

You can find documentation at the official website: http://pytorch.org/docs/0.3.0/.

Official website

NVidia PyTorch

NVidia PyTorch containers:

http://pytorch.org