Nextflow
Description
Nextflow enables scalable and reproducible scientific workflows using software containers. It allows the adaptation of pipelines written in the most common scripting languages.
License
Public
Usage
module add conda-modules-py37 conda activate nextflow nextflow
using of SW module:
Upcoming modulesystem change alert!
Due to large number of applications and their versions it is not practical to keep them explicitly listed at our wiki pages. Therefore an upgrade of modulefiles is underway. A feature of this upgrade will be the existence of default module for every application. This default choice does not need version number and it will load some (usually latest) version.
You can test the new version now by adding a line
source /cvmfs/software.metacentrum.cz/modulefiles/5.1.0/loadmodules
to your script before loading a module. Then, you can list all versions of nextflow and load default version of nextflow as
module avail nextflow/ # list available modules module load nextflow # load (default) module
If you wish to keep up to the current system, it is still possible. Simply list all modules by
module avail nextflow
and choose explicit version you want to use.
Simple example in Metacentrum PBS
Nexflow script with two processes for run as separate PBS jobs, with passing an intermediate results. Important is first line to run im Metacentrum. Parameters for jobs are limited for now, scratchdir, gpu and others should we test and prepare to run. Filename for this example is: basic.nexflow
#!/bin/bash /software/nextflow/21.04.3/nextflow
params.in = "$baseDir/sample.fa"
/*
* split a fasta file in multiple files
*/
process splitSequences {
executor = 'pbspro'
queue 'default@meta-pbs.metacentrum.cz'
cpus 1
memory '1 GB'
input:
path 'input.fa' from params.in
output:
path 'seq_*' into records
"""
csplit input.fa '%^>%' '/^>/' '{*}' -f seq_
"""
}
/*
* Simple reverse the sequences
*/
process reverse {
executor = 'pbspro'
queue 'default@meta-pbs.metacentrum.cz'
cpus 1
memory '2 GB'
input:
path x from records
output:
stdout into result
"""
cat $x | rev
"""
}
/*
* print the channel content
*/
result.subscribe { println it }
Run script for PBS, filename basic.qsub
#!/bin/bash #PBS -q default@meta-pbs.metacentrum.cz #PBS -l walltime=1:0:0 #PBS -l select=1:ncpus=1:mem=3gb #PBS -N NF-test module add nextflow-21.04.3 cd /storage/plzen1/home/hoidekr/nextflow/ #cd to work directory ./basic.nextflow
Run the job
qsub basic.qsub
The job starts and runs two other jobs as specified in nexflow script. All information about run jobs, stdout/errout job skripts, intermediate results, results etc. is saved into dir work and .nextflow under the working directory.