Biowulf
Jump to navigation
Jump to search
Quota
checkquota
freen
- https://hpc.nih.gov/docs/b2-userguide.html#partitions
- https://hpc.nih.gov/docs/b2-userguide.html#cluster_status
Biowulf nodes are grouped into partitions. A partition can be specified when submitting a job. The default partition is 'norm'. The freen command can be used to see free nodes and CPUs, and available types of nodes on each partition.
Environment modules
# What modules are available
module avail
module -d avail # default
module avail STAR
module spider bed # search by a case-insensitive keyword
# Load a module
module list # loaded modules
module load STAR
module load STAR/2.4.1a
module load plinkseq macs bowtie # load multiple modules
# Unload a module
module unload STAR/2.4.1a
# Switch to a different version of an application
# If you load a module, then load another version of the same module, the first one will be unloaded.
# Examine a modulefile
$ module display STAR
-----------------------------------------------------------------------------
/usr/local/lmod/modulefiles/STAR/2.5.1b.lua:
-----------------------------------------------------------------------------
help([[This module sets up the environment for using STAR.
Index files can be found in /fdb/STAR
]])
whatis("STAR: ultrafast universal RNA-seq aligner")
whatis("Version: 2.5.1b")
prepend_path("PATH","/usr/local/apps/STAR/2.5.1b/bin")
# Set up personal modulefiles
# Using modules in scripts
# Shared Modules
Single file - sbatch
- Note sbatch command does not support --module option. In sbatch case, the module command has to be put in the script file.
Don't use swarm for a single file
sbatch --cpus-per-task=2 --mem=4g MYSCRIPT
Multiple files - swarm
swarm -t 3 -g 20 -f run_seqtools_vc.sh --module samtools,picard,bwa --verbose 1 --devel # 3 commands run in 3 subjobs, each command requiring 20 gb and 3 threads, allocating 6 cores and 12 cpus swarm -t 3 -g 20 -f run_seqtools_vc.sh --module samtools,picard,bwa --verbose 1 # To change the default walltime, use --time=24:00:00 swarm -t 8 -g 24 --module tophat,samtools,htseq -f run_master.sh cat sw3n17156.o
Monitor/Delete jobs
sjobs scancel -u XXXXX scancel NNNNN scancel --state=PENDING scancel --state=RUNNING squeue -u XXXX jobhist 17500 # report the CPU and memory usage of completed jobs.
Exit code
https://hpc.nih.gov/docs/b2-userguide.html#exitcodes
Local disk and temporary files
https://hpc.nih.gov/docs/b2-userguide.html#local
Interactive debugging
4 CPUs on a single node
sinteractive --mem=8g -c 4