CODEWITHSUNDEEP
arraysparallel-processinghpclsf
LDT
LDT

Reputation: 3108

How to use ${LSB_JOB_INDEX} to run a for loop in LSF platform

I have a for loop that looks like this, and I want to convert it to an array job. I want to run 100 jobs in parallel, but I am not entirely sure where I can add the ${LSB_JOB_INDEX} parameter.

Any help or guidance is welcome!

#!/bin/bash
#BSUB -J My_array[1-100]      #job name AND job array
#BSUB -n 3               #number of cores
#BSUB -R span[ptile=1]
#BSUB -W 00:10            #walltime limit: hh:mm
###BSUB -q queuename      #specify queue is optional
#BSUB -o Output_%J_%I.out #output - %J is the job-id %I is the job-array index
#BSUB -e Error_%J_%I.err  #error - %J is the job-id %I is the job-array index 

path=./home/

for each in *.bam 
do 
samtools coverage ${each} -o ${each}_coverage.txt
done

Thank you for your time; any help is appreciated. I am a starter at LSF and quite confused

Upvotes: 0

Views: 468

Answers (1)

AndyT
AndyT

Reputation: 541

You may need to rename your *.bam files to contain the job array index number (or have another way of mapping the index number to the input filename). For example, if your input files are called data_1.bam, data_2.bam, ..., data_100.bam, then you can modify your LSF script to:

#!/bin/bash
#BSUB -J My_array[1-100]      #job name AND job array
#BSUB -n 3               #number of cores
#BSUB -R span[ptile=1]
#BSUB -W 00:10            #walltime limit: hh:mm
###BSUB -q queuename      #specify queue is optional
#BSUB -o Output_%J_%I.out #output - %J is the job-id %I is the job-array index
#BSUB -e Error_%J_%I.err  #error - %J is the job-id %I is the job-array index 

path=./home/

samtools coverage data_${LSB_JOBINDEX}.bam -o data_${LSB_JOBINDEX}_coverage.txt

This should make each job in the array operate on a different input fiel specified by the ${LSB_JOBINDEX} ID.

Upvotes: 1

Related Questions