CODEWITHSUNDEEP
pbsqsub
Hrishikesh Satarkar
Hrishikesh Satarkar

Reputation: 1

Is there any way by which we can save the output and error files of a PBS script with current time as filename?

I have tried multiple ways to save the output and error file to get current time as file names of respective files still couldnt match the expected outcome.


#!/bin/bash
#PBS -l select=1:ncpus=2:mem=1GB,walltime=00:02:00
#PBS -N PBS_test
#PBS -V
#PBS -o "path_to _dir/output_$(date +%Y%m%d_%H%M%S).log"
#PBS -e "path_to _dir/error_$(date +%Y%m%d_%H%M%S).log"
#PBS -q Hercules
#PBS -v CONTAINER_IMAGE=docker_img

#rest of my PBS commands.


current o/p:output_$(date +%Y%m%d_%H%M%S).log
expected o/p:
type here
output_20230417_060100.log

same goes with the error file.

Upvotes: 0

Views: 396

Answers (1)

adria
adria

Reputation: 1

I wrote a wrapper shell-script to compute date and then add this to the output. Once this is done, the wrapper submits the job, appending the precomputed date to the filename of the LOG/ERR. For example:

# Shellscript parses command-line arguments here
# (nº threads, job-name, etc.) and options.
while [ "$1" != "" ]; do
    case $1 in
        # Number of threads requested (-t integer)
        -t ) shift; ncpu="$1" ;;
        # Name given to the job submitted (-N string)
        -N ) shift; NAME="$1" ;;

        .......other variables........

        # Anything else is the actual job
        # (script, program, options and arguments for them...)
        * )     ARGS+="$1 "
    esac
    shift
done
# Remove the ending whitespace ("$1 ") in ARGS
ARGS=${ARGS:0: -1}

.... more things go here....

# Name for the stdout+stderr log-files, formatted as DATE-TITLE.log
# Prepends $(date) to 'job-name' ($NAME).
STDOUT="$(date +%y%m%d-%H%M)-${NAME}.log" 

# Echo information to the output file (at the moment non-existant)
    # write 'job-name' to file
    echo "==== Output for job '$NAME' ====" >> $STDOUT
    # write 'date' to file
    echo $(date) >> $STDOUT
    # ...
    # ...
    echo "Submitted job:" >> $STDOUT
    echo "qsub -cwd -V -N $NAME (......) -o $STDOUT $ARGS" >> $STDOUT
    # write a separator between qsub options and LOG file
    echo "-- -- -- -- -- -- -- -- -- --" >> $STDOUT

# Now the wrapper proceeds to submit the job
        qsub -cwd -V -N "$NAME" (......) -o "$STDOUT" $ARGS

All the suspension points (...) mark eluded code. The $ARGS variable is the program you would like to launch, including command-line options and stored as a string. To use it (I have it on $PATH):

qsub-wrapper.sh -N jobname123 -t 12 (...options...) script.py file1.tsv file2.tsv

I feel it can be cumbersome to employ sometimes, and I think you can sort log-files by using ls --time=birth -tl. I now have started to dislike the large filenames preppended with dates. However, I still love to have the qsub command, date, etc. as the header of the LOG file.

Upvotes: 0

Related Questions