CODEWITHSUNDEEP
snakemake
gregoruar
gregoruar

Reputation: 383

snakemake: How to log python scripts executed by script directive?

Is there any way to easily save logs coming from snakemake rule executing python script with the script directive? The script uses libraries, which already have some integrated logging, and I want to store their logs. I do not want to use shell or run directive as they both are not so comfortable, when working with python scripts. In addition, I hope that the answer will ask for minimum changes in the python scripts, the main changes being in the Snakemake file. Here is the example of code:

rule correcting_ids:
    input:
        "assembly_{type}/{sample}/{dataset}/final.fasta"
    output:
        "assembly_{type}/{sample}/{dataset}/final.corrected.fasta"
    log:
        "logs/{sample}/correct_{type}_ids_{dataset}.log"
    script:
        "scripts/correcting_ids.py"```

Upvotes: 3

Views: 2676

Answers (2)

KieserS
KieserS

Reputation: 61

I tried a solution where you can use the logggin module but still capture exceptions:


import sys,os
import logging, traceback
logging.basicConfig(filename=snakemake.log[0],
                    level=logging.INFO,
                    format='%(asctime)s %(message)s',
                    datefmt='%Y-%m-%d %H:%M:%S',
                    )
def handle_exception(exc_type, exc_value, exc_traceback):
    if issubclass(exc_type, KeyboardInterrupt):
        sys.__excepthook__(exc_type, exc_value, exc_traceback)
        return

    logger.error(''.join(["Uncaught exception: ",
                         *traceback.format_exception(exc_type, exc_value, exc_traceback)
                         ])
                 )
# Install exception handler
sys.excepthook = handle_exception

Upvotes: 3

Maarten-vd-Sande
Maarten-vd-Sande

Reputation: 3711

You could redirect all stdout and stderr to the logfile

python:

import sys

with open(snakemake.log[0], "w") as f:
    sys.stderr = sys.stdout = f
    [rest of the script]

R:

log <- file(snakemake@log[[1]], open="wt")
sink(log)
[rest of script]

Upvotes: 8

Related Questions