CODEWITHSUNDEEP
pythonoutput-buffering
LCMa
LCMa

Reputation: 445

Save console output in txt file as it happens

I want to save my console output in a text file, but I want it to be as it happens so that if the program crashes, logs will be saved. Do you have some ideas?

I can't just specify file in logger because I have a lot of different loggers that are printing into the console.

Upvotes: 1

Views: 3261

Answers (2)

Ziur Olpa
Ziur Olpa

Reputation: 2102

I think that you indeed can use a logger, just adding a file handler, from the logging module you can read this

As an example you can use something like this, which logs both to the terminal and to a file:

import logging
from pathlib import Path

root_path = <YOUR PATH>

log_level = logging.DEBUG

# Print to the terminal
logging.root.setLevel(log_level)
formatter = logging.Formatter("%(asctime)s | %(levelname)s | %(message)s", "%Y-%m-%d %H:%M:%S")
stream = logging.StreamHandler()
stream.setLevel(log_level)
stream.setFormatter(formatter)
log = logging.getLogger("pythonConfig")
if not log.hasHandlers():
    log.setLevel(log_level)
    log.addHandler(stream)

# file handler:
file_handler = logging.FileHandler(Path(root_path / "process.log"), mode="w")
file_handler.setLevel(log_level)
file_handler.setFormatter(formatter)
log.addHandler(file_handler)

log.info("test")

If you have multiple loggers, you can still use this solutions as loggers can inherit from other just put the handler in the root logger and ensure that the others take the handler from that one.

As an alternative you can use nohup command which will keep the process running even if the terminal closes and will return the outputs to the desired location:

nohup python main.py > log_file.out &

Upvotes: 5

Jib
Jib

Reputation: 1582

There are literally many ways to do this. However, they are not all suitable for different reasons (maintainability, ease of use, reinvent the wheel, etc.).

If you don't mind using your operating system built-ins you can:

  • forward standard output and error streams to a file of your choice with python3 -u ./myscript.py 2>&1 outputfile.txt.

  • forward standard output and error streams to a file of your choice AND display it to the console too with python3 -u ./myscript.py 2>&1 tee outputfile.txt. The -u option specifies the output is unbuffered (i.e.: whats put in the pipe goes immediately out).

If you want to do it from the Python side you can:

  • use the logging module to output the generated logs to a file handle instead of the standard output.

  • override the stdout and stderr streams defined in sys (sys.stdout and sys.stderr) so that they point to an opened file handle of your choice. For instance sys.stdout = open("log-stdout.txt", "w").

As a personnal preference, the simpler, the better. The logging module is made for the purpose of logging and provides all the necessary mechanisms to achieve what you want. So.. I would suggest that you stick with it. Here is a link to the logging module documentation which also provides many examples from simple use to more complex and advanced use.

Upvotes: 4

Related Questions