CODEWITHSUNDEEP
python-3.xscrapylog-level
Mah3sh
Mah3sh

Reputation: 35

Scrapy - How to set different log levels for Stdout and File

I would like to know how to set different log levels for log on terminal/stdout vs logging into file.

I would like to have INFO level for terminal and ERROR level for logging into file.

I am updating settings.py as below:

LOG_LEVEL = 'ERROR'
LOG_FORMAT = '%(levelname)s - %(message)s'
LOG_FILE = './log.txt'

However this will set ERROR level for both terminal as well as file.

Is there a way to set different log levels for scrapy.

Upvotes: 0

Views: 52

Answers (1)

Ivan STEPANIAN
Ivan STEPANIAN

Reputation: 161

You can add multiple handlers to your logger :

import logging

logger = logging.getLogger()

TERMINAL_LOG_LEVEL = logging.INFO
TERMINAL_LOG_FORMAT = 'your terminal log format'

# Terminal handler
terminalHandler = logging.StreamHandler()
terminalHandler.setLevel(TERMINAL_LOG_LEVEL)
terminalHandler.setFormatter(logging.Formatter(TERMINAL_LOG_FORMAT))
logger.addHandler(terminalHandler)

LOG_LEVEL = logging.ERROR # It is better to use logging's enum values
LOG_FORMAT = '%(levelname)s - %(message)s'
LOG_FILE = './log.txt'

# File handler
fileHandler = logging.FileHandler(LOG_FILE)
fileHandler.setLevel(LOG_LEVEL)
fileHandler.setFormatter(logging.Formatter(LOG_FORMAT))
logger.addHandler(fileHandler)

For more details, check out scrapy's logging docs.

If you still encounter a problem, post a comment !

Wish you good luck with your project ⭐️

Upvotes: 1

Related Questions