CODEWITHSUNDEEP
pythonloggingscrapystdoutstderr
Apogentus
Apogentus

Reputation: 6613

How to see output of print statements when scrapy logger enabled

For debug purposes I need to see both output of log messages and print statements when working with Scrapy. Hovewer, when I start Scrapy logger, I no longer see output of print statements! How should I modify the following sample code to see both print statements?

from scrapy import log

print 'PRINT OUTPUT BEFORE'
log.start(loglevel='DEBUG',logstdout=True)
print 'PRINT OUTPUT AFTER'

Output:

PRINT OUTPUT BEFORE

I beleive redirecting stderr or stdout may solve the problem. Any help would be greatly appreciated!

Upvotes: 6

Views: 6706

Answers (2)

mrtipale
mrtipale

Reputation: 959

I think above works for old version. They seems to have deprecated scrapy.log . Refer: https://docs.scrapy.org/en/latest/topics/logging.html

Eg:

import logging

logging.warning("This is a warning")
logging.info("This is an info")
logging.error("This is an error")

As my config was set to warn. I got only below

WARNING:root:This is a warning
ERROR:root:This is an error

Note that this is helpful if you are using scrapyd. As scrapy crawl will print all but scrapyd doesn't do same way.

Upvotes: 4

user650654
user650654

Reputation: 6128

logstdout must be set to False to disable stdout from being redirected.

from scrapy import log

print 'PRINT OUTPUT BEFORE'
log.start(loglevel='DEBUG', logstdout=False)
print 'PRINT OUTPUT AFTER'

With output:

PRINT OUTPUT BEFORE
PRINT OUTPUT AFTER

Upvotes: 6

Related Questions