CODEWITHSUNDEEP
pythonscrapyhierarchical-clustering
Haomao Wang
Haomao Wang

Reputation: 13

Python Scrapy: How to close cluster's logging

I use cluster.HierarchicalClustering in the Scrapy but there are too much logging in the console. How can I close the cluster's log but I don't want to use '--nolog' to close all the logging.

I tried to set LOG_LEVEL= 'INFO' in my Scrapy settings.py and it appeared other logging.

There is my code about cluster, if you have any way, please help me.

from difflib import SequenceMatcher
from cluster import HierarchicalClustering


def distance(url1, url2):
    ratio = SequenceMatcher(None, url1, url2).ratio()
    return 1.0 - ratio

def urls_clustering(urls):
    hc = HierarchicalClustering(urls, distance)
    clusters = hc.getlevel(0.2)
    # pprint.pprint(clusters)

    return clusters

And there are the logging:

2017-06-29 13:48:29 [cluster.matrix] DEBUG: Generating row 159/203 (78.33%)
2017-06-29 13:48:29 [cluster.matrix] DEBUG: Generating row 160/203 (78.82%)
2017-06-29 13:48:29 [cluster.matrix] DEBUG: Generating row 161/203 (79.31%)
2017-06-29 13:48:29 [cluster.matrix] DEBUG: Generating row 162/203 (79.80%)

Thanks in advance.

Upvotes: 0

Views: 67

Answers (1)

Granitosaurus
Granitosaurus

Reputation: 21406

Have you tried good ol' python's logging?

import logging
logging.getLogger('cluster.matrix').setLevel(logging.WARNING)

Upvotes: 1

Related Questions