CODEWITHSUNDEEP
pythonpython-3.xtwarc2
Adhithya Kiran
Adhithya Kiran

Reputation: 29

Class.main programming style in Python

Can anyone tell me what is this line of code is executing ?

if __name__ == "__main__":
    TwitterHarvester.main(TwitterHarvester, QUEUE, [SEARCH_ROUTING_KEY, TIMELINE_ROUTING_KEY])

This is a python py file containing a class called TwitterHarvester which has no main function defined. Can anyone tell me what this line is calling ?

This is the starting of the function ;

class TwitterHarvester(BaseHarvester):
    def __init__(self, working_path, stream_restart_interval_secs=30 * 60, mq_config=None, debug=False,
                 connection_errors=5, http_errors=5, debug_warcprox=False, tries=3):
        BaseHarvester.__init__(self, working_path, mq_config=mq_config,
                               stream_restart_interval_secs=stream_restart_interval_secs,
                               debug=debug, debug_warcprox=debug_warcprox, tries=tries)
        self.twarc = None
        self.connection_errors = connection_errors
        self.http_errors = http_errors

    def harvest_seeds(self):
        # Create a twarc
        self._create_twarc()

        # Dispatch message based on type.
        harvest_type = self.message.get("type")
        log.debug("Harvest type is %s", harvest_type)
        if harvest_type == "twitter_search":
            self.search()
        elif harvest_type == "twitter_filter":
            self.filter()
        elif harvest_type == "twitter_sample":
            self.sample()
        elif harvest_type == "twitter_user_timeline":
            self.user_timeline()
        else:
            raise KeyError

    def _create_twarc(self):

Upvotes: 0

Views: 47

Answers (1)

Nicolas Appriou
Nicolas Appriou

Reputation: 2331

TwitterHarvester extends BaseHarvester. So if TwitterHarvester does not have a main function, it should be in BaseHarvester.

A little googling told me that base harvester is part of sfm-utils package. The import line is here:

from sfmutils.harvester import BaseHarvester, Msg

So I went in the sfm-utls source to find:

class BaseHarvester(BaseConsumer):
    """
    Base class for a harvester, allowing harvesting from a queue or from a file.
    ...
    """
    ...
    @staticmethod
    def main(cls, queue, routing_keys):
        """
        A configurable main() for a harvester.

Here you go :)

Upvotes: 1

Related Questions