CODEWITHSUNDEEP
pythonmultithreadingpython-2.7queue
Pixel
Pixel

Reputation: 910

Communicate between two python app

I have done two app's :

  1. The first one is a spider that extract all links from a website.
  2. The second one do some checks on each link sent by the first app.

When the first app find a link, how can I send a notification or something else to the second app ?
The second app must listen continuously the data sent by the first app.

I found few post speaking of Queue but I don't really understand how that work.

Can someone explain me with a simple exemple how to communicate between the two app ?

Thank's

Upvotes: 0

Views: 1070

Answers (3)

martineau
martineau

Reputation: 123463

A Queue is just a container into which items may be put and retrieved, often in FIFO order. The Queue module in Python 2 is just an implementation of one that supports synchronized access, meaning that it supports multiple threads using it (putting and getting things) at the same time.

Upvotes: 0

user590028
user590028

Reputation: 11730

There are all sorts of ways to accomplish inter-process communication, but by far the simplest is to use the filesystem. Have your spider write it's output to a temp file. When it's finished, move it into a folder that your second process polls periodically and when it finds work, then process it.

The spider could like something like:

import tempfile, os
tmpname = ''
with tempfile.NamedTemporaryFile(delete=False) as tmp:
    tmpname = tmp.name
    tmp.write("spider output....\n")
tgt = os.path.join('incoming', os.path.basename(tmpname))
os.rename(tmpname, tgt)

The second process could look something like this:

import time, os
while 1:
    time.sleep(5)
    for item in os.listdir('incoming'):
        work_item = os.path.join('incoming', item)
        with open(work_item) as fin:
            # do something with item
        os.unlink(work_item)

Upvotes: 1

jxmorris12
jxmorris12

Reputation: 1372

You want to save one file as a "module" to be imported by the other file. Here this can be implemented with the the import keyword. For example, if you name the second part of your application listener.py, you can type import listener in your other file (remember to put them in the same folder!) and call any method from the second file. You can read more on Python modules here.

Upvotes: 0

Related Questions