CODEWITHSUNDEEP
pythongevent
inquisitor
inquisitor

Reputation: 89

How to tail a file with gevent

I have a problem constructing gevent tail function. In general, the code works when I comment gevent.sleep in loop, but then CPU utilization is 100%. When I leave gevent.sleep program works but nothing is happening. Gevent version is 1.0b1.

import os
import gevent

def follow(filename):
    fd = os.open(filename, os.O_RDONLY|os.O_NONBLOCK)
    os.lseek(fd, 0, os.SEEK_END)
    hub = gevent.get_hub()
    watcher = hub.loop.io(fd, 1)
    while True:
        hub.wait(watcher)
        lines = os.read(fd, 4096).splitlines()
        if not lines:
            #gevent.sleep(.1)
            continue
        else:
            for line in lines:
                print "%s:%s" % (filename, line)

    os.close(fd)

if __name__ == '__main__':
    job1 = gevent.spawn(follow, '/var/log/syslog')
    job2 = gevent.spawn(follow, '/var/log/messages')

    gevent.joinall([job1, job2])

Upvotes: 5

Views: 2332

Answers (3)

Josh C Josh C
Josh C Josh C

Reputation: 342

Well, that's code doesn't 'tail' the file, it's just prints the whole file, BUT it's show's how 'loop.stat' works. It wait's for file to change - or be just touched, and then print's content. When it's wait - it take's almost no resources!

import gevent,os

def follow(filename):
    hub = gevent.get_hub()
    watcher = hub.loop.stat(filename)
    while True:
        hub.wait(watcher)
        with open(filename) as f:
            print f.read()

if __name__ == '__main__':
    jobs=[gevent.spawn(follow,'/var/log/syslog')]
    jobs+=[gevent.spawn(follow,'/var/log/messages')]
    gevent.joinall(jobs)

Upvotes: 4

Denis
Denis

Reputation: 3780

Starting with gevent 1.0b2 you can use stat watchers to get notifications on file changes.

See the code and the libev documentation for stat watchers.

Upvotes: 7

inquisitor
inquisitor

Reputation: 89

Obviously wrong approach. This works perfectly:

import os
import gevent

def follow(filename):
    fd = os.open(filename, os.O_RDONLY|os.O_NONBLOCK)
    os.lseek(fd, 0, os.SEEK_END)
    while True:
        lines = os.read(fd, 4096).splitlines()
        if not lines:
            gevent.sleep(.5)
            continue
        else:
            for line in lines:
                print "%s:%s" % (filename, line)

    os.close(fd)

if __name__ == '__main__':
    job1 = gevent.spawn(follow, '/var/log/syslog')
    job2 = gevent.spawn(follow, '/var/log/messages')

    gevent.joinall([job1, job2])

Upvotes: 3

Related Questions