CODEWITHSUNDEEP
pythonmultithreadingfile-io
Seeker
Seeker

Reputation: 43

Read multiple log file in realtime

I have to read data from log files present in realtime. I used the code mentioned in Reading multiple files in real time? however I am able to read only the first file i.e system.log. How to read all the files iteratively i.e first system.log and then wifi.log and repeat the same process again.

import time
import glob
import threading

def follow(thefile):
   thefile.seek(0,2)
    while True:
        line = thefile.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line


 lock = threading.Lock()

 def printFile(logfile):
    loglines = follow(logfile)
    for line in loglines:
        lock.acquire()
        print (line)
        lock.release()



  if __name__ == '__main__':
      files=['system.log','wifi.log']
      for log in files:
          logfile = open(log, "r")
          t = threading.Thread(target = printFile,args = (logfile,))
          t.start()

Upvotes: 2

Views: 1245

Answers (1)

mama
mama

Reputation: 2227

You can use the asyncio library to create a concurrent function to follow the file's tail and then print it.

import asyncio

async def follow(thefile):
    thefile.seek(0,2)
    while True:
        line = thefile.readline()
        if not line:
            await asyncio.sleep(0.1)
            continue
        yield line.strip()

async def main(path):
    async for x in follow(open(path)):
        print(x)

for path in ['hej', 'med', 'dig']:
    asyncio.ensure_future(main(path))

loop = asyncio.get_event_loop()
loop.run_forever()

You could also skip the main function and the yield in the follow function, and just print the output directly in the follow function instead of passing it.

Upvotes: 3

Related Questions