CODEWITHSUNDEEP
pythonlinuxioparallel-processing
LRDPRDX
LRDPRDX

Reputation: 682

How to not write to file while reading and vise-versa

I have a python program (say reader.py) which uses file setting.py to read from:

while( True ):
  ...
  execfile( settings.py )
  ...

But there is other python program (say writer.py) that uses this file to write to:

...
try:
  settings = open('settings.py', 'w')
  settings.truncate()
  settings.write( 'some text')
except IOError:
  print('Cannot write to file')
finally:
  settings.close()
...

Note1: reader.py and writer.py do not ''know'' about each other.

Note2: reader.py reads settings.py cyclically, though writer.py writes to file when user wants to (not necessarily right after he/she clicked ''write'', it just means that there is no any rule when to write).

Question: What is the best way to cooperate two programs in order to avoid any contradiction? I know this might depend on platform. I am using Linux. Distributions are: Ubuntu, Scientific Linux.

EDIT1: If I choose to use FiFo I encounter the following problem: Once writer has write to settings file it will probably never write again but reader should have access to settings anyway in this case. In other words, reader should have an ability to read from file and not to wait for writer in this case. Otherwise reader has to wait for writer. Ordinary using of FiFo does not allow reader to read from file if writer does not write (until it has written). How to deal with this problem?

Upvotes: 1

Views: 338

Answers (2)

LRDPRDX
LRDPRDX

Reputation: 682

I found the following solution which seems to be working. I use flock to create locks.

Reader:

import errno                                                                                    
import fcntl
from time import *

path = "testLock.py"

f = open(path, "r")

while True: 
  try:
    fcntl.flock(f, fcntl.LOCK_EX | fcntl.LOCK_NB)
    break   
  except IOError as e:
    if e.errno != errno.EAGAIN:
    raise 
  else:   
    sleep(1)
    print 'Waiting...'

#reader's action      
execfile(path)  
#drop lock          
fcntl.flock(f, fcntl.LOCK_UN)

Writer:

import errno                                                                                    
import fcntl
from time import *

path = "testLock.py"

f = open(path, "w")

while True: 
  try:
    fcntl.flock(f, fcntl.LOCK_SH | fcntl.LOCK_NB)
    break   
  except IOError as e:
    if e.errno != errno.EAGAIN:
    raise 
  else:   
    sleep(1)
    print 'Waiting...'

#writer's action      
for i in (1,10,2):
  f.write('print "%d" % i')
  sleep(1)  
#drop lock          
fcntl.flock(f, fcntl.LOCK_UN)

I have some question here:

Qusetion 1: Is it correct usage of LOCK_EX and LOCK_SH I mean are they in the right place?

Question 2: Is the reader's action i.e execfile correct here? If the file is already opened is execfile try to open it anyway?

Upvotes: 0

datguy.dev
datguy.dev

Reputation: 31


    You may be interested in using a named pipe for your interprocess communications. Available in Linux, it is a special type of file designed for client (writer.py), server (reader.py), tasks. After writing to the pipe, the client will wait until the server has received the data. This allows you to sync the two processes somewhat.

Linux Manual for FiFo
Python doc: os.mkfifo(path[, mode])

Upvotes: 1

Related Questions