CODEWITHSUNDEEP
bashparallel-processingflock
thebananamancan
thebananamancan

Reputation: 91

Using flock to read and write to a file in bash

I have a bash script that may be run many times in parallel, and I need to be able to check the value in a file and modify it. Ideally, I'd like whichever script instance that gets there first to be able to do the reading and the writing without interference from another instance. I thought I could do this with flock, but it seems that some commands get ignored--I guess because they can't get a lock?

Here's what I have so far:

myfunc () {
    { flock -x 3 ; count=$(cat <&3); } 3< countfile
    { flock -x 3 ; echo $((count+1)) >&3; } 3> countfile
}

This is run from a subshell, so I have to do the counts via a file.

So, two things

  1. This is not using the same lock to read and write--I'd like that, but I'm not sure howto do it
  2. Why are my reads sometimes ignored?

Thanks!

Upvotes: 0

Views: 1917

Answers (2)

Charles Duffy
Charles Duffy

Reputation: 295650

You don't need to reuse the same descriptor for both read and write processes. In other languages, this would be a code smell, but bash doesn't have native support for seeking within an existing FD, making it a matter of unfortunate necessity.

myfunc () {
  {
    flock -x 3 || return
    count=$(<countfile)
    echo "$((count + 1 ))" >countfile
  } 3<>countfile
}

That said -- not all solutions are native! @telotortium has written an excellent C helper which can be used to seek within a pre-opened FD; if you were to use their code to seek back to the beginning of the file (or something similar to truncate it to 0 and move the FD there), you could reuse a single file descriptor for both the read and the write.

Upvotes: 1

thebananamancan
thebananamancan

Reputation: 91

i followed the example here. I'm not entirely sure why switching to using variable stream descriptors helped, but I accomplished what I needed. The other thing that's different, is that in the example there they are using the actual filename inside the subshell where it is locked, rather than actually referring to the stream.

This is new stuff to me, so I welcome comments.

Upvotes: 0

Related Questions