CODEWITHSUNDEEP
cfile-iofwrite
Kyle G.
Kyle G.

Reputation: 880

Limiting # of Lines in a Log/Buffer File

I've got a small program running on an OpenWRT router that logs to a remote MySQL database. In the event that the database becomes unavailable the program writes to a buffer file (/var/buffer) to protect against data loss. Thing is, since it's being stored on the router itself, there's a chance of running out of room fairly quickly if the database is down for too long.

I figure that if I keep the file to a max of 20,000 lines, discarding the oldest ones as new ones are written (once the max size has been reached), I can minimize my data losses and not have to worry about running out of storage space (a bit of loss isn't the end of the world, and I'd rather keep the newest stuff than the oldest).

From my research I understand that the first line of a file cannot be removed without rewriting the entire file (no good, too time-consuming), and every time I think I'm close to another solution it falls apart.

Is there a better way? Or is re-writing the 20k-line file every time I have a new line to add my only option?

Upvotes: 0

Views: 73

Answers (1)

0xF1
0xF1

Reputation: 6116

You can have a log_LastLineNo variable which will store line number of last line written in log at that instant (For the first time, at the very start it will be 0).

Keep writing to file until you write 20,000 lines, and keep updating log_LastLineNo.
After that start overwriting the file from start, and set a variable log_full = 1.

Now
Case 1: log_full = 0 & log_LastLineNo = [some value < 20000]
In this case read from start till log_LastLineNo

Case 2: log_full = 1 & log_LastLineNo = [some value < 20000]
In this case start reading from log_LastLineNo + 1 till line 20000 and again from start to log_LastLineNo.

Upvotes: 1

Related Questions