CODEWITHSUNDEEP
c#.netfile-io.net-4.0
amo
amo

Reputation: 325

Prune simple text log file using C# .NET 4.0

An external Windows service I work with maintains a single text-based log file that it continuously appends to. This log file grows unbounded over time. I'd like to prune this log file periodically to maintain, say the most recent 5mb of log entries. How can I efficiently implement the file I/O code in C# .NET 4.0 to prune the file to say 5mb?

Updated: The way service dependencies are set up, my service always starts before the external service. This means I get exclusive access to the log file to truncate it, if required. Once the external service starts up, I will not access the log file. I can gain exclusive access to the file on desktop startup. The problem is - the log file may a few gigabytes in size and I'm looking for an efficient way to truncate it.

Upvotes: 0

Views: 1825

Answers (3)

Monso
Monso

Reputation: 1123

It's going to take the amount of memory that you want to store to process the "new" log file but if you only want 5Mb then it should be fine. If you are talking about Gb+ then you probably have other problems; however, it could still be accomplished using a temp file and some locking.

As noted before, you may experience a race condition but that's not the case if this is the only thread writing to this file. This would replace your current writing to the file.

const int MAX_FILE_SIZE_IN_BYTES = 5 * 1024 * 1024; //5Mb;
const string LOG_FILE_PATH = @"ThisFolder\log.txt";
string newLogMessage = "Hey this happened";


#region Use one or the other, I mean you could use both below if you really want to.
//Use this one to save an extra character
if (!newLogMessage.StartsWith(Environment.NewLine))
    newLogMessage = Environment.NewLine + newLogMessage;

//Use this one to imitate a write line
if (!newLogMessage.EndsWith(Environment.NewLine))
    newLogMessage = newLogMessage + Environment.NewLine; 
#endregion

int newMessageSize = newLogMessage.Length*sizeof (char);
byte[] logMessage = new byte[MAX_FILE_SIZE_IN_BYTES];
//Append new log to end of "file"
System.Buffer.BlockCopy(newLogMessage.ToCharArray(), 0, logMessage, MAX_FILE_SIZE_IN_BYTES - newMessageSize, logMessage.Length);


FileStream logFile = File.Open(LOG_FILE_PATH, FileMode.Open, FileAccess.ReadWrite);

int sizeOfRetainedLog = (int)Math.Min(MAX_FILE_SIZE_IN_BYTES - newMessageSize, logFile.Length);
//Set start position/offset of the file
logFile.Position = logFile.Length - sizeOfRetainedLog;
//Read remaining portion of file to beginning of buffer
logFile.Read(logMessage, logMessage.Length, sizeOfRetainedLog);

//Clear the file
logFile.SetLength(0); 
logFile.Flush();

//Write the file
logFile.Write(logMessage, 0, logMessage.Length);

I wrote this really quick, I apologize if I'm off by 1 somewhere.

Upvotes: 1

user1959018
user1959018

Reputation: 101

You could use a file observer to monitor the file:

FileSystemWatcher logWatcher = new FileSystemWatcher();
        logWatcher.Path = @"c:\example.log"
        logWatcher.Changed += logWatcher_Changed;

Then when the event is raised you can use a StreamReader to read the file 

private void logWatcher_Changed(object sender, FileSystemEventArgs e)
    {
     using (StreamReader readFile = new StreamReader(path))
        {
            string line;
            string[] row;
            while ((line = readFile.ReadLine()) != null)
            {
               // Here you delete the lines you want or move it to another file, so that your log keeps small. Then save the file. 
            }
        }
    }

It´s an option.

Upvotes: 0

Paul McDowell
Paul McDowell

Reputation: 144

depending on how often it is written to I'd say you might be facing a race condition to modify the file without damaging the log. You could always try writing a service to monitor the file size, and once it reaches a certain point lock the file, dupe and clear the whole thing and close it. Then store the data in another file that the service controls the size of easily. Alternatively you could see if the external service has an option for logging to a database, which would make it pretty simple to roll out the oldest data.

Upvotes: 0

Related Questions