CODEWITHSUNDEEP
c++
vinnylinux
vinnylinux

Reputation: 7024

Thread-safe log buffer in C++?

I'm implementing my own logging system for performance purposes (and because i basically just need a buffer). What i currently have is something like this:

// category => messages
static std::unordered_map<std::string, std::ostringstream> log;

void main() {
    while (true) {
        log["info"] << "Whatever";
        log["192.168.0.1"] << "This is a dynamic entry";

        dump_logs();
    }
}

void dump_logs() {
    // i do something like this for each category, but they have different save strategies
    if (log["info"].tellp() > 1000000) {
        // save the ostringstream to a file

        // clear the log
        log["info"].str("")
    }
}

It works perfectly. However, i've just added threads and i'm not sure if this code is thread-safe. Any tips?

Upvotes: 1

Views: 1516

Answers (2)

Andrew Henle
Andrew Henle

Reputation: 1

On a POSIX system, if you're always writing data to the end of the file, the fastest way for multiple threads to write data to a file is to use low-level C-style open() in append mode, and just call write(), because the POSIX standard for write() states:

On a regular file or other file capable of seeking, the actual writing of data shall proceed from the position in the file indicated by the file offset associated with fildes. Before successful return from write(), the file offset shall be incremented by the number of bytes actually written. On a regular file, if the position of the last byte written is greater than or equal to the length of the file, the length of the file shall be set to this position plus one.

...

If the O_APPEND flag of the file status flags is set, the file offset shall be set to the end of the file prior to each write and no intervening file modification operation shall occur between changing the file offset and the write operation.

So, all write() calls from within a process to a file opened in append mode are atomic.

No mutexes are needed.

Almost. The only issue you have to be concerned with is

If write() is interrupted by a signal after it successfully writes some data, it shall return the number of bytes written.

If you have enough control over your environment that you can be sure that your calls to write() will not be interrupted by a signal after only a portion of the data is written, this is the fastest way to write data to a file from multiple threads - you're using the OS-provided lock on the file descriptor that ensures adherence to the POSIX-specified behavior, and as long as you generate the data to be written without any locking, that file descriptor lock is the only one in the entire data path. And that lock will be in your data path no matter what you do in your code.

Upvotes: 1

Galik
Galik

Reputation: 48625

You can make this thread safe by declaring your map thread_local. If you are going to use it across translation units then make it extern and define it in one translation unit, otherwise static is fine.

You will still need to synchronize writing the logs to disk. A mutex should fix that:

// category => messages (one per thread)
thread_local static std::unordered_map<std::string, std::ostringstream> log;

void main() {
    while (true) {
        log["info"] << "Whatever";
        log["192.168.0.1"] << "This is a dynamic entry";

        dump_logs();
    }
}

void dump_logs() {

    static std::mutex mtx; // mutex shared between threads

    // i do something like this for each category, but they have different save strategies
    if (log["info"].tellp() > 1000000) {

        // now I need to care about threads
        // use { to create a lock that will release at the end }
        {
            std::lock_guard<std::mutex> lock(mtx); // synchronized access

            // save the ostringstream to a file
        }

        // clear the log
        log["info"].str("");
    }
}

Upvotes: 1

Related Questions