How to overload << operator for thread-safe logging in C++?

Question

I'm writing a sketch of a simple thread-safe logging library and some things came on my mind. Here's the code:

#ifndef SimpleLogger_H
#define SimpleLogger_H
#include <iostream>
#include <mutex>
class SimpleLogger
{
public:
    template <typename T>
    static void log(T message)
    {
        mutex.lock();
        std::cout << message;
        mutex.unlock();
    }
private:
    static std::mutex mutex;
}LOG;

template <typename T>
SimpleLogger &operator<<(SimpleLogger &simpleLogger, T message)
{
    simpleLogger.log(message);
    return simpleLogger;
}
#endif //SimpleLogger_H

My idea is to use it like this:

LOG << "hello" << " world " << 8 << " I can mix integers and strings";

I understand that the line above is like the following:

auto a1 = LOG.operator<<("hello");
auto a2 = a1.operator<<(" world ");
//Another thread may want to use LOG here, and would print in the middle of my message
auto a3 = a2.operator<<(8);
auto a4 = a3.operator<<(" I can mix integers and strings");

As you can see, because << is broken into several funciton calls, there's a risk that a thread can use the LOG object in the middle of my message (I consider a message the entire cascade of << on one line)

Also, is there a way to automatically add an std::endl for the last << call? I couldn't think of a way to do this, but I saw that some logging libraries have this functionality

How do I solve these two problems?

I know it'd be preferable to use a logging library but I want to mix android, desktop and ios logging in one simple lib without need for high performance, and I'm also puzzled by how I can overcome the difficulties I encountered while writing my own

Answer 1

You could create a helper class that collects all the output and prints on destruction. Outline:

#include <string>
#include <iostream>

struct Msg;
struct Log {
    void print(const Msg &m);
};

struct Msg {
    std::string m;
    Log &l;
    Msg(Log &l) : l(l) {}
    ~Msg() {
        // Print the message on destruction
        l.print(*this);
    }
};

void Log::print(const Msg &m) {
    // Logger specific printing... here, append newline
    std::cout << m.m << std::endl;
}

Msg &&operator << (Msg &&m, const std::string &s) {
    // Append operator
    m.m += s;
    return std::move(m);
}


// Helper to log on a specific logger. Just creates the initial message
Msg log(Log &l) { return Msg(l); }

int main()
{
    Log l;
    log(l) << "a" << "b" << "c";
    return 0;
}

As the Msg is local, other threads will not interfere with it. Any necessary locking can be done in the Log.print method, which will receive the complete message

Answer 2

As others already mentioned, you need a local buffer to collect message before sending to the log file. In the example below, SimpleLoggerBuffer objects are designed to be used as temporary variable only. I.e. it gets destroyed at the end of the expression. The destructor flushes the buffer into the log so that you don't have to explicitly call a flush function (you may add endl there as well if you wish)

#include <iostream>
#include <sstream>
#include <mutex>

using namespace std;

class SimpleLogger
{
public:    
    template <typename T>
    static void log(T& message)
    {
        mutex.lock();
        std::cout << message.str();
        message.flush();
        mutex.unlock();
    }
private:
    static std::mutex mutex;
}LOG;
std::mutex SimpleLogger::mutex;

struct SimpleLoggerBuffer{
    stringstream ss;

     SimpleLoggerBuffer() = default;
     SimpleLoggerBuffer(const SimpleLoggerBuffer&) = delete;
     SimpleLoggerBuffer& operator=(const SimpleLoggerBuffer&) = delete;
     SimpleLoggerBuffer& operator=(SimpleLoggerBuffer&&) = delete;
     SimpleLoggerBuffer(SimpleLoggerBuffer&& buf): ss(move(buf.ss)) {
     }
     template <typename T>
     SimpleLoggerBuffer& operator<<(T&& message)
     {
          ss << std::forward<T>(message);
          return *this;
     }

    ~SimpleLoggerBuffer() {
        LOG.log(ss);
    }
};

template <typename T>
SimpleLoggerBuffer operator<<(SimpleLogger &simpleLogger, T&& message)
{
    SimpleLoggerBuffer buf;
    buf.ss << std::forward<T>(message);
    return buf;
}

int main() {
    LOG << "hello" << " world " << 8 << " I can mix integers and strings";
}

Answer 3

The simplest approach is to return a temporary proxy from the first << - this can either log your stream for the duration (and unlock on destruction), or simply build a local ostringstream and flush it in a single call (again, on destruction).

Holding a lock while logging isn't great for performance (although it's better than your existing log method, which should use std::lock_guard for exception safety).

Building and discarding a temporary ostringstream is probably better, but if you care about performance you'll need to benchmark, and may well end up requiring something more elaborate (per-thread circular buffers, mmapped files or something).

Answer 4

I think that you can simply use std::clog. It is thread safe and, in contrary of std::cout, designed to output instantly for logging. From the reference page :

Unless sync_with_stdio(false) has been issued, it is safe to concurrently access these objects from multiple threads for both formatted and unformatted output.

I recommend you this Jason Turner's video about cout, clog and cerror.

Answer 5

A simple solution is to write into files instead of standard output, and specifically, separate file for each thread. That way no locking or any other synchronization is needed. The files can later be merged if lines have parseable format.

Another solution is to write logs asynchronously from a single thread, and initially store the messages in a thread safe (possibly lock free) queue.

Also, is there a way to automatically add an std::endl for the last << call?

Unless I misunderstand, you can simply do stream << message << std::endl.