How to write log files correctly?

Question

How to reliably record logs to be sure of the atomicity of entries in multi-threaded applications? In addition, I would like to be able to rotate logs using the logrotate utility.

The simplest variant to write logs is next:

1. open/reopen log file
2. write entries by printf()
3. close log file at exit

Here is my example:

// default log level
static Cl_loglevl loglevel = LOGLEVEL_NONE;
// log file descriptor (open with Cl_openlog)
static FILE *logfd = NULL;

/**
 * @brief Cl_openlog - open log file
 * @param logfile - file name
 * @return FILE struct or NULL if failed
 */
FILE *Cl_openlog(const char *logfile, Cl_loglevl loglvl){
    if(logfd){
        Cl_putlog(LOGLEVEL_ERROR, "Reopen log file\n");
        fclose(logfd);
        logfd = NULL;
        char newname[PATH_MAX];
        snprintf(newname, PATH_MAX, "%s.old", logfile);
        if(rename(logfile, newname)) WARN("Can't rename old log file");
    }
    if(loglvl < LOGLEVEL_CNT) loglevel = loglvl;
    if(!logfile) return NULL;
    if(!(logfd = fopen(logfile, "w"))) WARN(_("Can't open log file"));
    return logfd;
}

/**
 * @brief Cl_putlog - put message to log file
 * @param lvl - message loglevel (if lvl > loglevel, message won't be printed)
 * @param fmt - format and the rest part of message
 * @return amount of symbols saved in file
 */
int Cl_putlog(Cl_loglevl lvl, const char *fmt, ...){
    if(lvl > loglevel || !logfd) return 0;
    char strtm[128];
    time_t t = time(NULL);
    struct tm *curtm = localtime(&t);
    strftime(strtm, 128, "%Y/%m/%d-%H:%M", curtm);
    int i = fprintf(logfd, "%s\t", strtm);
    va_list ar;
    va_start(ar, fmt);
    i += vfprintf(logfd, fmt, ar);
    va_end(ar);
    fflush(logfd);
    return i;
}

The call of Cl_openlog allows to rotate log once. I can call this function in SIG_USR1 handler and send this signal by logrotate. But it still remains unclear how to write to a file correctly in order to achieve atomicity of records.

I don't want to use external libraries like log4c for such simple problem.

Answer 1

Well, at last I did this:

// array with all opened logs - for error/warning messages
static Cl_log *errlogs = NULL;
static int errlogsnum = 0;

/**
 * @brief Cl_createlog - create log file: init mutex, test file open ability
 * @param log - log structure
 * @return 0 if all OK
 */
int Cl_createlog(Cl_log *log){
    if(!log || !log->logpath || log->loglevel <= LOGLEVEL_NONE || log->loglevel >= LOGLEVEL_CNT) return 1;
    FILE *logfd = fopen(log->logpath, "a");
    if(!logfd){
        WARN("Can't open log file");
        return 2;
    }
    fclose(logfd);
    pthread_mutex_init(&log->mutex, NULL);
    errlogs = realloc(errlogs, (++errlogsnum) *sizeof(Cl_log));
    if(!errlogs) errlogsnum = 0;
    else memcpy(&errlogs[errlogsnum-1], log, sizeof(Cl_log));
    return 0;
}

/**
 * @brief Cl_putlog - put message to log file with/without timestamp
 * @param timest - ==1 to put timestamp
 * @param log - pointer to log structure
 * @param lvl - message loglevel (if lvl > loglevel, message won't be printed)
 * @param fmt - format and the rest part of message
 * @return amount of symbols saved in file
 */
int Cl_putlogt(int timest, Cl_log *log, Cl_loglevl lvl, const char *fmt, ...){
    if(!log || !log->logpath || log->loglevel < 0 || log->loglevel >= LOGLEVEL_CNT) return 0;
    if(lvl > log->loglevel) return 0;
    if(pthread_mutex_lock(&log->mutex)) return 0;
    int i = 0;
    FILE *logfd = fopen(log->logpath, "a");
    if(!logfd) goto rtn;
    if(timest){
        char strtm[128];
        time_t t = time(NULL);
        struct tm *curtm = localtime(&t);
        strftime(strtm, 128, "%Y/%m/%d-%H:%M", curtm);
        i = fprintf(logfd, "%s\t", strtm);
    }
    va_list ar;
    va_start(ar, fmt);
    i += vfprintf(logfd, fmt, ar);
    va_end(ar);
    fclose(logfd);
rtn:
    pthread_mutex_unlock(&log->mutex);
    return i;
}

In Cl_createlog() I just test ability of opening of given file and init mutex. In Cl_putlogt() I open given file, write to it and close it. The mutex used to make writing atomic for any threads.

The array errlogs consists all opened logs for writing critical messages.

Answer 2

Either you can assure that all logging is done through a single thread, or you protect logging function with mutexes.
For first case, you can have a worker thread that polls on a pipe's read-end; then each thread will use pipe's write-end to wakeup worker thread and let it manage its log (allocated somewhere on heap and passed by address through pipe).
The same can be done for your SIGUSR1 handler (logrotate).

Please note that writing less than PIPE_BUF to a pipe() is guaranteed not to be interleaved (thus it is atomic).
Consequently writing just an heap storage address will always be atomic.

Last but not least, you may use different logfiles for each thread.