CODEWITHSUNDEEP
logginggo
FlowRaja
FlowRaja

Reputation: 727

How can I log in golang to a file with log rotation?

I am trying to write a web application which will run on a remote server. I need to log to capture errors/debug/audit. I find that multiple logging packages are available for golang including the standard "log" package. However, I need to fulfill three requirements:

  1. The log files need to be rotated
  2. It applies to the included packages which use "log"
  3. It needs to be cross-platform. Dev environment is Linux and needs to be deployed on Windows.

Upvotes: 23

Views: 68663

Answers (7)

Xaqron
Xaqron

Reputation: 30837

With logrus and lumberjack plugin for logrus:

package mypackage

import (
"io"
"os"
"path/filepath"
"time"

log "github.com/sirupsen/logrus"
"gopkg.in/natefinch/lumberjack.v2"
)

func SetupLogger() {

    lumberjackLogger := &lumberjack.Logger{
        // Log file abbsolute path, os agnostic
        Filename:   filepath.ToSlash("/path/to/log/file"), 
        MaxSize:    5, // MB
        MaxBackups: 10,
        MaxAge:     30,   // days
        Compress:   true, // disabled by default
    }

    // Fork writing into two outputs
    multiWriter := io.MultiWriter(os.Stderr, lumberjackLogger)

    logFormatter := new(log.TextFormatter)
    logFormatter.TimestampFormat = time.RFC1123Z // or RFC3339
    logFormatter.FullTimestamp = true

   log.SetFormatter(logFormatter)
   log.SetLevel(log.InfoLevel)
   log.SetOutput(multiWriter)
}

Use this function at early stage of your program (maybe inside init).

Import log "github.com/sirupsen/logrus" in any file and log with:

log.Info("some message")

Upvotes: 11

user27111987
user27111987

Reputation: 1085

We can also use a lib https://github.com/lestrrat/go-file-rotatelogs to achieve the same. It provides option to set

Max Age
Log Rotation Time

It can be hooked to any kind of logger too.

Source: https://golangbyexample.com/go-logger-rotation/

Upvotes: 3

kinshuk4
kinshuk4

Reputation: 3251

Though @Crast has given a very good answer, I want to also bring to the notice - lumberjack logger by Nate Finch which I ended up using.

Here is how to use it:

  1. First, clone the lumberjack repository OR get it somehow.
  2. Run the go install command on the folder.
  3. Now import go's "log" package and "lumberjack package".

import ( "log" "github.com/natefinch/lumberjack" )

  1. Now use it in your code like this:

Outside of main, declare your log variable.

var errLog *log.Logger

Inside main:

e, err := os.OpenFile("./foo.log", os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0666)

if err != nil {
    fmt.Printf("error opening file: %v", err)
    os.Exit(1)
}
errLog = log.New(e, "", log.Ldate|log.Ltime)
errLog.SetOutput(&lumberjack.Logger{
    Filename:   "./foo.log",
    MaxSize:    1,  // megabytes after which new file is created
    MaxBackups: 3,  // number of backups
    MaxAge:     28, //days
})

Now as soon as the file size get 1MB, a new file is created to keep the previous logs with the current timestamps, and the new logs will continue to log into foo.log file. Also, I have created the file using os.OpenFile but you may not need it as lumberjack internally does it, but I preferred it that way. Thanks, hope it helps. Once again thanks to @Crast and NateFinch.

Upvotes: 29

jame2981
jame2981

Reputation: 51

https://github.com/jame2981/log My package can help you. 

l1 := log.Pool.New("l1", "file:///tmp/test1.log")
l2 := log.Pool.New("l2", "file:///tmp/test2.log")
l3 := log.Pool.New("l3", "file:///tmp/test3.log")
l4 := log.Pool.New("l4", "file:///tmp/test4.log")

l1.Rotate() // rotate l1 only
log.Pool.Rotate() // was rotate all instances.

// rotate with signal
reopen := make(chan os.Signal, 1)
signal.Notify(reopen, syscall.SIGUSR1)
go func() {
    for{
       <-reopen
       l.Pool.Rotate()
    }
}()

set std logger writer so rotate work yet.

// std logger writer
import "log"
logger := log.New("test", "", 0)
logger.SetOutput(l1.Writer())

Upvotes: 0

Crast
Crast

Reputation: 16316

The best way to fulfill all your three requirements instead of creating an alternate logger struct, if you were satisfied using the base-level log.Log, is instead to set the output of the logger to your own io.Writer instance.

So basically what I'm going to do here is show an example where I create my own io.Writer:

import (
    "os"
    "sync"
    "time"
)

type RotateWriter struct {
    lock     sync.Mutex
    filename string // should be set to the actual filename
    fp       *os.File
}

// Make a new RotateWriter. Return nil if error occurs during setup.
func New(filename string) *RotateWriter {
    w := &RotateWriter{filename: filename}
    err := w.Rotate()
    if err != nil {
        return nil
    }
    return w
}

// Write satisfies the io.Writer interface.
func (w *RotateWriter) Write(output []byte) (int, error) {
    w.lock.Lock()
    defer w.lock.Unlock()
    return w.fp.Write(output)
}

// Perform the actual act of rotating and reopening file.
func (w *RotateWriter) Rotate() (err error) {
    w.lock.Lock()
    defer w.lock.Unlock()

    // Close existing file if open
    if w.fp != nil {
        err = w.fp.Close()
        w.fp = nil
        if err != nil {
            return
        }
    }
    // Rename dest file if it already exists
    _, err = os.Stat(w.filename)
    if err == nil {
        err = os.Rename(w.filename, w.filename+"."+time.Now().Format(time.RFC3339))
        if err != nil {
            return
        }
    }

    // Create a file.
    w.fp, err = os.Create(w.filename)
    return
}

You then create a RotateWriter and use log.SetOutput to set this writer (if other packages are using the standard logger instance) or alternately create your own instances using log.New to pass around.

I haven't solved the situation of when to call Rotate, I'll leave that to you to decide. It'd be fairly simple to trigger it based on time, or alternately do so after some amount of writes or some amount of bytes.

Upvotes: 21

antigloss
antigloss

Reputation: 41

Here is a light-weighted logging package that supports log rotation and auto purging

https://github.com/antigloss/go/tree/master/logger

// logger.Init must be called first to setup logger
logger.Init("./log", // specify the directory to save the logfiles
            400, // maximum logfiles allowed under the specified log directory
            20, // number of logfiles to delete when number of logfiles exceeds the configured limit
            100, // maximum size of a logfile in MB
            false) // whether logs with Trace level are written down
logger.Info("Failed to find player! uid=%d plid=%d cmd=%s xxx=%d", 1234, 678942, "getplayer", 102020101)
logger.Warn("Failed to parse protocol! uid=%d plid=%d cmd=%s", 1234, 678942, "getplayer")

Upvotes: 4

OneOfOne
OneOfOne

Reputation: 99215

One option that comes to mind is to wrap the logging in your own type and provide a reload function, something like:

type Logger struct {
    l *log.Logger
    f *os.File
    m sync.RWMutex
}

func NewLogger(fn string) (*Logger, error) {
    f, err := os.Create(fn)
    if err != nil {
        return nil, err
    }
    l := &Logger{
        l: log.New(f, "your-app", log.Lshortfile),
        f: f,
    }
    return l, nil
}

func (l *Logger) Logf(f string, args ...interface{}) {
    l.m.RLock()
    l.l.Printf(f, args...)
    l.m.RUnlock()
}

func (l *Logger) Reload() (err error) {
    l.m.Lock()
    defer l.m.Unlock()
    l.f.Close()
    if l.f, err = os.Create(l.f.Name()); err != nil {
        return
    }
    l.l = log.New(l.f, "your-app", log.Lshortfile)
    return
}

Then either listen for a signal (usually -HUP on *nix) or add an endpoint in your app that would call Logger.Reload().

Upvotes: 0

Related Questions