A 12-factor app logger built for performance and happy development

Overview

demo

logxi

log XI is a structured 12-factor app logger built for speed and happy development.

  • Simpler. Sane no-configuration defaults out of the box.
  • Faster. See benchmarks vs logrus and log15.
  • Structured. Key-value pairs are enforced. Logs JSON in production.
  • Configurable. Enable/disalbe Loggers and levels via env vars.
  • Friendlier. Happy, colorful and developer friendly logger in terminal.
  • Helpul. Traces, warnings and errors are emphasized with file, line number and callstack.
  • Efficient. Has level guards to avoid cost of building complex arguments.

Requirements

Go 1.3+

Installation

go get -u github.com/mgutz/logxi/v1

Getting Started

import "github.com/mgutz/logxi/v1"

// create package variable for Logger interface
var logger log.Logger

func main() {
    // use default logger
    who := "mario"
    log.Info("Hello", "who", who)

    // create a logger with a unique identifier which
    // can be enabled from environment variables
    logger = log.New("pkg")

    // specify a writer, use NewConcurrentWriter if it is not concurrent
    // safe
    modelLogger = log.NewLogger(log.NewConcurrentWriter(os.Stdout), "models")

    db, err := sql.Open("postgres", "dbname=testdb")
    if err != nil {
        modelLogger.Error("Could not open database", "err", err)
    }

    fruit := "apple"
    languages := []string{"go", "javascript"}
    if log.IsDebug() {
        // use key-value pairs after message
        logger.Debug("OK", "fruit", fruit, "languages", languages)
    }
}

logxi defaults to showing warnings and above. To view all logs

LOGXI=* go run main.go

Highlights

This logger package

  • Is fast in production environment

    A logger should be efficient and minimize performance tax. logxi encodes JSON 2X faster than logrus and log15 with primitive types. When diagnosing a problem in production, troubleshooting often means enabling small trace data in Debug and Info statements for some period of time.

    # primitive types
    BenchmarkLogxi          100000    20021 ns/op   2477 B/op    66 allocs/op
    BenchmarkLogrus          30000    46372 ns/op   8991 B/op   196 allocs/op
    BenchmarkLog15           20000    62974 ns/op   9244 B/op   236 allocs/op
    
    # nested object
    BenchmarkLogxiComplex    30000    44448 ns/op   6416 B/op   190 allocs/op
    BenchmarkLogrusComplex   20000    65006 ns/op  12231 B/op   278 allocs/op
    BenchmarkLog15Complex    20000    92880 ns/op  13172 B/op   311 allocs/op
    
  • Is developer friendly in the terminal. The HappyDevFormatter is colorful, prints file and line numbers for traces, warnings and errors. Arguments are printed in the order they are coded. Errors print the call stack.

    HappyDevFormatter is not too concerned with performance and delegates to JSONFormatter internally.

  • Logs machine parsable output in production environments. The default formatter for non terminals is JSONFormatter.

    TextFormatter may also be used which is MUCH faster than JSON but there is no guarantee it can be easily parsed.

  • Has level guards to avoid the cost of building arguments. Get in the habit of using guards.

    if log.IsDebug() {
        log.Debug("some ", "key1", expensive())
    }
    
  • Conforms to a logging interface so it can be replaced.

    type Logger interface {
        Trace(msg string, args ...interface{})
        Debug(msg string, args ...interface{})
        Info(msg string, args ...interface{})
        Warn(msg string, args ...interface{}) error
        Error(msg string, args ...interface{}) error
        Fatal(msg string, args ...interface{})
        Log(level int, msg string, args []interface{})
    
        SetLevel(int)
        IsTrace() bool
        IsDebug() bool
        IsInfo() bool
        IsWarn() bool
        // Error, Fatal not needed, those SHOULD always be logged
    }
    
  • Standardizes on key-value pair argument sequence

    
    

log.Debug("inside Fn()", "key1", value1, "key2", value2)

// instead of this log.WithFields(logrus.Fields{"m": "pkg", "key1": value1, "key2": value2}).Debug("inside fn()")

    logxi logs `FIX_IMBALANCED_PAIRS =>` if key-value pairs are imbalanced

    `log.Warn and log.Error` are special cases and return error:

    ```go
return log.Error(msg)               //=> fmt.Errorf(msg)
return log.Error(msg, "err", err)   //=> err
  • Supports Color Schemes (256 colors)

    log.New creates a logger that supports color schemes

    logger := log.New("mylog")
    

    To customize scheme

    # emphasize errors with white text on red background
    LOGXI_COLORS="ERR=white:red" yourapp
    
    # emphasize errors with pink = 200 on 256 colors table
    LOGXI_COLORS="ERR=200" yourapp
    
  • Is suppressable in unit tests

    
    

func TestErrNotFound() { log.Suppress(true) defer log.Suppress(false) ... }




## Configuration

### Enabling/Disabling Loggers

By default logxi logs entries whose level is `LevelWarn` or above when
using a terminal. For non-terminals, entries with level `LevelError` and
above are logged.

To quickly see all entries use short form

    # enable all, disable log named foo
    LOGXI=*,-foo yourapp

To better control logs in production, use long form which allows
for granular control of levels

    # the above statement is equivalent to this
    LOGXI=*=DBG,foo=OFF yourapp

`DBG` should obviously not be used in production unless for
troubleshooting. See `LevelAtoi` in `logger.go` for values.
For example, there is a problem in the data access layer
in production.

    # Set all to Error and set data related packages to Debug
    LOGXI=*=ERR,models=DBG,dat*=DBG,api=DBG yourapp

### Format

The format may be set via `LOGXI_FORMAT` environment
variable. Valid values are `"happy", "text", "JSON", "LTSV"`

    # Use JSON in production with custom time
    LOGXI_FORMAT=JSON,t=2006-01-02T15:04:05.000000-0700 yourapp

The "happy" formatter has more options

*   pretty - puts each key-value pair indented on its own line

    "happy" default to fitting key-value pair onto the same line. If
    result characters are longer than `maxcol` then the pair will be
    put on the next line and indented

*   maxcol - maximum number of columns before forcing a key to be on its
    own line. If you want everything on a single line, set this to high
    value like 1000. Default is 80.

*   context - the number of context lines to print on source. Set to -1
    to see only file:lineno. Default is 2.


### Color Schemes

The color scheme may be set with `LOGXI_COLORS` environment variable. For
example, the default dark scheme is emulated like this

    # on non-Windows, see Windows support below
    export LOGXI_COLORS=key=cyan+h,value,misc=blue+h,source=magenta,TRC,DBG,WRN=yellow,INF=green,ERR=red+h
    yourapp

    # color only errors
    LOGXI_COLORS=ERR=red yourapp

See [ansi](http://github.com/mgutz/ansi) package for styling. An empty
value, like "value" and "DBG" above means use default foreground and
background on terminal.

Keys

*   \*  - default color
*   TRC - trace color
*   DBG - debug color
*   WRN - warn color
*   INF - info color
*   ERR - error color
*   message - message color
*   key - key color
*   value - value color unless WRN or ERR
*   misc - time and log name color
*   source - source context color (excluding error line)

#### Windows

Use [ConEmu-Maximus5](https://github.com/Maximus5/ConEmu).
Read this page about [256 colors](https://code.google.com/p/conemu-maximus5/wiki/Xterm256Colors).

Colors in PowerShell and Command Prompt _work_ but not very pretty.

## Extending

What about hooks? There are least two ways to do this

*   Implement your own `io.Writer` to write to external services. Be sure to set
    the formatter to JSON to faciliate decoding with Go's built-in streaming
    decoder.
*   Create an external filter. See `v1/cmd/filter` as an example.

What about log rotation? 12 factor apps only concern themselves with
STDOUT. Use shell redirection operators to write to a file.

There are many utilities to rotate logs which accept STDIN as input. They can
do many things like send alerts, etc. The two obvious choices are Apache's `rotatelogs`
utility and `lograte`.

```sh
yourapp | rotatelogs yourapp 86400

Testing

# install godo task runner
go get -u gopkg.in/godo.v2/cmd/godo

# install dependencies
godo install -v

# run test
godo test

# run bench with allocs (requires manual cleanup of output)
godo bench-allocs

License

MIT License

Issues
  • Question: What's the purpose of this line of code?

    Question: What's the purpose of this line of code?

    I'm trying to make sense of this line of code but I dunno what it does. It seems the preceding code does enough for the current intent.

    https://github.com/mgutz/logxi/blob/master/v1/env.go#L107

    opened by oblitum 9
  • Option to disable colors

    Option to disable colors

    When running under LiteIDE the colors make it really hard to read the program output (the LiteIDE terminal does not handle the ANSI escape sequences, as such, we see them intermingled with the actual program output).

    Can we have an option to disable the colors? maybe LOGXI_COLORS=disable

    opened by rgl 8
  • Call stack is strange and shows up even on .Info() when passing an error variable.

    Call stack is strange and shows up even on .Info() when passing an error variable.

    Is this expected?

    ❯❯❯ GOMAXPROCS=4 LOGXI="*=INF" ./foo
    17:59:21.391877 INF foo Error on parsing message (continuing)
       err: invalid character 's' looking for beginning of value
       /home/francisco/.go/src/github.com/mgutz/logxi/v1/jsonFormatter.go:48 (0x4b58bf)
            (*JSONFormatter).writeError: jf.set(buf, KeyMap.CallStack, string(debug.Stack()))
    /home/francisco/.go/src/github.com/mgutz/logxi/v1/jsonFormatter.go:87 (0x4b5c4f)
            (*JSONFormatter).appendValue: jf.writeError(buf, err)
    /home/francisco/.go/src/github.com/mgutz/logxi/v1/jsonFormatter.go:114 (0x4b643c)
            (*JSONFormatter).set: jf.appendValue(buf, val)
    /home/francisco/.go/src/github.com/mgutz/logxi/v1/jsonFormatter.go:166 (0x4b6c36)
            (*JSONFormatter).Format: jf.set(buf, key, args[i+1])
    /home/francisco/.go/src/github.com/mgutz/logxi/v1/jsonFormatter.go:187 (0x4b6ffc)
            (*JSONFormatter).LogEntry: jf.Format(buf, level, msg, args)
    /home/francisco/.go/src/github.com/mgutz/logxi/v1/happyDevFormatter.go:273 (0x4b2b9b)
            (*HappyDevFormatter).Format: entry := hd.jsonFormatter.LogEntry(level, msg, args)
    /home/francisco/.go/src/github.com/mgutz/logxi/v1/defaultLogger.go:106 (0x4aed00)
            (*DefaultLogger).Log: l.formatter.Format(l.writer, level, msg, args)
    /home/francisco/.go/src/github.com/mgutz/logxi/v1/defaultLogger.go:70 (0x4ae8c3)
            (*DefaultLoggelogger.Info("Error on parsing message (continuing)", "err", err)r).Info: l.Log(LevelInfo, msg, args)
    /home/francisco/.go/src/github.com/foo/main.go:158 (0x402efe)
            main: logger.Info("Error on parsing message (continuing)", "err", err)
    /usr/local/go/src/runtime/proc.go:63 (0x4173a3)
            main: main_main()
    /usr/local/go/src/runtime/asm_amd64.s:2232 (0x42a411)
            goexit:
    

    This gets generated at logger.Info("Error on parsing message (continuing)", "err", err).

    If I do logger.Error("Error on parsing message (continuing)", "err", err.Error()) instead I get:

    ❯❯❯ ./timelapsee
    18:07:11.060864 ERR foo Error on parsing message (continuing)
       err: invalid character 'a' looking for beginning of value
    /home/francisco/.go/src/github.com/foo/main.go:158 (0x402f31)
            main: logger.Error("Error on parsing message (continuing)", "err", err.Error())
    /usr/local/go/src/runtime/proc.go:63 (0x4173d3)
            main: main_main()
    /usr/local/go/src/runtime/asm_amd64.s:2232 (0x42a441)
            goexit: 
    

    Which is the expected call stack for .Error, if I use .Info instead I get no call stack as expected.

    The docs at the README show samples passing error variables directly, but this seems to generate some unwanted behavior.

    opened by oblitum 8
  • Warn should return error object

    Warn should return error object

    Hey first of all I absolutely love logxi & feel bad about posting this first "issue" that is really a feature request!

    Request: can we have Warn() return the first error param like Error() does.

    Reasoning by real example:-

    I have a function that unpacks a packet received from the network, and analyzes its fields to make sure they are all legal in relation to the protocol and state transitions they will drive. In some circumstances, the packet will be illegal in the sense that it breaks a sequence of transitions, or something like that, and thus the function returns a predefined error describing what has happened, and higher level code resets the protocol. In this case the protocol is an overlay over and existing P2P network and such conditions are triggered quite naturally by various things and so resetting the virtual connection is quite normal and only merits a warning in the logs. Nonetheless, I still want to pass up the error so the protocol can be reset.

    Would there be any probs with having Warn return errors in the same way?

    Best, Dominic

    opened by dominicwilliams 5
  • Using HappyDevFormatter can cause race conditions not present in the jsonFormatter

    Using HappyDevFormatter can cause race conditions not present in the jsonFormatter

    No explicit statements are made concerning goroutine safety around this package.

    Generally the package does appear to be safe except for the col variable inside the HappyDevFormat struct which when the format is called concurrently can lead to failures coming from the GoLang race detector.

    opened by karlmutch 2
  • Use of isTerminal before initialization

    Use of isTerminal before initialization

    At line init.go#L190 it seems like isTerminal is being checked before it's initialized at line init.go#L197. This would make the internal logger to log always using json.

    opened by oblitum 2
  • Add LTSVFormatter

    Add LTSVFormatter

    I like to use logxi. Sometimes I want to LTSV log for parsing and collectioning. What I use / develop an system is still used to LTSV.

    $ export LOGXI_FORMAT="LTSV"
    $ go run main.go
    time:16:14:42.721886    name:test   level:INF   message:Process pid:15968
    time:16:14:42.722039    name:test   level:ERR   message:Process pid:15968   err:EOF _c:"/home/msshin/go/logxi/src/github.com/mgutz/logxi/v1/ltsvFormatter.go:58 (0x4434cf)\n\t(*LTSVFormatter).set: buf.WriteString(strconv.Quote(string(debug.Stack())))\n/home/msshin/go/logxi/src/github.com/mgutz/logxi/v1/ltsvFormatter.go:83 (0x443b3f)\n\t(*LTSVFormatter).Format: lf.set(buf, key, args[i+1])\n/home/msshin/go/logxi/src/github.com/mgutz/logxi/v1/defaultLogger.go:104 (0x43b100)\n\t(*DefaultLogger).Log: l.formatter.Format(l.writer, level, msg, args)\n/home/msshin/go/logxi/src/github.com/mgutz/logxi/v1/defaultLogger.go:84 (0x43aeb3)\n\t(*DefaultLogger).extractLogError: return fmt.Errorf(msg)\n/home/msshin/go/logxi/src/github.com/mgutz/logxi/v1/defaultLogger.go:89 (0x43af5b)\n\t(*DefaultLogger).Error: return l.extractLogError(LevelError, msg, args)\n/home/msshin/go/logxi/src/tmp/main.go:13 (0x400fea)\n\tmain: logger.Error(\"Process\", \"pid\", os.Getpid(), \"err\", io.EOF)\n/usr/local/go/src/runtime/proc.go:63 (0x411f43)\n/usr/local/go/src/runtime/asm_amd64.s:2232 (0x421c31)\n"
    
    opened by anarcher 2
  • fix panic when logging a nil pointer to a Stringer

    fix panic when logging a nil pointer to a Stringer

    jsonFormatter.go only checks whether the val is nil, but if val is a pointer, it's been cast to interface{}, and val == nil no longer works. See PR for failing test case.

    opened by ansel1 1
  • Feature: option to honor Stringer interface

    Feature: option to honor Stringer interface

    If I have a large struct, logxi's map rendering of it can be too verbose. I can implement String() on it, but logxi ignores it. If I pass s.String() as the arg to logxi, I incur the Sprintf cost even when the log message is ignored, so I need to do this:

    if log.isTrace() { log.Trace("the struct", s.String())}
    

    Would be cool if logxi had a flag to enable using String() if implemented.

    opened by ansel1 1
  • Error Installing

    Error Installing

    When i try to install the package i get this error:

    > go get -u github.com/mgutz/logxi/v1
    # github.com/mgutz/logxi/v1
    ../../mgutz/logxi/v1/callstack.go:11: import /home/deploy/golang/pkg/linux_amd64/github.com/mgutz/ansi.a: not a package file
    

    Additional Info:

    OS: Debian 7.1

    > go version
    go version go1.4.2 linux/amd64
    
    > go env
    GOARCH="amd64"
    GOBIN=""
    GOCHAR="6"
    GOEXE=""
    GOHOSTARCH="amd64"
    GOHOSTOS="linux"
    GOOS="linux"
    GOPATH="/home/deploy/golang"
    GORACE=""
    GOROOT="/usr/local/go"
    GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
    CC="gcc"
    GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
    CXX="g++"
    
    opened by pcriv 1
  • Safe concurrent logging

    Safe concurrent logging

    As far as I was able to check the internals of the package, there's no guarding mechanism when writting to stdout.

    If this is correct, then the library is not safe to use with concurrent code:

    • http://stackoverflow.com/a/14694630/1000282

    The standard log package does make use of locking mechanisms to safe-guard against this.

    bug 
    opened by oblitum 1
  • Suggestion. Remove unuseful panic

    Suggestion. Remove unuseful panic

    Usually:

    1. panic generate a lot of unuseful traces
    2. should be catch and logged additionally
    3. defer not make sense, cause it last line in the method

    This PR fix that behavior

    opened by gollariel 0
  • go modules build fails

    go modules build fails

    Under a project that depends on github.com/mgutz/logxi, when I migrate the dependency management to go mod, and run go build, then I see:

    go: finding github.com/mgutz/logxi latest
    build github.com/myorg/myproj/cmd: cannot load github.com/mgutz/logxi: cannot find module providing package github.com/mgutz/logxi
    

    It's probably because there's no source file in the top directory, but only under v1.

    See also a related issue: https://github.com/influxdata/influxdb1-client/issues/5

    opened by dongsupark 0
  • Maintainers?

    Maintainers?

    Are you interested in getting help maintaining logxi? I'd be happy to help with the outstanding issues if you are open to letting other people help you. There is little activity here and I'd preferably not fork the original project.

    @mgutz

    opened by KidLinus 2
  • Why does Warn and Error return the error?

    Why does Warn and Error return the error?

    Im failing to understand the purpose of returning back the error i sent to log.Error and log.Warn. I already have the error and the source code shows that you do not change anything about the error. What's the deal with this behaviour? I can't find any good documentation on this.

    opened by KidLinus 0
Owner
Mario Gutierrez
Mario Gutierrez
Logger - Simple logger without written with std pkg

Go-Logger Simple usage is: package main

MaskedTrench 2 Jan 2, 2022
Logger - A thin wrapper of uber-go/zap logger for personal project

a thin wraper of uber-go/zap logger for personal project 0. thanks uber-go/zap B

tsingson 1 Jun 16, 2022
A distributed logger micro-service built in go

Distributed Logger A distributed logger micro-service built in go using gRPC for internal communication with other micro-services and JSON for externa

Famous Ketoma 0 Nov 6, 2021
Golog is a logger which support tracing and other custom behaviors out of the box. Blazing fast and simple to use.

GOLOG Golog is an opinionated Go logger with simple APIs and configurable behavior. Why another logger? Golog is designed to address mainly two issues

Damiano Petrungaro 34 Jul 26, 2022
Dead simple, super fast, zero allocation and modular logger for Golang

Onelog Onelog is a dead simple but very efficient JSON logger. It is one of the fastest JSON logger out there. Also, it is one of the logger with the

Francois Parquet 399 Jun 16, 2022
A feature-rich and easy to use logger for golang

A feature-rich and easy to use logger for golang ?? Install ?? Common Logs lumber.Success() lumber.Info() lumber.Debug() lumber.Warning()

Matthew Gleich 50 Jul 20, 2022
A minimal and extensible structured logger

⚠️ PRE-RELEASE ⚠️ DO NOT IMPORT THIS MODULE YOUR PROJECT WILL BREAK package log package log provides a minimal interface for structured logging in ser

Go kit 111 Aug 2, 2022
🪵 A dead simple, pretty, and feature-rich logger for golang

?? lumber ?? A dead simple, pretty, and feature-rich logger for golang ?? Install ?? Logging Functions lumber.Success() lumber.Info() lumber.Debug() l

Matt Gleich 50 Jul 20, 2022
A customized GORM logger that implements the appropriate interface and uses Logrus to output logs

CryptoMath GORM Logger A customized GORM logger that implements the appropriate interface and uses Logrus to output logs. Install go get github.com/ma

Crypto Math 0 Nov 6, 2021
logger wraps uber/zap and trace with opentelemetry

logger 特性 支持 uber/zap 日志 支持 log rolling,使用 lumberjace 支持日志追踪 支持debug、info、warn、e

xiaolei 7 Jul 26, 2022
A simple and super power logger for golang

The most powerfull and faster logger for golang powered by DC ?? What is this? W

Teo 6 Jun 26, 2022
This package enables json output, level logging and so on to standard go logger.

logplug This package enables json output, level logging and so on to standard logger. Usage log.SetOutput(logplug.NewJSONPlug(os.Stderr, logplug.LogF

Koumei Mikuni 0 Dec 27, 2021
Goal is to generate logger and tracer wraps around a certain struct

Goal is to generate logger and tracer wraps around a certain struct

null 0 Feb 13, 2022
A logger, for Go

Go-Log A logger, for Go! It's sort of log and code.google.com/p/log4go compatible, so in most cases can be used without any code changes. Breaking cha

Ian Kent 38 May 12, 2022
Simple logger for Go programs. Allows custom formats for messages.

go-logger A simple go logger for easy logging in your programs. Allows setting custom format for messages. Preview Install go get github.com/apsdehal/

Amanpreet Singh 276 Jun 14, 2022
Loggly Hooks for GO Logrus logger

Loggly Hooks for Logrus Usage package main import ( "github.com/sirupsen/logrus" "github.com/sebest/logrusly" ) var logglyToken string = "YOUR_LOG

Sebest 27 Jul 27, 2021
A logger for Go SQL database driver without modify existing *sql.DB stdlib usage.

SQLDB-Logger A logger for Go SQL database driver without modify existing *sql.DB stdlib usage. Colored console writer output above only for sample/dev

Sarjono Mukti Aji 230 Aug 2, 2022
xlog is a logger for net/context aware HTTP applications

⚠️ Check zerolog, the successor of xlog. HTTP Handler Logger xlog is a logger for net/context aware HTTP applications. Unlike most loggers, xlog will

Olivier Poitrey 135 Nov 22, 2021
Zero Allocation JSON Logger

Zero Allocation JSON Logger The zerolog package provides a fast and simple logger dedicated to JSON output. Zerolog's API is designed to provide both

Olivier Poitrey 6.7k Aug 5, 2022