Zero Allocation JSON Logger

Overview

Zero Allocation JSON Logger

godoc license Build Status Coverage

The zerolog package provides a fast and simple logger dedicated to JSON output.

Zerolog's API is designed to provide both a great developer experience and stunning performance. Its unique chaining API allows zerolog to write JSON (or CBOR) log events by avoiding allocations and reflection.

Uber's zap library pioneered this approach. Zerolog is taking this concept to the next level with a simpler to use API and even better performance.

To keep the code base and the API simple, zerolog focuses on efficient structured logging only. Pretty logging on the console is made possible using the provided (but inefficient) zerolog.ConsoleWriter.

Pretty Logging Image

Who uses zerolog

Find out who uses zerolog and add your company / project to the list.

Features

Installation

go get -u github.com/rs/zerolog/log

Getting Started

Simple Logging Example

For simple logging, import the global logger package github.com/rs/zerolog/log

package main

import (
    "github.com/rs/zerolog"
    "github.com/rs/zerolog/log"
)

func main() {
    // UNIX Time is faster and smaller than most timestamps
    zerolog.TimeFieldFormat = zerolog.TimeFormatUnix

    log.Print("hello world")
}

// Output: {"time":1516134303,"level":"debug","message":"hello world"}

Note: By default log writes to os.Stderr Note: The default log level for log.Print is debug

Contextual Logging

zerolog allows data to be added to log messages in the form of key:value pairs. The data added to the message adds "context" about the log event that can be critical for debugging as well as myriad other purposes. An example of this is below:

package main

import (
    "github.com/rs/zerolog"
    "github.com/rs/zerolog/log"
)

func main() {
    zerolog.TimeFieldFormat = zerolog.TimeFormatUnix

    log.Debug().
        Str("Scale", "833 cents").
        Float64("Interval", 833.09).
        Msg("Fibonacci is everywhere")
    
    log.Debug().
        Str("Name", "Tom").
        Send()
}

// Output: {"level":"debug","Scale":"833 cents","Interval":833.09,"time":1562212768,"message":"Fibonacci is everywhere"}
// Output: {"level":"debug","Name":"Tom","time":1562212768}

You'll note in the above example that when adding contextual fields, the fields are strongly typed. You can find the full list of supported fields here

Leveled Logging

Simple Leveled Logging Example

package main

import (
    "github.com/rs/zerolog"
    "github.com/rs/zerolog/log"
)

func main() {
    zerolog.TimeFieldFormat = zerolog.TimeFormatUnix

    log.Info().Msg("hello world")
}

// Output: {"time":1516134303,"level":"info","message":"hello world"}

It is very important to note that when using the zerolog chaining API, as shown above (log.Info().Msg("hello world"), the chain must have either the Msg or Msgf method call. If you forget to add either of these, the log will not occur and there is no compile time error to alert you of this.

zerolog allows for logging at the following levels (from highest to lowest):

  • panic (zerolog.PanicLevel, 5)
  • fatal (zerolog.FatalLevel, 4)
  • error (zerolog.ErrorLevel, 3)
  • warn (zerolog.WarnLevel, 2)
  • info (zerolog.InfoLevel, 1)
  • debug (zerolog.DebugLevel, 0)
  • trace (zerolog.TraceLevel, -1)

You can set the Global logging level to any of these options using the SetGlobalLevel function in the zerolog package, passing in one of the given constants above, e.g. zerolog.InfoLevel would be the "info" level. Whichever level is chosen, all logs with a level greater than or equal to that level will be written. To turn off logging entirely, pass the zerolog.Disabled constant.

Setting Global Log Level

This example uses command-line flags to demonstrate various outputs depending on the chosen log level.

package main

import (
    "flag"

    "github.com/rs/zerolog"
    "github.com/rs/zerolog/log"
)

func main() {
    zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
    debug := flag.Bool("debug", false, "sets log level to debug")

    flag.Parse()

    // Default level for this example is info, unless debug flag is present
    zerolog.SetGlobalLevel(zerolog.InfoLevel)
    if *debug {
        zerolog.SetGlobalLevel(zerolog.DebugLevel)
    }

    log.Debug().Msg("This message appears only when log level set to Debug")
    log.Info().Msg("This message appears when log level set to Debug or Info")

    if e := log.Debug(); e.Enabled() {
        // Compute log output only if enabled.
        value := "bar"
        e.Str("foo", value).Msg("some debug message")
    }
}

Info Output (no flag)

$ ./logLevelExample
{"time":1516387492,"level":"info","message":"This message appears when log level set to Debug or Info"}

Debug Output (debug flag set)

$ ./logLevelExample -debug
{"time":1516387573,"level":"debug","message":"This message appears only when log level set to Debug"}
{"time":1516387573,"level":"info","message":"This message appears when log level set to Debug or Info"}
{"time":1516387573,"level":"debug","foo":"bar","message":"some debug message"}

Logging without Level or Message

You may choose to log without a specific level by using the Log method. You may also write without a message by setting an empty string in the msg string parameter of the Msg method. Both are demonstrated in the example below.

package main

import (
    "github.com/rs/zerolog"
    "github.com/rs/zerolog/log"
)

func main() {
    zerolog.TimeFieldFormat = zerolog.TimeFormatUnix

    log.Log().
        Str("foo", "bar").
        Msg("")
}

// Output: {"time":1494567715,"foo":"bar"}

Error Logging

You can log errors using the Err method

package main

import (
	"errors"

	"github.com/rs/zerolog"
	"github.com/rs/zerolog/log"
)

func main() {
	zerolog.TimeFieldFormat = zerolog.TimeFormatUnix

	err := errors.New("seems we have an error here")
	log.Error().Err(err).Msg("")
}

// Output: {"level":"error","error":"seems we have an error here","time":1609085256}

The default field name for errors is error, you can change this by setting zerolog.ErrorFieldName to meet your needs.

Error Logging with Stacktrace

Using github.com/pkg/errors, you can add a formatted stacktrace to your errors.

package main

import (
	"github.com/pkg/errors"
	"github.com/rs/zerolog/pkgerrors"

	"github.com/rs/zerolog"
	"github.com/rs/zerolog/log"
)

func main() {
	zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
	zerolog.ErrorStackMarshaler = pkgerrors.MarshalStack

	err := outer()
	log.Error().Stack().Err(err).Msg("")
}

func inner() error {
	return errors.New("seems we have an error here")
}

func middle() error {
	err := inner()
	if err != nil {
		return err
	}
	return nil
}

func outer() error {
	err := middle()
	if err != nil {
		return err
	}
	return nil
}

// Output: {"level":"error","stack":[{"func":"inner","line":"20","source":"errors.go"},{"func":"middle","line":"24","source":"errors.go"},{"func":"outer","line":"32","source":"errors.go"},{"func":"main","line":"15","source":"errors.go"},{"func":"main","line":"204","source":"proc.go"},{"func":"goexit","line":"1374","source":"asm_amd64.s"}],"error":"seems we have an error here","time":1609086683}

zerolog.ErrorStackMarshaler must be set in order for the stack to output anything.

Logging Fatal Messages

package main

import (
    "errors"

    "github.com/rs/zerolog"
    "github.com/rs/zerolog/log"
)

func main() {
    err := errors.New("A repo man spends his life getting into tense situations")
    service := "myservice"

    zerolog.TimeFieldFormat = zerolog.TimeFormatUnix

    log.Fatal().
        Err(err).
        Str("service", service).
        Msgf("Cannot start %s", service)
}

// Output: {"time":1516133263,"level":"fatal","error":"A repo man spends his life getting into tense situations","service":"myservice","message":"Cannot start myservice"}
//         exit status 1

NOTE: Using Msgf generates one allocation even when the logger is disabled.

Create logger instance to manage different outputs

logger := zerolog.New(os.Stderr).With().Timestamp().Logger()

logger.Info().Str("foo", "bar").Msg("hello world")

// Output: {"level":"info","time":1494567715,"message":"hello world","foo":"bar"}

Sub-loggers let you chain loggers with additional context

sublogger := log.With().
                 Str("component", "foo").
                 Logger()
sublogger.Info().Msg("hello world")

// Output: {"level":"info","time":1494567715,"message":"hello world","component":"foo"}

Pretty logging

To log a human-friendly, colorized output, use zerolog.ConsoleWriter:

log.Logger = log.Output(zerolog.ConsoleWriter{Out: os.Stderr})

log.Info().Str("foo", "bar").Msg("Hello world")

// Output: 3:04PM INF Hello World foo=bar

To customize the configuration and formatting:

output := zerolog.ConsoleWriter{Out: os.Stdout, TimeFormat: time.RFC3339}
output.FormatLevel = func(i interface{}) string {
    return strings.ToUpper(fmt.Sprintf("| %-6s|", i))
}
output.FormatMessage = func(i interface{}) string {
    return fmt.Sprintf("***%s****", i)
}
output.FormatFieldName = func(i interface{}) string {
    return fmt.Sprintf("%s:", i)
}
output.FormatFieldValue = func(i interface{}) string {
    return strings.ToUpper(fmt.Sprintf("%s", i))
}

log := zerolog.New(output).With().Timestamp().Logger()

log.Info().Str("foo", "bar").Msg("Hello World")

// Output: 2006-01-02T15:04:05Z07:00 | INFO  | ***Hello World**** foo:BAR

Sub dictionary

log.Info().
    Str("foo", "bar").
    Dict("dict", zerolog.Dict().
        Str("bar", "baz").
        Int("n", 1),
    ).Msg("hello world")

// Output: {"level":"info","time":1494567715,"foo":"bar","dict":{"bar":"baz","n":1},"message":"hello world"}

Customize automatic field names

zerolog.TimestampFieldName = "t"
zerolog.LevelFieldName = "l"
zerolog.MessageFieldName = "m"

log.Info().Msg("hello world")

// Output: {"l":"info","t":1494567715,"m":"hello world"}

Add contextual fields to the global logger

log.Logger = log.With().Str("foo", "bar").Logger()

Add file and line number to log

log.Logger = log.With().Caller().Logger()
log.Info().Msg("hello world")

// Output: {"level": "info", "message": "hello world", "caller": "/go/src/your_project/some_file:21"}

Thread-safe, lock-free, non-blocking writer

If your writer might be slow or not thread-safe and you need your log producers to never get slowed down by a slow writer, you can use a diode.Writer as follow:

wr := diode.NewWriter(os.Stdout, 1000, 10*time.Millisecond, func(missed int) {
		fmt.Printf("Logger Dropped %d messages", missed)
	})
log := zerolog.New(wr)
log.Print("test")

You will need to install code.cloudfoundry.org/go-diodes to use this feature.

Log Sampling

sampled := log.Sample(&zerolog.BasicSampler{N: 10})
sampled.Info().Msg("will be logged every 10 messages")

// Output: {"time":1494567715,"level":"info","message":"will be logged every 10 messages"}

More advanced sampling:

// Will let 5 debug messages per period of 1 second.
// Over 5 debug message, 1 every 100 debug messages are logged.
// Other levels are not sampled.
sampled := log.Sample(zerolog.LevelSampler{
    DebugSampler: &zerolog.BurstSampler{
        Burst: 5,
        Period: 1*time.Second,
        NextSampler: &zerolog.BasicSampler{N: 100},
    },
})
sampled.Debug().Msg("hello world")

// Output: {"time":1494567715,"level":"debug","message":"hello world"}

Hooks

type SeverityHook struct{}

func (h SeverityHook) Run(e *zerolog.Event, level zerolog.Level, msg string) {
    if level != zerolog.NoLevel {
        e.Str("severity", level.String())
    }
}

hooked := log.Hook(SeverityHook{})
hooked.Warn().Msg("")

// Output: {"level":"warn","severity":"warn"}

Pass a sub-logger by context

ctx := log.With().Str("component", "module").Logger().WithContext(ctx)

log.Ctx(ctx).Info().Msg("hello world")

// Output: {"component":"module","level":"info","message":"hello world"}

Set as standard logger output

log := zerolog.New(os.Stdout).With().
    Str("foo", "bar").
    Logger()

stdlog.SetFlags(0)
stdlog.SetOutput(log)

stdlog.Print("hello world")

// Output: {"foo":"bar","message":"hello world"}

Integration with net/http

The github.com/rs/zerolog/hlog package provides some helpers to integrate zerolog with http.Handler.

In this example we use alice to install logger for better readability.

log := zerolog.New(os.Stdout).With().
    Timestamp().
    Str("role", "my-service").
    Str("host", host).
    Logger()

c := alice.New()

// Install the logger handler with default output on the console
c = c.Append(hlog.NewHandler(log))

// Install some provided extra handler to set some request's context fields.
// Thanks to that handler, all our logs will come with some prepopulated fields.
c = c.Append(hlog.AccessHandler(func(r *http.Request, status, size int, duration time.Duration) {
    hlog.FromRequest(r).Info().
        Str("method", r.Method).
        Stringer("url", r.URL).
        Int("status", status).
        Int("size", size).
        Dur("duration", duration).
        Msg("")
}))
c = c.Append(hlog.RemoteAddrHandler("ip"))
c = c.Append(hlog.UserAgentHandler("user_agent"))
c = c.Append(hlog.RefererHandler("referer"))
c = c.Append(hlog.RequestIDHandler("req_id", "Request-Id"))

// Here is your final handler
h := c.Then(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    // Get the logger from the request's context. You can safely assume it
    // will be always there: if the handler is removed, hlog.FromRequest
    // will return a no-op logger.
    hlog.FromRequest(r).Info().
        Str("user", "current user").
        Str("status", "ok").
        Msg("Something happened")

    // Output: {"level":"info","time":"2001-02-03T04:05:06Z","role":"my-service","host":"local-hostname","req_id":"b4g0l5t6tfid6dtrapu0","user":"current user","status":"ok","message":"Something happened"}
}))
http.Handle("/", h)

if err := http.ListenAndServe(":8080", nil); err != nil {
    log.Fatal().Err(err).Msg("Startup failed")
}

Multiple Log Output

zerolog.MultiLevelWriter may be used to send the log message to multiple outputs. In this example, we send the log message to both os.Stdout and the in-built ConsoleWriter.

func main() {
	consoleWriter := zerolog.ConsoleWriter{Out: os.Stdout}

	multi := zerolog.MultiLevelWriter(consoleWriter, os.Stdout)

	logger := zerolog.New(multi).With().Timestamp().Logger()

	logger.Info().Msg("Hello World!")
}

// Output (Line 1: Console; Line 2: Stdout)
// 12:36PM INF Hello World!
// {"level":"info","time":"2019-11-07T12:36:38+03:00","message":"Hello World!"}

Global Settings

Some settings can be changed and will by applied to all loggers:

  • log.Logger: You can set this value to customize the global logger (the one used by package level methods).
  • zerolog.SetGlobalLevel: Can raise the minimum level of all loggers. Call this with zerolog.Disabled to disable logging altogether (quiet mode).
  • zerolog.DisableSampling: If argument is true, all sampled loggers will stop sampling and issue 100% of their log events.
  • zerolog.TimestampFieldName: Can be set to customize Timestamp field name.
  • zerolog.LevelFieldName: Can be set to customize level field name.
  • zerolog.MessageFieldName: Can be set to customize message field name.
  • zerolog.ErrorFieldName: Can be set to customize Err field name.
  • zerolog.TimeFieldFormat: Can be set to customize Time field value formatting. If set with zerolog.TimeFormatUnix, zerolog.TimeFormatUnixMs or zerolog.TimeFormatUnixMicro, times are formated as UNIX timestamp.
  • zerolog.DurationFieldUnit: Can be set to customize the unit for time.Duration type fields added by Dur (default: time.Millisecond).
  • zerolog.DurationFieldInteger: If set to true, Dur fields are formatted as integers instead of floats (default: false).
  • zerolog.ErrorHandler: Called whenever zerolog fails to write an event on its output. If not set, an error is printed on the stderr. This handler must be thread safe and non-blocking.

Field Types

Standard Types

  • Str
  • Bool
  • Int, Int8, Int16, Int32, Int64
  • Uint, Uint8, Uint16, Uint32, Uint64
  • Float32, Float64

Advanced Fields

  • Err: Takes an error and renders it as a string using the zerolog.ErrorFieldName field name.
  • Timestamp: Inserts a timestamp field with zerolog.TimestampFieldName field name, formatted using zerolog.TimeFieldFormat.
  • Time: Adds a field with time formatted with zerolog.TimeFieldFormat.
  • Dur: Adds a field with time.Duration.
  • Dict: Adds a sub-key/value as a field of the event.
  • RawJSON: Adds a field with an already encoded JSON ([]byte)
  • Hex: Adds a field with value formatted as a hexadecimal string ([]byte)
  • Interface: Uses reflection to marshal the type.

Most fields are also available in the slice format (Strs for []string, Errs for []error etc.)

Binary Encoding

In addition to the default JSON encoding, zerolog can produce binary logs using CBOR encoding. The choice of encoding can be decided at compile time using the build tag binary_log as follows:

go build -tags binary_log .

To Decode binary encoded log files you can use any CBOR decoder. One has been tested to work with zerolog library is CSD.

Related Projects

  • grpc-zerolog: Implementation of grpclog.LoggerV2 interface using zerolog
  • overlog: Implementation of Mapped Diagnostic Context interface using zerolog

Benchmarks

See logbench for more comprehensive and up-to-date benchmarks.

All operations are allocation free (those numbers include JSON encoding):

BenchmarkLogEmpty-8        100000000    19.1 ns/op     0 B/op       0 allocs/op
BenchmarkDisabled-8        500000000    4.07 ns/op     0 B/op       0 allocs/op
BenchmarkInfo-8            30000000     42.5 ns/op     0 B/op       0 allocs/op
BenchmarkContextFields-8   30000000     44.9 ns/op     0 B/op       0 allocs/op
BenchmarkLogFields-8       10000000     184 ns/op      0 B/op       0 allocs/op

There are a few Go logging benchmarks and comparisons that include zerolog.

Using Uber's zap comparison benchmark:

Log a message and 10 fields:

Library Time Bytes Allocated Objects Allocated
zerolog 767 ns/op 552 B/op 6 allocs/op
zap 848 ns/op 704 B/op 2 allocs/op
zap (sugared) 1363 ns/op 1610 B/op 20 allocs/op
go-kit 3614 ns/op 2895 B/op 66 allocs/op
lion 5392 ns/op 5807 B/op 63 allocs/op
logrus 5661 ns/op 6092 B/op 78 allocs/op
apex/log 15332 ns/op 3832 B/op 65 allocs/op
log15 20657 ns/op 5632 B/op 93 allocs/op

Log a message with a logger that already has 10 fields of context:

Library Time Bytes Allocated Objects Allocated
zerolog 52 ns/op 0 B/op 0 allocs/op
zap 283 ns/op 0 B/op 0 allocs/op
zap (sugared) 337 ns/op 80 B/op 2 allocs/op
lion 2702 ns/op 4074 B/op 38 allocs/op
go-kit 3378 ns/op 3046 B/op 52 allocs/op
logrus 4309 ns/op 4564 B/op 63 allocs/op
apex/log 13456 ns/op 2898 B/op 51 allocs/op
log15 14179 ns/op 2642 B/op 44 allocs/op

Log a static string, without any context or printf-style templating:

Library Time Bytes Allocated Objects Allocated
zerolog 50 ns/op 0 B/op 0 allocs/op
zap 236 ns/op 0 B/op 0 allocs/op
standard library 453 ns/op 80 B/op 2 allocs/op
zap (sugared) 337 ns/op 80 B/op 2 allocs/op
go-kit 508 ns/op 656 B/op 13 allocs/op
lion 771 ns/op 1224 B/op 10 allocs/op
logrus 1244 ns/op 1505 B/op 27 allocs/op
apex/log 2751 ns/op 584 B/op 11 allocs/op
log15 5181 ns/op 1592 B/op 26 allocs/op

Caveats

Note that zerolog does no de-duplication of fields. Using the same key multiple times creates multiple keys in final JSON:

logger := zerolog.New(os.Stderr).With().Timestamp().Logger()
logger.Info().
       Timestamp().
       Msg("dup")
// Output: {"level":"info","time":1494567715,"time":1494567715,"message":"dup"}

In this case, many consumers will take the last value, but this is not guaranteed; check yours if in doubt.

Comments
  • Refactored zerolog.ConsoleWriter to allow customization

    Refactored zerolog.ConsoleWriter to allow customization

    As outlined in https://github.com/rs/zerolog/issues/84, this patch refactors the zerolog.ConsoleWriter in two ways:

    • The default formatting has been changed to be more sparse and visually attractive
    • The formatting logic has been extracted to separate functions, so it can be changed from the user code

    I had to change the design of the prototype in order to maintain the backwards compatibility: the prototype has defined default formatting functions (formatters) on the concrete instance of ConsoleWriter, but without using an initializer, these would be empty. Therefore, I've added a number of defensive checks into ConsoleWriter.Write(), and a ConsoleWriter.Reset() method. This is not ideal, but I agree that maintaining backwards compatibility is an important goal, and the behaviour is predictable in my testing.

    I've added a benchmark test to make sure I inadverently don't harm the performance: the performance has stayed roughly the same. The tests and examples have been run with the -race parameter.

    I've updated the screenshot to reflect the new look, and added a comprehensive example to the README.

    opened by karmi 26
  • Binary format support

    Binary format support

    Submitting the PR for #21 with two minor things in the todo items:

    1. Changes to documentation (README.md) Because after code review, will write the docs.

    2. When doing reflection, skip using JSON and write a binary based encoder Writing an equivalent of json.Marshal() would take more time and careful testing.

    opened by toravir 21
  • Teach zerolog how to return errors, with and without stack traces

    Teach zerolog how to return errors, with and without stack traces

    When zerolog logs an error message with an Error() finalizer, it returns a error. If the StackTrace() event has been called, the error will be a wrapped error using github.com/pkg/errors and will include the stack trace.

    There are a few subtle differences introduced with this change:

    1. It is possible to defeat this behavior using the Fields() method.
    2. Only one error can be handled by an Event at a time.

    Time was spent considering introducing an accumulator for errors that would be evaluated at finalizer-time, but this seemed unnecessary given errors should be wrapped and multiple calls to Err() seemed unlikely in practice.

    The behavior of the Msg() finalizer was left in tact in order to avoid any performance penalty in non-error handling conditions. The deferral of error handling can best be observed by looking at the performance hit of an Err() Event with and without StackTrace():

    BenchmarkErrorContextFields-8                  	20000000	        84.8 ns/op
    BenchmarkErrorContextFields_WithStackTrace-8   	  500000	      2552 ns/op
    BenchmarkLogContextFields-8                    	30000000	        47.6 ns/op
    PASS
    ok  	github.com/rs/zerolog	4.642s
    

    Fixes: #9

    opened by sean- 21
  • Context.Stack() not working

    Context.Stack() not working

    What happened: Using Context.Stack().Logger() does not work, the stack is not added to each log entries.

    What you expected to happen: The stack field should be added to each log entries

    How to reproduce it (as minimally and precisely as possible):

    package main
    
    import (
    	"github.com/pkg/errors"
    
    	"github.com/rs/zerolog"
    	"github.com/rs/zerolog/log"
    	"github.com/rs/zerolog/pkgerrors"
    )
    
    func myFunc() {
    
    	err := errors.New("my error")
    	err = errors.WithStack(errors.Wrap(err, "something failed"))
    
    	log.Error().Err(err).Msg("hello world")
    }
    
    func main() {
    	zerolog.ErrorStackMarshaler = pkgerrors.MarshalStack
    
    	log.Logger = log.With().Str("my", "field").Stack().Logger()
    	myFunc()
    }
    

    Anything else we need to know?:

    I think it comes from the fact that Context.Stack() setup an Hook which is called after the event is completely built, so Err never see that Stack() was enabled. We can see it because the following example works as expected:

    package main
    
    import (
    	"github.com/pkg/errors"
    
    	"github.com/rs/zerolog"
    	"github.com/rs/zerolog/log"
    	"github.com/rs/zerolog/pkgerrors"
    )
    
    func myFunc() {
    
    	err := errors.New("my error")
    	err = errors.WithStack(errors.Wrap(err, "something failed"))
    
    	log.Error().Stack().Err(err).Msg("hello world")
    }
    
    func main() {
    	zerolog.ErrorStackMarshaler = pkgerrors.MarshalStack
    
    	log.Logger = log.With().Str("my", "field").Logger()
    	myFunc()
    }
    

    I'm not sure how to fix this. One solution could be to add a stack field to Logger which would be transferred to the event in logger.newEvent.

    opened by skerkour 19
  • Add context hook support

    Add context hook support

    I've got an idea to retranslate logger messages to Opentracing. It needs to add some mechanics to be able to propagate the span context from context.Context to the logger.

    opened by ndk 18
  • Add support for stack trace extration of error fields

    Add support for stack trace extration of error fields

    Proposal for #11.

    @sean-: what do you think about this implementation?

    I'm not super happy of the API. I'm not sure having Stack and Err separated is great design. What would you think about a new set of error-with-stack methods that would package the error message and the stack together.

    For instance:

    log.Log().ErrStack("error", err).Msg("")
    

    would generate:

    {"error": {"message": "error message", "stack": [{...}, {...}]}}
    
    opened by rs 18
  • Updates to README.md to help with readability

    Updates to README.md to help with readability

    Hey - I've recently decided to switch to zerolog and wanted to try and help by adding some organization to the README.md file. I love the logger so far! As I'm teaching myself the library and learning the APIs, it seems like there's some room for improvement in the instructions for the layman (that would be me)... I have more edits I'd like to make as I go, but wanted to see if you considered this helpful before I spent more time on it! Thanks for the logger and your consideration!

    You can see the changes fully realized at my fork, which may be easier to read than all the code merge: https://github.com/gilcrest/zerolog

    Dan

    opened by gilcrest 14
  • Add TraceLevel

    Add TraceLevel

    Sometimes you want finer-grained informational events and need to debug something that outputs way too much data to log outside of a specific build when you're targeting that particular thing and do not care about errors or other logging info (since the volume of trace info will obscure them).

    I think this could encroach on loggers like journald and that's why I'm asking for your opinion on adding this new log level.

    opened by crazy-max 13
  • The ConsoleWriter doesn't work well  on windows10

    The ConsoleWriter doesn't work well on windows10

    The Code:

    image

    The messy output in Windows 10 (git bash & powershell & cmd): image

    And it‘s normal in Linux。 image

    It seems like the spaces are not displayed properly...

    help wanted 
    opened by Alecyrus 12
  • export buf

    export buf

    While using hook capability, in my use case, I had to access buf member in zerolog.Event struct, which is currently available only using reflection, which affects the performance.

    opened by evgenigourvitch 10
  • default output to stdout

    default output to stdout

    Hi,

    I spent an hour or so debugging zerolog, so let's see if i'm wrong or there is a bug :-)

    In the sample application code (without testing.T to keep it as simple as possible) below, the default output does not go to stderr, but stdout. When I debug, I do see initialization within zerolog to stderr, but the end result is not in stderr...

    package main
    
    import (
    	"bytes"
    	"fmt"
    	"io"
    	"os"
    	"strings"
    
    	"github.com/rs/zerolog/log"
    )
    
    
    func captureStderr(f func()) (string, error) {
    	var errOccurred error // in case an error occurred
    	old := os.Stderr      // keep backup of the real stderr
    	r, w, err := os.Pipe()
    	if err != nil {
    		return "", err
    	}
    	os.Stderr = w
    
    	outC := make(chan string)
    	// copy the output in a separate goroutine so printing can't block indefinitely
    	go func() {
    		var buf bytes.Buffer
    		_, cpErr := io.Copy(&buf, r)
    		if cpErr != nil {
    			errOccurred = cpErr
    		}
    		outC <- buf.String()
    	}()
    
    	// calling function which stderr we are going to capture:
    	f()
    
    	// back to normal state
    	err = w.Close()
    	if err != nil {
    		_, err = old.Write([]byte("Could not close file, error: " + err.Error()))
    		if err != nil {
    			errOccurred = err
    		}
    	}
    	os.Stderr = old // restoring the real stderr
    	if errOccurred != nil {
    		fmt.Printf("CaptureStderr, error occurred: %v\n", err)
    	}
    	return <-outC, nil
    }
    
    
    func main() {
    	stdErr, err := captureStderr(func() {
    		log.Error().Msg("sw")
    	})
    	if err != nil {
    		fmt.Println("error: " + err.Error())
    	}
    	if !strings.Contains(stdErr, "sw") {
    		fmt.Println("stderr does not contain, output: " + stdErr)
    	} else {
    		fmt.Println("ok, stderr contains: " + stdErr)
    	}
    }
    
    module "awesomeProject"
    
    go 1.14
    
    require (
        github.com/rs/zerolog v1.19.0
    )
    
    opened by stephanwesten 9
  • Double quotes are escaped (while using ConsoleWriter)

    Double quotes are escaped (while using ConsoleWriter)

    I'm new to using zerolog. I'm using ConsoleWriter, as it's indented to be viewable to the end-user, while running the program. However, double quotes seem to be escaped.

    	logger.Fatal(). // Fatal because this is the last attempt at a ping
    		Err(err).
    		Str("help", "Privileged pings are disabled. To enable, run \"sudo sysctl -w net.ipv4.ping_group_range=\"0 2147483647\"\" For more information, check https://github.com/prometheus-community/pro-bing#linux").
    		Msg("Privileged ping failed")
    

    Outputs the followig: 2022-11-05T03:23:29-05:00 FTL Privileged ping failed error="socket: permission denied" help="Privileged pings are disabled. To enable, run \"sudo sysctl -w net.ipv4.ping_group_range=\"0 2147483647\"\" For more information, check https://github.com/prometheus-community/pro-bing#linux"

    My logger configuration is logger = zerolog.New(zerolog.ConsoleWriter{Out: os.Stderr, TimeFormat: time.RFC3339}).Level(zerolog.DebugLevel).With().Timestamp().Logger()

    opened by slashtechno 0
  • Event.ctx to support OpenTelemetry

    Event.ctx to support OpenTelemetry

    opened by wcarmon 0
  • UpdateContext data race

    UpdateContext data race

    Hi,

    I am getting data race for this implementation. All I need is to generate and add a request id to the logger so that each log entry has it by default. How do I solve this issue please? I am coming from logrus where I can do the same thing without a problem but if I cannot solve this issue I will have to go back which is what I prefer to avoid for different reasons.

    Thanks

    type requestID string
    
    const requestIDKey = requestID("request_id")
    
    func RequestID(next http.Handler) http.Handler {
    	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    		id := xid.New().String()
    
    		log.Logger.UpdateContext(func(c zerolog.Context) zerolog.Context {
    		 	return c.Str(string(requestIDKey), id)
    		})
    
    		w.Header().Set("X-Request-ID", id)
    
    		next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), requestIDKey, id)))
    	})
    }
    
    ==================
    WARNING: DATA RACE
    Write at 0x00c0004d5900 by goroutine 603:
      github.com/rs/zerolog/internal/json.Encoder.AppendString()
          /rs/[email protected]/internal/json/string.go:46 +0xa8
      github.com/rs/zerolog/internal/json.Encoder.AppendKey()
          /rs/[email protected]/internal/json/base.go:18 +0x153
      github.com/rs/zerolog.Context.Str()
          /rs/[email protected]/context.go:77 +0x73
      /blog/pkg/middleware.RequestID.func1.1()
          /blog/pkg/middleware/request_id.go:24 +0xb3
      github.com/rs/zerolog.(*Logger).UpdateContext()
          /rs/[email protected]/log.go:288 +0x29d
      /blog/pkg/middleware.RequestID.func1()
          /blog/pkg/middleware/request_id.go:23 +0x184
      net/http.HandlerFunc.ServeHTTP()
          /usr/local/go/src/net/http/server.go:2109 +0x4d
      /blog/pkg/middleware.PanicRecoverer.func1()
          /blog/pkg/middleware/panic_recoverer.go:29 +0xeb
      net/http.HandlerFunc.ServeHTTP()
          /usr/local/go/src/net/http/server.go:2109 +0x4d
      net/http.serverHandler.ServeHTTP()
          /usr/local/go/src/net/http/server.go:2947 +0x641
      net/http.(*conn).serve()
          /usr/local/go/src/net/http/server.go:1991 +0xbe4
      net/http.(*Server).Serve.func3()
          /usr/local/go/src/net/http/server.go:3102 +0x58
    
    Previous write at 0x00c0004d5900 by goroutine 604:
      github.com/rs/zerolog/internal/json.Encoder.AppendString()
          /rs/[email protected]/internal/json/string.go:46 +0xa8
      github.com/rs/zerolog/internal/json.Encoder.AppendKey()
          /rs/[email protected]/internal/json/base.go:18 +0x153
      github.com/rs/zerolog.Context.Str()
          /rs/[email protected]/context.go:77 +0x73
      /blog/pkg/middleware.RequestID.func1.1()
          /blog/pkg/middleware/request_id.go:24 +0xb3
      github.com/rs/zerolog.(*Logger).UpdateContext()
          /rs/[email protected]/log.go:288 +0x29d
      /blog/pkg/middleware.RequestID.func1()
          /blog/pkg/middleware/request_id.go:23 +0x184
      net/http.HandlerFunc.ServeHTTP()
          /usr/local/go/src/net/http/server.go:2109 +0x4d
      /blog/pkg/middleware.PanicRecoverer.func1()
          /blog/pkg/middleware/panic_recoverer.go:29 +0xeb
      net/http.HandlerFunc.ServeHTTP()
          /usr/local/go/src/net/http/server.go:2109 +0x4d
      net/http.serverHandler.ServeHTTP()
          /usr/local/go/src/net/http/server.go:2947 +0x641
      net/http.(*conn).serve()
          /usr/local/go/src/net/http/server.go:1991 +0xbe4
      net/http.(*Server).Serve.func3()
          /usr/local/go/src/net/http/server.go:3102 +0x58
    
    Goroutine 603 (running) created at:
      net/http.(*Server).Serve()
          /usr/local/go/src/net/http/server.go:3102 +0x837
      net/http.(*Server).ListenAndServe()
          /usr/local/go/src/net/http/server.go:2999 +0xc4
      /blog/pkg/server.HTTP.Start()
          /blog/pkg/server/http.go:40 +0x34
      main.main.func1()
          /blog/cmd/api/main.go:53 +0x3f
    
    Goroutine 604 (running) created at:
      net/http.(*Server).Serve()
          /usr/local/go/src/net/http/server.go:3102 +0x837
      net/http.(*Server).ListenAndServe()
          /usr/local/go/src/net/http/server.go:2999 +0xc4
      /blog/pkg/server.HTTP.Start()
          /blog/pkg/server/http.go:40 +0x34
      main.main.func1()
          /blog/cmd/api/main.go:53 +0x3f
    ==================
    
    opened by bentcoder 3
  • Bump actions/cache from 3.0.5 to 3.0.11

    Bump actions/cache from 3.0.5 to 3.0.11

    Bumps actions/cache from 3.0.5 to 3.0.11.

    Release notes

    Sourced from actions/cache's releases.

    v3.0.11

    What's Changed

    New Contributors

    Full Changelog: https://github.com/actions/cache/compare/v3...v3.0.11

    v3.0.10

    • Fix a bug with sorting inputs.
    • Update definition for restore-keys in README.md

    v3.0.9

    • Enhanced the warning message for cache unavailability in case of GHES.

    v3.0.8

    What's Changed

    • Fix zstd not working for windows on gnu tar in issues.
    • Allow users to provide a custom timeout as input for aborting cache segment download using the environment variable SEGMENT_DOWNLOAD_TIMEOUT_MIN. Default is 60 minutes.

    v3.0.7

    What's Changed

    • Fix for the download stuck problem has been added in actions/cache for users who were intermittently facing the issue. As part of this fix, new timeout has been introduced in the download step to stop the download if it doesn't complete within an hour and run the rest of the workflow without erroring out.

    v3.0.6

    What's Changed

    • Add example for clojure lein project dependencies by @​shivamarora1 in PR actions/cache#835
    • Update toolkit's cache npm module to latest. Bump cache version to v3.0.6 by @​pdotl in PR actions/cache#887
    • Fix issue #809 where cache save/restore was failing for Amazon Linux 2 runners due to older tar version
    • Fix issue #833 where cache save was not working for caching github workspace directory

    New Contributors

    Full Changelog: https://github.com/actions/cache/compare/v3...v3.0.6

    Changelog

    Sourced from actions/cache's changelog.

    3.0.5

    • Removed error handling by consuming actions/cache 3.0 toolkit, Now cache server error handling will be done by toolkit. (PR)

    3.0.6

    • Fixed #809 - zstd -d: no such file or directory error
    • Fixed #833 - cache doesn't work with github workspace directory

    3.0.7

    • Fixed #810 - download stuck issue. A new timeout is introduced in the download process to abort the download if it gets stuck and doesn't finish within an hour.

    3.0.8

    • Fix zstd not working for windows on gnu tar in issues #888 and #891.
    • Allowing users to provide a custom timeout as input for aborting download of a cache segment using an environment variable SEGMENT_DOWNLOAD_TIMEOUT_MIN. Default is 60 minutes.

    3.0.9

    • Enhanced the warning message for cache unavailablity in case of GHES.

    3.0.10

    • Fix a bug with sorting inputs.
    • Update definition for restore-keys in README.md

    3.0.11

    • Update toolkit version to 3.0.5 to include @actions/[email protected]^1.10.0
    • Update @actions/cache to use updated saveState and setOutput functions from @actions/[email protected]^1.10.0
    Commits
    • 9b0c1fc Merge pull request #956 from actions/pdotl-version-bump
    • 18103f6 Fix licensed status error
    • 3e383cd Update RELEASES
    • 43428ea toolkit versioon update and version bump for cache
    • 1c73980 3.0.11
    • a3f5edc Merge pull request #950 from rentziass/rentziass/update-actions-core
    • 831ee69 Update licenses
    • b9c8bfe Update @​actions/core to 1.10.0
    • 0f20846 Merge pull request #946 from actions/Phantsure-patch-2
    • 862fc14 Update README.md
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies github_actions 
    opened by dependabot[bot] 0
  • Console writer does not use defined timestamp formatter

    Console writer does not use defined timestamp formatter

    The console writer is not using the timestamp formatter for fields:

    log.Ctx(ctx).Warn().Time("expiredAt", time.Now()).Msgf("something expired")
    

    Outputs:

    2022-10-13T10:34:16+02:00 WRN something expired expiredAt=1665650056
    

    Imo, expected would be:

    2022-10-13T10:34:16+02:00 WRN something expired expiredAt=2022-10-13T10:34:16+02:00
    
    opened by gucki 0
Releases(v1.0.0)
Owner
Olivier Poitrey
Director of Engineering at Netflix Co-Founder & ex-CTO of Dailymotion Co-Founder of NextDNS
Olivier Poitrey
Get JSON values quickly - JSON parser for Go

get json values quickly GJSON is a Go package that provides a fast and simple way to get values from a json document. It has features such as one line

Josh Baker 11.4k Nov 27, 2022
JSON diff library for Go based on RFC6902 (JSON Patch)

jsondiff jsondiff is a Go package for computing the diff between two JSON documents as a series of RFC6902 (JSON Patch) operations, which is particula

William Poussier 215 Dec 1, 2022
Fast JSON encoder/decoder compatible with encoding/json for Go

Fast JSON encoder/decoder compatible with encoding/json for Go

Masaaki Goshima 1.9k Nov 23, 2022
Package json implements encoding and decoding of JSON as defined in RFC 7159

Package json implements encoding and decoding of JSON as defined in RFC 7159. The mapping between JSON and Go values is described in the documentation for the Marshal and Unmarshal functions

High Performance, Kubernetes Native Object Storage 2 Jun 26, 2022
Json-go - CLI to convert JSON to go and vice versa

Json To Go Struct CLI Install Go version 1.17 go install github.com/samit22/js

Samit Ghimire 7 Jul 29, 2022
JSON Spanner - A Go package that provides a fast and simple way to filter or transform a json document

JSON SPANNER JSON Spanner is a Go package that provides a fast and simple way to

null 6 Sep 14, 2022
Abstract JSON for golang with JSONPath support

Abstract JSON Abstract JSON is a small golang package provides a parser for JSON with support of JSONPath, in case when you are not sure in its struct

Stepan Pyzhov 137 Nov 22, 2022
Fast JSON parser and validator for Go. No custom structs, no code generation, no reflection

fastjson - fast JSON parser and validator for Go Features Fast. As usual, up to 15x faster than the standard encoding/json. See benchmarks. Parses arb

Aliaksandr Valialkin 1.7k Nov 23, 2022
Small utility to create JSON objects

gjo Small utility to create JSON objects. This was inspired by jpmens/jo. Support OS Mac Linux Windows Requirements Go 1.1.14~ Git Installtion Build $

skanehira 114 Oct 24, 2022
A Go package for handling common HTTP JSON responses.

go-respond A Go package for handling common HTTP JSON responses. Installation go get github.com/nicklaw5/go-respond Usage The goal of go-respond is to

Nick Law 50 Sep 26, 2022
JSON query in Golang

gojq JSON query in Golang. Install go get -u github.com/elgs/gojq This library serves three purposes: makes parsing JSON configuration file much easie

Qian Chen 184 Sep 26, 2022
Automatically generate Go (golang) struct definitions from example JSON

gojson gojson generates go struct definitions from json or yaml documents. Example $ curl -s https://api.github.com/repos/chimeracoder/gojson | gojson

Aditya Mukerjee 2.6k Nov 26, 2022
A JSON diff utility

JayDiff A JSON diff utility. Install Downloading the compiled binary Download the latest version of the binary: releases extract the archive and place

Guillaume de Sagazan 93 Oct 24, 2022
Fast and flexible JSON encoder for Go

Jettison Jettison is a fast and flexible JSON encoder for the Go programming language, inspired by bet365/jingo, with a richer features set, aiming at

William Poussier 138 Nov 20, 2022
Create go type representation from json

json2go Package json2go provides utilities for creating go type representation from json inputs. Json2go can be used in various ways: CLI tool Web pag

Maciek Zając 110 Nov 6, 2022
Console JSON formatter with query feature

Console JSON formatter with query feature. Install: $ go get github.com/miolini/jsonf Usage: Usage of jsonf: -c=true: colorize output -d=false: de

Artem Andreenko 64 Sep 26, 2022
Fluent API to make it easier to create Json objects.

Jsongo Fluent API to make it easier to create Json objects. Install go get github.com/ricardolonga/jsongo Usage To create this: { "name":"Ricar

Ricardo Longa 107 Nov 7, 2022
Arbitrary transformations of JSON in Golang

kazaam Description Kazaam was created with the goal of supporting easy and fast transformations of JSON data with Golang. This functionality provides

Qntfy 241 Nov 16, 2022
Parsing JSON is a hassle in golang

GoJSON Parsing JSON is a hassle in golang. This package will allow you to parse and search elements in a json without structs. Install gojson go get g

swaraj18 25 Nov 12, 2021