Simple profiling for Go

Related tags

Performance profile
Overview

profile

Simple profiling support package for Go

Build Status GoDoc

installation

go get github.com/pkg/profile

usage

Enabling profiling in your application is as simple as one line at the top of your main function

import "github.com/pkg/profile"

func main() {
    defer profile.Start().Stop()
    ...
}

options

What to profile is controlled by config value passed to profile.Start. By default CPU profiling is enabled.

import "github.com/pkg/profile"

func main() {
    // p.Stop() must be called before the program exits to
    // ensure profiling information is written to disk.
    p := profile.Start(profile.MemProfile, profile.ProfilePath("."), profile.NoShutdownHook)
    ...
    // You can enable different kinds of memory profiling, either Heap or Allocs where Heap
    // profiling is the default with profile.MemProfile.
    p := profile.Start(profile.MemProfileAllocs, profile.ProfilePath("."), profile.NoShutdownHook)
}

Several convenience package level values are provided for cpu, memory, and block (contention) profiling.

For more complex options, consult the documentation.

contributing

We welcome pull requests, bug fixes and issue reports.

Before proposing a change, please discuss it first by raising an issue.

Issues
  • Proposal

    Proposal

    Because you liked my first suggestion, I took the liberty of suggesting some more changes to simplify readability. Hope they are welcome and I'd love to hear what you think of them. I also hope I didn't break anything :100:

    opened by gbbr 14
  • Multi-modal profiling

    Multi-modal profiling

    Problem Statement

    From my viewpoint, the biggest thing stopping me from using github.com/pkg/profile is its hard stance on "no more than one kind of profiling at a time". Especially since my usual starting assumption is "show me both a CPU profile and a memory (allocs) profile".

    Taking a CPU + memory profile, or a CPU + memory + goroutines + mutexes profile is actually quite a sane thing to do in my viewpoint. Further I do sometimes want to get both a trace and a cpu profile from the same long-running program, rather than go to all the trouble of running it twice.

    So with that introduction, let me lay out my understanding of why this is an okay thing to do (conversely, why the restriction on "one at a time" is actually necessary and overly restrictive).

    Kinds of Profiling

    For the sake of my argument I'm going to focus primarily on the distinction between active and passive profiles.

    There are two kinds of active profiles: CPU profiling and execution tracing. Here the cost of instrumentation is so high that they only run for a set amount of time (say take a 10-second CPU profile from a server, or re-running a tool with trace profiling turned on for the duration).

    On the other hand there are passive profiles including allocs, block, goroutine, heap, mutex, and threadcreate. Here the instrumentation cost is low enough that these profiles are latently always on; their counts are collected all the time, and can be collected at any point in time.

    NOTE this model is of course complicated by the fact that there is a tuning knob for the passive/always-on memory profiler, which pkg/profile is fairly unique in making such a user-facing API-out of (to say nothing of the basically unusable CPU profiler hz tuning knob...)

    Common Combinations

    Of all those lovely modes of profiling CPU, memory, and tracing are the most generally known, used, and useful; as evidenced by the prevalent flag idioms -cpuprofile FILE, -memprofile FILE, and -trace FILE.

    There's really no problem in combining CPU profiling and memory profiling (at normal rates); to the contrary: since the memory profiler is always on any how, you may as well dump it at the end (in the context of an offline tool, which seems to be the use-case that pkg/profile is most suited for). The same goes for any/all of the other passive profilers: they already have their counts just sitting there, you're only doing yourself a harm by not dumping the data.

    The most concern comes when combining CPU profiling and tracing. But even there, at least in my experience, any cross-chatter is fairly easy to pick out or compensate for:

    • in the event trace, you can at least see that a CPU profile was going on, since there will be at least one dedicated goroutine writing it out over some time span (you should even be able to see the os-interrupt timing... but I've not actually tried to do that, and now I'm purely speculating in parenthetical...)
    • there is real concern when it comes to skewing the CPU profile however, since every trace instrumentation branch is now hot, further inflating (the probably already dominant) impact of runtime functions
    • in practice, for online servers, I always sequence tracing and CPU profiling for this reason, rather than do them in parallel; however for an offline tool where your profiling its entire lifecycle, there are times when you want to see both concurrently (even if you also re-run it to also get a "pure" trace and cpu profile)

    Why I Care

    In my view pkg/profile is very close to fully solving the use case of "lifetime profiling for an offline tool / batch task". I'd like to wrap some flag.Value implementation around it and use it as a dependency (maybe even send a pull request for adding the flag convenience).

    However in its current form, not being able to take several at once is a bit of a blocker for that use case.

    opened by jcorbin 11
  • parsing profile: unrecognized profile format under MacOs Sierra 10.12.5

    parsing profile: unrecognized profile format under MacOs Sierra 10.12.5

    Hi, I'm trying to profile my application using your package following the documentation. I have a cpu.pprof non empty file generated but pprof is unable to read it

    $ go tool pprof —pdf ./app /var/folders/x7/wnp2zjn563j2dr45dsfpvdz80000gn/T/profile464041113/cpu.pprof > out.pdf              [email protected]
    parsing profile: unrecognized profile format
    

    I've tried multiple configuration from the default one defer profile.Start().Stop() to a more elaborated defer profile.Start(profile.BlockProfile, profile.MemProfile, profile.CPUProfile).Stop() but pprof is still unable to parse it. Any ideas?

    FYI

    $ go version                                                                                                                  
    go version go1.8.3 darwin/amd64
    

    Thanks

    opened by nsitbon 11
  • Empty CPU Profile

    Empty CPU Profile

    I'm trying to profile a simple application:

    package main
    
    import (
        "github.com/pkg/profile"
        "github.com/spf13/cobra"
        "strings"
    )
    
    /*
    Tokenize TODO
    */
    func Tokenize(text string) []string {
        return strings.Split(text, "\n")
    }
    
    func main() {
        profileCommand := &cobra.Command{
            Use: "profile",
            Run: func(cmd *cobra.Command, args []string) {
                defer profile.Start(profile.CPUProfile, profile.ProfilePath(".")).Stop()
    
                Tokenize("Hello\nWorld")
            },
        }
    
        rootCommand := &cobra.Command{Use: "bern"}
        rootCommand.AddCommand(profileCommand)
        rootCommand.Execute()
    }
    

    But when I look at the cpu.pprof file it simply contains ' and nothing else. What am I doing incorrectly?

    opened by ianwalter 8
  • 10 byte profile file can't be opened

    10 byte profile file can't be opened

    After profiling I get a 10 byte file (cpu.pprof.zip) that when read produces:

    • cpu.pprof: decompressing profile: unexpected EOF ( With go tool pprof cpu.pprof)
    • Unexpected end of ZLIB input stream (With Goland)

    Tried both:

    defer profile.Start(profile.ProfilePath(os.Getenv("HOME"))).Stop()
    

    as well as:

    p := profile.Start(profile.ProfilePath(os.Getenv("HOME")), profile.NoShutdownHook)
    // ...
    p.Stop()
    

    As of 3704c8d23324a3fc0771ad2c1cef0aa4e9459f6f on macOS Big Sur 11.1.

    With the shutdown hook I saw the library was intercepting shutdown to stop profiling. Without it I see my code called to .Stop().

    Troubleshooting ideas?

    Thanks for your work.

    opened by thnk2wn 7
  • profile.Start() should only be called once

    profile.Start() should only be called once

    Now that we've made it impossible to activate more than one profile at a time, users may attempt to work around this with something like

     defer profile.Start(profile.CPUProfile).Stop
     defer profile.Start(profile.MemProfile).Stop
    

    We should decide to support this, and if so, make sure that we move anything which is shared across profiles into the *Profile, see #5. Or, we should prohibit it with some package global lock.

    opened by davecheney 7
  • Switched to switch

    Switched to switch

    I was looking over this with the hope to learn something new and I found it was a great example of using the functional options you spoke about during your talk!

    I thought I'd make this change because I personally find it more readable to use a switch when there are multiple if's, it kind of prepares the reader for a lot of conditions. The fact that Go also allows us to have a "no parameter" switch gives us the opportunity to do such things.

    This is obviously just a personal opinion and you don't have to merge it if you don't agree.

    opened by gbbr 6
  • TestProfile fails

    TestProfile fails

    I'm trying to create a Debian package for profile. The test TestProfile fails during the build of the packages:

    go test -v -p 4 github.com/pkg/profile
    === RUN   TestProfile
    --- FAIL: TestProfile (1.75s)
    profile_test.go:174: default profile (cpu)
    profile_test.go:174: memory profile
    profile_test.go:174: memory profile (rate 2048)
    profile_test.go:174: double start
    profile_test.go:174: block profile
    profile_test.go:174: profile path
    profile_test.go:174: profile path error
    profile_test.go:194: stderr: wanted '[could not create initial output]', got '2017/05/09 06:50:47 profile: cpu profiling enabled, README.md/cpu.pprof
    	2017/05/09 06:50:47 profile: cpu profiling disabled, README.md/cpu.pprof
    	'
    profile_test.go:209: expected error
    profile_test.go:174: multiple profile sessions
    profile_test.go:174: profile quiet
    
    opened by legrostdg 5
  • 0 Byte files when attempting to generate CPU and Memory profiles

    0 Byte files when attempting to generate CPU and Memory profiles

    I have been trying to collect CPU and Memory profiles for a program I wrote. I have imported "github.com/pkg/profile" and added the appropriate line for either CPU or Memory profile.

    import (
        ...
        "github.com/pkg/profile"
        ...
    )
    
    ...
    
    func main() {
        // memory profile
        defer profile.Start(profile.MemProfile, profile.ProfilePath("./profiles/")).Stop()
        // OR cpu profile
        defer profile.Start(profile.CPUProfile, profile.ProfilePath("./profiles/")).Stop()
    }
    

    When I run the program, I see messages like the following on the command line:

    2019/10/11 12:33:24 profile: memory profiling enabled (rate 4096), profiles/mem.pprof
    

    However, the cpu.pprof or mem.pprof files are always 0 bytes. This used to work before for the same program, I do not understand why this no longer works.

    ls -alhrt profiles
    total 8.0K
    -rw-rw-r-- 1 manas manas    0 Oct 11 12:33 mem.pprof
    drwxrwxr-x 9 manas manas 4.0K Oct 11 12:33 ..
    drwxrwxr-x 2 manas manas 4.0K Oct 11 12:33 .
    
    opened by manasbhatnagar 4
  • How to profile loading of libraries ?

    How to profile loading of libraries ?

    Fantastic tool.

    One thing, say I have a binary that is loading slowly from a massive amount of imports that may possibly be having side effects upon code being loaded.

    I don't see how to use profile here because I put defer profile.Start().Stop() at the beginning of main but that doesn't capture the loading/code execution of library code.

    thank you

    opened by fxfactorial 4
  • Where do I go to learn what the output means?

    Where do I go to learn what the output means?

    I'm writing a web app and using /wg/wrk to stress test it along with this tool but don't understand the output. Here's the CPU profile output after running wrk -t12 -c400 -d7s http://127.0.0.1:8080

    question 
    opened by xDinomode 4
  • Move the CI to Github Actions

    Move the CI to Github Actions

    Travis-ci.org is ceased. I just did the migration to GH actions for two of my repos so when I saw this I thought if you also need to move this can help and took me just few minutes.

    Ignore if you have other plans.

    opened by kavehmz 0
  • Doesn't releases the file even after calling profile.Stop()

    Doesn't releases the file even after calling profile.Stop()

    Failed to remove the files created by the profile pkg "error":"remove mem.pprof: The process cannot access the file because it is being used by another process."

    opened by prakashettigar 1
  • Named interface for Stop()

    Named interface for Stop()

    Problem

    Currently the signature of the Start function is

    func Start(options ...func(*Profile)) interface {
    	Stop()
    } 
    

    Use case

    The lifecycle in our case is managed by another library (fluent-bit-go)

    • The Start() has to happen in lifecycle function FLBPluginInit
    • The Stop() has to happen in lifecycle function FLBPluginExit

    Proposal

    have a named interface for Stop

    type Stopper interface {
    	Stop()
    } 
    

    So the Start will become

    func Start(options ...func(*Profile)) Stopper
    
    opened by kishaningithub 10
  • Timing profiling?

    Timing profiling?

    Hi. I hope everyone is having a great new year so far.

    I was told by a colleague I could use this package for determining how much time is being spent in each function during an application's run. Is this correct and, if so, how? I saw nothing in the documentation which makes this clear if so.

    Thanks in advance.

    opened by FrankDMartinez 7
  • added power support arch ppc64le on yml file.

    added power support arch ppc64le on yml file.

    Hi Team, Added power support for the travis.yml file with ppc64le. This is part of the Ubuntu distribution for ppc64le. This helps us simplify testing later when distributions are re-building and re-releasing. For more info tag @gerrith3. continuous integration build has passed after adding power support arch ppc64le without any failures. please check and close this component.

    opened by srinivas32 0
  • Add ProfileFilename

    Add ProfileFilename

    Every time I use this package, I think that I can do something like:

    var flagCPUProfile = flag.String("cpuprofile", "", "write a cpuprofile to `cpuprofile`")
    
    func main() {
    	if *flagCPUProfile != "" {
    		defer profile.Start(profile.ProfilePath(*flagCPUProfile)).Stop()
    	}
    

    And then I run the program with -cpuprofile=somename.pprof. And there is no profile to be found.

    And then I dig through the docs and discover ProfilePath is supposed to be a directory. And then I wish there was a way to provide a filename. This is because I sometimes do multiple runs, and I want to write the results to different profiles so that I can combine them, and I don't want to have to deal with creating and cleaning up a directory per profile.

    May I send a PR to add ProfileFilename, or something like it?

    opened by josharian 2
Releases(v1.6.0)
  • v1.5.0(May 20, 2020)

  • v1.4.0(Nov 21, 2019)

  • v1.2.1(May 10, 2017)

    New features

    • Version 1.2.1 adds support for the Go mutex profile. Thanks @valyala

    Bug fixes

    • Version 1.2.1 also fixes a bug executing tests in some environments where README.md had been removed. Fixes #40
    • Missing Stop() call in ExampleProfilePath was fixed. Thanks @mark-rushakoff
    Source code(tar.gz)
    Source code(zip)
  • v1.2.0(Aug 1, 2016)

    Version 1.2.0 adds support for the Go execution tracer. This feature was added to Go 1.5 but hasn't really been usable til Go 1.7. Support for profile.Start(profile.TraceProfile) is only available in Go 1.7 and later.

    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Jun 1, 2016)

    Version 1.1.0 adds explicit support for stopping and starting a profile multiple times. This was previously support, but untested, and contained a race condition. The race has been fixed and profiling multiple times during the execution of a program is now supported. Thanks to @valyala for adding this feature.

    Fixes #18, #19

    Source code(tar.gz)
    Source code(zip)
Owner
Artisanal, hand crafted, barrel aged, Go packages
null
Simple profiling for Go

profile Simple profiling support package for Go installation go get github.com/pkg/profile usage Enabling profiling in your application is as simple

null 1.7k Jun 27, 2022
Simple profiling for Go

profile Simple profiling support package for Go installation go get github.com/pkg/profile usage Enabling profiling in your application is as simple

null 1.7k Jun 20, 2022
Simple profiling for Go

profile Simple profiling for Go. Easy management of Go's built-in profiling and tracing Based on the widely-used pkg/profile: mostly-compatible API Su

Michael McLoughlin 70 Apr 27, 2022
🔥 Continuous profiling platform — debug performance issues in your code!

Pyroscope is an open source continuous profiling platform.

Pyroscope 6k Jun 25, 2022
A GNU/Linux monitoring and profiling tool focused on single processes.

Uroboros is a GNU/Linux monitoring tool focused on single processes. While utilities like top, ps and htop provide great overall details, they often l

Simone Margaritelli 649 Jun 30, 2022
pprof is a tool for visualization and analysis of profiling data

Introduction pprof is a tool for visualization and analysis of profiling data. pprof reads a collection of profiling samples in profile.proto format a

Google 5.5k Jun 29, 2022
perfessor - Continuous Profiling Sidecar

perfessor - Continuous Profiling Sidecar About Perfessor is a continuous profiling agent that can profile running programs using perf It then converts

null 52 Feb 19, 2022
Continuous profiling for long-term postmortem analysis

profefe, a continuous profiling system, collects profiling data from a fleet of running applications and provides API for querying profiling samples for postmortem performance analysis.

profefe - continuous profiling 531 Jun 30, 2022
A tool suite for Redis profiling

Insecticide Insecticide is a tool suite for Redis profiling. It finds ambiguous values in your redis configuration.

Сити-Мобил 8 Dec 13, 2021
conprof - Continuous Profiling

conprof - Continuous Profiling Conprof is a continuous profiling project. Continuous profiling is the act of taking profiles of programs in a systemat

Parca 4 Feb 10, 2022
Continuous profiling for analysis of CPU, memory usage over time, and down to the line number. Saving infrastructure cost, improving performance, and increasing reliability.

Continuous profiling for analysis of CPU, memory usage over time, and down to the line number. Saving infrastructure cost, improving performance, and increasing reliability.

Parca 2.3k Jun 24, 2022
A profiling tool to peek and profile the memory or cpu usage of a process

Peekprof Get the CPU and Memory usage of a single process, monitor it live, and extract it in CSV and HTML. Get the best out of your optimizations. Us

Apostolis A. 5 Oct 10, 2021
Continuous profiling of golang program based on pprof

基于 pprof 的 Golang 程序连续分析 Demo 点击 point Quick Start 需要被收集分析的golang程序,需要提供net/http/pprof端点,并配置在collector.yaml配置文件中 #run server :8080 go run ser

xyctruth 106 Jun 23, 2022
Go-xrayprofile - Selective profiling of AWS Lambda functions

go-xrayprofile AWS X-Ray is handy for understanding the overall performance of y

Aidan Steele 7 May 18, 2022
simple-jwt-provider - Simple and lightweight provider which exhibits JWTs, supports login, password-reset (via mail) and user management.

Simple and lightweight JWT-Provider written in go (golang). It exhibits JWT for the in postgres persisted user, which can be managed via api. Also, a password-reset flow via mail verification is available. User specific custom-claims also available for jwt-generation and mail rendering.

Max 25 May 18, 2022
Simple Bank is a simple REST API that allows users to perform transferences with each other.

Simple Bank is a simple REST API that allows users to perform transferences with each other. ?? Technologies Golang Docker PostgreSQ

Matheus Mosca 12 Feb 15, 2022
bf.go - A dead simple brainfuck interpreter Slow and simple

bf.go - A dead simple brainfuck interpreter Slow and simple. Can execute pretty much all tested Brainfuck scripts. Installation If you have Go install

Chris 0 Oct 15, 2021
A simple project (which is visitor counter) on kubernetesA simple project (which is visitor counter) on kubernetes

k8s playground This project aims to deploy a simple project (which is visitor counter) on kubernetes. Deploy steps kubectl apply -f secret.yaml kubect

null 11 Mar 23, 2022
A simple http-web server logging incoming requests to stdout with simple http-interface.

http-cli-echo-logger A simple http-web server logging incoming requests to stdout with simple http-interface. Run locally go run ./cmd/main.go Default

Andrii Bosonchenko 4 Jun 7, 2022
Simple to do list API with Gin and Gorm (with Postgres)Simple to do list API with Gin and Gorm (with Postgres)

go-todo Simple to do list API with Gin and Gorm (with Postgres) Docker Clone this repository and run: docker-compose up You can then hit the followin

ansh-dev 2 Apr 28, 2022
go-simple-geo is a library for simple geo calculations.

go-simple-geo is a library for simple geo calculations. Installation go get github.com/EpicStep/go-simple-geo Example package main import ( "github.

Stepan Rabotkin 19 Mar 16, 2022
Go-generics-simple-doubly-linked-list - A simple doubly linked list implemented using generics (Golang)

Welcome to Go-Generics-Simple-Doubly-Linked-List! Hi, This repository contains a

Behrad Ravanbod 4 Jan 6, 2022
Simple-crm-system - Simple CRM system CRUD backend using Go, Fiber, SQLite, Gorm

Simple CRM system CRUD backend using GO, Fiber, Gorm, SQLite Developent go mod t

Oleg Luganskiy 3 Apr 21, 2022
Simple-go-api - This porject deploys a simple go app inside a EKS Cluster

SimpleGoApp This porject deploys a simple go app inside a EKS Cluster Prerequisi

null 0 Jan 19, 2022
Simple-api - Create Simple `Hello World` APIs from Various Programming Languages

simple-api Create Simple `Hello World` APIs from Various Programming Languages.

I am Groot 1 Jan 18, 2022
Simple wget - Simple wget written as test for Scorum

simple_wget simple wget written as test for Scorum Task: Implement in Go (http:/

Andrey Solomenniy 0 Jan 24, 2022
Simple-lisp - A Simple Lisp Interpreter written in Go

Simple Lisp A simple Lisp interpreter written in Go. The fixed-precision-numbers

Oxygen 5 Jun 21, 2022
Simple-Weather-API - Simple weather api app created using golang and Open Weather API key

Simple Weather API Simple weather api app created using golang and Open Weather

Siva Prakash 3 Feb 6, 2022
Hetzner-dns-updater - A simple tool to update a DNS record via Hetzner DNS API. Used for simple HA together with Nomad

hetzner-dns-updater A small utility tool to update a single record via Hetzner D

Patrick Pacher 0 Feb 12, 2022