Shotizam analyzes the size of Go binaries

Overview

Shotizam

Shotizam analyzes the size of Go binaries and outputs SQL with size info for analysis in SQLite3.

$ shotizam --sqlite /some/go.binary
SQLite version 3.28.0 2019-04-15 14:49:49
Enter ".help" for usage hints.
sqlite> .width 40
sqlite> .mode column

sqlite> select func, sum(size) from bin where func <> '' group by 1 order by 2 desc limit 20;
unicode.init                              21528
debug/dwarf.init                          13764
fmt.(*pp).printValue                      11941
debug/macho.NewFile                       10017
time.Time.AppendFormat                    9949
runtime.gentraceback                      8278
debug/elf.NewFile                         7631
runtime.selectgo                          6722
encoding/binary.Read                      6555
main.main                                 6476
internal/fmtsort.compare                  6373
time.LoadLocationFromTZData               5795
fmt.(*pp).doPrintf                        5520
runtime.findrunnable                      5287
runtime.typesEqual                        4843
time.nextStdChunk                         4736
time.ParseDuration                        4440
time.loadTzinfoFromZip                    4234
runtime.(*pageAlloc).find                 4109
runtime.heapBitsSetType                   3938

sqlite> select what, sum(size) from bin group by 1;
TODO                                      1370985
fixedheader                               101800
funcdata                                  99656
funcname                                  63062
pcdata0-regmap                            54361
pcdata1-stackmap                          47977
pcdata2-inltree                           40024
pcfile                                    38544
pcln                                      141103
pcsp                                      36478
text                                      896722

sqlite> select pkg, sum(size) from bin where pkg <> '' group by 1 order by 2 desc limit 20;
runtime               621236
reflect               187926
fmt                   61727
time                  58786
strconv               55817
syscall               39481
debug/elf             37070
os                    36707
compress/flate        31462
debug/macho           27117
encoding/binary       26637
os/exec               26599
internal/reflectlite  23687
unicode               22316
sync                  21322
flag                  20871
debug/dwarf           17692
internal/poll         17092
strings               15511
sort                  14992

sqlite> select func, length(func) from bin order by 2 desc limit 5;
type..eq.[67]struct { runtime.size uint3  89
type..eq.[67]struct { runtime.size uint3  89
type..eq.[67]struct { runtime.size uint3  89
type..eq.[67]struct { runtime.size uint3  89
type..eq.[67]struct { runtime.size uint3  89

sqlite> select func, sum(size) from bin where what = 'pcfile' and func <> '' group by 1 order by 2 desc limit 20;
internal/fmtsort.compare                  268
fmt.(*pp).printValue                      227
encoding/binary.Read                      195
os.Getwd                                  174
runtime.sighandler                        172
runtime.(*pageAlloc).scavengeOne          153
runtime.findrunnable                      148
debug/macho.NewFile                       131
runtime.selectgo                          130
syscall.forkAndExecInChild                129
main.main                                 123
runtime.(*mspan).sweep                    123
internal/fmtsort.nilCompare               119
runtime.growslice                         119
debug/elf.NewFile                         112
fmt.intFromArg                            107
runtime.greyobject                        105
runtime.evacuate_fast64                   103
runtime.mapaccess2_faststr                101
runtime.evacuate                          99

sqlite> select func, sum(size) from bin where what = 'pcln' and func <> '' group by 1 order by 2 desc limit 20;
time.Time.AppendFormat                    1347
runtime.gentraceback                      1132
runtime.selectgo                          1102
fmt.(*pp).printValue                      1027
runtime.findrunnable                      830
runtime.heapBitsSetType                   794
internal/fmtsort.compare                  720
runtime.memmove                           720
debug/macho.NewFile                       692
fmt.(*pp).doPrintf                        638
encoding/binary.Read                      616
runtime.runGCProg                         613
syscall.forkAndExecInChild                552
debug/elf.NewFile                         539
runtime.mallocgc                          532
time.LoadLocationFromTZData               530
runtime.duffcopy                          528
aeshashbody                               518
runtime.(*mspan).sweep                    502
runtime.sighandler                        478

sqlite> select * from bin limit 30;
go.buildid                                            fixedheader  40
go.buildid                                            text         112
go.buildid                                            funcname     11
internal/cpu.Initialize                   internal/c  fixedheader  40
internal/cpu.Initialize                   internal/c  pcsp         13
internal/cpu.Initialize                   internal/c  pcfile       5
internal/cpu.Initialize                   internal/c  pcln         21
internal/cpu.Initialize                   internal/c  pcdata0-reg  22
internal/cpu.Initialize                   internal/c  pcdata1-sta  21
internal/cpu.Initialize                   internal/c  text         80
internal/cpu.Initialize                   internal/c  funcname     24
internal/cpu.processOptions               internal/c  fixedheader  40
internal/cpu.processOptions               internal/c  pcsp         24
internal/cpu.processOptions               internal/c  pcfile       6
internal/cpu.processOptions               internal/c  pcln         167
internal/cpu.processOptions               internal/c  pcdata0-reg  30
internal/cpu.processOptions               internal/c  pcdata1-sta  60
internal/cpu.processOptions               internal/c  text         1792
internal/cpu.processOptions               internal/c  funcname     28
internal/cpu.indexByte                    internal/c  fixedheader  40
internal/cpu.indexByte                    internal/c  pcsp         9
internal/cpu.indexByte                    internal/c  pcfile       5
internal/cpu.indexByte                    internal/c  pcln         21
internal/cpu.indexByte                    internal/c  pcdata0-reg  22
internal/cpu.indexByte                    internal/c  text         64
internal/cpu.indexByte                    internal/c  funcname     23
internal/cpu.doinit                       internal/c  fixedheader  40
internal/cpu.doinit                       internal/c  pcsp         22
internal/cpu.doinit                       internal/c  pcfile       5
internal/cpu.doinit                       internal/c  pcln         159

etc

For fun bugs to make Go smaller, see https://github.com/golang/go/labels/binary-size

Issues
  • no __gopclntab section found in ELF file when building with PIE enabled

    no __gopclntab section found in ELF file when building with PIE enabled

    I ran (current HEAD, i.e. c9a147c6aa3d4767214712d5886e8a9703f6085b):

    % shotizam --sqlite blaze-bin/…/testsrv
    no __gopclntab section found in ELF file
    

    When searching for __gopclntab, I only find matches in go/src/cmd/internal/objfile/macho.go. Is this a mac-related thing that I’m missing on Linux?

    opened by stapelberg 5
  • Over-Escaping Quotes?

    Over-Escaping Quotes?

    I ran shotizam against Caddy 2 (v2.0.0-rc.3) executable and got about 18 lines of syntax error from SQLite. This is the first offending line and the line that precedes it:

    INSERT INTO Bin VALUES ("golang.org/x/oauth2/jwt.(*jwtSource).Token", "golang.org/x/oauth2/jwt", "funcname", 43);
    INSERT INTO Bin VALUES ("type..eq.struct { AccessToken string \"json:\\\"access_token\\\"\"; TokenType string \"json:\\\"token_type\\\"\"; IDToken string \"json:\\\"id_token\\\"\"; ExpiresIn int64 \"json:\\\"expires_in\\\"\" }", "", "fixedheader", 40);
    

    SQLite's complaint: Error: near line 159205: near "json": syntax error

    opened by mohammed90 3
  • Add support for 1.16

    Add support for 1.16

    This pretty much just consists of integrating the changes to upstream debug/gosym.

    Not very well tested, but results seem OK, and all 0 unit tests pass!

    Fixes #10

    opened by prattmic 2
  • Add support for linux

    Add support for linux

    The linux objdump (at least on ubuntu) uses <> instead of _. Example output from objdump -D on ubuntu - 0000000000402bf0 <runtime.c64hash>:

    This way it works for compiled binaries on linux (checked on Ubuntu only) with CGO_ENABLE=1 and CGO_ENABLE=0

    opened by egorse 2
  • gosym: PCData comes from pctab, not funcdata

    gosym: PCData comes from pctab, not funcdata

    See LineTable.pcvalue and runtime.pcvalue as reference.

    This fixes values from PCData and occassional crashes if the garbage we were reading before happened to be bad varints.

    opened by prattmic 1
  • Cannot analyze binaries built with Go 1.16

    Cannot analyze binaries built with Go 1.16

    Trying to analyze a binary built with Go 1.16.2 I get:

    $ shotizam --sqlite daemon/cilium-agent
    not a go1.2+ line table
    

    It looks like this is due to the linker changes in Go 1.16. See e.g.https://golang.org/cl/246497 for some of the corresponding changes to the debug/gosym package.

    opened by tklauser 0
Owner
Brad Fitzpatrick
Xoogler. Ex @golang team (2010-2020). Currently making WireGuard easier and more magical @Tailscale.
Brad Fitzpatrick
Shotizam analyzes the size of Go binaries

Shotizam Shotizam analyzes the size of Go binaries and outputs SQL with size info for analysis in SQLite3. $ shotizam --sqlite /some/go.binary SQLite

Brad Fitzpatrick 571 Jul 29, 2022
Shotizam analyzes the size of Go binaries

Shotizam Shotizam analyzes the size of Go binaries and outputs SQL with size info for analysis in SQLite3. $ shotizam --sqlite /some/go.binary SQLite

Brad Fitzpatrick 571 Jul 29, 2022
go-to64 analyzes Golang main package to convert int/uint to int64/uint64.

go-to64 About go-to64 analyzes Golang main package to convert int/uint to int64/uint64. This is an experiment tool, so be very careful. In a 32-bit en

Tomohiro Takezawa 1 Oct 31, 2021
Small tool that analyzes the data of my crappy file format for catalogging things

Things Catalog Analyzer I recently started catalogging all the things I own. I simply wanted to have an overview over my things, think about what I ca

Marcel Schramm 0 Nov 7, 2021
Dev Lake is the one-stop solution that integrates, analyzes, and visualizes software development data

Dev Lake is the one-stop solution that integrates, analyzes, and visualizes software development data throughout the software development life cycle (SDLC) for engineering teams.

Merico 40 Aug 6, 2022
A prediction program which analyzes given numbers and calculates new values

Guess-It-2 About This is a prediction program which analyzes given numbers and calculates new values. Usage To test the program, download this zip fil

null 0 Nov 30, 2021
A tool to analyze and troubleshoot a Go binary size.

goweight A tool to analyze and troubleshoot a Go binary size. For more, see this blog post ✅ Get a breakdown of all modules inside a binary ✅ Supports

Dotan J. Nahum 1.5k Aug 7, 2022
A simple thread-safe, fixed size LRU written in Go. Based on dominictarr's Hashlru Algorithm. 🔃

go-hashlru A simple thread-safe, fixed size LRU written in Go. Based on dominictarr's Hashlru Algorithm. ?? Uses map[interface{}]interface{} to allow

Saurabh Pujari 70 Jul 3, 2022
A tool to filter URLs by parameter count or size

GoFilter A tool to filter URLs by parameter count or size. This tool requires unique sorted URL list. For example: cat hosts.txt | sort -u > sorted &&

Ayberk ESER 7 Sep 10, 2021
A Connected Graph Generator tool that construct graphs of some given size

graph graph is a Connected Graph Generator tool that construct graphs of some given size. Notice that it generates all possible connected, undirected

Nicolas A Perez 0 Nov 5, 2021
A lightweight replacement for the standard fmt package, reduces binary size by roughly 400kb in a hello world

console This is a lightweight replacement for the fmt package, reduces the binary size by roughly 400kb in a hello world program. Please note: This pa

null 1 Nov 7, 2021
Converts a number to its English counterpart. Uses arbitrary precision; so a number of any size can be converted.

Converts a number to its English counterpart. Uses arbitrary precision; so a number of any size can be converted.

null 0 Dec 14, 2021
Removes unnecessarily saved git objects to optimize the size of the .git directory.

Git Repo Cleaner Optimizes the size of the .git directory by removing all of the files that are unnecessarily-still-saved as part of the git history.

Omar Yasser 2 Mar 24, 2022
This provides the lru package which implements a fixed-size thread safe LRU cache

golang-lru This provides the lru package which implements a fixed-size thread sa

byx 0 Dec 22, 2021
Image size analyzer for jpg/png/gif/webp

imgsz Image size analyzer for jpg/png/gif/webp Usage // DecodeSize decodes the dimensions of an image that has // been encoded in a registered format.

源文雨 0 Jan 8, 2022
Go-binsize-treemap - Go binary size SVG treemap

?? Go binary size SVG treemap Make treemap breakdown of Go executable binary $ g

Nikolay Dubina 389 Aug 8, 2022
YouTube'da altyazısı olan veya otomatik olarak oluşturulmuş altyazılı videolarda istediğiniz kelimenin hangi saat, dakika ve saniye de geçtiğini size gösterip aradığınız şeyi hızlıca bulmanızı sağlar.

YouTube Subtitles YouTube'da altyazısı olan veya otomatik olarak oluşturulmuş altyazılı videolarda istediğiniz kelimenin hangi saat, dakika ve saniye

İbrahim Can Mercan 2 Mar 4, 2022
A tool for testing, building, signing, and publishing binaries.

gomason Tool for testing, building, signing and publishing binaries. Think of it as an on premesis CI/CD system- that also performs code signing and p

Nik Ogura 53 Apr 7, 2022
The simple and easy way to embed static files into Go binaries.

NOTICE: Please consider migrating your projects to github.com/markbates/pkger. It has an idiomatic API, minimal dependencies, a stronger test suite (t

Buffalo - The Go Web Eco-System 3.4k Aug 4, 2022
The forgotten go tool that executes and caches binaries included in go.mod files.

The forgotten go tool that executes and caches binaries included in go.mod files. This makes it easy to version cli tools in your projects such as gol

Dustin Blackman 23 Jul 25, 2022
A CLI tool that generates OpenTelemetry Collector binaries based on a manifest.

OpenTelemetry Collector builder This program generates a custom OpenTelemetry Collector binary based on a given configuration. TL;DR $ go get github.c

OpenTelemetry - CNCF 51 Aug 3, 2022
GoReleaser builds Go binaries as fast and easily as possible

GoReleaser builds Go binaries for several platforms, creates a GitHub release and then pushes a Homebrew formula to a tap repository. All that wrapped in your favorite CI.

GoReleaser 10.4k Aug 7, 2022
📦 Package Node.js applications into executable binaries 📦

caxa ?? Package Node.js applications into executable binaries ?? Support Recurring support on Patreon: https://patreon.com/leafac One-time support on

Leandro Facchinetti 562 Aug 5, 2022
⚡️ A lightweight service that will build and store your go projects binaries, Integrated with Github, Gitlab, Bitbucket and Bitbucket Server.

Rabbit A lightweight service that will build and store your go projects binaries. Rabbit is a lightweight service that will build and store your go pr

Ahmed 195 Jul 29, 2022
Monitorable, gracefully restarting, self-upgrading binaries in Go (golang)

overseer overseer is a package for creating monitorable, gracefully restarting, self-upgrading binaries in Go (golang). The main goal of this project

Jaime Pillora 1.9k Aug 8, 2022
Embed static files in Go binaries (replacement for gobuffalo/packr)

Pkger github.com/markbates/pkger is a tool for embedding static files into Go binaries. It will, hopefully, be a replacement for github.com/gobuffalo/

Mark Bates 1.2k Jul 27, 2022
Golang binaries compiled on-demand for your system

Go Binaries Go Binaries is an on-demand binary server, allowing non-Go users to quickly install tools written in Go without installing go itself, and

TJ Holowaychuk 747 Aug 10, 2022
Scan and analyze OSS dependencies and licenses from compiled Go binaries

golicense - Go Binary OSS License Scanner golicense is a tool that scans compiled Go binaries and can output all the dependencies, their versions, and

Mitchell Hashimoto 627 Jul 27, 2022
Find strings in Go binaries

gostringsr2 gostringsr2 extracts strings from a Go binary using radare2. Tested with radare2 3.7.0, Python 3.7, r2pipe 1.4.1, on OS X and Linux. Teste

Carve Systems 44 Jul 27, 2022