Scan and analyze OSS dependencies and licenses from compiled Go binaries

Related tags

Security golicense
Overview

golicense - Go Binary OSS License Scanner

golicense is a tool that scans compiled Go binaries and can output all the dependencies, their versions, and their respective licenses (if known). golicense only works with Go binaries compiled using Go modules for dependency management.

golicense determines the dependency list quickly and with exact accuracy since it uses metadata from the Go compiler to determine the exact set of dependencies embedded in a compiled Go binary. This excludes dependencies that are not used in the final binary. For example, if a library depends on "foo" in function "F" but "F" is never called, then the dependency "foo" will not be present in the final binary.

golicense is not meant to be a complete replacement for open source compliance companies such as FOSSA or BlackDuck, both of which provide hundreds of additional features related to open source compliance.

Warning: The binary itself must be trusted and untampered with to provide accurate results. It is trivial to modify the dependency information of a compiled binary. This is the opposite side of the same coin with source-based dependency analysis where the source must not be tampered.

Features

  • List dependencies and their associated licenses
  • Cross-reference dependency licenses against an allow/deny list
  • Output reports in the terminal and Excel (XLSX) format
  • Manually specify overrides for specific dependencies if the detection is incorrect.

Example

The example below runs golicense against itself from a recent build.

golicense Example

Installation

To install golicense, download the appropriate release for your platform from the releases page.

You can also compile from source using Go 1.11 or later using standard go build. Please ensure that Go modules are enabled (GOPATH not set or GO111MODULE set to "on").

Usage

golicense is used with one or two required arguments. In the one-argument form, the dependencies and their licenses are listed. In the two-argument form, a configuration file can be given to specify an allow/deny list of licenses and more.

$ golicense [flags] [BINARY]
$ golicense [flags] [CONFIG] [BINARY]

You may also pass mutliple binaries (but only if you are providing a CONFIG).

Configuration File

The configuration file can specify allow/deny lists of licenses for reports, license overrides for specific dependencies, and more. The configuration file format is HCL or JSON.

Example:

allow = ["MIT", "Apache-2.0"]
deny  = ["GNU General Public License v2.0"]
{
  "allow": ["MIT", "Apache-2.0"],
  "deny": ["GNU General Public License v2.0"]
}

Supported configurations:

  • allow (array<string>) - A list of names or SPDX IDs of allowed licenses.
  • deny (array<string>) - A list of names or SPDX IDs of denied licenses.
  • override (map<string, string>) - A mapping of Go import identifiers to translate into a specific license by SPDX ID. This can be used to set the license of imports that golicense cannot detect so that reports pass.
  • translate (map<string, string>) - A mapping of Go import identifiers to translate into alternate import identifiers. Example: "gopkg.in/foo/bar.v2" to "github.com/foo/bar". If the map key starts and ends with / then it is treated as a regular expression. In this case, the map value can use \1, \2, etc. to reference capture groups.

GitHub Authentication

golicense uses the GitHub API to look up licenses. This doesn't require any authentication out of the box but will be severely rate limited. It is recommended that you generate a personal access token to increase the rate limit. The personal access token doesn't require any special access unless it needs to look at private repositories you have access to, in which case it should be granted the repo permission. Specify your token using the GITHUB_TOKEN environment variable.

$ export GITHUB_TOKEN=abcd1234
$ golicense ./binary

Excel (XLSX) Reporting Output

If the -out-xlsx flag is specified, then an Excel report is generated and written to the path specified in addition to the terminal output.

$ golicense -out-xlsx=report.xlsx ./my-program

The Excel report contains the list of dependencies, their versions, the detected license, and whether the license is allowed or not. The dependencies are listed in alphabetical order. The row of the dependency will have a green background if everything is okay, a yellow background if a license is unknown, or a red background is a license is denied. An example screenshot is shown below:

Excel Report

Limitations

There are a number of limitations to golicense currently. These are fixable but work hasn't been done to address these yet. If you feel like taking a stab at any of these, please do and contribute!

GitHub API: The license detected by golicense may be incorrect if a GitHub project changes licenses. golicense uses the GitHub API which only returns the license currently detected; we can't lookup licenses for specific commit hashes.

Comments
  • "This executable was compiled without using Go modules"

    I just gave it a run with ddev and get

    $ golicense ddev
    ⚠️  "ddev" ⚠️
    
    This executable was compiled without using Go modules or has
    zero dependencies. golicense considers this an error (exit code 1).
    

    Well, it uses dep, not go modules... but it has lots in the vendor directory. Most projects haven't converted to modules yet. To test, you can just download ddev from releases.

    opened by rfay 4
  • Release 0.1.2?

    Release 0.1.2?

    @mitchellh First: thanks for the great project! I found it as I wanted to add an OS license scanner to a CI pipeline. When do you think there will be a new release? Without the fix for #10 it is quite cumbersome to have a pipeline fail on a forbidden license.

    opened by s-spindler 3
  • golicense seems to be not `go get`-able at the moment

    golicense seems to be not `go get`-able at the moment

    Trying to go get in a fresh container, I'm getting this:

    $ docker run --rm -it golang:1.11
    [...]
    Status: Downloaded newer image for golang:1.11
    [email protected]:/go# go version
    go version go1.11.2 linux/amd64
    [email protected]:/go# go get github.com/mitchellh/golicense
    package github.com/google/go-github/v18/github: cannot find package "github.com/google/go-github/v18/github" in any of:
    	/usr/local/go/src/github.com/google/go-github/v18/github (from $GOROOT)
    	/go/src/github.com/google/go-github/v18/github (from $GOPATH)
    
    opened by rtfb 3
  • Can not handle kubectl

    Can not handle kubectl

    Hi, it look like can not handle kubectl (the cli tool of kubernetes):

    $ which kubectl
    /usr/local/bin/kubectl
    $ ./golicense /usr/local/bin/kubectl
    ⚠️  "/usr/local/bin/kubectl" ⚠️
    
    This executable was compiled without using Go modules or has
    zero dependencies. golicense considers this an error (exit code 1).
    $
    $ kubectl version
    Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:17:28Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
    Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0", GitCommit:"fc32d2f3698e36b93322a3465f63a14e9f0eaead", GitTreeState:"clean", BuildDate:"2018-04-10T12:46:31Z", GoVersion:"go1.9.4", Compiler:"gc", Platform:"linux/amd64"}
    $
    

    And can detect its own:

    $ ./golicense ./golicense
    github.com/hashicorp/go-multierror        Mozilla Public License 2.0
    github.com/hashicorp/go-cleanhttp         Mozilla Public License 2.0
    github.com/hashicorp/errwrap              Mozilla Public License 2.0
    ...
    github.com/google/go-github               BSD 3-Clause "New" or "Revised" License
    github.com/gosuri/uilive                  MIT License
    github.com/360EntSecGroup-Skylar/excelize BSD 3-Clause "New" or "Revised" License
    github.com/apparentlymart/go-textseg      Apache License 2.0
    $
    

    But they are the same statically linked, so what's wrong with processing kubeclt? Thanks!

    $ file /usr/local/bin/kubectl
    /usr/local/bin/kubectl: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
    $
    $ file ./golicense
    ./golicense: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped
    $
    
    opened by warmchang 2
  • Allow ignoring specific packages

    Allow ignoring specific packages

    Awesome project! Thank you :)

    I tried running this on a binary that depended on a package in a private repository, and the license for that package could not be found, surely.

    It would be nice if the configuration allowed a list of packages to ignore when fetching licenses.

    opened by happyalu 1
  • not compiling with goModules on

    not compiling with goModules on

    from root:

    GO111MODULE=on go build -o /bin/main main.go
    # command-line-arguments
    ./main.go:36:14: undefined: TermOutput
    ./main.go:112:10: undefined: MultiOutput
    ./main.go:114:38: undefined: XLSXOutput
    ./main.go:151:9: undefined: NewSemaphore
    ./main.go:162:42: undefined: StatusListener
    [go-build] Error 2
    

    go env:

    x-MacBook-Pro:bin-parse apple$ go env
    GOARCH="amd64"
    GOBIN=""
    GOCACHE="/Users/apple/Library/Caches/go-build"
    GOEXE=""
    GOFLAGS=""
    GOHOSTARCH="amd64"
    GOHOSTOS="darwin"
    GOOS="darwin"
    GOPATH="/Users/apple/workspace/go"
    GOPROXY=""
    GORACE=""
    GOROOT="/usr/local/opt/go/libexec"
    GOTMPDIR=""
    GOTOOLDIR="/usr/local/opt/go/libexec/pkg/tool/darwin_amd64"
    GCCGO="gccgo"
    CC="clang"
    CXX="clang++"
    CGO_ENABLED="1"
    GOMOD=""
    CGO_CFLAGS="-g -O2"
    CGO_CPPFLAGS=""
    CGO_CXXFLAGS="-g -O2"
    CGO_FFLAGS="-g -O2"
    CGO_LDFLAGS="-g -O2"
    PKG_CONFIG="pkg-config"
    GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/wp/ff6sz9qs6g71jnm12nj2kbyw0000gp/T/go-build332320505=/tmp/go-build -gno-record-gcc-switches -fno-common"
    
    opened by ghost 1
  •  Error parsing dependencies

    Error parsing dependencies

    https://github.com/gohugoio/hugo

    ▶ golicense ./hugo
    ❗️ Error parsing dependencies: Unexpected raw dependency format: dep	github.com/markbates/inflect	v1.0.0%
    
    opened by bep 1
  • Add support for SBOM reporting

    Add support for SBOM reporting

    • It's now possible to generate the report as SBOM - software bill-of-material: https://cyclonedx.org/
    • it's possible to set custom license name in override section in the config (before only SPDX IDs were allowed)
    • it's possible to optionally specify licence url for SBOM report in the config using sbomLicenseURLs section
    • take a look at docs for sample usage
    opened by ikawalec 0
  • Add third-party-licenses.txt Output Type

    Add third-party-licenses.txt Output Type

    Adds a new output type via -output-licensefile that dumps the following information for each dependency to the file name specified:

    1. Import path
    2. Version
    3. Full license text

    Also added is a new configuration type to allow for "pre-approved" licenses (e.g. imports with non-standard licenses that need manual review). This is different from "overrides", which forces a given import path to be treated as having a specific license type (e.g. MIT).

    opened by arittr 0
  • Support running over multiple binaries at once

    Support running over multiple binaries at once

    For my use case I just need the aggregate license analysis from multiple binaries. The goal here would be to avoid a lot of the duplication that comes from checking multiple binaries individually, which have a lot of shared dependencies.

    opened by alex 0
  • Support -version argument

    Support -version argument

    Since golicense is distributed in binary form, it would help debugging and differentiation for it to respond to -version in the conventional way.

    While go version -m -v /path/to/golicense can provide some detail, this is not necessarily the same thing as a known SemVer identifier. Example:

    go version -m -v `which golicense`
    /usr/local/bin/golicense: go1.13.4
    	path	github.com/mitchellh/golicense
    	mod	github.com/mitchellh/golicense	(devel)	
    	dep	github.com/360EntSecGroup-Skylar/excelize	v1.4.0	h1:43rak9uafmwSJpXfFO1heKQph8tP3nlfWJWFQQtW1R0=
    	...
    

    This is particularly telling because no devel git-ref currently exists in the project.

    opened by atz 0
  • ModError:replace a third_party pkg

    ModError:replace a third_party pkg

    image it's a very common use when you use a thrid party and you do you own changes on this pkg. you use : replace xxxxx => ./xxxxx and the golicense take error

    opened by Arasinaka 0
  • Is golicense abandoned?

    Is golicense abandoned?

    @mitchellh Can you please shed some light on the status of this project? Do you think you will be able to look into some of the open issues & PRs in the near-ish future or are you busy? In case of the latter: would you be open to adding contributors to the project?

    opened by s-spindler 4
  • Add templating

    Add templating

    I needed a way of creating different output than the default or xlsx - an asciidoc table to be more specific. I hope this is considered useful by other people :)

    opened by s-spindler 0
  • Output the license URL to the xslx output

    Output the license URL to the xslx output

    Added a column to the xslx output that contain the URL of the license file.

    This is the license file as detected by GitHub, but in the specific tag of the module version.

    opened by mlitvin 0
Releases(v0.2.0)
Owner
Mitchell Hashimoto
Mitchell Hashimoto
Use golang.org/x/vuln to scan your dependencies

vulnz - Use golang.org/x/vuln to scan your dependencies vulnz uses golang.org/x/vuln to scan the dependencies in your go.mod. Install go install githu

Miki Tebeka 1 Mar 19, 2022
DockerSlim (docker-slim): Don't change anything in your Docker container image and minify it by up to 30x (and for compiled languages even more) making it secure too! (free and open source)

Minify and Secure Docker containers (free and open source!) Don't change anything in your Docker container image and minify it by up to 30x making it

docker-slim 15.1k Oct 1, 2022
crowdsec 5.6k Sep 20, 2022
Take a list of domains and scan for endpoints, secrets, api keys, file extensions, tokens and more...

Take a list of domains and scan for endpoints, secrets, api keys, file extensions, tokens and more... Coded with ?? by edoardottt. Share on Twitter! P

gilfoyle97 577 Sep 20, 2022
Scan your pictures and videos for corruption, and sort them by EXIF or modification time

scanogram Scan your pictures and videos for corruption, and sort them by EXIF or modification time. Introduction This tool is a fast and lightweight s

Victor 3 Mar 30, 2022
Static binary analysis tool to compute shared strings references between binaries and output in JSON, YAML and YARA

StrTwins StrTwins is a binary analysis tool, powered by radare, that is capable to find shared code string references between executables and output i

Anderson 2 May 3, 2022
A fast tool to mass scan for a vulnerability on Microsoft Exchange Server that allows an attacker bypassing the authentication and impersonating as the admin (CVE-2021-26855).

proxylogscan This tool to mass scan for a vulnerability on Microsoft Exchange Server that allows an attacker bypassing the authentication and imperson

dw1 140 Aug 28, 2022
Git watchdog will scan your public repository and find out the vulnerabilities

Dependencies Docker Go 1.17 MySQL 8.0.25 Bootstrap Run chmod +x start.sh if start.sh script does not have privileged to run Run ./start.sh --bootstrap

Quang Nguyen 2 Dec 30, 2021
A port scan and service weakpass brute tool build by golang.

A port scan and service weakpass brute tool build by golang.

M1ku 6 Aug 26, 2022
Scan systems and docker images for potential spring4shell vulnerabilities.

Scan systems and docker images for potential spring4shell vulnerabilities. Will detect in-depth (layered archives jar/zip/tar/war and scans for vulnerable Spring4shell versions. Binaries for Windows, Linux and OsX, but can be build on each platform supported by supported Golang.

null 11 May 6, 2022
Analyse binaries for missing security features, information disclosure and more.

extrude Analyse binaries for missing security features, information disclosure and more. ?? Extrude is in the early stages of development, and current

Liam Galvin 45 Sep 22, 2022
A fast tool to scan CRLF vulnerability written in Go

CRLFuzz A fast tool to scan CRLF vulnerability written in Go Resources Installation from Binary from Source from GitHub Usage Basic Usage Flags Target

dw1 814 Sep 24, 2022
Carbon Black Harbor Adapter is a scanner to scan images in Harbor Registry with the help of Carbon Black Cloud.

carbon-black-adapter-for-harbor Overview Carbon Black adapter for Harbor integrates your Harbor Registry with the Carbon Black Cloud. It leverages Har

VMware 3 Apr 18, 2022
WIP. Converts Azure Container Scan Action output to SARIF, for an easier integration with GitHub Code Scanning

container-scan-to-sarif container-scan-to-sarif converts Azure Container Scan Action output to Static Analysis Results Interchange Format (SARIF), for

Armel Soro 2 Jan 25, 2022
Go network scan tool.

gns Go network scan tool. Version: v0.6.0 Usage: gns [Options] <IP or domain> eg: gns -r 22-8080 -s 300 localhost Options: -a All ports, 1-6553

Sean Zhang 2 Dec 9, 2021
Fast scan for redtools

scaninfo by 华东360安服团队 注意的点 漏洞扫描的时候有时候最后几个任务会卡住,是因为ftp爆破模块,这个fscan也一样目前没有好的解决办法,后续更新.先阶段可以-eq 21跳过ftp,或者control+c 主动停止不影响结果保存。 有时候扫外网的全端口会漏掉端口可以使用-n 指定

kb_24 559 Oct 1, 2022
Scan all AWS EC2 instances in a region for potentially vulnerable log4j versions

ec2-log4j-scan Scan all AWS EC2 instances in a region for potentially vulnerable log4j versions. This is a clumsy but effective tool which takes outpu

null 2 Dec 28, 2021
Log4j-scanner tools - Support for multiple scan method

Log4j-scanner URL mode (fuzzing url with header, payload) go run . url -h Usage

BREAK TEAM - Hacking Team 2 Sep 7, 2022