Support CI generation of SBOMs via golang tooling.

Overview

SPDX Software Bill of Materials (SBOM) Generator

Overview

Software Package Data Exchange (SPDX) is an open standard for communicating software bill of materials (SBOM) information that supports accurate identification of software components, explicit mapping of relationships between components, and the association of security and licensing information with each component.

spdx-sbom-generatortool to help those in the community that want to generate SPDX Software Bill of Materials (SBOMs) with current package managers. It has a command line Interface (CLI) that lets you generate SBOM information, including components, licenses, copyrights, and security references of your software using SPDX v2.2 specification and aligning with the current known minimum elements from NTIA. It automatically determines which package managers or build systems are actually being used by the software.

spdx-sbom-generatoris supporting the following package managers:

  • GoMod (go)
  • Cargo (Rust)
  • Composer (PHP)
  • DotNet (.NET)
  • Maven (Java)
  • NPM (Node.js)
  • Yarn (Node.js)
  • PIP (Python)
  • Pipenv (Python)
  • Gems (Ruby)

Installation:

Note: The spdx-sbom-generator CLI is under development. You may expect some breakages and stability issues with the current release. A stable version is under development and will be available to the open source community in the upcoming beta release.

Available command Options

Run help:

Target schema version (default: '2.2') (default "2.2") -f, --format string output file format (default: 'spdx') ">
./spdx-sbom-generator -h

Output Package Manager dependency on SPDX format

Usage:
  spdx-sbom-generator [flags]

Flags:
  -h, --help                   help for spdx-sbom-generator
  -i, --include-license-text   include full license text (default: false)
  -o, --output-dir string      directory to write output file to (default: current directory)
  -p, --path string            the path to package file or the path to a directory which will be recursively analyzed for the package files (default '.') (default ".")
  -s, --schema string          <version> Target schema version (default: '2.2') (default "2.2")
  -f, --format string          output file format (default: 'spdx')

Output Options

  • spdx (Default format)

  • JSON (In progress)

  • RDF (In progress)

Command output sample option:

./spdx-sbom-generator -o /out/spdx/

Output Sample

The following snippet is a sample SPDX SBOM file:

SPDXVersion: SPDX-2.2
DataLicense: CC0-1.0
SPDXID: SPDXRef-DOCUMENT
DocumentName: spdx-sbom-generator
DocumentNamespace: http://spdx.org/spdxpackages/spdx-sbom-generator--57918521-3212-4369-a8ed-3d681ec1d7a1
Creator: Tool: spdx-sbom-generator-XXXXX
Created: 2021-05-23 11:25:29.1672276 -0400 -04 m=+0.538283001

##### Package representing the Go distribution

PackageNam: go
SPDXID: SPDXRef-Package-go
PackageVersion: v0.46.3
PackageSupplier: NOASSERTION
PackageDownloadLocation: pkg:golang/cloud.google.com/[email protected]
FilesAnalyzed: false
PackageChecksum: TEST: SHA-1 224ffa55932c22cef869e85aa33e2ada43f0fb8d
PackageHomePage: pkg:golang/cloud.google.com/[email protected]
PackageLicenseConcluded: NOASSERTION
PackageLicenseDeclared: NOASSERTION
PackageCopyrightText: NOASSERTION
PackageLicenseComments: NOASSERTION
PackageComment: NOASSERTION

Relationship: SPDXRef-DOCUMENT DESCRIBES SPDXRef-Package-go

##### Package representing the Bigquery Distribution

PackageNam: bigquery
SPDXID: SPDXRef-Package-bigquery
PackageVersion: v1.0.1
PackageSupplier: NOASSERTION
PackageDownloadLocation: pkg:golang/cloud.google.com/go/[email protected]
FilesAnalyzed: false
PackageChecksum: TEST: SHA-1 8168e852b675afc9a63b502feeefac90944a5a2a
PackageHomePage: pkg:golang/cloud.google.com/go/[email protected]
PackageLicenseConcluded: NOASSERTION
PackageLicenseDeclared: NOASSERTION
PackageCopyrightText: NOASSERTION
PackageLicenseComments: NOASSERTION
PackageComment: NOASSERTION

Relationship: SPDXRef-Package-go CONTAINS SPDXRef-Package-bigquery

Docker Images

Currently few Docker images are supported:

spdx/spdx-sbom-generator - Alpine image and spdx-sbom-generator binary installed

$ docker run -it --rm \
    -v "/path/to/repository:/repository" \
    -v "$(pwd)/out:/out" \
    spdx/spdx-sbom-generator -p /repository -o /out/spdx/

Data Contract

The interface requires the following functions:

type IPlugin interface {
  SetRootModule(path string) error
  GetVersion() (string, error)
  GetMetadata() PluginMetadata
  GetRootModule(path string) (*Module, error)
  ListUsedModules(path string) ([]Module, error)
  ListModulesWithDeps(path string) ([]Module, error)
  IsValid(path string) bool

Module model definition:

type Module struct {
  Version          string `json:"Version,omitempty"`
  Name             string
  Path             string `json:"Path,omitempty"`
  LocalPath        string `json:"Dir,noempty"`
  Supplier         SupplierContact
  PackageURL       string
  CheckSum         *CheckSum
  PackageHomePage  string
  LicenseConcluded string
  LicenseDeclared  string
  CommentsLicense  string
  OtherLicense     []*License
  Copyright        string
  PackageComment   string
  Root             bool
  Modules          map[string]*Module
}```

`PluginMetadata` model definition:
​```GO
type PluginMetadata struct {
    Name       string
    Slug       string
    Manifest   []string
    ModulePath []string
}

How to Generate Module Values

  • CheckSum: We have built an internal method that calculates CheckSum for a given content (in bytes) using algorithm that is defined on models.CheckSum. You now have an option to provide Content field for models.CheckSum{} and CheckSum will calculate automatically, but if you want to calculate CheckSum on your own you still can provide Value field for models.CheckSum{}.

Also, you can generate a manifest from a given directory tree using utility/helper method BuildManifestContent, and that is what is used for gomod plugin as Content value.

Interface Definitions

The following list provides the interface definitions:

  • GetVersion: returns version of current project platform (development language) version i.e: go version

    Input: None

    Output: version in string format and error (null in case of successful process)

  • GetMetadata: returns metadata of identify ecosystem pluging

    Input: None

    Output: plugin metadata

PluginMetadata{
    Name:       "Go Modules",
    Slug:       "go-mod",
    Manifest:   []string{"go.mod"},
    ModulePath: []string{"vendor"},
}
  • SetRootModule: sets root package information base on path given

    Input: The working directory to read the package from

    Output: returns error

  • GetRootModule: returns root package information base on path given

    Input: The working directory to read the package from

    Output: returns the Package Information of the root Module

  • ListUsedModules: fetches and lists all packages required by the project in the given project directory, this is a plain list of all used modules (no nested or tree view)

    Input: The working directory to read the package from

    Output: returns the Package Information of the root Module, and its dependencies in flatten format

  • ListModulesWithDeps: fetches and lists all packages (root and direct dependencies) required by the project in the given project directory (side-by-side), this is a one level only list of all used modules, and each with its direct dependency only (similar output to ListUsedModules but with direct dependency only)

    Input: The working directory to read the package from

    Output: returns the Package Information of the root Module, and its direct dependencies

  • IsValid: check if the project dependency file provided in the contract exists

    Input: The working directory to read the package from

    Output: True or False

  • HasModulesInstalled: check whether the current project(based on given path) has the dependent packages installed

    Input: The working directory to read the package from

    Output: True or False

Module Structure JSON Example

The sample module structure JSON Code snippet is provided in the following code snippet:

{
       "Version": "v0.0.1-2019.2.3",
       "Name": "honnef.co/go/tools",
       "Path": "honnef.co/go/tools",
       "LocalPath": "",
       "Supplier": {
               "Type": "",
               "Name": "",
               "EMail": ""
       },
       "PackageURL": "pkg:golang/honnef.co/go/[email protected]",
       "CheckSum": {
               "Algorithm": "SHA-1",
               "Value": "66ed272162df8ef5f9e6d7bece3da6828a4ef3eb"
       },
       "PackageHomePage": "",
       "LicenseConcluded": "",
       "LicenseDeclared": "",
       "CommentsLicense": "",
       "OtherLicense": null,
       "Copyright": "",
       "PackageComment": "",
       "Root": false,
       "Modules": {
               "github.com/BurntSushi/toml": {
                       "Version": "v0.3.1",
                       "Name": "github.com/BurntSushi/toml",
                       "Path": "github.com/BurntSushi/toml",
                       "LocalPath": "",
                       "Supplier": {
                               "Type": "",
                               "Name": "",
                               "EMail": ""
                       },
                       "PackageURL": "pkg:golang/github.com/BurntSushi/[email protected]",
                       "CheckSum": {
                               "Algorithm": "SHA-1",
                               "Value": "38263d2f264e90324c9e9b3b1933f0e94fde1c7e"
                       },
                       "PackageHomePage": "",
                       "LicenseConcluded": "",
                       "LicenseDeclared": "",
                       "CommentsLicense": "",
                       "OtherLicense": null,
                       "Copyright": "",
                       "PackageComment": "",
                       "Root": false,
                       "Modules": null
               }
        }
}

For a more complete JSON example look at modules.json.

Utility Methods

The following list provide the utility methods:

  • BuildManifestContent walks through a given directory tree, and generates a content based on file paths

    Input: Directory to walk through

    Output: directory tree in bytes

  • GetLicenses: returns the detected license object

    Input: The working directory of the package licenses

    Output: The package license object

type License struct {
	ID            string
	Name          string
	ExtractedText string
	Comments      string
	File          string
}
  • LicenseSPDXExists: Check if the package license is a valid SPDX reference

    Input: The package license

    Output: True or False

How to Register a New Plugin

To register for a new plugin, perform the following steps:

  1. Clone a project.

    git clone [email protected]:LF-Engineering/spdx-sbom-generator.git
    
  2. Create a new directory into ./internal/modules/ with package manager name, for example: npm, you should end with a directory:

    /internal/modules/npm
    
    
  3. Create a Handler file, for example: handler.go, and follow Data Contract section above. Define package name, and import section as explained in the following code snippet:

    package npm
    
    import (
    	"path/filepath"
    
    	"spdx-sbom-generator/internal/helper"
    	"spdx-sbom-generator/internal/models"
    )
    
    // rest of the file below
    
    
  4. In handler.go, define the plugin struct with at least the plugin metadata info as explained in the following code snippet:

    type npm struct {
    	metadata models.PluginMetadata
    }
    
    
  5. Define plugin registration method (New func) with metadata values as explained in the following code snippet:

    // New ...
    func New() *npm {
    	return &npm{
    		metadata: models.PluginMetadata{
    			Name:       "Node Package Manager",
    			Slug:       "npm",
    			Manifest:   []string{"package.json"},
    			ModulePath: []string{"node_modules"},
    		},
    	}
    }
    
    
  6. In handler.go, create the required interface function (Data contract definition above).

    // GetMetadata ...
    func (m *npm) GetMetadata() models.PluginMetadata {
      return m.metadata
    }
    
    // IsValid ...
    func (m *npm) IsValid(path string) bool {
      for i := range m.metadata.Manifest {
        if helper.Exists(filepath.Join(path, m.metadata.Manifest[i])) {
          return true
        }
      }
      return false
    }
    
    // HasModulesInstalled ...
    func (m *npm) HasModulesInstalled(path string) error {
      for i := range m.metadata.ModulePath {
        if helper.Exists(filepath.Join(path, m.metadata.ModulePath[i])) {
          return nil
        }
      }
      return errDependenciesNotFound
    }
    
    // GetVersion ...
    func (m *npm) GetVersion() (string, error) {
      output, err := exec.Command("npm", "--version").Output()
      if err != nil {
        return "", err
      }
    
      return string(output), nil
    }
    
    // SetRootModule ...
    func (m *npm) SetRootModule(path string) error {
      return nil
    }
    
    // GetRootModule ...
    func (m *npm) GetRootModule(path string) (*models.Module, error) {
      return nil, nil
    }
    
    // ListUsedModules...
    func (m *npm) ListUsedModules(path string) ([]models.Module, error) {
      return nil, nil
    }
    
    // ListModulesWithDeps ...
    func (m *npm) ListModulesWithDeps(path string) ([]models.Module, error) {
      return nil, nil
    }
    
    
  7. In modules.go at ./internal/modules/ directory, register the new plugin. Add the plugin to register to the existing definition.

    func init() {
        registeredPlugins = append(registeredPlugins,
                gomod.New(),
                npm.New(),
        )
    }
    
    

How to Work With SPDX SBOM Generator

A Makefile for the spdx-sbom-generator is described below with ability to run, test, lint, and build the project binary for different platforms (Linux, Mac, and Windows).

Perform the following steps to work with SPDX SBOM Generator:

  1. Run project on current directory.

    make generate
    

    you can provide the CLI parameters that will be passed along the command, for example:

    ARGS="--path /home/ubuntu/projects/expressjs" make generate
    
  2. Build Linux Intel/AMD 64-bit binary.

    make build
    
  3. Build Mac Intel/AMD 64-bit binary.

    make build-mac
    
  4. Build Mac ARM 64-bit binary.

    make build-mac-arm64
    
  5. Build Windows Intel/AMD 64-bit binary.

    make build-win
    

Licensing

docker/cli is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Additional Information

SPDX

SPDX SBOM

SPDX Tools

SPDX License List

SPDX GitHub Repos

Issues
  • Ability to output to JSON

    Ability to output to JSON

    This pull request adds support for creating JSON SPDX SBOMs (thereby resolving #117 if merged)

    This is achieved by

    • Making the formatter/renderer modular (now implemented by a go interface)
    • Passing down the -f argument to the renderer so that it can use the appropriate implementation
    • Updating the Document and Package structs (which are now annotated as per the official JSON spec/example) to better resemble the structure specified by the SPDX spec
    • Updating the filename resolution logic to take into consideration the format passed down by the user

    Additionally, this pull request also completes the todo item of reimplementing the tag-value format (.spdx) renderer as a go template.

    opened by amithkk 9
  • Java - Maven - NOASSERTION is displayed for both PackageSupplier and PackageDownloadLocation even when values exists as per the conditions mentioned in specification

    Java - Maven - NOASSERTION is displayed for both PackageSupplier and PackageDownloadLocation even when values exists as per the conditions mentioned in specification

    @prathapbproximabiz Tool Version Tested with v.0.0.8 and as well as tested with binaries built from cloned code from main branch of https://github.com/spdx/spdx-sbom-generator on 27-06-2021 Test Repo https://github.com/mybatis/mybatis-3 OS Windows 10

    Observed that NOASSERTION is displayed for both PackageSupplier and PackageDownloadLocation for all packages even when values exists as per the conditions mentioned in specification SPDX files bom-Java-Maven_generated with Latest code.spdx.txt bom-Java-Maven_mybatis-3_v0.0.8_27-Jun-2021.spdx.txt bom-Java-Maven_sample-java-programs_v0.0.8_27-Jun-2021.spdx.txt bom-Java-Maven_zxing_v0.0.8_27-Jun-2021.spdx.txt

    Specification image

    Example image

    image

    bug java maven 
    opened by niruautomation 8
  • .net - Warning message is displayed when SPDX file validated in the SPDX validator

    .net - Warning message is displayed when SPDX file validated in the SPDX validator

    @proximapc Tool Version v0.0.6 Test Repo https://github.com/dotnet-architecture/eShopOnWeb OS Windows 10

    1. Clone the repo
    2. Generate the SPDX file with command ./spdx-sbom-generator image

    Validate the SPDX file generated for rust in https://tools.spdx.org/app/validate/ Observed that warnings are displayed SPDX File bom-nuget.spdx.txt

    image

    bug .net 
    opened by niruautomation 5
  • Python(Go) - poetry - dependencies listed in METADAT file are not displayed in SPDX file

    Python(Go) - poetry - dependencies listed in METADAT file are not displayed in SPDX file

    @lfpratik Tool Version Cloned code from main branch of https://github.com/spdx/spdx-sbom-generator on 11-06-2021 and built the tool Test Repo https://github.com/lfpratik/spdx-poetry-demo OS Windows 10

    1. Followed all prerequisite steps as per https://confluence.linuxfoundation.org/display/PROD/SPDX+-+Python+Module+-+Prerequisites+For+Windows
    2. Followed Prerequisite and Steps as per below screenshot image
    3. Execute ./spdx-sbom-generator
    4. Observed that all dependencies listed in METADAT file are not displayed in SPDX file Example1 image

    image

    Example2 image

    image

    bug golang python poetry 
    opened by niruautomation 5
  • Java - Maven - Warning message is displayed when SPDX file validated in the SPDX validator

    Java - Maven - Warning message is displayed when SPDX file validated in the SPDX validator

    Test Repo used for testing https://github.com/mlehotskylf/sample-java-programs

    1. Clone the https://github.com/spdx/spdx-sbom-generator.git from main branch (Since latest version tool is not available followed this approach for testing)
    2. Execute the make build-win to build the tool
    3. Generate the SPDX file for JAVA module
    4. Validate the generated SPDX file in https://tools.spdx.org/app/validate/
    5. Observed that the warning message is displayed. PFA SPDX file for reference image

    bom-Java-Maven.txt

    opened by niruautomation 5
  • .NET - Value for PackageDownloadLocation is displayed as NOASSERTION even when value exists for {package.repository.url}

    .NET - Value for PackageDownloadLocation is displayed as NOASSERTION even when value exists for {package.repository.url}

    @proximapc Tool Version v0.0.8 Test Repo https://github.com/jasontaylordev/CleanArchitecture OS Windows 10

    Observed that value for PackageDownloadLocation is displayed as NOASSERTION even when value exists for {package.repository.url}. As per specification file {package.repository.url should be displayed for PackageDownloadLocation

    Specification file image

    Example1 Package-FluentValidation-9.3.0 image

    image

    Example2 Package-MediatR-9.0.0 image

    image

    Example3 Package-AutoMapper-10.0.0 image

    image

    bug .net 
    opened by niruautomation 4
  • Rust - SPDX file validation failed in the SPDX validator

    Rust - SPDX file validation failed in the SPDX validator

    @niravpatel27 Tool Version v0.0.6 Test Repo https://github.com/rust-random/rand OS Windows 10

    Validate the SPDX file generated for rust in https://tools.spdx.org/app/validate/ Observed that the validation failed

    SPDX file bom-cargo.spdx.txt

    image

    bug rust 
    opened by niruautomation 3
  • Python(Go) - pipenv/venv - Details of Document and root package are not matching with the repo against which SPDX file is generated

    Python(Go) - pipenv/venv - Details of Document and root package are not matching with the repo against which SPDX file is generated

    @lfpratik Tool Version I cloned the code from master on 14-06-2021, build the tool and verified the ticket Test Repo https://github.com/lfpratik/spdx-pipenv-demo OS Windows 10

    1. Followed all prerequisite steps as per https://confluence.linuxfoundation.org/display/PROD/SPDX+-+Python+Module+-+Prerequisites+For+Windows
    2. Followed Prerequisite and Steps as per below screenshot image
    3. Execute ./spdx-sbom-generator
    4. Observed that SPDX file is generated but details of Document and root package are not matching with the repo against which SPDX file is generated

    image

    image

    bug golang python pipenv 
    opened by niruautomation 3
  • Python(Go) - pipenv - Not able to generate SPDX file

    Python(Go) - pipenv - Not able to generate SPDX file

    @lfpratik Tool Version v0.0.6 Test Repo https://github.com/lfpratik/spdx-pipenv-demo OS Windows 10

    1. Followed all prerequisite steps as per https://confluence.linuxfoundation.org/display/PROD/SPDX+-+Python+Module+-+Prerequisites+For+Windows
    2. Followed Prerequisite and Steps as per below screenshot image
    3. Execute ./spdx-sbom-generator
    4. Observed that SPDX file is not generated and below error is displayed
    [email protected] MINGW64 /d/LFX/Projects/SPDX/Python/Go/spdx-pipenv-demo (main)
    $ pipenv sync
    Installing dependencies from Pipfile.lock (5a4c19)...
    To activate this project's virtualenv, run pipenv shell.
    Alternatively, run a command inside the virtualenv with pipenv run.
    All dependencies are now up-to-date!
    
    [email protected] MINGW64 /d/LFX/Projects/SPDX/Python/Go/spdx-pipenv-demo (main)
    $ pipenv install
    Installing dependencies from Pipfile.lock (5a4c19)...
    To activate this project's virtualenv, run pipenv shell.
    Alternatively, run a command inside the virtualenv with pipenv run.
    
    [email protected] MINGW64 /d/LFX/Projects/SPDX/Python/Go/spdx-pipenv-demo (main)
    $ spdx-sbom-generator
    bash: spdx-sbom-generator: command not found
    
    [email protected] MINGW64 /d/LFX/Projects/SPDX/Python/Go/spdx-pipenv-demo (main)
    $ ./spdx-sbom-generator
    INFO[2021-06-10T19:07:23+05:30] Starting to generate SPDX ...
    INFO[2021-06-10T19:07:23+05:30] Running generator for Module Manager: `pip` with output `bom-pip.spdx`
           21-06-10T19:07:24+05:30] Current Language Version Python 3.9.5
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal 0xc0000005 code=0x0 addr=0x30 pc=0x7940cd]
    
    goroutine 1 [running]:
    spdx-sbom-generator/internal/models.(*CheckSum).String(0x0, 0x0, 0x0)
            /github/workspace/internal/models/models.go:76 +0x2d
    spdx-sbom-generator/internal/format.(*Format).convertToPackage(0xc002eb9b08, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
            /github/workspace/internal/format/format.go:180 +0xd0
    spdx-sbom-generator/internal/format.(*Format).buildPackages(0xc002eb9b08, 0xc000075800, 0x33, 0x34, 0xc002fbea86, 0x13, 0xc0034d33e0, 0x2a, 0xc003333ab0, 0x61)
            /github/workspace/internal/format/format.go:153 +0x125
    spdx-sbom-generator/internal/format.(*Format).Render(0xc002eb9b08, 0xc0030144e0, 0x0)
            /github/workspace/internal/format/format.go:59 +0x13b
    spdx-sbom-generator/internal/handler.(*spdxHandler).Run(0xc002ee6240, 0x6, 0xd311d5)
            /github/workspace/internal/handler/spdx.go:89 +0x548
    main.generate(0x1644880, 0x166c0e8, 0x0, 0x0)
            /github/workspace/cmd/generator/generator.go:105 +0x43a
    github.com/spf13/cobra.(*Command).execute(0x1644880, 0xc0000581a0, 0x0, 0x0, 0x1644880, 0xc0000581a0)
            /go/pkg/mod/github.com/spf13/[email protected]/command.go:856 +0x2b1
    github.com/spf13/cobra.(*Command).ExecuteC(0x1644880, 0x0, 0x0, 0x0)
            /go/pkg/mod/github.com/spf13/[email protected]/command.go:960 +0x350
    github.com/spf13/cobra.(*Command).Execute(...)
            /go/pkg/mod/github.com/spf13/[email protected]/command.go:897
    main.main()
            /github/workspace/cmd/generator/generator.go:37 +0x64
    
    bug golang python pipenv 
    opened by niruautomation 3
  • Java - Maven - Duplicate Package Information is displayed for modules in SPDX

    Java - Maven - Duplicate Package Information is displayed for modules in SPDX

    Repo used for testing https://github.com/mlehotskylf/sample-java-programs

    Observed that duplicate Package details are displayed for modules in SPDX. PFA SPDX file for reference bom-Java-Maven.txt

    For e.g. Package-base-0.0.2-SNAPSHOT

    Record1 image

    Record2 image

    bug java maven 
    opened by niruautomation 3
  • Not to be a troll - but what's in this thing?

    Not to be a troll - but what's in this thing?

    Summary

    Since golang compiles down to an EXE, and I cannot easily look at the pieces that go into spdx-sbom-generator, I think it would be good for you to generate an SPDX compliant SBOM for reach release - then I can perhaps trust the download.

    Background

    Steps to reproduce:

    1. Download sbom-spdx-generator for MacOS
    2. Expand archive generating the file "sbom-spdx-generator"
    3. Realize this is software I found on the internet at github - do I really want to run it?
    4. Try to figure out what kind of script it is: file sbom-spdx-generator
    5. Conclude I am not allowed as a government employee to run it because it is an executable that I did not create

    User Story

    • As a software architect, I want to be able comply with SBOM requirements using open source, but it is hard to trust an open source tool that is installed as an executable not as a pip package (which lists dependencies although not as an SBOM), or npm package (which lists dependencies although not as an SBOM). This is especially true when the executable is hosted on github rather than in an package repository such as rpm, etc.

    Acceptance Criteria

    There are probably a lot of ways to make it better... and having an SBOM for each release is only the "eat your own dog food" approach.

    References

    (lrrdjango) NLM01991848OCCS:lrrdjango davisda4$ tar tvf ~/Downloads/spdx-sbom-generator-v0.0.10-darwin-amd64.tar.gz 
    -rwxr-xr-x  0 root   root 24905488 Jun 29 12:09 spdx-sbom-generator
    (lrrdjango) NLM01991848OCCS:lrrdjango davisda4$ tar xzvf ~/Downloads/spdx-sbom-generator-v0.0.10-darwin-amd64.tar.gz 
    x spdx-sbom-generator
    (lrrdjango) NLM01991848OCCS:lrrdjango davisda4$ file spdx-sbom-generatr
    spdx-sbom-generatr: cannot open `spdx-sbom-generatr' (No such file or directory)
    (lrrdjango) NLM01991848OCCS:lrrdjango davisda4$ file spdx-sbom-generator
    spdx-sbom-generator: Mach-O 64-bit executable x86_64
    
    enhancement 
    opened by danizen 0
  • docs: Add reference to `scoop install spdx-sbom-generator` for Windows users

    docs: Add reference to `scoop install spdx-sbom-generator` for Windows users

    Impacted Docs

    I added the ability for Windows users to install the latest version directly from Scoop with the command scoop install spdx-sbom-generator so it might be worth adding this to the README.

    Tasks

    • [ ] Review help documents for impacts beyond what this issue explicitly calls out

    Verbiage

    n/a

    Acceptance Criteria

    The "done" criteria when this feature or problem is resolved. Such as:

    1. Documentation changes submitted as a Pull Request
    2. Pull Request Reviewed and Approved by Product Owner
    3. Documentation changes merged to 'master' branch

    Images

    n/a

    documentation 
    opened by stevehipwell 1
  • NPM - No support for package-lock v2 files

    NPM - No support for package-lock v2 files

    Summary

    As of February 2021, npm 7 is now generally available. By default npm 7 utilizes v2 lockfile. (Which are backwards compatible but have a slightly different structure)

    However, attempting to generate SBOMs with a package-lock generated by npm7 causes a crash in spdx-sbom-generator. This has been attempted with node red

    Background

    Provide context to the issue - provide steps to reproduce the behavior, such as:

    1. Download sbom-spdx-generator version 0.0.15
    2. Clone repository https://github.com/node-red/node-red
    3. Install dependencies with npm i
    4. Run ./sbom-spdx-generator
    5. Observe the following error:
    INFO[2021-10-03T16:10:31+05:30] Starting to generate SPDX ...
    INFO[2021-10-03T16:10:31+05:30] Running generator for Module Manager: `npm` with output `bom-npm.spdx` 
    INFO[2021-10-03T16:10:34+05:30] Current Language Version 7.24.1
    panic: interface conversion: interface {} is string, not map[string]interface {}
    
    goroutine 1 [running]:
    github.com/spdx/spdx-sbom-generator/pkg/modules/npm.appendDependencies(0xc11bc0, 0xc000cd3c50, 0xc00220fe00)
            <PATH-REDACTED>/spdx-sbom-generator/pkg/modules/npm/handler.go:372 +0x285
    github.com/spdx/spdx-sbom-generator/pkg/modules/npm.appendNestedDependencies(0xc000a0f6e0, 0x14c6d80)
            <PATH-REDACTED>/spdx-sbom-generator/pkg/modules/npm/handler.go:359 +0x2ce
    github.com/spdx/spdx-sbom-generator/pkg/modules/npm.(*npm).buildDependencies(0xc00006e1e0, 0xc000028110, 0xa, 0xc000a0f6e0, 0xc000134428, 0xb6352b, 0xc0032d1b80, 0x1a, 0x1)
            <PATH-REDACTED>/spdx-sbom-generator/pkg/modules/npm/handler.go:209 +0x505
    github.com/spdx/spdx-sbom-generator/pkg/modules/npm.(*npm).ListModulesWithDeps(0xc00006e1e0, 0xc000028110, 0xa, 0x0, 0x0, 0x1, 
    0x1, 0xccf860)
            <PATH-REDACTED>/spdx-sbom-generator/pkg/modules/npm/handler.go:185 +0x20e
    github.com/spdx/spdx-sbom-generator/pkg/modules.(*Manager).Run(0xc000514040, 0x4, 0xd05a7f)
            <PATH-REDACTED>/spdx-sbom-generator/pkg/modules/modules.go:99 +0x15e
    github.com/spdx/spdx-sbom-generator/pkg/handler.(*spdxHandler).Run(0xc000594180, 0xb, 0xc000028110)
            <PATH-REDACTED>/spdx-sbom-generator/pkg/handler/spdx.go:85 +0x35b
    main.generate(0x14759c0, 0xc000332060, 0x0, 0x2)
            <PATH-REDACTED>/spdx-sbom-generator/cmd/generator/generator.go:118 +0x446
    github.com/spf13/cobra.(*Command).execute(0x14759c0, 0xc00003e090, 0x2, 0x3, 0x14759c0, 0xc00003e090)
            <PATH-REDACTED>/spdx-sbom-generator/vendor/github.com/spf13/cobra/command.go:856 +0x2c2
    github.com/spf13/cobra.(*Command).ExecuteC(0x14759c0, 0x44bd01, 0x0, 0x0)
            <PATH-REDACTED>/spdx-sbom-generator/vendor/github.com/spf13/cobra/command.go:960 +0x375
    github.com/spf13/cobra.(*Command).Execute(...)
            <PATH-REDACTED>/spdx-sbom-generator/vendor/github.com/spf13/cobra/command.go:897
    main.main()
            <PATH-REDACTED>/spdx-sbom-generator/cmd/generator/generator.go:39 +0x68
    

    Expected behavior

    The SBOM is generated

    Repository

    Which repository causes this error?

    • node-red (Any environment where npm v7 is installed)

    Acceptance Criteria

    The "done" criteria when this feature or problem is resolved. Such as:

    When v2 lockfiles generated by NPM v7 can be used to generate SBOMs

    References

    Here is an example; package-lock.json

    bug 
    opened by amithkk 0
  • go-mod: Incorrect PackageDownloadLocation

    go-mod: Incorrect PackageDownloadLocation

    Summary

    Running spdx-sbom-generator (built from HEAD) on https://github.com/google/ko prints PackageDownloadLocations that are invalid, e.g.:

    ##### Package representing the github.com/opencontainers/image-spec
    
    PackageName: github.com/opencontainers/image-spec
    SPDXID: SPDXRef-Package-github.com.opencontainers.image-spec-v1.0.2-0.20210730191737-8e42a01fb1b7
    PackageVersion: v1.0.2-0.20210730191737-8e42a01fb1b7
    PackageSupplier: Organization: github.com/opencontainers/image-spec
    PackageDownloadLocation: https://github.com/opencontainers/image-spec/releases/tag/v1.0.2-0.20210730191737-8e42a01fb1b7
    FilesAnalyzed: false
    PackageChecksum: SHA256: fc54332d8eaf1fb435b840238aa286757e0a181bc117f2e1324f4fb97bc282d6
    PackageHomePage: https://github.com/opencontainers/image-spec
    
    • https://github.com/opencontainers/image-spec/releases/tag/v1.0.2-0.20210730191737-8e42a01fb1b7 is not found
    • a valid download URL would be https://github.com/opencontainers/image-spec/archive/8e42a01fb1b7.zip

    (dependencies referenced by commit SHA won't have a GitHub release, but can be downloaded as a zip using the /archive/ URL)

    ##### Package representing the github.com/docker/distribution
    
    PackageName: github.com/docker/distribution
    SPDXID: SPDXRef-Package-github.com.docker.distribution-v2.7.1+incompatible
    PackageVersion: v2.7.1+incompatible
    PackageSupplier: Organization: github.com/docker/distribution
    PackageDownloadLocation: https://github.com/docker/distribution/releases/tag/v2.7.1+incompatible
    FilesAnalyzed: false
    PackageChecksum: SHA256: 305d2f1be0a274519bf2f7b32c4f780f352d5c1aed0ed44adfbfc60a657be739
    PackageHomePage: https://github.com/docker/distribution
    
    • https://github.com/docker/distribution/releases/tag/v2.7.1+incompatible is not found
    • https://github.com/docker/distribution/releases/tag/v2.7.1 would be valid

    (the +incompatible suffix is unnecessary)

    Repository

    Which repository causes this error?

    • https://github.com/google/ko
    • any repository that has a dep on a module by commit SHA, or +incompatible

    Acceptance Criteria

    The "done" criteria when this feature or problem is resolved. Such as:

    1. Unit Tests added and running in CI
    2. Functional Tests updated to cover feature, if applicable

    References

    • https://stackoverflow.com/questions/57355929/what-does-incompatible-in-go-mod-mean-will-it-cause-harm
    • https://stackoverflow.com/questions/53682247/how-to-point-go-module-dependency-in-go-mod-to-a-latest-commit-in-a-repo
    bug 
    opened by imjasonh 1
  • SPDX SBOM Generator Bug Report -  Plugin pipenv return error failed to read modules

    SPDX SBOM Generator Bug Report - Plugin pipenv return error failed to read modules

    Summary

    Got error message Unable to fetch package details when I try to generate the SBOM, the error has no more details or verbose log, I have no idea what's happening or what's required to make it work.

    $ spdx-sbom-generator 
    INFO[2021-08-16T17:10:17+08:00] Starting to generate SPDX ...                
    INFO[2021-08-16T17:10:17+08:00] Running generator for Module Manager: `pipenv` with output `bom-pipenv.spdx` 
    INFO[2021-08-16T17:10:17+08:00] Current Language Version Python 3.7.5        
    ERRO[2021-08-16T17:10:18+08:00] Unable to fetch package details              
    INFO[2021-08-16T17:10:18+08:00] Command has completed with errors for some package managers, see details below 
    INFO[2021-08-16T17:10:18+08:00] Plugin pipenv return error failed to read modules 
    

    Background

    Environment:

    • Ubuntu 18.04.5 LTS
    • Python 3.6.9 / 3.7.5 (From Ubuntu apt packages)
    • sbom-spdx-generator v0.0.13 for linux-amd64

    Steps to get the problem:

    1. Download sbom-spdx-generator from https://github.com/spdx/spdx-sbom-generator/releases/tag/v0.0.13, extract the binary to $PATH
    2. Run sbom-spdx-generator binary in a private project path(It's a python project, using pipenv to help manage the packages)
    3. Observe the following error:
    INFO[2021-08-16T17:10:17+08:00] Starting to generate SPDX ...                
    INFO[2021-08-16T17:10:17+08:00] Running generator for Module Manager: `pipenv` with output `bom-pipenv.spdx` 
    INFO[2021-08-16T17:10:17+08:00] Current Language Version Python 3.7.5        
    ERRO[2021-08-16T17:10:18+08:00] Unable to fetch package details              
    INFO[2021-08-16T17:10:18+08:00] Command has completed with errors for some package managers, see details below 
    INFO[2021-08-16T17:10:18+08:00] Plugin pipenv return error failed to read modules 
    

    Expected behavior

    Expect to produce SBOM.

    Screenshots

    image

    Repository

    It's a private repository, but I might be able to provide the Pipfile file of pipenv, if it's something will help to reproduce the bug.

    Acceptance Criteria

    • The SBOM can be generated.
    bug 
    opened by PeterDaveHello 0
  • [Question] How to solve the error message:

    [Question] How to solve the error message: "error in getting mvn transitive dependency tree and parsing it"?

    Hello,

    I got an error message "error in getting mvn transitive dependency tree and parsing it" when I tried to generator from a simple Maven project:

    $ ./spdx-sbom-generator -p /home/ubuntu/maven -o /home/ubuntu/out2
    INFO[2021-08-05T11:42:35+09:00] Starting to generate SPDX ...
    INFO[2021-08-05T14:37:01+09:00] Running generator for Module Manager: `Java-Maven` with output `/home/ubuntu/out2/bom-Java-Maven.spdx`
    INFO[2021-08-05T14:37:01+09:00] Current Language Version Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)
    Maven home: /home/ubuntu/apache-maven-3.8.1
    Java version: 11.0.11, vendor: Ubuntu, runtime: /usr/lib/jvm/java-11-openjdk-amd64
    Default locale: ja_JP, platform encoding: UTF-8
    OS name: "linux", version: "4.15.0-23-generic", arch: "amd64", family: "unix"
    error in getting mvn transitive dependency tree and parsing it
    ERRO[2021-08-05T17:22:18+09:00] exit status 1
    INFO[2021-08-05T17:22:18+09:00] Command has completed with errors for some package managers, see details below
    INFO[2021-08-05T17:22:18+09:00] Plugin Java-Maven return error failed to read modules
    

    Did anyone get the same error message before?

    I checked the source code:

    https://github.com/spdx/spdx-sbom-generator/blob/90ec05b20557e3cda6fd12cf214ad02b83c02f87/pkg/modules/javamaven/handler.go#L119-L135 and: https://github.com/spdx/spdx-sbom-generator/blob/90ec05b20557e3cda6fd12cf214ad02b83c02f87/pkg/modules/javamaven/decoder.go#L427-L444

    Then I run "mvn dependency:tree" command and it was successful. So I have no idea about what is wrong here. I will be very thankful if anyone can solve my problem.

    bug 
    opened by shi9qiu 1
  • Please tweak so it works for CII Best Practices badge project

    Please tweak so it works for CII Best Practices badge project

    Summary

    The OpenSSF CII Best Practices badge uses Ruby on Rails. The spdx-sbom-generator tool should work, but doesn't. I separately sent an email about this. This is a GitHub issue so we can track this.

    bug 
    opened by david-a-wheeler 0
  • GetCopyright Index Out of Range Error

    GetCopyright Index Out of Range Error

    Summary

    Helper - GetCopyright function - Runtime error: index out of range [] with length

    Background

    Index out of range occurs when providing the string "copyright" to the function:

    panic: runtime error: index out of range [1] with length 1
    
    goroutine 1 [running]:
    runtime/debug.Stack(0xc00310e510, 0xc002ec3c00, 0x33)
    	runtime/debug/stack.go:24 +0x9f
    git.fuzzbuzz.io/fuzz.(*F).Close(0xc003103800)
    	git.fuzzbuzz.io/fuzz/f_obj.go:657 +0x7d2
    panic(0xa24940, 0xc00310e510)
    	runtime/panic.go:965 +0x1b9
    github.com/spdx/spdx-sbom-generator/pkg/helper.GetCopyright(0xc00310acd7, 0x9, 0x9, 0xc003110380)
    	/src/pkg/helper/helper.go:158 +0x754
    github.com/spdx/spdx-sbom-generator/pkg/helper.FuzzGetCopyright(0xc003103800)
    	/src/pkg/helper/fuzzbuzz_autogen.go:8 +0x7e
    github.com/spdx/spdx-sbom-generator/fuzzing/fuzzbuzzauto.FuzzFunction0(...)
    	/src/fuzzing/fuzzbuzzauto/fuzz.go:9
    github.com/spdx/spdx-sbom-generator/fuzzing/fuzzbuzzauto.FuzzFunction0_FuzzWrapper(0x6608e1b147c38b75, 0x9aaac0, 0xc0030f4918, 0x0)
    	github.com/spdx/spdx-sbom-generator/fuzzing/fuzzbuzzauto/fuzzwrappers.go:12 +0x91
    git.fuzzbuzz.io/fuzz/endpoint.(*StandardFuzzEndpoint).StartMain(0xc000167e08)
    	git.fuzzbuzz.io/fuzz/endpoint/endpoint.go:150 +0x4de
    base-dep.Main(0x10c2418, 0x0, 0x0, 0xc002774bf0, 0x4, 0x4, 0xc00007e0f0, 0xa, 0xa)
    	base-dep/main.go:50 +0x385
    main.main()
    	github.com/spdx/spdx-sbom-generator/fuzzing/fuzzbuzzauto/go.fuzz.main/main.go:36 +0x125
    

    Expected behavior

    The function should not crash/die when provided invalid input.

    Screenshots

    Screen Shot 2021-07-27 at 10 37 07 AM

    Repository

    Which repository causes this error?

    Additional Context

    Optional - add any other context about the problem here.

    Acceptance Criteria

    The "done" criteria when this feature or problem is resolved. Such as:

    1. Unit Tests added and running in CI
    2. Functional Tests updated to cover feature, if applicable
    3. Demonstrate the set of capabilities to the product team

    References

    • Provide any code or specification references that would be helpful for the developer implementing this feature.
    bug 
    opened by dealako 1
  • [Question] What does 'No module manager found' error mean?

    [Question] What does 'No module manager found' error mean?

    Hello,

    I'm trying to generate a sample SBOM on a python project directory, but I'm getting this error:

    $ ./spdx-sbom-generator ../sample_python_project -o out
    
    INFO[2021-07-15T09:10:31-07:00] Starting to generate SPDX ...                
    FATA[2021-07-15T09:10:31-07:00] Failed to run command: No module manager found
    

    This is the first time I'm generating an SBOM, so the chances that I'm missing something trivial are high.

    Thanks

    opened by Vafa-Andalibi 0
  • Calculate sha of artifact contents, not name

    Calculate sha of artifact contents, not name

    Summary

    https://github.com/spdx/spdx-sbom-generator/blob/main/pkg/modules/javamaven/decoder.go#L173

    the artifact hash should not be on the name of the module. For example the artifact com.google.guava:guava:30.1.1-jre should have a sha1 of https://repo1.maven.org/maven2/com/google/guava/guava/30.1.1-jre/guava-30.1.1-jre-javadoc.jar.sha1

    bug 
    opened by loosebazooka 2
Releases(v0.0.13)
Owner
SPDX
A standard format for communicating the components, licenses and copyrights associated with a software package.
SPDX
Personal notetaking tooling

jot CLI Task List and Journal jot is a simple journaling program for CLI that helps you keep track of your life. Config File ~/.jot.yaml is read on st

Gabe Conradi 5 Aug 6, 2021
:sunglasses:Package captcha provides an easy to use, unopinionated API for captcha generation

Package captcha provides an easy to use, unopinionated API for captcha generation. Why another captcha generator? I want a simple and framework-indepe

Weilin Shi 86 Oct 22, 2021
Set of functions/methods that will ease GO code generation

Set of functions/methods that will ease GO code generation

Matheus Leonel Balduino 1 Nov 19, 2021
:chart_with_upwards_trend: Monitors Go MemStats + System stats such as Memory, Swap and CPU and sends via UDP anywhere you want for logging etc...

Package stats Package stats allows for gathering of statistics regarding your Go application and system it is running on and sent them via UDP to a se

Go Playgound 157 Aug 27, 2021
DSV Parallel Processor takes input files and query specification via a spec file

DSV Parallel Processor Spec file DSV Parallel Processor takes input files and query specification via a spec file (conventionally named "spec.toml").

Wattanit Hotrakool 0 Oct 9, 2021
using go search the Marvel universe characters via marvel api

go-marvel-api using go search the Marvel universe characters via marvel api Build and run tests on the local environemnt Build the project $ go build

Burak KÖSE 1 Oct 5, 2021
CoreFoundation Property List support for Go

PACKAGE package plist import "github.com/kballard/go-osx-plist" Package plist implements serializing and deserializing of property list

Lily Ballard 27 May 21, 2021
Prometheus support for go-metrics

go-metrics-prometheus This is a reporter for the go-metrics library which will post the metrics to the prometheus client registry . It just updates th

Csergő Bálint 60 Oct 7, 2021
Library to work with MimeHeaders and another mime types. Library support wildcards and parameters.

Mime header Motivation This library created to help people to parse media type data, like headers, and store and match it. The main features of the li

Anton Ohorodnyk 25 Aug 24, 2021
🏆 A decentralized layer to support NFT on Mixin Messenger and Kernel.

NFO A decentralized layer to support NFT on Mixin Kernel. This MTG sends back an NFT to the receiver whenever it receives a transaction with valid min

Mixin Network 11 Nov 30, 2021
Golang CS:GO external base. Development currently halted due to compiler/runtime Golang bugs.

gogo Golang CS:GO External cheat/base. Also, my first Golang project. Wait! Development momentarily halted due to compiler/runtime bugs. Disclaimer Th

cristei 3 Nov 14, 2021
Belajar Golang Install Golang

Golang belajar Golang Install Golang = download di https://golang.org/dl/ = pilih yg Zip = extract file zipnya = buka foldernya - copy folder go = pas

Arif Fadilah 1 Nov 15, 2021
Minimalistic, pluggable Golang evloop/timer handler with dependency-injection

Anagent Minimalistic, pluggable Golang evloop/timer handler with dependency-injection - based on codegangsta/inject - go-macaron/inject and chuckpresl

Ettore Di Giacinto 13 Jun 22, 2021
GoLang Library for Browser Capabilities Project

Browser Capabilities GoLang Project PHP has get_browser() function which tells what the user's browser is capable of. You can check original documenta

Maksim N. 39 Nov 13, 2021
Golang counters for readers/writers

Datacounter Golang counters for readers/writers. Examples ReaderCounter buf := bytes.Buffer{} buf.Write(data) counter := datacounter.NewReaderCounter(

Artem Andreenko 37 Nov 15, 2021
Golang beautify data display for Humans

Golang beautify data display for Humans English 简体中文 Install # Stable version go get -u -v gopkg.in/ffmt.v1 # Latest version go get -u -v github.com/

ffmt 244 Nov 19, 2021
a generic object pool for golang

Go Commons Pool The Go Commons Pool is a generic object pool for Golang, direct rewrite from Apache Commons Pool. Features Support custom PooledObject

jolestar 996 Dec 6, 2021
Resiliency patterns for golang

go-resiliency Resiliency patterns for golang. Based in part on Hystrix, Semian, and others. Currently implemented patterns include: circuit-breaker (i

Evan Huus 1.3k Dec 6, 2021
psutil for golang

gopsutil: psutil for golang This is a port of psutil (https://github.com/giampaolo/psutil). The challenge is porting all psutil functions on some arch

shirou 7k Nov 28, 2021