Generate spreadsheets based on GitHub contributions

Related tags

Miscellaneous github
Overview

pullsheet

pullsheet generates a CSV (comma separated values) & HTML output about GitHub activity across a series of repositories.

It currently supports CSV exports for:

  • Merged Pull Requests: --kind=prs (default)
  • Pull Request Reviews: --kind=reviews
  • Opening/Closing Issues: --kind=issues
  • Issue Comments: --kind=issue-comments

As well as a new HTML leaderboard mode: --kind=leaderboard

This tool was created as a brain-tickler for what PR's to discuss when asking for that big promotion.

Usage

go run pullsheet --repos --since 2006-01-02 --token-path [--users=]

You will need a GitHub authentication token from https://github.com/settings/tokens

Example: Merged PRs for 1 person across repos

go run pullsheet.go --repos kubernetes/minikube,GoogleContainerTools/skaffold --since 2019-10-01 --token-path /path/to/github/token/file --user someone > someone.csv

Example: Merged PR Reviews for all users in a repo

go run pullsheet.go --repos kubernetes/minikube --kind=reviews --since 2020-12-24 --token-path /path/to/github/token/file > reviews.csv

CSV fields

Merged Pull Requests

	URL         string
	Date        string
	User        string
	Project     string
	Type        string
	Title       string
	Delta       int
	Added       int
	Deleted     int
	FilesTotal  int
	Files       string // newline delimited
	Description string

Merged Pull Request Reviews

	URL            string
	Date           string
	Reviewer       string
	PRAuthor       string
	Project        string
	Title          string
	PRComments     int
	ReviewComments int
	Words          int

Closed/Opened Issues

	URL     string
	Date    string
	Author  string
	Closer  string
	Project string
	Type    string
	Title   string

Issue Comments

	URL         string
	Date        string
	Project     string
	Commenter   string
	IssueAuthor string
	IssueState  string
	Comments    int
	Words       int
	Title       string
Issues
  • Changes to enable JSON output | FIX for ISSUE_5

    Changes to enable JSON output | FIX for ISSUE_5

    @tstromberg @cpanato I have added a new "out" flag Usage: --out JSON or --out CSV default: CSV

    I have noticed that in cmd/review, cmd/prs, cmd/issues, cmd/issues-comments files, we are calling the Marshall function. So instead of redundantly checking for JSON/CSV output in each of these files, I have created a print package under pkg/print where we handle whether to print JSON/CSV. We can now call this function from all of those folders mentioned above.

    Example: go run pullsheet.go prs --repos kubernetes/minikube --since 2021-06-10 --out JSON --token-path > prs.json

    opened by bharathk005 8
  • Switch logging library from logrus to klog.

    Switch logging library from logrus to klog.

    fixes #35.

    Previously, the primary logging library was logrus. However, triage-party (one of our dependencies) used klog. Since this is more standard for kubernetes, this switches us to klog. Some minor behavior has changed, for example the few lines using logrus.Debugf have been replaced with klog.Infof.

    Note: We still implicitly depend on logrus through github.com/spf13/viper. In my use case, no log messages were produced by logrus though. We likely could remove viper since its only purpose is allowing variables to be defaulted to environment variables (which is a very limited usage).

    opened by andriyDev 6
  • refactor: using cobra and create subcommands

    refactor: using cobra and create subcommands

    @tstromberg

    This is a refactor to make it a bit easier for future changes. If you like this I will continue on that and make it cleaner and remove some duplicate code. Just in this way now to open this WIP PR to see what do you think.

    But instead of having the --kind we split in subcommand and then we can have this

    $ ./pullsheet
    pullsheet - Generate spreadsheets based on GitHub contributions
    
    pullsheet generates a CSV (comma separated values) & HTML output about GitHub activity across a series of repositories.
    
    Usage:
      pullsheet [command]
    
    Available Commands:
      help           Help about any command
      issue-comments Generate data around issues
      issues         Generate data around issues
      leaderboard    Generate leaderboard data
      prs            Generate data around pull requests
      reviews        Generate data around reviews
    
    Flags:
      -h, --help                help for pullsheet
          --log-level string    the logging verbosity, either 'panic', 'fatal', 'error', 'warning', 'info', 'debug', 'trace' (default "info")
          --repos strings       comma-delimited list of repositories. ex: kubernetes/minikube, google/pullsheet
          --since string        when to query from
          --title string        Title to use for output pages
          --token-path string   GitHub token path
          --until string        when to query till
          --users strings       comma-delimiited list of users
    
    Use "pullsheet [command] --help" for more information about a command.
    

    and using cobra we have some handy helpers 😄

    please let me know your thoughts, if you disagree feel free to close it. thanks!

    opened by cpanato 3
  • add JSON support for the leaderboard command

    add JSON support for the leaderboard command

    Changes:

    • Replaced leftover instances of k8s.io/klog with k8s.io/klog/v2
      • Resulted in k8s.io/klog module being removed from project
    • Added support for Go 1.18
      • Done by running go get -u golang.org/x/sys
    • Added hide-command flag to leaderboard command
      • Hides the Command-line line from the HTML output specifying what args were passed in
      • This is useful if the output is going to be public
    • Added json-output flag to leaderboard command
      • This will write the output in JSON to the file specified
      • Will skip writing JSON if flag is empty (default)
    • Added json-files flag to leaderboard command
      • Will append data from previously generated JSON files (--json-output) to new JSON and HTML output
        • ie. Assuming your have a JSON file already containing data from January - March, you can query April from GitHub and then append the previous months, resulting in a leaderboard from January - April
      • This is useful if you want to update the results say every month, you can save the previous data in JSON and then combine it with the new data and generate an updated HTML
    • Added since-display & until-display flags to leaderboard command
      • These flags will override the since and until flag values that are listed as the date range on the leaderboard
      • This is so if appending previous data the date will match the true data
      • This does not modify the query dates for GitHub, just the display on the HTML
    opened by spowelljr 1
  • HTML Output shows Data column(s) for axis #0 cannot be of type string in google visualization when there is no rows but the column is present

    HTML Output shows Data column(s) for axis #0 cannot be of type string in google visualization when there is no rows but the column is present

    The output of the following command will yield in no data for some of the boards which is not handled by google visualizations.

    go run pullsheet.go leaderboard --repos google/pullsheet --since 2021-05-06 --token-path gh_token.txt > test.html

    The visualization in browser gives the error Data column(s) for axis #0 cannot be of type string

    Line 102 in test.html should look like this [{label:'',type:'string'},{label: '# of Merged PRs reviewed', type: 'number'}, { role: 'annotation' }], but curretnly looks like this ['', '# of Merged PRs reviewed', { role: 'annotation' }],

    opened by vibecoder 1
  • Change how tokens are handled

    Change how tokens are handled

    There are now 2 ways to handle giving pullsheet a github token. The original way is to use the --token-path flag or PULLSHEET_TOKEN-PATH environment variable, which will cause pullsheet to open that file and use its contents as the token. The new way is to use the PULLSHEET_TOKEN environment variable, which directly takes the contents of the token and uses that.

    opened by MarlonGamez 1
  • update template to be a const and avoid this dependency when running …

    update template to be a const and avoid this dependency when running …

    …the service outside of the project repo

    I did the go get github.com/google/pullsheet to install the binary in my path and then I run the pullsheet outside of the project repo and got an error:

    F0226 16:11:54.215001 26719 pullsheet.go:157] generate failed: parsefiles: open pkg/leaderboard/leaderboard.tmpl: no such file or directory

    This PR address this by making a const var to store the template. Maybe in the future, we can pass it as a flag to a custom template

    /assign @tstromberg

    opened by cpanato 1
  • Allow disabling caching for leaderboard pages `--no-caching`

    Allow disabling caching for leaderboard pages `--no-caching`

    This adds the --no-caching flag to disable caching of the HTML files produced.

    Excluding "--no-caching":

    <html>
    <head>
        <title>kubernetes/minikube - Leaderboard</title>
                    
        <link rel="preconnect" href="https://fonts.gstatic.com">
        <link href="https://fonts.googleapis.com/css2?family=Open+Sans:[email protected];400;600;700&display=swap" rel="stylesheet">
        <script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>
        <script type="text/javascript">
    ...
    

    Including "--no-caching":

    <html>
    <head>
        <title>kubernetes/minikube - Leaderboard</title>
                    
        <meta http-equiv="CacheControl" content="no-cache, no-store, must-revalidate"/>
        <meta http-equiv="Pragma" content="no-cache"/>
        <meta http-equiv="Expires" content="0"/>
                    
        <link rel="preconnect" href="https://fonts.gstatic.com">
        <link href="https://fonts.googleapis.com/css2?family=Open+Sans:[email protected];400;600;700&display=swap" rel="stylesheet">
        <script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>
        <script type="text/javascript">
    ...
    
    opened by andriyDev 0
  • Add option to include bots through `--include-bots` flag

    Add option to include bots through `--include-bots` flag

    after this PR you can include bots with --include-bots

    for example:

    go run pullsheet.go prs --include-bots --token-path ~/token.txt --repos kubernetes/minikube,medyagh/gopogh,GoogleContainerTools/gcp-auth-webhook,GoogleContainerTools/minikube-image-benchmark --since 2021-04-01 --until 2021-08-23 --logtostderr=false  >> ~/Desktop/perf_prs.csv
    
    opened by medyagh 0
  • bugfix if number of issue or pr is zero

    bugfix if number of issue or pr is zero

    • to prevent from out of index error

    for now, if prs or issues are empty, out of index error occured by https://github.com/google/pullsheet/blob/585be57d72bd42b4a3843ff0a8f3d3ee55a7aed3/pkg/repo/issue.go#L90 and https://github.com/google/pullsheet/blob/585be57d72bd42b4a3843ff0a8f3d3ee55a7aed3/pkg/repo/pr.go#L72

    opened by anencore94 0
  • Switch to klog

    Switch to klog

    Currently we use logrus. But one of our dependencies is triage-party which uses klog. We should prefer to use klog anyway as it is more in line with the rest of our ecosystem.

    opened by andriyDev 0
  • wrong date range in leaderboard

    wrong date range in leaderboard

    in command line I am passing since 04/01/2021 until 08/23/2021

    but I see Screen Shot 2021-08-23 at 2 38 18 PM

    go run pullsheet.go leaderboard --token-path ~/token.txt --repos kubernetes/minikube,medyagh/gopogh,GoogleContainerTools/gcp-auth-webhook,GoogleContainerTools/minikube-image-benchmark since 04/01/2021 until 08/23/2021 --logtostderr=false --stderrthreshold=2 | sed -r -e "/Command\-line/,/pullsheet/d" >> ~/Desktop/perf_21_aug_2021.html
    
    
    opened by medyagh 1
  • Add server mode

    Add server mode

    Add web server mode that serves periodically leaderboard/CSV exports. Would use the command refactor introduced by #14

    This could ideally support a multi-tenant YAML configuration approach, but happy to split that into a second issue.

    opened by tstromberg 0
  • Add GitLab as extra provider

    Add GitLab as extra provider

    Maybe we can add GitLab as well as an option to generate the same thing and in the command line can be passed something like

    pullsheet --provider gitlab ...
    or 
    pullsheet --provider github ...
    
    
    opened by cpanato 2
Releases(v0.1.0)
Owner
Google
Google ❤️ Open Source
Google
HTTP service to generate PDF from Json requests

pdfgen HTTP service to generate PDF from Json requests Install and run The recommended method is to use the docker container by mounting your template

Hyperboloide 57 Apr 14, 2022
Automatically generate Go test boilerplate from your source code.

gotests gotests makes writing Go tests easy. It's a Golang commandline tool that generates table driven tests based on its target source files' functi

Charles Weill 4k Jun 21, 2022
generate fake data in go

Faker for Go Usage package main import ( "github.com/manveru/faker" ) func main() { fake, err := faker.New("en") if err != nil { panic(err

Michael Fellinger 164 Jun 20, 2022
:runner:runs go generate recursively on a specified path or environment variable and can filter by regex

Package generate Package generate runs go generate recursively on a specified path or environment variable like $GOPATH and can filter by regex Why wo

Go Playgound 26 Jun 6, 2022
A command line tool to generate sequence diagrams

goseq - text based sequence diagrams A small command line utility used to generate UML sequence diagrams from a text-base definition file. Inspired by

Leon Mika 179 Jun 22, 2022
Generate type-safe Go converters by simply defining an interface

goverter a "type-safe Go converter" generator goverter is a tool for creating type-safe converters. All you have to do is create an interface and exec

Jannis Mattheis 123 Jun 20, 2022
Generate random, pronounceable, sometimes even memorable, "superhero like" codenames - just like Docker does with container names.

Codename an RFC1178 implementation to generate pronounceable, sometimes even memorable, "superheroe like" codenames, consisting of a random combinatio

Luca Sepe 81 May 12, 2022
An application that is developed to generate application by API specification

GO boilerplate is an application that is developed to generate application by API specification and Database schema with the collaboration with opn-generator.

Rafi Mahmud 0 Oct 14, 2021
A tool to generate Pulumi Package schemas from Go type definitions

MkSchema A tool to generate Pulumi Package schemas from Go type definitions. This tool translates annotated Go files into Pulumi component schema meta

Joe Duffy 2 Jun 5, 2022
Generate FIRST/FOLLOW/PREDICT Set from BNF.

Generate FIRST/FOLLOW/PREDICT Set from BNF. We can use it to study parser theory. Feature FirstSet generate. Output pretty. FollowSet generate. Output

imtf 1 Oct 30, 2021
Generate droppers with encrypted payloads automatically.

This tool started out as a simple Python script. After discovering Python just couldn't cut it for my intended use I decided to learn and move to Golang. So far I'm quite happy with that decision.

ByteJunkies 47 Jan 17, 2022
Program to generate ruins using the Numenera Ruin Mapping Engine

Ruin Generator This is my attempt to build a program to generate ruins for Numenera using the rules from the Jade Colossus splatbook. The output only

Sean Hagen 0 Nov 7, 2021
Library to generate TOTP/HOTP codes

otpgen Library to generate TOTP/HOTP codes Installation go get -u github.com/grijul/otpgen Usage Here is a sample demonstration package main import (

Rijul Gulati 120 Jun 15, 2022
Go library for generate serial numbers according to rules.

go-sn - Serial Number Generator Go library for generate serial numbers according to rules. This library is also useful for generating serial numbers i

Kenkyu 5 May 25, 2022
Generate possible AD usernames from names like John Doe > J.Doe or JDoe

GOtusernames Generate possible AD usernames from names like John Doe > J.Doe or JDoe Example user file: ~/programming/golang/src/gotyourusername ❯ cat

null 4 Jan 4, 2022
Quickly collect data from thousands of exposed Elasticsearch or Kibana instances and generate a report to be reviewed.

elasticpwn Quickly collects data from exposed Elasticsearch or Kibana instances and generates a report to be reviewed. It mainly aims for sensitive da

Joel M 17 Jun 27, 2022
gal - generate authors file from git log

[日本語] gal - generate authors file from git log gal command generate AUTHORS.md file at current directory. gal command gets the author name and email a

CHIKAMATSU Naohiro 0 Feb 23, 2022
Script to generate a web page for your Aliucord plugins repo.

Aliucord-Store Script used to generate a website front-end for your plugins. Usage: go run cmds/store/main.go -dir string Your repository's

Zoey 3 Jan 31, 2022
Go package and associated command line utility to generate random yet human-readable names and identifiers

namegen | What's this? Go package and associated command line utility to generate random yet human-readable names and identifiers. Somewhat inspired b

Anand Varma 3 Jun 8, 2022