@tstromberg @cpanato I have added a new "out" flag Usage: --out JSON or --out CSV default: CSV

I have noticed that in cmd/review, cmd/prs, cmd/issues, cmd/issues-comments files, we are calling the Marshall function. So instead of redundantly checking for JSON/CSV output in each of these files, I have created a print package under pkg/print where we handle whether to print JSON/CSV. We can now call this function from all of those folders mentioned above.

Example: go run pullsheet.go prs --repos kubernetes/minikube --since 2021-06-10 --out JSON --token-path > prs.json

fixes #35.

Previously, the primary logging library was logrus. However, triage-party (one of our dependencies) used klog. Since this is more standard for kubernetes, this switches us to klog. Some minor behavior has changed, for example the few lines using logrus.Debugf have been replaced with klog.Infof.

Note: We still implicitly depend on logrus through github.com/spf13/viper. In my use case, no log messages were produced by logrus though. We likely could remove viper since its only purpose is allowing variables to be defaulted to environment variables (which is a very limited usage).

@tstromberg

This is a refactor to make it a bit easier for future changes. If you like this I will continue on that and make it cleaner and remove some duplicate code. Just in this way now to open this WIP PR to see what do you think.

But instead of having the --kind we split in subcommand and then we can have this

$ ./pullsheet
pullsheet - Generate spreadsheets based on GitHub contributions

pullsheet generates a CSV (comma separated values) & HTML output about GitHub activity across a series of repositories.

Usage:
  pullsheet [command]

Available Commands:
  help           Help about any command
  issue-comments Generate data around issues
  issues         Generate data around issues
  leaderboard    Generate leaderboard data
  prs            Generate data around pull requests
  reviews        Generate data around reviews

Flags:
  -h, --help                help for pullsheet
      --log-level string    the logging verbosity, either 'panic', 'fatal', 'error', 'warning', 'info', 'debug', 'trace' (default "info")
      --repos strings       comma-delimited list of repositories. ex: kubernetes/minikube, google/pullsheet
      --since string        when to query from
      --title string        Title to use for output pages
      --token-path string   GitHub token path
      --until string        when to query till
      --users strings       comma-delimiited list of users

Use "pullsheet [command] --help" for more information about a command.

and using cobra we have some handy helpers 😄

please let me know your thoughts, if you disagree feel free to close it. thanks!

Changes:

• Replaced leftover instances of k8s.io/klog with k8s.io/klog/v2
  • Resulted in k8s.io/klog module being removed from project
• Added support for Go 1.18
  • Done by running go get -u golang.org/x/sys
• Added hide-command flag to leaderboard command
  • Hides the Command-line line from the HTML output specifying what args were passed in
  • This is useful if the output is going to be public
• Added json-output flag to leaderboard command
  • This will write the output in JSON to the file specified
  • Will skip writing JSON if flag is empty (default)
• Added json-files flag to leaderboard command
  • Will append data from previously generated JSON files (--json-output) to new JSON and HTML output
    • ie. Assuming your have a JSON file already containing data from January - March, you can query April from GitHub and then append the previous months, resulting in a leaderboard from January - April
  • This is useful if you want to update the results say every month, you can save the previous data in JSON and then combine it with the new data and generate an updated HTML
• Added since-display & until-display flags to leaderboard command
  • These flags will override the since and until flag values that are listed as the date range on the leaderboard
  • This is so if appending previous data the date will match the true data
  • This does not modify the query dates for GitHub, just the display on the HTML

The output of the following command will yield in no data for some of the boards which is not handled by google visualizations.

go run pullsheet.go leaderboard --repos google/pullsheet --since 2021-05-06 --token-path gh_token.txt > test.html

The visualization in browser gives the error Data column(s) for axis #0 cannot be of type string

Line 102 in test.html should look like this [{label:'',type:'string'},{label: '# of Merged PRs reviewed', type: 'number'}, { role: 'annotation' }], but curretnly looks like this ['', '# of Merged PRs reviewed', { role: 'annotation' }],

There are now 2 ways to handle giving pullsheet a github token. The original way is to use the --token-path flag or PULLSHEET_TOKEN-PATH environment variable, which will cause pullsheet to open that file and use its contents as the token. The new way is to use the PULLSHEET_TOKEN environment variable, which directly takes the contents of the token and uses that.

…the service outside of the project repo

I did the go get github.com/google/pullsheet to install the binary in my path and then I run the pullsheet outside of the project repo and got an error:

F0226 16:11:54.215001 26719 pullsheet.go:157] generate failed: parsefiles: open pkg/leaderboard/leaderboard.tmpl: no such file or directory

This PR address this by making a const var to store the template. Maybe in the future, we can pass it as a flag to a custom template

/assign @tstromberg

This adds the --no-caching flag to disable caching of the HTML files produced.

Excluding "--no-caching":

<html>
<head>
    <title>kubernetes/minikube - Leaderboard</title>
                
    <link rel="preconnect" href="https://fonts.gstatic.com">
    <link href="https://fonts.googleapis.com/css2?family=Open+Sans:wght@300;400;600;700&display=swap" rel="stylesheet">
    <script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>
    <script type="text/javascript">
...

Including "--no-caching":

<html>
<head>
    <title>kubernetes/minikube - Leaderboard</title>
                
    <meta http-equiv="CacheControl" content="no-cache, no-store, must-revalidate"/>
    <meta http-equiv="Pragma" content="no-cache"/>
    <meta http-equiv="Expires" content="0"/>
                
    <link rel="preconnect" href="https://fonts.gstatic.com">
    <link href="https://fonts.googleapis.com/css2?family=Open+Sans:wght@300;400;600;700&display=swap" rel="stylesheet">
    <script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>
    <script type="text/javascript">
...

after this PR you can include bots with --include-bots

for example:

go run pullsheet.go prs --include-bots --token-path ~/token.txt --repos kubernetes/minikube,medyagh/gopogh,GoogleContainerTools/gcp-auth-webhook,GoogleContainerTools/minikube-image-benchmark --since 2021-04-01 --until 2021-08-23 --logtostderr=false  >> ~/Desktop/perf_prs.csv

• to prevent from out of index error

for now, if prs or issues are empty, out of index error occured by https://github.com/google/pullsheet/blob/585be57d72bd42b4a3843ff0a8f3d3ee55a7aed3/pkg/repo/issue.go#L90 and https://github.com/google/pullsheet/blob/585be57d72bd42b4a3843ff0a8f3d3ee55a7aed3/pkg/repo/pr.go#L72

Currently we use logrus. But one of our dependencies is triage-party which uses klog. We should prefer to use klog anyway as it is more in line with the rest of our ecosystem.

in command line I am passing since 04/01/2021 until 08/23/2021

but I see

go run pullsheet.go leaderboard --token-path ~/token.txt --repos kubernetes/minikube,medyagh/gopogh,GoogleContainerTools/gcp-auth-webhook,GoogleContainerTools/minikube-image-benchmark since 04/01/2021 until 08/23/2021 --logtostderr=false --stderrthreshold=2 | sed -r -e "/Command\-line/,/pullsheet/d" >> ~/Desktop/perf_21_aug_2021.html

Add web server mode that serves periodically leaderboard/CSV exports. Would use the command refactor introduced by #14

This could ideally support a multi-tenant YAML configuration approach, but happy to split that into a second issue.

Maybe we can add GitLab as well as an option to generate the same thing and in the command line can be passed something like

pullsheet --provider gitlab ...
or 
pullsheet --provider github ...