Explore Docker registries and manipulate Docker images!

Overview

CircleCI Go Report Card Coverage Status

L/S tags

Utility and API to manipulate (analyze, synchronize and aggregate) images across different Docker registries.

Example invocation

$ lstags alpine~/^3\\./
<STATE>      <DIGEST>                                   <(local) ID>    <Created At>          <TAG>
ABSENT       sha256:9363d03ef12c8c25a2def8551e609f146   n/a             2017-09-13T16:32:00   alpine:3.1
CHANGED      sha256:9866438860a1b28cd9f0c944e42d3f6cd   39be345c901f    2017-09-13T16:32:05   alpine:3.2
ABSENT       sha256:ae4d16d132e3c93dd09aec45e4c13e9d7   n/a             2017-09-13T16:32:10   alpine:3.3
CHANGED      sha256:0d82f2f4b464452aac758c77debfff138   f64255f97787    2017-09-13T16:32:15   alpine:3.4
PRESENT      sha256:129a7f8c0fae8c3251a8df9370577d9d6   074d602a59d7    2017-09-13T16:32:20   alpine:3.5
PRESENT      sha256:f006ecbb824d87947d0b51ab8488634bf   76da55c8019d    2017-09-13T16:32:26   alpine:3.6

NB! You can specify many images to operate on, e.g: lstags nginx~/^1\\.13/ mesosphere/chronos alpine~/^3\\./

Why would someone use this?

You could use lstags, if you ...

  • ... aggregate images from different external registries into your own registry for speed and locality reasons.
  • ... compare images present locally with the registry ones (e.g.: know if image tagged "latest" was re-pushed).
  • ... continuously pull Docker images from some public or private registry to speed-up Docker run on your system.

How?

... pull Ubuntu 14.04 & 16.04, all the Alpine images and Debian "stretch" to have the latest software to play with:

lstags --pull ubuntu~/^1[46]\\.04$/ alpine debian~/stretch/

... pull and re-push CoreOS-related images from quay.io to your own registry (in case these hipsters will break everything):

lstags -P /quay -r registry.company.io quay.io/coreos/hyperkube quay.io/coreos/flannel

NB! In case you use private registry with authentication, make sure your Docker client knows how to authenticate against it! lstags will reuse credentials saved by Docker client in its config.json file, one usually found at ~/.docker/config.json

Possible image states

lstags distinguishes five states of Docker image:

  • ABSENT - present in registry, but absent locally
  • PRESENT - present in registry, present locally, with local and remote digests being equal
  • CHANGED - present in registry, present locally, but with different local and remote digests
  • LOCAL_ONLY - present locally, absent in registry
  • NOT_FOUND - absent in registry, absent locally, probably does not exist at all

Authentication

You can either:

  • rely on lstags discovering credentials "automagically" 🎩
  • load credentials from any Docker JSON config file specified

Assume tags

Sometimes registry may contain tags not exposed to any kind of search though still existing. lstags is unable to discover these tags, but if you need to pull or push them, you may "assume" they exist and make lstags blindly try to pull these tags from the registry. To inject assumed tags into the registry query you need to extend repository specification with a = followed by a comma-separated list of tags you want to assume.

e.g. we assume tags v1.6.1 and v1.7.0 exist like this: lstags quay.io/calico/cni=v1.6.1,v1.7.0

Repository specification

Full repository specification looks like this:

[REGISTRY[:PORT]/]REPOSITORY[~/FILTER_REGEXP/][=TAG1,TAG2,TAGn]

You may provide infinite number of repository specifications to lstags

Push prefix

When you [re]push images to your "push" registry, you can control the destination repository path prefix:

  • by default, repository path prefix will be auto-generated from the source registry hostname, e.g.:
    • alpine ▶️ /registry/hub/docker/com/
    • localhost:5000/nginx ▶️ /localhost/
    • registry.company.com/hype/kubernetes ▶️ /registry/company/com/
  • passing --push-prefix=/ will push images "as is", with no additional repository path prefix
  • passing --push-prefix=/my/prefix/ will push images appending /my/prefix/ to the repository path
  • specifying /my/prefix without trailing slash is OK, as long as path would still be formatted correctly by API ✨
  • passing --push-prefix="" would trigger "default" behavior with prefix being auto-generated

To fail or not to fail?

By default application exits after encountering any errors. To make it more tolerant to subsequent failures, you may use CLI option -N, --do-not-fail or set environment variable DO_NOT_FAIL=true before running application. HINT: Option -d, --daemon-mode always implies activation of --do-not-fail.

YAML

💡 You can load repositories from the YAML file just like you do it from the command line arguments:

lstags -f file.yaml

A valid YAML file looks like this (mandatory lstags root key is here to be able to use "shared" YAMLs):

lstags:
  repositories:
    - busybox
    - nginx:stable
    - mesosphere/marathon-lb~/^v1/
    - quay.io/coreos/awscli=master,latest,edge
    - gcr.io/google-containers/hyperkube~/^v1\.(9|10)\./

NB! lstags can load repositories from YAML or from CLI args, but not from both at the same time!

Install: Binaries

https://github.com/ivanilves/lstags/releases

Install: Wrapper

git clone [email protected]:ivanilves/lstags.git
cd lstags
sudo make wrapper
lstags -h

A special wrapper script will be installed to manage lstags invocation and updates. 😎

Install: From source

git clone [email protected]:ivanilves/lstags.git
cd lstags
dep ensure
go build
./lstags -h

NB! I assume you have current versions of Go & dep installed and also have set up GOPATH correctly.

Using it with Docker

docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock ivanilves/lstags
Usage:
  lstags [OPTIONS] REPO1 REPO2 REPOn...

Application Options:
  -j, --docker-json=          JSON file with credentials (default:
                              ~/.docker/config.json) [$DOCKER_JSON]
  -p, --pull                  Pull Docker images matched by filter (will use
                              local Docker deamon) [$PULL]
  -P, --push                  Push Docker images matched by filter to some
                              registry (See 'push-registry') [$PUSH]
  -r, --push-registry=        [Re]Push pulled images to a specified remote
                              registry [$PUSH_REGISTRY]

--- OUTPUT WAS CUT HERE TO SAVE SPACE ---

Analyze an image

docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock ivanilves/lstags alpine~/^3\\./
ANALYZE alpine
FETCHED alpine
-
<STATE>   <DIGEST>                                  <(local) ID>    <Created At>            <TAG>
CHANGED   sha256:b40e202395eaec699f2d0c5e01e6d6cb8  76da55c8019d    2017-10-25T23:19:51Z    alpine:3.6
ABSENT    sha256:d95da16498d5d6fb4b907cbe013f95032  n/a             2017-10-25T23:20:18Z    alpine:3.1
ABSENT    sha256:cb275b62f789b211114f28b391fca3cc2  n/a             2017-10-25T23:20:32Z    alpine:3.2
ABSENT    sha256:27af7da847283a947c008592f2b2cd6d2  n/a             2017-10-25T23:20:45Z    alpine:3.3
CHANGED   sha256:246bbbaa81b28837b64cb9dfc574de958  1a19a71e5d38    2017-10-25T23:20:59Z    alpine:3.4
CHANGED   sha256:aa96c8dc3815c44d4aceaf1ee7903ce58  37c7be7a096b    2017-10-25T23:21:13Z    alpine:3.5
-

Development

You are very welcome to open pull requests to this repository! 😉

⚠️ CI build will fail, if your commit messages are not semantic!

To maximize our collaboration efficiency we would humbly ask you to follow these recommendations:

  • Please add reasonable description (what?/why?/etc) to your pull request ❗
  • Your code should pass CI (CircleCI) and a [pretty liberal] code review 🔍
  • If code adds or changes some logic, it should be covered by a unit test :neckbeard:
  • Please, put meaningful and semantic messages on your commits 🙏

NB! Not a requirement, but a GIF included in PR description would make our world a happier place!

'NORELEASE' branches and commits

We have automatic release system. Every PR merge will create a new application release with a changelog generated from PR branch commits. For the most cases it is OK. However, if you work with things that do not need to be released (e.g. non user-facing changes), you have following options:

  • If you don't want to create release from your PR, make it from branch containing "NORELEASE" keyword in its name.
  • If you want to prevent single commit from appearing in a changelog, please start commit message with "NORELEASE".

⚠️ We don't build RPMs/DEBs/etc, as we see no need for it. We ship lstags as a single binary or as a Docker container.

API

You may use lstags either as a standalone CLI or as a Golang package inside your own application.

Set up and build PoC application with our v1 API:

make poc-app APP_PATH=../lstags-api
cd ../lstags-api
go build
# run "./lstags-api" binary to see PoC in action (examine main.go first to ensure no "rm -rf /" is there)
  • This installs all necessary dependencies and sets up PoC application at the path ../lstags-api/
  • We assume you already have recent Golang version installed on your system https://golang.org/dl/

GoDoc

NB! Far more complete API usage example could be found in main.go 😉

Issues
  • Bad response status: 401 Unauthorized

    Bad response status: 401 Unauthorized

    Hello, When I do a docker pull from our repos using docker pull everything works fine. When using lstags docker the pull doesnt work. Please help.

    [email protected] config]# docker pull gcr.io/anyvision-training/templates Using default tag: latest Trying to pull repository gcr.io/anyvision-training/templates ... latest: Pulling from gcr.io/anyvision-training/templates Digest: sha256:7fc087838e4da276debc2b20ba57c865d25192a39597b7dcc572c9fd396f97f9 Status: Image is up to date for gcr.io/anyvision-training/templates:latest

    [email protected] config]# docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock ivanilves/lstags -p gcr.io/anyvision-training/templates INFO[0000] ANALYZE gcr.io/anyvision-training/templates
    Bad response status: 401 Unauthorized >> https://gcr.io/v2/anyvision-training/templates/tags/list

    [email protected] lstags]# docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock ivanilves/lstags -j /root/.gcp/docker-registry-ro-training.json -p gcr.io/anyvision-training/templates open /root/.gcp/docker-registry-ro-training.json: no such file or directory

    [[email protected] lstags]# ls -ltr /root/.gcp/docker-registry-ro-training.json -rw-r--r-- 1 root root 2327 Oct 2 06:13 /root/.gcp/docker-registry-ro-training.json

    The xml file is good, it is used when using the plain docker command.

    Thanks,

    Meir

    bug critical 
    opened by meirhazonAnyVision 20
  • goroutine fatal when pushing big amount of images

    goroutine fatal when pushing big amount of images

    in similar context as https://github.com/ivanilves/lstags/issues/153, but different scenario, got goroutine fatal errors when pushing ~400 images to registry.

    the registry:5000 is insecure registry confured via env: INSECURE_REGISTRY_EX=registry:5000

    worked fine with 1 image with same configuration.

    $ lstags '--docker-json=/root/.docker/config.json' '--yaml-config=lstags.yml' --push '--push-prefix=/' '--push-registry=registry:5000'
    INFO[0000] BATCH 1 of 13
    fatal error: concurrent map writes
    
    goroutine 1346 [running]:
    runtime.throw(0x7f16c1, 0x15)
        /home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/panic.go:619 +0x81 fp=0xc4203d5d60 sp=0xc4203d5d40 pc=0x429951
    runtime.mapassign_faststr(0x77a200, 0xc42008b650, 0xc42030a1b0, 0x2f, 0xc420f91d40)
        /home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/hashmap_fast.go:703 +0x3e9 fp=0xc4203d5dd0 sp=0xc4203d5d60 pc=0x40bb39
    github.com/ivanilves/lstags/api/v1.(*API).CollectTags.func1(0xc4202c6320, 0xc42008b650, 0xc420d9a9c0, 0xc420e339e0)
        /home/travis/gopath/src/github.com/ivanilves/lstags/api/v1/v1.go:143 +0x609 fp=0xc4203d5fc0 sp=0xc4203d5dd0 pc=0x6f0709
    runtime.goexit()
        /home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc4203d5fc8 sp=0xc4203d5fc0 pc=0x455791
    created by github.com/ivanilves/lstags/api/v1.(*API).CollectTags
        /home/travis/gopath/src/github.com/ivanilves/lstags/api/v1/v1.go:120 +0x24f
    

    More complete output: push_b_1a.log

    bug critical handicap 
    opened by glensc 17
  • Problem with great amount of entries.

    Problem with great amount of entries.

    We have an other problem that happens with 1.2.7 till 1.2.11. We mirror a lot of images from Docker Hub and the yaml file with the images/tags is large. The problem was, that lstags stopped working after a specific amount of images/tags. We enabled trace logging but it does not show anything useful. The "batches" all work fine and the "state" is shown. But during "[PULL/PUSH] PUSHING" there is a moment when the time to the next "[PULL/PUSH] PUSHING" increases heavily from image to image until some image where lstags waits forever. We looked at the process and it is not crashed but it does not run anymore.

    We found a workaround by splitting the yaml file from Docker Hub but it seems that there is a problem when the amount of images, tags or size reaches some limit after that lstags stops working.

    We also found out that the lstags process does not terminate when run from gitlab-ci and terminated through pipeline timeout. We found hanging lstags processes on the runner. But sending them a regular sigterm works to to terminate them.

    help wanted 
    opened by nudgegoonies 14
  • Support Insecure docker registries

    Support Insecure docker registries

    lstags private-host/cassandra
    Get https://private-host/v2: http: server gave HTTP response to HTTPS client
    
    bug critical question 
    opened by msoedov 11
  • handicapped pull

    handicapped pull

    there's no progress update what is lstags doing, even with verbosity set to max.

    and there's bug that when one of the images specified in yaml/commandline is not pullable. the process never finishes. and i have no indication has it finished and idling due the bug, or actually doing the hard work of pulling.

    # ./lstags --version
    VERSION: v1.0.61
    # ./lstags --pull alpine:2  -vvv
    DEBU[0000] [New()] API config: {DockerJSONConfigFile:~/.docker/config.json ConcurrentRequests:32 TraceRequests:false RetryRequests:2 RetryDelay:30s InsecureRegistryEx: VerboseLogging:true}
    INFO[0000] BATCH 1 of 1
    DEBU[0000] [CollectTags()] references: [alpine:2]
    DEBU[0000] [CollectTags()] repository: &{ref:alpine:2 registry:registry.hub.docker.com fullRepo:registry.hub.docker.com/alpine repoTags:[2] filterRE:<nil> isSecure:true isSingle:true}
    INFO[0000] ANALYZE alpine:2
    DEBU[0001] [func1():alpine:2] remote tags: map[]
    DEBU[0001] [func1():alpine:2] local tags: map[]
    DEBU[0001] [func1():alpine:2] joined tags: map[2:0xc4200b4900]
    INFO[0001] FETCHED alpine:2
    DEBU[0001] [CollectTags()] tags: map[alpine:2:[0xc4200b4900]]
    -
    <STATE>      <DIGEST>                                      <(local) ID>    <Created At>              <IMAGE>:<TAG>
    ASSUMED      n/a                                           n/a             1970-01-01T03:00:00       alpine:2
    -
    DEBU[0001] [PullTags()] collection: &{refs:[alpine:2] repos:map[alpine:2:0xc4200b4ae0] tags:map[alpine:2:[0xc4200b4900]]} (1 repos / 1 tags)
    DEBU[0001] [PullTags()] repository: &{ref:alpine:2 registry:registry.hub.docker.com fullRepo:registry.hub.docker.com/alpine repoTags:[2] filterRE:<nil> isSecure:true isSingle:true}
    DEBU[0001] [PullTags()] tag: &{name:2 digest:n/a imageID:n/a state:ASSUMED created:0 containerID:}
    INFO[0001] PULLING alpine:2
    INFO[0001] PULLING alpine:2
    ^C
    
    # docker pull alpine:2
    Error response from daemon: manifest for alpine:2 not found
    
    critical handicap 
    opened by glensc 10
  • Unable to clear

    Unable to clear "push-prefix".

    i want to copy images from one registry to other registry without altering their prefix. i seem not to succeed on that.

    # lstags.yaml
    lstags:
      repositories:
        - gitlab.example.net:4567/ed/php~/^(5\.[36]|7\.[12])-/
    

    with --push-prefix=/ it creates url with double slash and gets auth failure.

    $ ./lstags --docker-json=$HOME/.docker/config.json --yaml-config=lstags.yaml  --push-registry=registry.gitlab.test.example.net --push-prefix=/
    INFO[0000] ANALYZE gitlab.example.net:4567/ed/php~/^(5\.[36]|7\.[12])-/
    INFO[0001] FETCHED gitlab.example.net:4567/ed/php~/^(5\.[36]|7\.[12])-/
    -
    <STATE>      <DIGEST>                                      <(local) ID>    <Created At>              <IMAGE>:<TAG>
    PRESENT      sha256:4c90cc845b6774236878e81a86099270d      518f80c5b6eb    2018-04-14T04:11:40       gitlab.example.net:4567/ed/php:5.3-cli
    PRESENT      sha256:f0f4fd471768435a9542bf70c2b015e51      2a6486ae26ab    2018-04-14T04:11:40       gitlab.example.net:4567/ed/php:5.6-cli
    PRESENT      sha256:cf95b3b716aea34d7121ac9b499b1d4ac      4770d7ee458d    2018-04-14T04:11:42       gitlab.example.net:4567/ed/php:7.1-cli
    PRESENT      sha256:a416582b3bd748ce8ecfcf289afbc9332      728c2d641db5    2018-04-14T04:13:01       gitlab.example.net:4567/ed/php:7.2-cli
    PRESENT      sha256:d97cd6f5603bfda42c00e6eb7b25cce6d      a71b22877f5c    2018-04-14T04:13:57       gitlab.example.net:4567/ed/php:5.3-fpm
    PRESENT      sha256:c523a8822c4207aa45c30d1f21bbb8738      a62dd9c0f1a8    2018-04-14T04:13:59       gitlab.example.net:4567/ed/php:5.6-fpm
    PRESENT      sha256:6e42a119faab447e8ca15e985e4c3f62a      036e59ff29da    2018-04-14T04:13:59       gitlab.example.net:4567/ed/php:7.1-fpm
    PRESENT      sha256:8613ce851ab4a2c25b3c6601e5c4f5ac6      575bb1c83b10    2018-04-14T04:14:41       gitlab.example.net:4567/ed/php:7.2-fpm
    PRESENT      sha256:ea4a8fb5f10eaffe016fb57b38016a9df      1708314e97dd    2018-04-14T04:15:48       gitlab.example.net:4567/ed/php:5.3-apache
    PRESENT      sha256:7a69db8b42b8f3c9e870fd63678d164e1      0ff1700cc718    2018-04-14T04:15:48       gitlab.example.net:4567/ed/php:5.6-apache
    PRESENT      sha256:c3c958966f6f11fca4ebe2123de1d59cd      b11ee7c8dc3e    2018-04-14T04:15:49       gitlab.example.net:4567/ed/php:7.1-apache
    PRESENT      sha256:59dd8d1a58f5946199113bf3714f2ae10      cdcffea8acad    2018-04-14T04:16:35       gitlab.example.net:4567/ed/php:7.2-apache
    -
    INFO[0001] [PULL/PUSH] ANALYZE gitlab.example.net:4567/ed/php~/^(5\.[36]|7\.[12])-/ => registry.gitlab.test.example.net//ed/php~/.*/
    Bad response status: 401 Unauthorized >> https://registry.gitlab.test.example.net/v2//ed/php/tags/list
    

    with --push-prefix= it seems to ignore argument, defaulting to some "dynamic prefix from pushed image" aka the default behavior:

    $ ./lstags --docker-json=$HOME/.docker/config.json --yaml-config=lstags.yaml  --push-registry=registry.gitlab.test.example.net --push-prefix=
    INFO[0000] ANALYZE gitlab.example.net:4567/ed/php~/^(5\.[36]|7\.[12])-/
    INFO[0001] FETCHED gitlab.example.net:4567/ed/php~/^(5\.[36]|7\.[12])-/
    -
    <STATE>      <DIGEST>                                      <(local) ID>    <Created At>              <IMAGE>:<TAG>
    PRESENT      sha256:4c90cc845b6774236878e81a86099270d      518f80c5b6eb    2018-04-14T04:11:40       gitlab.example.net:4567/ed/php:5.3-cli
    PRESENT      sha256:f0f4fd471768435a9542bf70c2b015e51      2a6486ae26ab    2018-04-14T04:11:40       gitlab.example.net:4567/ed/php:5.6-cli
    PRESENT      sha256:cf95b3b716aea34d7121ac9b499b1d4ac      4770d7ee458d    2018-04-14T04:11:42       gitlab.example.net:4567/ed/php:7.1-cli
    PRESENT      sha256:a416582b3bd748ce8ecfcf289afbc9332      728c2d641db5    2018-04-14T04:13:01       gitlab.example.net:4567/ed/php:7.2-cli
    PRESENT      sha256:d97cd6f5603bfda42c00e6eb7b25cce6d      a71b22877f5c    2018-04-14T04:13:57       gitlab.example.net:4567/ed/php:5.3-fpm
    PRESENT      sha256:c523a8822c4207aa45c30d1f21bbb8738      a62dd9c0f1a8    2018-04-14T04:13:59       gitlab.example.net:4567/ed/php:5.6-fpm
    PRESENT      sha256:6e42a119faab447e8ca15e985e4c3f62a      036e59ff29da    2018-04-14T04:13:59       gitlab.example.net:4567/ed/php:7.1-fpm
    PRESENT      sha256:8613ce851ab4a2c25b3c6601e5c4f5ac6      575bb1c83b10    2018-04-14T04:14:41       gitlab.example.net:4567/ed/php:7.2-fpm
    PRESENT      sha256:ea4a8fb5f10eaffe016fb57b38016a9df      1708314e97dd    2018-04-14T04:15:48       gitlab.example.net:4567/ed/php:5.3-apache
    PRESENT      sha256:7a69db8b42b8f3c9e870fd63678d164e1      0ff1700cc718    2018-04-14T04:15:48       gitlab.example.net:4567/ed/php:5.6-apache
    PRESENT      sha256:c3c958966f6f11fca4ebe2123de1d59cd      b11ee7c8dc3e    2018-04-14T04:15:49       gitlab.example.net:4567/ed/php:7.1-apache
    PRESENT      sha256:59dd8d1a58f5946199113bf3714f2ae10      cdcffea8acad    2018-04-14T04:16:35       gitlab.example.net:4567/ed/php:7.2-apache
    -
    INFO[0001] [PULL/PUSH] ANALYZE gitlab.example.net:4567/ed/php~/^(5\.[36]|7\.[12])-/ => registry.gitlab.test.example.net/gitlab/example.net/ed/php~/.*/
    Bad response status: 401 Unauthorized >> https://registry.gitlab.test.example.net/v2/gitlab/example.net/ed/php/tags/list
    
    bug handicap 
    opened by glensc 9
  • invalid image in spec skips any pulling

    invalid image in spec skips any pulling

    invalid, aka inexistent image. should not abort whole process. just do as much work as possible, report errors summary if needed.

    [email protected] ~/scm/docker# time ./lstags --pull alpine:2 alpine:3.3
    INFO[0000] BATCH 1 of 1
    INFO[0000] ANALYZE alpine:3.3
    INFO[0000] ANALYZE alpine:2
    INFO[0001] FETCHED alpine:2
    INFO[0002] FETCHED alpine:3.3
    -
    <STATE>      <DIGEST>                                      <(local) ID>    <Created At>              <IMAGE>:<TAG>
    ASSUMED      n/a                                           n/a             1970-01-01T03:00:00       alpine:2
    ABSENT       sha256:36b22b4ceb2128310ed800fdb28c52bf8      n/a             2018-01-09T23:12:20       alpine:3.3
    -
    INFO[0002] PULLING alpine:3.3
    INFO[0002] PULLING alpine:2
    Error response from daemon: manifest for alpine:2 not found
    
    real    0m3,681s
    user    0m0,137s
    sys     0m0,030s
    
    [email protected] ~/scm/docker# ./lstags --version
    VERSION: v1.0.63
    
    bug critical handicap 
    opened by glensc 7
  • Better world: First refactor of the lstags

    Better world: First refactor of the lstags

    We have reached the state when we need to:

    • stop adding new functionality for a while.
    • work more on automated testing, code quality and various fixes.
    • add performance, benefit more of concurrency Golang gives us [almost] for free.

    TODO

    • [x] Add black-box shell tests for "image pull" functionality.
    • [x] Fix lame ("double"?) help messages.
    • [x] Make registry and operations event more concurrent.
    • [x] Fix authenticated pull.

    GIF

    critical enhancement 
    opened by ivanilves 7
  • index out of range error

    index out of range error

    Just trying to get things up and running–

    $ lstags --version
    VERSION: v30-d15b77b
    $ lstags alpine~/^3\\./
    panic: runtime error: index out of range
    
    goroutine 1 [running]:
    github.com/ivanilves/lstags/docker/config.Load(0x1360289, 0x15, 0x0, 0x0, 0x0)
    	/home/travis/gopath/src/github.com/ivanilves/lstags/docker/config/config.go:79 +0x4fe
    main.main()
    	/home/travis/gopath/src/github.com/ivanilves/lstags/main.go:98 +0x8d
    
    possible bug 
    opened by ryanking 7
  • TCP option for the docker engine connection

    TCP option for the docker engine connection

    I think the idea from @solidnerd PR https://github.com/ivanilves/lstags/pull/92 should be implemented.

    critical enhancement handicap 
    opened by ivanilves 7
  • panic: value method ... bearer.Token.Method called using nil *Token pointer

    panic: value method ... bearer.Token.Method called using nil *Token pointer

    • I am using latest release 1.2.20 (built from sources on macOS 11.4 using go version go1.16.5 darwin/amd64)
    • However I saw the same happening with 1.2.18 on macOS and 1.2.20 downloaded from GitHub on Linux
    • I try to check the attached file but get the following (this does not happen all the times but on Linux it never succeeds):
    $ YAML_CONFIG=registry.gitlab.com.txt CONCURRENT_REQUESTS=16 WAIT_BETWEEN=0s RETRY_DELAY=1s RETRY_REQUESTS=2
    time="2021-07-09T15:23:37Z" level=info msg="BATCH 1 of 1"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/spotbugs=2,latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/flawfinder=2,latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/mobsf=latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/kubesec=2,latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/bandit=2,latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/phpcs-security-audit=2,latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/eslint=2,latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/brakeman=2,latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/nodejs-scan=2,latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/semgrep=2,latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/security-code-scan=2,latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/sobelow=2,latest"
    time="2021-07-09T15:23:37Z" level=info msg="ANALYZE registry.gitlab.com/gitlab-org/security-products/analyzers/pmd-apex=2,latest"
    panic: value method github.com/ivanilves/lstags/api/v1/registry/client/auth/bearer.Token.Method called using nil *Token pointer
    
    goroutine 254 [running]:
    github.com/ivanilves/lstags/api/v1/registry/client/auth/bearer.(*Token).Method(0x0, 0xc000790bc0, 0x4170de)
    	<autogenerated>:1 +0x49
    github.com/ivanilves/lstags/api/v1/registry/client.(*RegistryClient).repoToken(0xc00035fc70, 0xc0000be0f0, 0x2f, 0xc000790cb8, 0x40bc76, 0xc00038a660, 0x60)
    	/go/src/github.com/ivanilves/lstags/api/v1/registry/client/client.go:179 +0x47b
    github.com/ivanilves/lstags/api/v1/registry/client.(*RegistryClient).v1TagOptions(0xc00035fc70, 0xc0000be0f0, 0x2f, 0xc000142a94, 0x1, 0xc000142a94, 0x1, 0xc00038a600)
    	/go/src/github.com/ivanilves/lstags/api/v1/registry/client/client.go:333 +0x5d
    github.com/ivanilves/lstags/api/v1/registry/client.(*RegistryClient).Tag(0xc00035fc70, 0xc0000be0f0, 0x2f, 0xc000142a94, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    	/go/src/github.com/ivanilves/lstags/api/v1/registry/client/client.go:380 +0x130
    github.com/ivanilves/lstags/tag/remote.FetchTags.func1(0xc00035fc70, 0xc00024ff80, 0xc000142a94, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    	/go/src/github.com/ivanilves/lstags/tag/remote/remote.go:104 +0x8b
    created by github.com/ivanilves/lstags/tag/remote.FetchTags
    	/go/src/github.com/ivanilves/lstags/tag/remote/remote.go:98 +0x46f
    panic: value method github.com/ivanilves/lstags/api/v1/registry/client/auth/bearer.Token.Method called using nil *Token pointer
    
    goroutine 278 [running]:
    github.com/ivanilves/lstags/api/v1/registry/client/auth/bearer.(*Token).Method(0x0, 0xce5d20, 0x9d3da0)
    	<autogenerated>:1 +0x49
    github.com/ivanilves/lstags/api/v1/registry/client.(*RegistryClient).repoToken(0xc00035fc70, 0xc0000be0f0, 0x2f, 0xc000001680, 0xc00036cec0, 0x4051aa, 0xc00039c660)
    	/go/src/github.com/ivanilves/lstags/api/v1/registry/client/client.go:179 +0x47b
    github.com/ivanilves/lstags/api/v1/registry/client.(*RegistryClient).tagDigest(0xc00035fc70, 0xc0000be0f0, 0x2f, 0xc000142a94, 0x1, 0xc000412180, 0xc000318aa8, 0xc00036cfb0, 0x6c46c2)
    	/go/src/github.com/ivanilves/lstags/api/v1/registry/client/client.go:274 +0x5d
    github.com/ivanilves/lstags/api/v1/registry/client.(*RegistryClient).Tag.func1(0xc00035fc70, 0xc0000be0f0, 0x2f, 0xc000142a94, 0x1, 0xc00038a600, 0xc00038a660)
    	/go/src/github.com/ivanilves/lstags/api/v1/registry/client/client.go:371 +0x70
    created by github.com/ivanilves/lstags/api/v1/registry/client.(*RegistryClient).Tag
    	/go/src/github.com/ivanilves/lstags/api/v1/registry/client/client.go:370 +0xe0
    

    registry.gitlab.com.txt

    bug critical 
    opened by mfriedenhagen 1
  • bearer response should also allow for access_token

    bearer response should also allow for access_token

    via https://docs.docker.com/registry/spec/auth/token/#requesting-a-token

    token: An opaque Bearer token that clients should supply to subsequent requests in the Authorization header.

    access_token: For compatibility with OAuth 2.0, we will also accept token under the name access_token. At least one of these fields must be specified, but both may also appear (for compatibility with older clients). When both are specified, they should be equivalent; if they differ the client's choice is undefined.

    diff --git a/api/v1/registry/client/auth/bearer/bearer.go b/api/v1/registry/client/auth/bearer/bearer.go
    index 3a99170..fc1ea5b 100644
    --- a/api/v1/registry/client/auth/bearer/bearer.go
    +++ b/api/v1/registry/client/auth/bearer/bearer.go
    @@ -9,7 +9,8 @@ import (
     
     // Token implementation for Bearer authentication
     type Token struct {
    -       T string `json:"token"`
    +       T string `json:"token,omitempty"`
    +       A string `json:"access_token,omitempty"`
            E int    `json:"expires_in"`
     }
     
    @@ -20,7 +21,10 @@ func (tk Token) Method() string {
     
     // String form of Bearer token
     func (tk Token) String() string {
    -       return tk.T
    +       if tk.A == "" {
    +               return tk.T
    +       }
    +       return tk.A
     }
    

    bit dirty but it does the trick @ivanilves

    enhancement 
    opened by Deshke 1
  • Create missing repositories

    Create missing repositories

    I would be really great if this tool had the ability to create missing repositories, in our case this would require integrating with the ECR API. Currently I have to do this manually for each new repository I add to the configuration.

    enhancement feature 
    opened by cablespaghetti 1
  • 401 error while pushing since 1.2.11

    401 error while pushing since 1.2.11

    Since our update to lstags 1.2.11 we get 401 errors on pushing when images have changed.

    I turned on tracing and verbosity with 1.2.10 and 1.2.11 and copied the relevant sections for the image/tag that fails with 1.2.11. This is the working 1.2.10:

    b6874ea|@URL: https://docker.elastic.co/v2/kibana/kibana/tags/list
    b6874ea|@HEADER: Date                                     = [Thu, 09 Jan 2020 08:09:08 GMT]
    b6874ea|@HEADER: Docker-Distribution-Api-Version          = [registry/2.0]
    b6874ea|@HEADER: X-Content-Type-Options                   = [nosniff]
    b6874ea|@HEADER: Content-Length                           = [2278]
    b6874ea|@HEADER: Connection                               = [keep-alive]
    b6874ea|@HEADER: Content-Type                             = [application/json; charset=utf-8]
    b6874ea|--- BODY BEGIN ---
    b6874ea|{"name":"kibana/kibana","tags":["5.0.0-731e78df","5.0.0-86a0b164","5.0.0-alpha5","5.0.0-beta1","5.0.0-ccd69424","5.0.0-rc1","5.0.0","5.0.1-6d75f4de","5.0.1-c068af18","5.0.1","5.0.2","5.1.1-71988514","5.1.1","5.1.2","5.2.0-11b63c8d","5.2.0-6ee3d9ba","5.2.0-a9f63967","5.2.0","5.2.1","5.2.2","5.3.0-d5b30bd7","5.3.0","5.3.1","5.3.2","5.3.3","5.4.0","5.4.1","5.4.2","5.4.3","5.5.0-e210c18b","5.5.0","5.5.1","5.5.2","5.5.3","5.6.0","5.6.1","5.6.10","5.6.11","5.6.12","5.6.13","5.6.14","5.6.15","5.6.16","5.6.2","5.6.3","5.6.4-1fdad85","5.6.4","5.6.5","5.6.6","5.6.7","5.6.8","5.6.9","6.0.0-alpha1","6.0.0-alpha2","6.0.0-beta1","6.0.0-beta2-d2816397","6.0.0-beta2","6.0.0-rc1","6.0.0-rc2","6.0.0","6.0.1","6.1.0","6.1.1","6.1.2","6.1.3","6.1.4","6.2.0","6.2.1","6.2.2","6.2.3","6.2.4","6.3.0","6.3.1","6.3.2","6.3.3-SNAPSHOT","6.4-SNAPSHOT","6.4.0-SNAPSHOT","6.4.0","6.4.1-SNAPSHOT","6.4.1","6.4.2-SNAPSHOT","6.4.2","6.4.3-SNAPSHOT","6.4.3","6.4.4-SNAPSHOT","6.5-SNAPSHOT","6.5.0-SNAPSHOT","6.5.0","6.5.1-109","6.5.1-SNAPSHOT","6.5.1","6.5.2-SNAPSHOT","6.5.2","6.5.3-SNAPSHOT","6.5.3","6.5.4-SNAPSHOT","6.5.4","6.5.5-SNAPSHOT","6.6-SNAPSHOT","6.6.0-SNAPSHOT","6.6.0","6.6.1-SNAPSHOT","6.6.1","6.6.2-SNAPSHOT","6.6.2","6.6.3-SNAPSHOT","6.7-SNAPSHOT","6.7.0-SNAPSHOT","6.7.0","6.7.1-SNAPSHOT","6.7.1","6.7.2-SNAPSHOT","6.7.2","6.7.3-SNAPSHOT","6.8-SNAPSHOT","6.8.0-SNAPSHOT","6.8.0","6.8.1-SNAPSHOT","6.8.1-e43faf64","6.8.1","6.8.2-SNAPSHOT","6.8.2","6.8.3-SNAPSHOT","6.8.3","6.8.4-SNAPSHOT","6.8.4","6.8.5-SNAPSHOT","6.8.5","6.8.6-SNAPSHOT","6.8.6","6.8.7-SNAPSHOT","6.x-SNAPSHOT","7.0-SNAPSHOT","7.0.0-SNAPSHOT","7.0.0-alpha1-SNAPSHOT","7.0.0-alpha1","7.0.0-alpha2","7.0.0-beta1","7.0.0-rc1","7.0.0-rc2","7.0.0","7.0.1-SNAPSHOT","7.0.1","7.0.2-SNAPSHOT","7.1-SNAPSHOT","7.1.0-SNAPSHOT","7.1.0","7.1.1-SNAPSHOT","7.1.1","7.1.2-SNAPSHOT","7.2-SNAPSHOT","7.2.0-SNAPSHOT","7.2.0","7.2.1-SNAPSHOT","7.2.1","7.2.2-SNAPSHOT","7.3-SNAPSHOT","7.3.0-SNAPSHOT","7.3.0","7.3.1-SNAPSHOT","7.3.1","7.3.2-SNAPSHOT","7.3.2","7.3.3-SNAPSHOT","7.4-SNAPSHOT","7.4.0-SNAPSHOT","7.4.0","7.4.1-SNAPSHOT","7.4.1","7.4.2-SNAPSHOT","7.4.2","7.4.3-SNAPSHOT","7.5-SNAPSHOT","7.5.0-SNAPSHOT","7.5.0","7.5.1-SNAPSHOT","7.5.1","7.5.2-SNAPSHOT","7.6.0-SNAPSHOT","7.x-SNAPSHOT","8.0.0-SNAPSHOT","master-SNAPSHOT"]}
    b6874ea|
    b6874ea|--- BODY END ---
    [...]
    12c6fdb|@URL: https://$ARTIFACTORY/v2/kibana/kibana/tags/list
    12c6fdb|@HEADER: Connection                               = [keep-alive]
    12c6fdb|@HEADER: Server                                   = [Artifactory/6.16.0]
    12c6fdb|@HEADER: X-Artifactory-Id                         = [$ID]
    12c6fdb|@HEADER: Docker-Distribution-Api-Version          = [registry/2.0]
    12c6fdb|@HEADER: Date                                     = [Thu, 09 Jan 2020 08:09:15 GMT]
    12c6fdb|@HEADER: Content-Type                             = [application/json]
    

    This is the failing 1.2.11:

    d55957a|@URL: https://docker.elastic.co/v2/kibana/kibana/tags/list
    d55957a|@HEADER: Content-Type                             = [application/json; charset=utf-8]
    d55957a|@HEADER: Date                                     = [Thu, 09 Jan 2020 08:03:50 GMT]
    d55957a|@HEADER: Docker-Distribution-Api-Version          = [registry/2.0]
    d55957a|@HEADER: X-Content-Type-Options                   = [nosniff]
    d55957a|@HEADER: Content-Length                           = [2278]
    d55957a|@HEADER: Connection                               = [keep-alive]
    d55957a|--- BODY BEGIN ---
    d55957a|{"name":"kibana/kibana","tags":["5.0.0-731e78df","5.0.0-86a0b164","5.0.0-alpha5","5.0.0-beta1","5.0.0-ccd69424","5.0.0-rc1","5.0.0","5.0.1-6d75f4de","5.0.1-c068af18","5.0.1","5.0.2","5.1.1-71988514","5.1.1","5.1.2","5.2.0-11b63c8d","5.2.0-6ee3d9ba","5.2.0-a9f63967","5.2.0","5.2.1","5.2.2","5.3.0-d5b30bd7","5.3.0","5.3.1","5.3.2","5.3.3","5.4.0","5.4.1","5.4.2","5.4.3","5.5.0-e210c18b","5.5.0","5.5.1","5.5.2","5.5.3","5.6.0","5.6.1","5.6.10","5.6.11","5.6.12","5.6.13","5.6.14","5.6.15","5.6.16","5.6.2","5.6.3","5.6.4-1fdad85","5.6.4","5.6.5","5.6.6","5.6.7","5.6.8","5.6.9","6.0.0-alpha1","6.0.0-alpha2","6.0.0-beta1","6.0.0-beta2-d2816397","6.0.0-beta2","6.0.0-rc1","6.0.0-rc2","6.0.0","6.0.1","6.1.0","6.1.1","6.1.2","6.1.3","6.1.4","6.2.0","6.2.1","6.2.2","6.2.3","6.2.4","6.3.0","6.3.1","6.3.2","6.3.3-SNAPSHOT","6.4-SNAPSHOT","6.4.0-SNAPSHOT","6.4.0","6.4.1-SNAPSHOT","6.4.1","6.4.2-SNAPSHOT","6.4.2","6.4.3-SNAPSHOT","6.4.3","6.4.4-SNAPSHOT","6.5-SNAPSHOT","6.5.0-SNAPSHOT","6.5.0","6.5.1-109","6.5.1-SNAPSHOT","6.5.1","6.5.2-SNAPSHOT","6.5.2","6.5.3-SNAPSHOT","6.5.3","6.5.4-SNAPSHOT","6.5.4","6.5.5-SNAPSHOT","6.6-SNAPSHOT","6.6.0-SNAPSHOT","6.6.0","6.6.1-SNAPSHOT","6.6.1","6.6.2-SNAPSHOT","6.6.2","6.6.3-SNAPSHOT","6.7-SNAPSHOT","6.7.0-SNAPSHOT","6.7.0","6.7.1-SNAPSHOT","6.7.1","6.7.2-SNAPSHOT","6.7.2","6.7.3-SNAPSHOT","6.8-SNAPSHOT","6.8.0-SNAPSHOT","6.8.0","6.8.1-SNAPSHOT","6.8.1-e43faf64","6.8.1","6.8.2-SNAPSHOT","6.8.2","6.8.3-SNAPSHOT","6.8.3","6.8.4-SNAPSHOT","6.8.4","6.8.5-SNAPSHOT","6.8.5","6.8.6-SNAPSHOT","6.8.6","6.8.7-SNAPSHOT","6.x-SNAPSHOT","7.0-SNAPSHOT","7.0.0-SNAPSHOT","7.0.0-alpha1-SNAPSHOT","7.0.0-alpha1","7.0.0-alpha2","7.0.0-beta1","7.0.0-rc1","7.0.0-rc2","7.0.0","7.0.1-SNAPSHOT","7.0.1","7.0.2-SNAPSHOT","7.1-SNAPSHOT","7.1.0-SNAPSHOT","7.1.0","7.1.1-SNAPSHOT","7.1.1","7.1.2-SNAPSHOT","7.2-SNAPSHOT","7.2.0-SNAPSHOT","7.2.0","7.2.1-SNAPSHOT","7.2.1","7.2.2-SNAPSHOT","7.3-SNAPSHOT","7.3.0-SNAPSHOT","7.3.0","7.3.1-SNAPSHOT","7.3.1","7.3.2-SNAPSHOT","7.3.2","7.3.3-SNAPSHOT","7.4-SNAPSHOT","7.4.0-SNAPSHOT","7.4.0","7.4.1-SNAPSHOT","7.4.1","7.4.2-SNAPSHOT","7.4.2","7.4.3-SNAPSHOT","7.5-SNAPSHOT","7.5.0-SNAPSHOT","7.5.0","7.5.1-SNAPSHOT","7.5.1","7.5.2-SNAPSHOT","7.6.0-SNAPSHOT","7.x-SNAPSHOT","8.0.0-SNAPSHOT","master-SNAPSHOT"]}
    d55957a|
    d55957a|--- BODY END ---
    
    Bad response status: 401 Unauthorized >> https://$ARTIFACTORY/v2/kibana/kibana/tags/list
    

    Because this error comes from authentication there is no useful log from Artifactory. The nginx log shows these:

    [09/Jan/2020:08:03:15 +0000] "GET /artifactory/api/docker/docker-external-local/v2/kibana/kibana/tags/list HTTP/1.1" 200 119 "-" "Go-http-client/1.1"
    [09/Jan/2020:08:09:56 +0000] "GET /artifactory/api/docker/docker-external-local/v2/kibana/kibana/tags/list HTTP/1.1" 401 101 "-" "Go-http-client/1.1"
    

    I suspect this has to do with the new token handling that was merged into 1.2.11.

    possible bug 
    opened by nudgegoonies 7
  • Unauthorized on gcr.io

    Unauthorized on gcr.io

    Trying to use lstag (CLI and lib) against private repo hosted on gcr.io. lstags cannot connect to the repo (but the corresponding docker pull ... works).

    After some quick analysis looks like:

    • the credentials helper part works well and is able to grab a fresh token from the gcr helper
    • the API should use the bearer token here and not username/password.
    help wanted 
    opened by fredbi 3
  • Allow synchronization with sha syntax

    Allow synchronization with sha syntax

    For images that only have a latest tag it would be useful to sync an image via SHA. This is an example image that we stumbled on where we mirror the SHA manually:

    solsson/[email protected]:6f82e2b0464f50da8104acd7363fb9b995001ddff77d248379f8788e78946143
    
    feature 
    opened by nudgegoonies 1
  • Multi-platform images do not show

    Multi-platform images do not show

    Hello,

    I noticed an error with multi-platform images (at least in my configuration w/ Ubuntu 18.04 out of the box install).

    When mirroring https://quay.io/repository/prometheus/alertmanager the images sync perfectly, until image v0.17.0, where the images seem to download. But upon re-run, the images never get to "PRESENT" but remain in "CHANGED" state each run.

    To reproduce:

    lstags -v quay.io/prometheus/alertmanager:v0.18.0

    CHANGED sha256:3c0f0ea52a662342282fc7236ce058ab4 ce3c87f17369 2019-07-08T08:07:19-07:00 quay.io/prometheus/alertmanager:v0.18.0

    This above command never succeeds in updating v0.18.0, despite docker being tasked with a new download.

    docker images --digests | grep v0.18.0

    quay.io/prometheus/alertmanager v0.18.0 sha256:3d52ab7ef3a2e23e26a73cfd0ce13189e3796fb276b9af3e79664e4f743d8a1d ce3c87f17369 2 months ago 51.9MB

    I believe this issue is due to Docker using the top level checksum, while lstags uses the container specific sha256. Looking at https://quay.io/repository/prometheus/alertmanager?tag=latest&tab=tags I noticed the lstags checksum matches the child manifest checksum.

    Furthermore, this behavior for prometheus/alertmanager started with v0.17.0, the same time that alertmanager shows multi-platform containers added.

    Thank you for any suggestions, or if I can provide more info or test, I would be happy to.

    Matt

    bug handicap 
    opened by mattsimonsen 1
  • Second refactor of lstags

    Second refactor of lstags

    • Remove "magic" from remote and client code.
    • Add interoperability tests for major registries: quay, GCR, DockerHub etc.
    enhancement future testing 
    opened by ivanilves 0
  • Option to clean up local images after synchronizing remote registries

    Option to clean up local images after synchronizing remote registries

    Option to clean up local images after synchronizing remote registries - would be nice to have it.

    enhancement 
    opened by ivanilves 0
  • Update documentation for API 1.X

    Update documentation for API 1.X

    We need to clearly highlight API 1.X features in our documentation.

    • [ ] Update GoDoc / comments
    • [ ] Update project README
    documentation request 
    opened by vonrabbe 0
Releases(v1.2.20)
Owner
Ivan Ilves
Seasoned Linux system admin and DevOps engineer. Sceptic, realistic, but still in love with the industry. ;)
Ivan Ilves
GitHub中文排行榜,帮助你发现高分优秀中文项目、更高效地吸收国人的优秀经验成果;榜单每周更新一次,敬请关注!

榜单设立目的 ???? GitHub中文排行榜,帮助你发现高分优秀中文项目; 各位开发者伙伴可以更高效地吸收国人的优秀经验、成果; 中文项目只能满足阶段性的需求,想要有进一步提升,还请多花时间学习高分神级英文项目; 榜单设立范围 设立1个总榜(所有语言项目汇总排名)、18个分榜(单个语言项目排名);

kon9chunkit 36.7k Jul 25, 2021
Explore Docker registries and manipulate Docker images!

L/S tags Utility and API to manipulate (analyze, synchronize and aggregate) images across different Docker registries. Example invocation $ lstags alp

Ivan Ilves 282 Jul 12, 2021
Kubei is a flexible Kubernetes runtime scanner, scanning images of worker and Kubernetes nodes providing accurate vulnerabilities assessment, for more information checkout:

Kubei is a vulnerabilities scanning and CIS Docker benchmark tool that allows users to get an accurate and immediate risk assessment of their kubernet

Portshift 414 Jul 21, 2021
Fast docker image distribution plugin for containerd, based on CRFS/stargz

[ ⬇️ Download] [ ?? Browse images] [ ☸ Quick Start (Kubernetes)] [ ?? Quick Start (nerdctl)] Stargz Snapshotter Read also introductory blog: Startup C

containerd 361 Jul 23, 2021
A Simple and Comprehensive Vulnerability Scanner for Container Images, Git Repositories and Filesystems. Suitable for CI

A Simple and Comprehensive Vulnerability Scanner for Containers and other Artifacts, Suitable for CI. Table of Contents Abstract Features Installation

Aqua Security 8k Jul 23, 2021
Docker App Development & Command Practice

Docker App Development & Command Practice What is Docker? A platform for building,running and shipping applications. in a consistent manner so if your

MD MOSTAIN BILLAH 33 Jul 8, 2021
Progressive delivery Kubernetes operator (Canary, A/B Testing and Blue/Green deployments)

flagger Flagger is a progressive delivery tool that automates the release process for applications running on Kubernetes. It reduces the risk of intro

Flux project 3k Jul 24, 2021
Build and deploy Go applications on Kubernetes

ko: Easy Go Containers ko is a simple, fast container image builder for Go applications. It's ideal for use cases where your image contains a single G

Google 3k Jul 20, 2021
Container Registry Synchronization made easy and fast

?? booster - Makes synchronization of container images between registries faster.

Silvio Moioli 10 Jul 17, 2021
Kubedock is a minimal implementation of the docker api that will orchestrate containers on a Kubernetes cluster, rather than running containers locally.

Kubedock Kubedock is an minimal implementation of the docker api that will orchestrate containers on a kubernetes cluster, rather than running contain

Vincent van Dam 15 Jul 8, 2021
🐻 The Universal Service Mesh. CNCF Sandbox Project.

Kuma is a modern Envoy-based service mesh that can run on every cloud, in a single or multi-zone capacity, across both Kubernetes and VMs. Thanks to i

Kuma 2.3k Jul 28, 2021
Terraform provider to help with various AWS automation tasks (mostly all that stuff we cannot accomplish with the official AWS terraform provider)

terraform-provider-awsutils Terraform provider for performing various tasks that cannot be performed with the official AWS Terraform Provider from Has

Cloud Posse 6 Jul 19, 2021
A beginner friendly introduction to prometheus 🔥

Prometheus-Basics A beginner friendly introduction to prometheus. Table of Contents What is prometheus ? What are metrics and why is it important ? Ba

S Santhosh Nagaraj 1.5k Jul 26, 2021
Hassle-free minimal CI/CD for git repositories with docker or docker-compose projects.

GIT-PIPE Hassle-free minimal CI/CD for git repos for docker-based projects. Features: zero configuration for repos by default automatic encrypted back

Aleksandr Baryshnikov 32 Jul 18, 2021