Small wrapper for containers/image which exposes a HTTP API to fetch

Overview

CLI to expose containers/image fetching via HTTP

This is a small CLI program which vendors the containers/image Go library and exposes a HTTP API to fetch manifests and blobs.

Eventually, this should probably be folded into containers/skopeo but for now we'll iterate here.

Why?

First, assume one is operating on a codebase that isn't Go, but wants to interact with container images - we can't just include the Go containers/image library.

The primary intended use case of this is for things like ostree-containers where we're using container images to encapsulate host operating system updates, but we don't want to involve the containers/image storage layer.

What we do want from the containers/image library is support for things like signatures and offline mirroring. More on this below.

Forgetting things like ostree exist for a second - imagine that you wanted to encapsulate a set of Debian/RPM/etc packages inside a container image to ship for package-based operating systems. You could use this to stream out the layer containing those packages and extract them directly, rather than serializing everything to disk in the containers/storage disk location, only to copy it out again and delete the first.

Another theoretical use case could be something like krustlet, which fetches WebAssembly blobs inside containers. Here again, we don't want to involve containers/storage.

Desired containers/image features

There are e.g. Rust libraries like dkregistry-rs, and similar for other languages. However, the containers/image Go library has a lot of additional infrastructure that will impose a maintenance burden to replicate:

  • Signatures (man containers-auth.json)
  • Mirroring/renaming (man containers-registries.conf)
  • Support for ~/.docker/config.json for authentication as well as /run

Status

We have a 0.1 release that works. However, in the future this will hopefully move into skopeo.

Usage

The intended production use of this is:

  • Parent process creates a socketpair (e.g. Rust tokio)
  • Parent passes one half of socketpair to child via e.g. fd 3 - container-image-proxy --sockfd 3 docker://quay.io/cgwalters/exampleos:latest
  • Parent makes HTTP (1.1) requests on its half of the socketpair

APIs

GET /manifest

Returns the manifest converted into OCI format, plus the original manifest digest in a Manifest-Digest header.

At the moment, when presented with an image index AKA "manifest list", this request will choose the image matching the current operating system and processor.

GET /blobs/<digest>

Fetch a blob as is - no decompression is performed if relevant. The digest will be verified.

POST /quit

Gracefully shut down the server and exit the process.

Python demo code

See demo.py.

You might also like...
fetch data from different databses

how to install make a folder with: - dbAgent.exe - config - log *config and log are folders make a config as mentioned below and put it inside config

Connect directly to Docker-for-Mac containers via IP address 🐳 💻
Connect directly to Docker-for-Mac containers via IP address 🐳 💻

Docker Mac Net Connect Connect directly to Docker-for-Mac containers via IP address. Features L3 connectivity: Connect to Docker containers from macOS

Docker4ssh: Docker containers and more via ssh

docker4ssh - docker containers and more via ssh docker4ssh is an ssh server that

Furui - A process-based communication control system for containers

furui Communication control of the container runtime environment(now only docker

Small round tripper to avoid triggering the
Small round tripper to avoid triggering the "attention required" status of CloudFlare for HTTP requests

CloudFlare-ByPass-Go. Small round tripper to avoid triggering the "attention req

go http wrapper for boomer

什么是 go-httpwrapper? 如果你想快速实现http协议的分布式压测,那么go-httpwrapper将会是一个不错的选择! Boomer 是Locust框架worker端的go实现,它很好地弥补了Locust使用Python实现而导致性能不佳的缺陷。 go-httpwrapper对Bo

golang http server wrapper

Yong Simple Web Framework This project benchmarked gin-gonic. Installation Go command to install Yong. $ go get -u github.com/rladyd818/yong Import it

A http proxy server chaining a upstream which needs authentication headers.

Normalize HTTP Proxy A http proxy server chaining a upstream which needs authentication headers. local - [np] - upstream - destination Usage Norma

Log4Shell is a middleware plugin for Traefik which blocks JNDI attacks based on HTTP header values.

Log4Shell Mitigation Log4Shell is a middleware plugin for Traefik which blocks JNDI attacks based on HTTP header values. Related to the Log4J CVE: htt

Comments
  • Plan for merge into skopeo

    Plan for merge into skopeo

    I think this project is probably close to suitable to propose for inclusion into github.com/containers/skopeo/

    There's stuff that are somewhere between nice-to-have and must-have:

    • API documentation
    • API review for stabilization (use of HTTP/1.1 seems OK, but is there an even simpler option?)
    • done: ~Sample code in e.g. Python (and/or extract the Rust code from https://github.com/ostreedev/ostree-rs-ext/pull/98 )~
    • done: ~Maybe drop the "run via listening on port" since it's an anti-pattern, even if useful for debugging?~

    Also: should we think about trying to support writes via this proxy too? It would be a large scope increase and not important for my use cases right now. But it'd be good to avoid ruling it out.

    opened by cgwalters 10
Owner
Colin Walters
@openshift & Fedora/RHEL @coreos engineer at @RedHatOfficial
Colin Walters
Lobby - A Nox game lobby which exposes a simple HTTP API for both listing and registering Nox game servers

Nox lobby server This project provides a Nox game lobby which exposes a simple H

NoX World 3 Mar 6, 2022
Go-fetch-words - Fetch 5 letter words from dictionary.com

Go-fetch-words This GO app fetches 5 letter words from dictionary.com and saves

Royson 1 Jun 23, 2022
Automatically exposes the remote container's listening ports back to the local machine

Auto-portforward (apf) A handy tool to automatically set up proxies that expose the remote container's listening ports back to the local machine. Just

Ruoshan Huang 278 Sep 9, 2022
A Go client used to fetch information from the Go API

Hilo Client Go Hilo API Go client, Introduction This is a (partial) Go client used to fetch information from the Go API, most of the reverse engineeri

Valentin T. 0 Nov 27, 2021
Whats-da-twilio - Small Golang HTTP Server that leverages Twilio's API for calling and SMS messaging

What's da twilio Description Small Golang HTTP Server that leverages Twilio's API for calling and SMS messaging Note: trial Twilio accounts and number

null 0 Dec 31, 2021
A small tool used to correspond to the IP address according to the name, id, and network alias of the docker container, which can be run as a DNS server

A small tool used to correspond to the IP address according to the name, id, and network alias of the docker container, which can be run as a DNS server

Swift 5 Apr 4, 2022
A small GoLang app which can bruteforce ssh credentials

A small GoLang app which can bruteforce ssh credentials, was used before for a ctf and is now optimized for future ctf events.

Vincent Schneider 4 Mar 7, 2022
🐈📦 nyaa.si client library for Go. Fetch Anime, Manga, Music and more torrents

?? ?? go-nyaa nyaa.si client library for Go Built on top of: gofeed - search using RSS colly - scrap torrent details page Original idea: ejnshtein/nya

Ilya Revenko 25 Sep 23, 2022
fetch papers from p2p network

sci-hub P2P A project aims to fetch paper from the BitTorrent network. According to this Reddit post, currently, all `sci-hub's papers are available i

Trim21 185 Sep 5, 2022
fetch and serve papers in p2p network

sci-hub P2P A project aims to fetch paper from the BitTorrent network. This is not a cli client of sci-hub website. English Introduction 中文简介 Contribu

Sci-Hub on P2P 186 Sep 27, 2022