Runwasi - A containerd shim which runs wasm workloads in wasmtime

Related tags

DevOps Tools runwasi
Overview

containerd-shim-wasmtime-v1

This is a containerd shim which runs wasm workloads in wasmtime. You can use it with containerd's ctr by specifying --runtime=io.containerd.wasmtime.v1 when creating the container. The shim binary must be in $PATH (that is the $PATH that containerd sees).

You can use the test image provided in this repo to have test with, use make load to load it into containerd. Run it with ctr run --rm --runtime=io.containerd.wasmtime.v1 docker.io/library/wasmtest:latest testwasm. You should see some output like:

Hello from wasm!

The test binary supports some other commands, see test/image/wasm.go to play around more.

Build

$ make build

Install

$ sudo make install
Comments
  • Support for cgroups

    Support for cgroups

    This is a series of patches to add support for running wasm code in a cgroup. While some cgroup controllers can support running different threads in different cgroups, the main one we are interested in, the cgroup memory controller, this doesn't make sense since threads all share memory. Because of this we'll fork off a new process (which should have CoW version of the main shim's memory) to run the wasm code.

    Of course all the issues of fork apply here since this is fairly unsafe from a multithreaded program (which the shim is always multi-threaded), so care must be taken to not try do things like take a lock in the new process because this can cause a deadlock. There may be some interesting things to test out with the wasmtime engine just to make sure we aren't going to deadlock if there's multiple things happening when the fork occurs.

    opened by cpuguy83 8
  • Add troubleshooting guide

    Add troubleshooting guide

    Currently there is no troubleshotting guide. People might find it hard to follow readme to produce a hello world example.

    Known issues are, but not limited to:

    1. containerd currently only support Linux. So in order to build runwasi, either you need to have a linux machien or run it in WSL on Windows
    2. docker buildx is a dependency
    3. make load is broken
    enhancement 
    opened by Mossaka 5
  • Move wasi impl to separate crate

    Move wasi impl to separate crate

    The repo has a few binaries and a wasi implementation that is fairly tied to wasmtime. #15 makes the core library runtime agnostic, meaning it does not depend on wasmtime.

    In order to completely remove wasmtime as a dependency from the core library it may be useful to move the binaries along with the Wasi instance implementation into a separate crate (of course both crates can be in this repo).

    opened by cpuguy83 1
  • feat: generic engine in the shim library

    feat: generic engine in the shim library

    This PR implements a generic engine to the shim library. This removes the dependency on wasmtime::Engine in the library entirely, and it is up to the implementation to decide what instance of the engine should be.

    opened by Mossaka 0
  • add the runner os release env var

    add the runner os release env var

    The RUNNER_OS env var was not set in the release job, so it made the release artifact name look like it has an extra "-". This adds the env var.

    This var will likely be useful in the future when building for multiple architectures / OSes.

    opened by devigned 0
  • rust: add sandboxing daemon

    rust: add sandboxing daemon

    This runs the shim in a mode where there is 1 daemon that runs multiple shims in-process. This allows a host implementation to be shared and has reduced overhead.

    opened by cpuguy83 0
  • thiserror and anyhow

    thiserror and anyhow

    This project uses two libraries for error handling, maybe we can choose only one and remove the other? I'm not sure why both are needed. If I had to choose I would keep anyhow, I like their context function. WDYT?

    opened by rumpl 0
  • Full Linux OCI runtime spec support

    Full Linux OCI runtime spec support

    Right now we have only partial support for the OCI runtime spec. While some things in the spec may not make sense for running wasm code itself, it is useful for sandboxing for the wasm runtime and/or the execution of the wasm for defense-in-depth as well as ensuring fewer surprises for users expecting their settings to actually apply.

    Some things missing:

    • [x] cgroups: In progress: https://github.com/deislabs/runwasi/pull/21
    • [ ] Lifecycle Hooks: https://github.com/opencontainers/runtime-spec/blob/main/runtime.md#lifecycle https://github.com/opencontainers/runtime-spec/blob/86290f6a00fbdc6d561e14b2e6a11788a1a5f29c/config.md#posix-platform-hooks (Docker has at least some of this in their fork: https://github.com/second-state/runwasi/pull/13/files @rumpl)
    • [ ] hybrid cgroup: In #21 there is support for cgroupv1 and cgroupv2, but not hybrid mode
    • [ ] systemd cgroup: #21 now implements cgroupfs mode but does not support systemd cgroups
    • [ ] namespaces: We do setup the network namespace, but other namespace support is still needed https://github.com/opencontainers/runtime-spec/blob/main/config-linux.md#namespaces
    • [ ] seccomp: https://github.com/opencontainers/runtime-spec/blob/main/config-linux.md#seccomp
    • [ ] apparmor: https://github.com/opencontainers/runtime-spec/blob/86290f6a00fbdc6d561e14b2e6a11788a1a5f29c/config.md#linux-process
    • [ ] selinux: https://github.com/opencontainers/runtime-spec/blob/main/config-linux.md#mount-label https://github.com/opencontainers/runtime-spec/blob/86290f6a00fbdc6d561e14b2e6a11788a1a5f29c/config.md#linux-process
    • [ ] devices: https://github.com/opencontainers/runtime-spec/blob/main/config-linux.md#devices
    • [ ] sysctl: https://github.com/opencontainers/runtime-spec/blob/main/config-linux.md#sysctl
    • [ ] masked paths: https://github.com/opencontainers/runtime-spec/blob/main/config-linux.md#masked-paths
    • [ ] personality: https://github.com/opencontainers/runtime-spec/blob/main/config-linux.md#personality
    • [ ] rlimits: https://github.com/opencontainers/runtime-spec/blob/86290f6a00fbdc6d561e14b2e6a11788a1a5f29c/config.md#posix-process
    • [ ] capabilities: https://github.com/opencontainers/runtime-spec/blob/86290f6a00fbdc6d561e14b2e6a11788a1a5f29c/config.md#linux-process
    • [ ] oom score: https://github.com/opencontainers/runtime-spec/blob/86290f6a00fbdc6d561e14b2e6a11788a1a5f29c/config.md#linux-process
    • [ ] no-new-privileges: https://github.com/opencontainers/runtime-spec/blob/86290f6a00fbdc6d561e14b2e6a11788a1a5f29c/config.md#linux-process
    • [ ] Users/Groups: https://github.com/opencontainers/runtime-spec/blob/86290f6a00fbdc6d561e14b2e6a11788a1a5f29c/config.md#user
    • [ ] IntelRDT: This is pretty low-priority and is very new in OCI https://github.com/opencontainers/runtime-spec/blob/main/config-linux.md#intelrdt
    opened by cpuguy83 2
  • Document full set up with k3s

    Document full set up with k3s

    There is a large amount of assumed knowledge and set up in the current instructions so it would be useful to have documentation of a full run through of setup and usage with k3s.

    I'm working on getting this running in my lab using k3s. If I get it all working, I can write up the commands I used.

    documentation question 
    opened by agracey 2
Releases(v0.1.0)
Owner
Brian Goff
Brian Goff
Fast docker image distribution plugin for containerd, based on CRFS/stargz

[ ⬇️ Download] [ ?? Browse images] [ ☸ Quick Start (Kubernetes)] [ ?? Quick Start (nerdctl)] Stargz Snapshotter Read also introductory blog: Startup C

containerd 740 Nov 15, 2022
Repositório para a aula sobre integração do containerd com Golang

Integrando containers nativamente usando Golang Este é o código finalizado da aplicação Já pensou em uma alternativa ao Docker? Que tal manipular cont

Rocketseat Experts Club 3 May 4, 2021
Container-Explorer is a tool to explore containerd installation on a mounted image.

Container-Explorer Container-Explorer is a tool to explore containerd installation on a mounted image. Container-Explorer attempts to provide the simi

Google 25 Sep 20, 2022
Installs containerd on Windows, optionally with default CNI plugins

containerd-installer Installs containerd on Windows, optionally with default CNI plugins Usage NAME: containerd-installer.exe - Install containerd

Markus Lippert 13 Nov 23, 2022
Nydus-snapshotter - A containerd snapshotter with capability of on-demand read

Nydus Snapshotter Nydus-snapshotter is a non-core sub-project of containerd. Pul

containerd 47 Nov 22, 2022
This process installs onto kubernetes cluster(s) and provisions workloads designated by the uffizzi interface

Uffizzi Cloud Resource Controller This application connects to a Kubernetes (k8s) Cluster to provision Uffizzi users' workloads on their behalf. While

Uffizzi 13 Nov 8, 2022
OpenAIOS is an incubating open-source distributed OS kernel based on Kubernetes for AI workloads

OpenAIOS is an incubating open-source distributed OS kernel based on Kubernetes for AI workloads. OpenAIOS-Platform is an AI development platform built upon OpenAIOS for enterprises to develop and deploy AI applications for production.

4Paradigm 79 Nov 16, 2022
Natural-deploy - A natural and simple way to deploy workloads or anything on other machines.

Natural Deploy Its Go way of doing Ansibles: Motivation: Have you ever felt when using ansible or any declarative type of program that is used for dep

Akilan Selvacoumar 0 Jan 3, 2022
Kube-step-podautoscaler - Controller to scale workloads based on steps

Refer controller/*controller.go for implementation details and explanation for a better understanding.

Danish Prakash 5 Sep 5, 2022
Feels like Cloud Foundry. Runs on Kubernetes.

Migrate Cloud Foundry applications to Kubernetes using Kf As your teams standardize on Kubernetes, migrating applications from existing platforms like

Google 374 Nov 21, 2022
vcluster - Create fully functional virtual Kubernetes clusters - Each cluster runs inside a Kubernetes namespace and can be started within seconds

Website • Quickstart • Documentation • Blog • Twitter • Slack vcluster - Virtual Clusters For Kubernetes Lightweight & Low-Overhead - Based on k3s, bu

Loft Labs 2.2k Nov 24, 2022
rld is a tiny tool that runs a go program and watch changes on it.

RLD rld is a tiny tool that runs a go program and watch changes on it. rld was inspired by Nodemon Installation Clone the git repository and build: $

Francis Sunday 10 Jun 13, 2022
Opinionated platform that runs on Kubernetes, that takes you from App to URL in one step.

Epinio Opinionated platform that runs on Kubernetes, that takes you from App to URL in one step. Contents Epinio Contents What problem does Epinio sol

Julien ADAMEK 2 Nov 13, 2022
GoScanPlayers - Hypixel online player tracker. Runs as an executable and can notify a Discord Webhook

GoScanPlayers Hypixel online player tracker. Runs as an executable and can notif

null 2 Oct 16, 2022
A simple project (which is visitor counter) on kubernetesA simple project (which is visitor counter) on kubernetes

k8s playground This project aims to deploy a simple project (which is visitor counter) on kubernetes. Deploy steps kubectl apply -f secret.yaml kubect

null 12 Nov 10, 2022
Tool which gathers basic info from apk, which can be used for Android penetration testing.

APKSEC Tool which gathers basic info from apk, which can be used for Android penetration testing. REQUIREMENTS AND INSTALLATION Build APKSEC: git clon

Jayateertha Guruprasad 3 Sep 2, 2022
An operator which complements grafana-operator for custom features which are not feasible to be merged into core operator

Grafana Complementary Operator A grafana which complements grafana-operator for custom features which are not feasible to be merged into core operator

Snapp Cab Incubators 6 Aug 16, 2022
A kubernetes plugin which enables dynamically add or remove GPU resources for a running Pod

GPU Mounter GPU Mounter is a kubernetes plugin which enables add or remove GPU resources for running Pods. This Introduction(In Chinese) is recommende

XinYuan 80 Nov 23, 2022
A sub module of EdgeGallery MECM which responsible for the app lifecycle management

mecm-applcm Description Application life cycle manager is part of MEP manager whose responsibility is to handle the host level life cycle management i

EdgeGallery 21 Jan 10, 2022