Framework for building distributed services with Web Assembly

Overview

Tarmac

Framework for building distributed services with Web Assembly

PkgGoDev Go Report Card Documentation

Tarmac is a unique framework designed for the next generation of distributed systems. At its core, like many other microservice frameworks, Tarmac is focused on abstracting the complexities of building cloud-native services allowing users to focus more on business logic and less on boilerplate code.

What makes Tarmac unique is that, unlike most microservice frameworks, Tarmac is language agnostic. Using Web Assembly (WASM), Tarmac users can write their business logic in many different languages such as Rust, Go, Javascript, or even Swift; and run it all using the same core framework.

Tarmac vs. Serverless Functions

Tarmac shares many traits with Serverless Functions and Functions as a Service (FaaS) platforms. Tarmac makes it easy for developers to deploy functions and microservices without writing repetitive boilerplate code. As a developer, you can create a production-ready service in less than 100 lines of code.

But Tarmac takes Serverless Functions further. In general, FaaS platforms provide a simple runtime for user code. If a function requires any dependency (i.e., a Database), the developer-provided function code must maintain the database connectivity and query calls.

Using the power of Web Assembly, Tarmac not only provides functions a secure sandboxed runtime environment, but it also provides abstractions that developers can use to interact with platform capabilities such as Databases, Caching, Metrics, and even Dynamic Configuration.

In many ways, Tarmac is more akin to a microservices framework with the developer experience of a FaaS platform.

Quick Start

At the moment, Tramac is executing WASM functions by executing a defined set of function signatures. When Tarmac receives an HTTP GET request, it will call the function's registered under the http:GET signature.

As part of the WASM Function, users must register their handlers using the pre-defined function signatures.

To understand this better, look at one of our simple examples written in Rust (found in example/).

// Tac is a small, simple Rust program that is an example WASM function for Tarmac.
// This program will accept a Tarmac server request, log it, and echo back the payload
// but with the payload reversed.
extern crate wapc_guest as guest;
extern crate base64;
use serde::{Deserialize, Serialize};
use serde_json;
use std::collections::HashMap;
use guest::prelude::*;

#[derive(Serialize, Deserialize)]
struct ServerRequest {
  headers: HashMap<String, String>,
  payload: String,
}

#[derive(Serialize, Deserialize)]
struct ServerResponse {
  headers: HashMap<String, String>,
  status: Status,
  payload: String,
}

#[derive(Serialize, Deserialize)]
struct Status {
  code: u32,
  status: String,
}

fn main() {}

#[no_mangle]
pub extern "C" fn wapc_init() {
  // Add Handler for the GET request
  register_function("http:GET", fail_handler);
  // Add Handler for the POST request
  register_function("http:POST", handler);
  // Add Handler for the PUT request
  register_function("http:PUT", handler);
  // Add Handler for the DELETE request
  register_function("http:DELETE", fail_handler);
}

// fail_handler will accept the server request and return a server response
// which rejects the client request
fn fail_handler(_msg: &[u8]) -> CallResult {
  // Create the response
  let rsp = ServerResponse {
      status: Status {
        code: 503,
        status: "Not Implemented".to_string(),
      },
      payload: "".to_string(),
      headers: HashMap::new(),
  };

  // Marshal the response
  let r = serde_json::to_vec(&rsp).unwrap();

  // Return JSON byte array
  Ok(r)
}

// handler is a simple example of a Tarmac WASM function written in Rust.
// This function will accept the server request, log it, and echo back the payload
// but with the payload reversed.
fn handler(msg: &[u8]) -> CallResult {
  // Perform a host callback to log the incoming request
  let _res = host_call("tarmac", "logger", "debug", &msg.to_vec());

  // Unmarshal the request
  let rq: ServerRequest = serde_json::from_slice(msg).unwrap();

  // Decode Payload
  let b = base64::decode(rq.payload).unwrap();
  // Convert to a String
  let s = String::from_utf8(b).expect("Found Invalid UTF-8");
  // Reverse it and re-encode
  let enc = base64::encode(s.chars().rev().collect::<String>());

  // Create the response
  let rsp = ServerResponse {
      status: Status {
        code: 200,
        status: "OK".to_string(),
      },
      payload: enc,
      headers: HashMap::new(),
  };

  // Marshal the response
  let r = serde_json::to_vec(&rsp).unwrap();

  // Return JSON byte array
  Ok(r)
}

Tarmac passes the HTTP Context and Payload to the WASM function via the incoming msg. The msg is a JSON that contains Headers and a Payload which is Base64 encoded but otherwise untouched.

To compile the example above, run:

$ cd example/tac/rust
$ make build

Once compiled, users can run Tarmac via Docker using the following command:

$ docker run -p 8080:8080 \
  -e "APP_ENABLE_TLS=false" -e "APP_LISTEN_ADDR=0.0.0.0:8080" \
  -v ./functions:/functions madflojo/tarmac

With Tarmac now running, we can access our WASM function using any HTTP Client such as curl.

$ curl -v --data "Tarmac Example" http://localhost:8080
Issues
  • WASI Module not instantiated error when running example

    WASI Module not instantiated error when running example

    Running on WSL2 Ubuntu Docker version 20.10.14, build a224086

    The source was cloned and the example build instructions were followed

    $ cd example/tac/go
    $ make build
    make build
    mkdir -p functions
    ## Run TinyGo build via Docker because waPC requires a <0.18.0 version of TinyGo to work right
    docker run -v `pwd`/:/build -w /build tinygo/tinygo:0.17.0 tinygo build -o /build/functions/tarmac.wasm -target wasi /build/main.go
    go: downloading github.com/wapc/wapc-guest-tinygo v0.3.0
    

    The tarmac wasm file is generated after running the build command:

    $ ls -l functions/
    total 220
    -rwxr-xr-x 1 root root 221650 Jul 18 11:21 tarmac.wasm
    

    Running the tarmac container:

    docker run -p 8080:8080  \
        -e "APP_ENABLE_TLS=false" \
        -e "APP_LISTEN_ADDR=0.0.0.0:8080" \
        -v $(pwd)/functions:/functions madflojo/tarmac
    
    time="2022-07-18T15:00:33Z" level=warning msg="No Config file found, loaded config from Environment - Default path ./conf"
    time="2022-07-18T15:00:33Z" level=info msg="KV Store not configured, skipping"
    time="2022-07-18T15:00:33Z" level=info msg="SQL DB not configured, skipping"
    time="2022-07-18T15:00:33Z" level=fatal msg="Service stopped - unable to create module pool for wasm file /functions/tarmac.wasm - module[wasi_unstable] not instantiated"
    
    bug good first issue 
    opened by jakemaguy 6
  •  GGO free wasm runner

    GGO free wasm runner

    Wazero is faster than the others due to less overhead

    https://wazero.io/

    It can run within envoy or alone Config can be file based or off envoy.

    It can also run wasi and wasm compiled golang .

    enhancement 
    opened by gedw99 3
  • Updating wapc to use wazero

    Updating wapc to use wazero

    This PR closes #30 and updates the wapc-go package to use a C-free WASM runtime engine https://github.com/tetratelabs/wazero

    One Caveat for this PR is that WASM modules will need to be compiled with a higher version of TinyGo than used in the past. I.E. TinyGo 0.24.

    opened by madflojo 1
  • Adding m-TLS for Authentication

    Adding m-TLS for Authentication

    After looking at #34 I started looking at the different Auth N/Z options available. I will be adding JWT-based authentication later but I wanted to get started with m-TLS which is pretty straightforward.

    opened by madflojo 0
  • Fixing WASI Module error

    Fixing WASI Module error

    When looking more into the error and reading some of the maintainers for wapc comments on previous Pull Requests, I came across the release notes for this: https://github.com/wapc/wapc-guest-tinygo/releases/tag/v0.3.1

    There were some changes in TinyGo v0.19 that the older version of wapc-guest-tinygo didn't work well with. But with the new version v0.3.1, everything seems to be working fine.

    opened by madflojo 0
  • Changing to TinyGo v18

    Changing to TinyGo v18

    As reported in #35, after updating wapc and using wazero POST requests fail for all TinyGo versions greater than 0.17 except for 0.18. So, for now, I am updating the Makefiles to use v0.18.

    opened by madflojo 0
  • Revamping function inputs

    Revamping function inputs

    This pull request updates quite a bit of how Tarmac works. The goal is to make it a little more default and simple to configure/use. Expect some more changes like this as it goes.

    opened by madflojo 0
  • Toolkit

    Toolkit

    This pull request is mainly a refactor in that I'm moving most of the callback functionality into their packages. The cool thing about this is, it starts opening the door for users to use Tarmac as a toolkit to add functionality to their services running WASM.

    Later I'll add some examples and documentation of just that.

    opened by madflojo 0
  • HTTP Callback

    HTTP Callback

    This Pull Request adds an HTTP Client Callback that allows WASM functions to execute HTTP calls. This is loosely molded after how Envoy exposes an HTTP Call function to WASM plugins.

    opened by madflojo 0
  • Adding Metrics Callbacks for WASM Functions

    Adding Metrics Callbacks for WASM Functions

    This pull request adds a Metrics Callback for WASM Functions. This PR includes a Counter, Gauge, and Histogram (really Summary) which seems to be common across the various metrics capabilities like OTEL and Prometheus.

    As an FYI, I chose to use Prometheus because when I was looking at the OTEL Go package it stated metrics are on hold...

    opened by madflojo 0
  • Adding System Metrics

    Adding System Metrics

    This pull request adds system-level metrics to Tarmac. This included the standard items such as goroutine count, GC duration, memstats as well as custom ones such as WASM execution time, Scheduled Task execution times, Callback Counters, etc.

    opened by madflojo 0
  • Add tags to GitHub Actions

    Add tags to GitHub Actions

    https://github.com/madflojo/tarmac/blob/7f6349796c6612c8d45f66edbeb4c66d70752798/.github/workflows/docker.yml#L3-L5

    Currently, release tags don't have automatic docker publishing, tests, or any other actions being performed.

    opened by madflojo 0
  • Auth - authz

    Auth - authz

    As far as I can see there is not any auth or authz in tarmac.

    i generally use NATS for rerouting events into tarmac . Nats just wants a jwt to control Auth and Authz. But even without NATS , Tarnac needs to assert who can do what.

    https://github.com/pocketbase/pocketbase looks like a nice solution to this. .Or at least to use as one way to add auth / authz. It’s probable that others might want a different solution and that’s why jwt is loose coupled.

    the cool thing about pocketbaae is that it’s real time and simple.

    have a look

    enhancement 
    opened by gedw99 5
  • embed

    embed

    tinygo now supports embed package which might be useful for functions that need resources.

    See: https://github.com/tinygo-org/tinygo/releases/tag/v0.24.0

    opened by gedw99 2
  • More Examples

    More Examples

    Tarmac is still pretty new, and while I know Go pretty well Rust and AssemblyScript are still fairly new to me. It would be great to have some more examples created for Tarmac.

    https://github.com/madflojo/tarmac/tree/main/example

    good first issue 
    opened by madflojo 0
Releases(v0.3.0)
  • v0.3.0(Jul 29, 2022)

    What's Changed

    • Updated to leverage wazero as the underlying Web Assembly runtime engine.
    • Added m-TLS authentication support and configuration options.
    • Minor refactoring, documentation updates, and bug fixes.

    New Contributors

    • @ColinEberhardt made their first contribution in https://github.com/madflojo/tarmac/pull/28

    Full Changelog: https://github.com/madflojo/tarmac/compare/v0.2.0...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Mar 31, 2022)

    Made some significant interface changes for WASM guests, updated documentation. The goal is to make building basic functions, dead simple.

    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Oct 17, 2021)

    This release is an initial release of Tarmac. While Tarmac is still under active development and many contracts are subject to change, the basic functionality is stable and ready for others to start using.

    Source code(tar.gz)
    Source code(zip)
Owner
Benjamin Cane
Thoughts and opinions are my own.
Benjamin Cane
WebAssembly Lightweight Javascript Framework in Go (AngularJS Inspired)

Tango Lightweight WASM HTML / Javascript Framework Intro WebAssembly is nice, Go on the web is nice, so I ported Tangu to Go and WebAssembly. Tangu is

enimatek 4 Jun 8, 2022
A package to build progressive web apps with Go programming language and WebAssembly.

Go-app is a package for building progressive web apps (PWA) with the Go programming language (Golang) and WebAssembly (Wasm). Shaping a UI is done by

Maxence Charriere 6.3k Aug 4, 2022
A package to build progressive web apps with Go programming language and WebAssembly.

Go-app is a package for building progressive web apps (PWA) with the Go programming language (Golang) and WebAssembly (Wasm). Shaping a UI is done by

Maxence Charriere 6.3k Jul 29, 2022
Bed and Breakfast web app written in Go

BOOKINGS AND RESERVATIONS This repository contains the files for my RareBnB application RareBnB is an "AirBnB" style clone providing a user the abilit

null 0 Jan 11, 2022
Dom - A Go API for different Web APIs for WebAssembly target

Go DOM binding (and more) for WebAssembly This library provides a Go API for dif

Denys Smirnov 458 Jul 15, 2022
The Bhojpur Ara is a software product used for automated resource assembly within Bhojpur.NET Platform ecosystem to enable delivery of applications and services.

Bhojpur Ara - Automated Resource Assembly The Bhojpur Ara is a service product used for automated resource assembly within the Bhojpur.NET Platform ec

Bhojpur Consulting 0 Apr 28, 2022
Distributed-Services - Distributed Systems with Golang to consequently build a fully-fletched distributed service

Distributed-Services This project is essentially a result of my attempt to under

Hamza Yusuff 6 Jun 1, 2022
A Go framework for building JSON web services inspired by Dropwizard

Tiger Tonic A Go framework for building JSON web services inspired by Dropwizard. If HTML is your game, this will hurt a little. Like the Go language

Richard Crowley 1k Jul 25, 2022
A Go framework for building JSON web services inspired by Dropwizard

Tiger Tonic A Go framework for building JSON web services inspired by Dropwizard. If HTML is your game, this will hurt a little. Like the Go language

Richard Crowley 1k Jul 25, 2022
Mortar is a GO framework/library for building gRPC (and REST) web services.

Mortar Mortar is a GO framework/library for building gRPC (and REST) web services. Mortar has out-of-the-box support for configuration, application me

null 624 Aug 3, 2022
Go Lang Web Assembly bindings for DOM, HTML etc

WebAPI Go Language Web Assembly bindings for DOM, HTML etc WARNING: The current API is in very early state and should be consider to be expremental. T

Go Web API 116 Jul 4, 2022
Amazon Web Services (AWS) providerAmazon Web Services (AWS) provider

Amazon Web Services (AWS) provider The Amazon Web Services (AWS) resource provider for Pulumi lets you use AWS resources in your cloud programs. To us

William Garcia Jacobo 0 Nov 10, 2021
This library contains utilities that are useful for building distributed services.

Grafana Dskit This library contains utilities that are useful for building distributed services. Current state This library is still in development. D

Grafana Labs 224 Jul 29, 2022
7 days golang programs from scratch (web framework Gee, distributed cache GeeCache, object relational mapping ORM framework GeeORM, rpc framework GeeRPC etc) 7天用Go动手写/从零实现系列

7 days golang programs from scratch README 中文版本 7天用Go从零实现系列 7天能写什么呢?类似 gin 的 web 框架?类似 groupcache 的分布式缓存?或者一个简单的 Python 解释器?希望这个仓库能给你答案

Dai Jie 11.1k Aug 8, 2022
High-performance framework for building redis-protocol compatible TCP servers/services

Redeo The high-performance Swiss Army Knife for building redis-protocol compatible servers/services. Parts This repository is organised into multiple

Black Square Media 416 Aug 3, 2022
⚡️ A Go framework for rapidly building powerful graphql services

Thunder is a Go framework for rapidly building powerful graphql servers. Thunder has support for schemas automatically generated from Go types, live q

null 1.5k Aug 1, 2022
package for building REST-style Web Services using Go

go-restful package for building REST-style Web Services using Google Go Code examples using v3 REST asks developers to use HTTP methods explicitly and

Ernest Micklei 4.5k Aug 1, 2022
package for building REST-style Web Services using Go

go-restful package for building REST-style Web Services using Google Go Code examples using v3 REST asks developers to use HTTP methods explicitly and

Ernest Micklei 4.5k Aug 1, 2022
Distributed Commit Log from Travis Jeffery's Distributed Services book

proglog A distributed commit log. This repository follows the book "Distributed Services with Go" by Travis Jeffrey. The official repository for this

Arindam Das 2 May 23, 2022
Golang evasion tool, execute-assembly .Net file

?? Frog For Automatic Scan ?? Doge For Defense Evasion&Offensive Security Doge-Assembly Golang evasion tool, execute-assembly .Net file Intro Are you

TimWhite 84 Jul 11, 2022
Compiler for a small language into x86-64 Assembly

Compiler This project is a small compiler, that compiles my own little language into X86-64 Assembly. It then uses yasm and ld to assemble and link in

Maurice Tollmien 237 Jul 17, 2022
Generate x86 Assembly with Go

Generate x86 Assembly with Go avo makes high-performance Go assembly easier to write, review and maintain. The avo package presents a familiar assembl

Michael McLoughlin 2k Jul 27, 2022
Compiler for a small language into x86-64 Assembly

Compiler This project is a small compiler, that compiles my own little language into X86-64 Assembly. It then uses yasm and ld to assemble and link in

Maurice Tollmien 237 Jul 17, 2022
Assembly syntax that makes you feel like you're writing code in a high-level language.

shasm Assembly syntax that makes you feel like you're writing code in a high-level language. Shasm is not an Assembler. Shasm simply compiles Shasm sy

Shoyaaa 14 Jun 5, 2021
Assembly-optimized MD4 hash algorithm in Go

md4 MD4 hash algorithm in Go. Assembly-optimized for amd64 platforms. MD4 is cryptographically broken and should should only be used where compatibili

Michael McLoughlin 15 Apr 14, 2022
A program to create assembly 8086 strings to print without using any printing/strings related function but only mov-xchg-int and loops

Assembly String builder tool A program to create assembly 8086 strings to print without using any printing/strings related function but only mov-xchg-

Reg 2 Feb 1, 2022
An efficient Go Rapid Product Assembly system used within the Bhojpur.NET Platform ecosystem.

Bhojpur GoRPA - Builder, Packager, Assembler An efficient Go-based Rapid Product Assembly software tool used within the Bhojpur.NET Platform ecosystem

Bhojpur Consulting 0 Apr 28, 2022
RISC-V meta assembler that adds quality of life features to assembly

Lox language TODOs (Partially complete) unreachable code. if a "return" has been found in a local scope and we encounter other code directly following

William Cleveland 0 Jan 11, 2022
Skynet is a framework for distributed services in Go.

##Introduction Skynet is a communication protocol for building massively distributed apps in Go. It is not constrained to Go, so it will lend itself n

null 2k Jul 25, 2022