Go driven rpc code generation tool for right now.

Overview

Welcome to oto project

Go driven rpc code generation tool for right now.

  • 100% Go
  • Describe services with Go interfaces (with structs, methods, comments, etc.)
  • Generate server and client code
  • Production ready templates (or copy and modify)

Who's using Oto?

  • Pace.dev - Oto came out of the Pace project
  • Firesearch.dev - Firesearch uses Oto to offer full-text services to the web, mobile, and the backend

Templates

These templates are already being used in production.

Learn

Tutorial

Install the project:

go install github.com/pacedotdev/oto

Create a project folder, and write your service definition as a Go interface:

// definitions/definitons.go
package definitions

// GreeterService makes nice greetings.
type GreeterService interface {
    // Greet makes a greeting.
    Greet(GreetRequest) GreetResponse
}

// GreetRequest is the request object for GreeterService.Greet.
type GreetRequest struct {
    // Name is the person to greet.
    // example: "Mat Ryer"
    Name string
}

// GreetResponse is the response object containing a
// person's greeting.
type GreetResponse struct {
    // Greeting is the greeting that was generated.
    // example: "Hello Mat Ryer"
    Greeting string
}

Download templates from otohttp

mkdir templates \
    && wget https://raw.githubusercontent.com/pacedotdev/oto/master/otohttp/templates/server.go.plush -q -O ./templates/server.go.plush \
    && wget https://raw.githubusercontent.com/pacedotdev/oto/master/otohttp/templates/client.js.plush -q -O ./templates/client.js.plush

Use the oto tool to generate a client and server:

mkdir generated
oto -template ./templates/server.go.plush \
    -out ./generated/oto.gen.go \
    -ignore Ignorer \
    -pkg generated \
    ./definitions
gofmt -w ./generated/oto.gen.go ./generated/oto.gen.go
oto -template ./templates/client.js.plush \
    -out ./generated/oto.gen.js \
    -ignore Ignorer \
    ./definitions
  • Run oto -help for more information about these flags

Implement the service in Go:

// greeter_service.go
package main

// GreeterService makes nice greetings.
type GreeterService struct{}

// Greet makes a greeting.
func (GreeterService) Greet(ctx context.Context, r GreetRequest) (*GreetResponse, error) {
    resp := &GreetResponse{
        Greeting: "Hello " + r.Name,
    }
    return resp, nil
}

Use the generated Go code to write a main.go that exposes the server:

// main.go
package main

func main() {
    g := GreeterService{}
    server := otohttp.NewServer()
    generated.RegisterGreeterService(server, g)
    http.Handle("/oto/", server)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Use the generated client to access the service in JavaScript:

import { GreeterService } from "oto.gen.js";

const greeterService = new GreeterService();

greeterService
    .greet({
        name: "Mat"
    })
    .then(response => alert(response.greeting))
    .catch(e => alert(e));

Use json tags to control the front-end facing name

You can control the name of the field in JSON and in front-end code using json tags:

// Thing does something.
type Thing struct {
    SomeField string `json:"some_field"
}
  • The SomeField field will appear as some_field in json and front-end code
  • The name must be a valid JavaScript field name

Specifying additional template data

You can provide strings to your templates via the -params flag:

oto \
    -template ./templates/server.go.plush \
    -out ./oto.gen.go \
    -params "key1:value1,key2:value2" \
    ./path/to/definition

Within your templates, you may access these strings with <%= params["key1"] %>.

Comment metadata

It's possible to include additional metadata for services, methods, objects, and fields in the comments.

// Thing does something.
// field: "value"
type Thing struct {
    //...
}

The Metadata["field"] value will be the string value.

  • The value must be valid JSON (for strings, use quotes)

Examples are officially supported, but all data is available via the Metadata map fields.

Examples

To provide an example value for a field, you may use the example: prefix line in a comment.

// GreetRequest is the request object for GreeterService.Greet.
type GreetRequest struct {
    // Name is the person to greet.
    // example: "Mat Ryer"
    Name string
}
  • The example must be valid JSON

The example is extracted and made available via the Field.Example field.

Contributions

Special thank you to:

  • @mgutz - for struct tag support
  • @sethcenterbar - for comment metadata support

A PACE. project

Issues
  • Handling Cookies / Status Codes / any other HTTP related headers

    Handling Cookies / Status Codes / any other HTTP related headers

    How would one drop down to the http.Request or http.Response structs from inside service implementations (as shown below) - for purposes such as accessing r.Header / r.AddCookie() / w.WriteHeader(400) etc?

    https://github.com/pacedotdev/oto/blob/df5a36ea40c3912971d1134c1911324a60751454/example/main.go#L17-L22

    opened by fr3fou 8
  • Small documentation inconsistency

    Small documentation inconsistency

    Hi guys. I was just testing oto and so far so good. I love it. It's simple and does with it suppose to do.

    There is a small glitch in understanding on handle mapping that might put the brains of the week of heart in an infinite loop :)

    Given the following code

    srv := otohttp.NewServer()
    RegisterHelloWorld(srv, helloWorldService)
    http.Handle("/oto/", srv)
    

    Works, but if I want to do this

    srv := otohttp.NewServer()
    RegisterHelloWorld(srv, helloWorldService)
    http.Handle("/api/v2/", srv)
    

    I get a 404. Took me a solid 30-45 mins until I decided to take a peek inside the lib to see the oto does a base map that defaults to /oto/ The only workaround I found but is ugly and redundant is to match patterns by overwriting the Basepath

    srv := otohttp.NewServer()
    srv.Basepath = "/api/v1/"
    
    RegisterHelloWorld(srv, helloWorldService)
    
    http.Handle("/api/v1/", srv)
    

    Is there another way to do this?

    opened by metacatdud 7
  • Byte slice type not handled

    Byte slice type not handled

    The parser doesn't handle byte and []byte types when parsing the JSType. This will cause the generated code to have a type of [] if it is a []byte. According to the JSON encoding documentation, []byte types are base64 encoded to a string. The proper JSType should be a string.

    This can be handled in the template by checking if the "go type" is of byte and that it is a multiple. This snippet can handle the case:

    <%= for (object) in def.Objects { %>
    <%= format_comment_text(object.Comment) %>export interface <%= object.Name %> {
    <%= for (field) in object.Fields { %>
            <%= format_comment_text(field.Comment) %>       <%= field.NameLowerCamel %>: <%= if (field.Type.IsObject) { %><%= field.Type.TypeName %><% } else if (field.Type.TypeName == "byte" && field.Type.Multiple) { %>string<% } else { %><%= field.Type.JSType %><% } %><%= if (field.Type.Multiple && field.Type.TypeName != "byte") { %>[]<% } %>;
    <% } %>
    }
    <% } %>
    

    The cleaner way to handle this would be to do it in the parser but I couldn't find a nice way to do it with a major redesign. It would need to the determination between single and multi-type as it's parsing the JSType.

    opened by TcM1911 7
  • Autogenerated documentation doesn't handle special characters properly

    Autogenerated documentation doesn't handle special characters properly

    Oto's autogenerated documentation doesn't properly encode special characters such as double quotes. I found this problem while reading pace's docs and reproduced it by myself. You can find an example here:

    • Service definition with double quoted text: https://github.com/pacedotdev/pace/blob/d241b53b81fa4d2457bab150a66442263587298b/oto/cards.go#L50-L52 (left browser)
    • Generated Go code with encoded double quotes: https://github.com/pacedotdev/pace/blob/d241b53b81fa4d2457bab150a66442263587298b/pace.gen.go#L743-L745 (right browser)

    image

    • Docs as displayed on pkg.go.dev: https://pkg.go.dev/github.com/pacedotdev/pace?tab=doc#AddCommentRequest

    image

    Please, feel free to close this issue if this is the intended behavior. I was just trying the tool and noticed it.

    Best, Miguel

    opened by jimen0 5
  • Embed gofmt

    Embed gofmt

    To reduce the steps needed to render templates, this patch embeds gofmt in oto and applies it when output is a .go file. Documentation and example are updated accordingly.

    opened by dolmen 4
  • Error handling server/client ?

    Error handling server/client ?

    First, thanks for this tool! I really like it so far.

    I'm wondering a bit though how errors are supposed to be handled? Maybe I'm missing something, but what I'm seeing is this:

    If I return a non-nil error from one of the service endpoints, a 500 is returned to the client with the body being the error message (https://github.com/pacedotdev/oto/blob/master/otohttp/templates/server.go.plush#L46-L47). On the client side, the response is assumed to be JSON though (https://github.com/pacedotdev/oto/blob/master/otohttp/templates/client.ts.plush#L38), which leads to a JS error like SyntaxError: Unexpected token c in JSON at position 0. So out-of-the-box that seems a bit broken ... and I'm a bit confused why the "service endpoint error" is not simply handled with s.server.OnErr(w, r, err), like in the other error cases? Would be great if you could shed some light on this!

    opened by michaelsauter 3
  • With oto 0.11.0, TypeScript definitions with objects are wrong

    With oto 0.11.0, TypeScript definitions with objects are wrong

    With the following API definition and the client.ts.plush template from the otohttp project, we get wrong TypeScript output. This can be reproduced with the current version, v0.11.0, but was not occurring with an older version, in my case v.0.10.14.

    package definitions
    
    type SessionService interface {
    	UpdateSession(UpdateSessionRequest) UpdateSessionResponse
    }
    
    type UpdateSessionRequest struct { Customer *Customer }
    type UpdateSessionResponse struct {}
    type Customer struct {}
    

    The wrong TypeScript code:

    // removed for brevity
    export class UpdateSessionRequest {
    	constructor(data?: any) {
    		if (data) {
    			this.customer = new *Customer(data.customer);
    		}
    	}
    
    	customer?: *Customer;
    }
    // removed for brevity
    

    Noticing the star before the Customer? That's causing syntax errors.

    opened by ghost 2
  • Comments are not copied correctly

    Comments are not copied correctly

    Given the following input:

    type MyResponse struct {
    	IsOpen bool `json:"is_open,omitempty"`
    
    	// ClosedReasonCode defines the reason, why a shop can be closed. It is empty, when IsOpen is
    	// true. Possible values are: "holiday", "same_day_deadline", "out_of_season".
    	// The ClosedReasonMessage provides a user message that can be shown.
    	ClosedReasonCode    string `json:"closed_reason_code,omitempty"`
    }
    

    the following output is generated:

    type MyResponse struct {
    	IsOpen bool `json:"is_open,omitempty"`
    	// ClosedReasonCode defines the reason, why a shop can be closed. It is empty, when
    	// IsOpen is The ClosedReasonMessage provides a user message that can be shown.
    	ClosedReasonCode    string `json:"closed_reason_code,omitempty"`
    	// Error is string explaining what went wrong. Empty if everything was fine.
    	Error string `json:"error,omitempty"`
    }
    

    I think this is related to the parsing of the : " in the second comment line.

    expected

    The comment should be copied as is.

    opened by ghost 2
  • [question] How should errors be handled?

    [question] How should errors be handled?

    So, oto generates an error property with each response. Should I set this manually, or doing the go way? So, what of these approaches is better:

    // this:
    	if err != nil {
    		return &MyServiceResponse{Error: "not found"}, nil
    	}
    
    // or this?
    	if err != nil {
    		return nil, errors.New("not found")
    	}
    
    opened by ghost 2
  • Copy struct tags to generated server.go

    Copy struct tags to generated server.go

    When defining additional struct tags, like the ones from the go-playground/validator package, they aren't copied to the generated oto.gen.go file. I could define them again or do some other type of validation in the implementation, but I think this is a nice and clean way for things like that.

    opened by ghost 2
  • Unsupported time.Time (json.Marshaler/Unmarshaler?)

    Unsupported time.Time (json.Marshaler/Unmarshaler?)

    Good morning!

    Yesterday evening I was giving oto a try and found that time.Time is not supported which made me think it was because I was on Windows at that time.

    I just tried on Ubuntu with the same result: parse input object type: parse type: parse type: /usr/local/go/src/time/time.go:139:2: wall must be exported

    Which led me to think oto is not checking if a type satifies json.Marshaler and/or json.Unmarshaler. Is this something that I should solve in my template?

    Here's my sample definition:

    package definition
    
    import "time"
    
    type ThingService interface {
    	CreateThing(CreateThingRequest) CreateThingResponse
    }
    
    type CreateThingRequest struct {
    	Thing Thing
    }
    
    type CreateThingResponse struct {
    	Result bool
    }
    
    type Thing struct {
    	ID      int64
    	OwnerID int64
    	Type    string
    	Props   map[string]interface{}
    	Created time.Time
    	Enabled bool
    }
    

    I know I could fix this by using a string as type for Created instead but this feels weird as time.Time is supported by encoding/json.

    opened by toqueteos 2
  • Adopting Go generics

    Adopting Go generics

    In the following article, it mentioned generics. Now the go 1.18 is released and has generics support, I was wondering if there is plan to take advantage of generics?

    How code generation wrote our API and CLI.

    When generics lands in Go, we will be able to reduce the amount of boilerplate code generated in favour of generic methods. In this world, we may not need to generate much plumbing code at all.

    opened by michaelwu0505 1
  • Configurable templating engine

    Configurable templating engine

    I've been playing around with oto and one of the first things I wondered was why would the templates not be written in the text/template or the html/template?

    And with that in mind I was thinking if a feature like a configurable templating engine would be interesting from your point of view.

    The benefit is that new comers to oto, who don't know plush (like me), don't have to go and learn a new templating language. There's another, more subjective thing that could be considered: "if the standard lib has what I need, why go to a third party alternative".

    The interface for the command line could be a new optional parameter --engine {text/template|html/template[|plush]}. The absence would mean the plush engine and that would preserve compatibility.


    I've glanced at plush but at first sight couldn't see the need for it over the std lib (could be an historical thing, like at some point in the past the Go templating engines weren't enough), so if its something obvious I'd love to understand, should you know it.

    opened by pedromss 0
  • Is there a way to do matching error on client side ?

    Is there a way to do matching error on client side ?

    I'm trying oto and thanks it's made creating a client lib so easy. I wonder could oto create an error object for the client-side ? I imagine something like this

    // definition.go
    type ServiceError int
    
    const (
    	ErrInternal          ServiceError = 1000
    	ErrPermissionDenined ServiceError = 1001
    	ErrInvalidArgument   ServiceError = 1002
    )
    
    func (s ServiceError) Error() string {
    	switch s {
    	default: // ErrInternal
    		return "internal"
            case ErrInvalidArgument:
                    return "invalid_argument" 
    	case ErrPermissionDenined:
    		return "permission_denied"
    	}
    }
    
    // client.js
    const ErrInvalidArgument = new Error('permission_denied')
    
    // index.js
    try {
       res = await greeter.Greets(req)
    } catch (err) {
       switch (err.message) {
        case greeter.ErrInvalidArgument:
            // do something
         case greeter.ErrPermissionDenined:
            // do something
         case greeter.ErrInternal:
            // do something
       }
    }
    
    
    opened by fahmifan 2
  • Add python client generation to /example

    Add python client generation to /example

    1. Add python client to /example image

    2. [WIP] Generate python classes from def.Objects Requires:

    • (Probably) Adding a Python mapping for (*Parser).parseFieldType ✅
    • Creating classes for def.Objects ✅
    • Create Constructor from object.Fields ✅
    • Adding type hints via field.type 🚧
    • ???
    opened by sethcenterbar 5
Releases(v0.12.1)
Owner
pace.dev
Minimalist project management built specifically for software teams
pace.dev
rpc/v2 support for JSON-RPC 2.0 Specification.

rpc rpc/v2 support for JSON-RPC 2.0 Specification. gorilla/rpc is a foundation for RPC over HTTP services, providing access to the exported methods of

High Performance, Kubernetes Native Object Storage 3 Jul 4, 2021
Go Substrate RPC Client (GSRPC)Go Substrate RPC Client (GSRPC)

Go Substrate RPC Client (GSRPC) Substrate RPC client in Go. It provides APIs and types around Polkadot and any Substrate-based chain RPC calls. This c

Chino Chang 1 Nov 11, 2021
v2 of the veai listener. Now with channels!

veai-listener This is a pipeline tool to allow for watching a dir that takes in video that needs to be upscaled. The dir is accessed via Dropbox and i

Brendan P. 0 Dec 8, 2021
🚀 gnet is a high-performance, lightweight, non-blocking, event-driven networking framework written in pure Go./ gnet 是一个高性能、轻量级、非阻塞的事件驱动 Go 网络框架。

English | ???? 中文 ?? Introduction gnet is an event-driven networking framework that is fast and lightweight. It makes direct epoll and kqueue syscalls

Andy Pan 6.6k Jun 30, 2022
webrpc is a schema-driven approach to writing backend services for modern Web apps and networks

webrpc is a schema-driven approach to writing backend servers for the Web. Write your server's api interface in a schema format of RIDL or JSON, and t

null 457 Jul 3, 2022
Fast and idiomatic client-driven REST APIs.

Vulcain is a brand new protocol using HTTP/2 Server Push to create fast and idiomatic client-driven REST APIs. An open source gateway server which you

Kévin Dunglas 3.3k Jun 29, 2022
High-performance, non-blocking, event-driven, easy-to-use networking framework written in Go, support tls/http1.x/websocket.

High-performance, non-blocking, event-driven, easy-to-use networking framework written in Go, support tls/http1.x/websocket.

lesismal 706 Jul 3, 2022
Fake server, Consumer Driven Contracts and help with testing performance from one configuration file with zero system dependencies and no coding whatsoever

mockingjay server Mockingjay lets you define the contract between a consumer and producer and with just a configuration file you get: A fast to launch

Chris James 519 Jun 18, 2022
A sample web API in GO (with GIn) under a domain driven architecture.

Golang Sample API Domain Driven Design Pattern 1. About This sample project presents a custom made domain driven API architecture in Golang using the

George Batagiannis ⚡ 1 Jan 10, 2022
Event driven modular status-bar for dwm; written in Go & uses Unix sockets for signaling.

dwmstat A simple event-driven modular status-bar for dwm. It is written in Go & uses Unix sockets for signaling. The status bar is conceptualized as a

Navaz Alani 1 Dec 25, 2021
A yaml data-driven testing format together with golang testing library

Specified Yaml data-driven testing Specified is a yaml data format for data-driven testing. This enforces separation between feature being tested the

Design it, Run it 0 Jan 9, 2022
An event driven remote access trojan for experimental purposes.

erat An event driven remote access trojan for experimental purposes. This example is very simple and leverages ssh failed login events to trigger erat

siovador 0 Jan 16, 2022
Package event-driven makes it easy for you to drive events between services

Event-Driven Event-driven architecture is a software architecture and model for application design. With an event-driven system, the capture, communic

Ramooz 3 Apr 20, 2022
hostkey generation for your next golang ssh server

hostkeys A host key manager for your golang ssh daemons hostkeys will manage private keys for an ssh.ServerConfig. It creates missing private keys if

Kristian Mide 14 Nov 14, 2021
HttpRunner+ is the next generation of HttpRunner, written in golang

hrp (HttpRunner+) hrp is a golang implementation of HttpRunner. Ideally, hrp will be fully compatible with HttpRunner, including testcase format and u

HttpRunner 81 Jun 16, 2022
A simulation to see what's the result among normal people、rich-second generation、hard-working people

A simulation to see what's the result of competion among normal people、rich-second generation and hard-working people. 假设: 一个社会集体中有部分富二代,部分努力的人,多数是普通人

Myrainhua 0 Feb 20, 2022
Core is the next-generation digital data engine.

tKeel-Core The digital engine of world ?? Core is the data centre of the tKeel IoT Open Platform, a high-performance, scalable and lightweight next-ge

null 21 Mar 28, 2022
hopefully the the next-generation backend server of bgm.tv

基于 python 的新 api server 开发环境 python 版本: 3.8 依赖管理: poetry web 框架: fastapi quick start: git clone https://github.com/bangumi/server bangumi-server cd ba

Bangumi 343 Jun 27, 2022