go-zero is a web and rpc framework that with lots of engineering practices builtin.

Overview

go-zero

English | 简体中文

Go codecov Go Report Card Release License: MIT

0. what is go-zero

go-zero is a web and rpc framework that with lots of engineering practices builtin. It’s born to ensure the stability of the busy services with resilience design, and has been serving sites with tens of millions users for years.

go-zero contains simple API description syntax and code generation tool called goctl. You can generate Go, iOS, Android, Kotlin, Dart, TypeScript, JavaScript from .api files with goctl.

Advantages of go-zero:

  • improve the stability of the services with tens of millions of daily active users
  • builtin chained timeout control, concurrency control, rate limit, adaptive circuit breaker, adaptive load shedding, even no configuration needed
  • builtin middlewares also can be integrated into your frameworks
  • simple API syntax, one command to generate couple of different languages
  • auto validate the request parameters from clients
  • plenty of builtin microservice management and concurrent toolkits

Architecture

1. Backgrounds of go-zero

At the beginning of 2018, we decided to re-design our system, from monolithic architecture with Java+MongoDB to microservice architecture. After researches and comparison, we chose to:

  • Golang based
    • great performance
    • simple syntax
    • proven engineering efficiency
    • extreme deployment experience
    • less server resource consumption
  • Self-designed microservice architecture
    • I have rich experience on designing microservice architectures
    • easy to location the problems
    • easy to extend the features

2. Design considerations on go-zero

By designing the microservice architecture, we expected to ensure the stability, as well as the productivity. And from just the beginning, we have the following design principles:

  • keep it simple
  • high availability
  • stable on high concurrency
  • easy to extend
  • resilience design, failure-oriented programming
  • try best to be friendly to the business logic development, encapsulate the complexity
  • one thing, one way

After almost half a year, we finished the transfer from monolithic system to microservice system, and deployed on August 2018. The new system guaranteed the business growth, and the system stability.

3. The implementation and features of go-zero

go-zero is a web and rpc framework that integrates lots of engineering practices. The features are mainly listed below:

  • powerful tool included, less code to write
  • simple interfaces
  • fully compatible with net/http
  • middlewares are supported, easy to extend
  • high performance
  • failure-oriented programming, resilience design
  • builtin service discovery, load balancing
  • builtin concurrency control, adaptive circuit breaker, adaptive load shedding, auto trigger, auto recover
  • auto validation of API request parameters
  • chained timeout control
  • auto management of data caching
  • call tracing, metrics and monitoring
  • high concurrency protected

As below, go-zero protects the system with couple layers and mechanisms:

Resilience

4. Future development plans of go-zero

  • auto generate API mock server, make the client debugging easier
  • auto generate the simple integration test for the server side just from the .api files

5. Installation

Run the following command under your project:

go get -u github.com/tal-tech/go-zero

6. Quick Start

  1. full examples can be checked out from below:

    Rapid development of microservice systems

    Rapid development of microservice systems - multiple RPCs

  2. install goctl

    goctlcan be read as go control. goctl means not to be controlled by code, instead, we control it. The inside go is not golang. At the very beginning, I was expecting it to help us improve the productivity, and make our lives easier.

    GO111MODULE=on go get -u github.com/tal-tech/go-zero/tools/goctl

    make sure goctl is executable.

  3. create the API file, like greet.api, you can install the plugin of goctl in vs code, api syntax is supported.

    type Request struct {
      Name string `path:"name,options=you|me"` // parameters are auto validated
    }
    
    type Response struct {
      Message string `json:"message"`
    }
    
    service greet-api {
      @handler GreetHandler
      get /greet/from/:name(Request) returns (Response);
    }

    the .api files also can be generate by goctl, like below:

    goctl api -o greet.api
  4. generate the go server side code

    goctl api go -api greet.api -dir greet

    the generated files look like:

    ├── greet
    │   ├── etc
    │   │   └── greet-api.yaml        // configuration file
    │   ├── greet.go                  // main file
    │   └── internal
    │       ├── config
    │       │   └── config.go         // configuration definition
    │       ├── handler
    │       │   ├── greethandler.go   // get/put/post/delete routes are defined here
    │       │   └── routes.go         // routes list
    │       ├── logic
    │       │   └── greetlogic.go     // request logic can be written here
    │       ├── svc
    │       │   └── servicecontext.go // service context, mysql/redis can be passed in here
    │       └── types
    │           └── types.go          // request/response defined here
    └── greet.api                     // api description file
    

    the generated code can be run directly:

    cd greet
    go mod init
    go mod tidy
    go run greet.go -f etc/greet-api.yaml

    by default, it’s listening on port 8888, while it can be changed in configuration file.

    you can check it by curl:

    curl -i http://localhost:8888/greet/from/you

    the response looks like:

    HTTP/1.1 200 OK
    Date: Sun, 30 Aug 2020 15:32:35 GMT
    Content-Length: 0
  5. Write the business logic code

    • the dependencies can be passed into the logic within servicecontext.go, like mysql, reds etc.
    • add the logic code in logic package according to .api file
  6. Generate code like Java, TypeScript, Dart, JavaScript etc. just from the api file

    goctl api java -api greet.api -dir greet
    goctl api dart -api greet.api -dir greet
    ...

7. Benchmark

benchmark

Checkout the test code

8. Documents (adding)

9. Chat group

Join the chat via https://join.slack.com/t/go-zeroworkspace/shared_invite/zt-m39xssxc-kgIqERa7aVsujKNj~XuPKg

Comments
  • 访问 rpc 时为什么不能直接使用 protobuf 生成的 Client,而要封装一层

    访问 rpc 时为什么不能直接使用 protobuf 生成的 Client,而要封装一层

    例如以下代码 https://github.com/zeromicro/zero-examples/blob/main/bookstore/api/internal/svc/servicecontext.go

    type ServiceContext struct {
    	Config  config.Config
    	Adder   adder.Adder
    	Checker checker.Checker
    }
    
    func NewServiceContext(c config.Config) *ServiceContext {
    	return &ServiceContext{
    		Config:  c,
    		Adder:   adder.NewAdder(zrpc.MustNewClient(c.Add)),
    		Checker: checker.NewChecker(zrpc.MustNewClient(c.Check)),
    	}
    }
    

    中的 Adder 可否改成 protobuf 生成的 AdderClient,这样写会有什么问题吗?

    type ServiceContext struct {
    	Config  config.Config
    	Adder   add.AdderClient
    	Checker checker.Checker
    }
    
    func NewServiceContext(c config.Config) *ServiceContext {
    	return &ServiceContext{
    		Config:  c,
    		Adder:   add.NewAdderClient(zrpc.MustNewClient(c.Add).Conn()),
    		Checker: checker.NewChecker(zrpc.MustNewClient(c.Check)),
    	}
    }
    
    stale 
    opened by wuyazi 3
  •  cannot use transformer.NewTransformer(zrpc.MustNewClient(c.Transform)) (type transformer.Transformer) as type zrpc.RpcClientConf in field value

    cannot use transformer.NewTransformer(zrpc.MustNewClient(c.Transform)) (type transformer.Transformer) as type zrpc.RpcClientConf in field value

    internal/svc/servicecontext.go:17:3: cannot use transformer.NewTransformer(zrpc.MustNewClient(c.Transform)) (type transformer.Transformer) as type zrpc.RpcClientConf in field value

    stale 
    opened by Kevin-Ben 3
  • bloom bug?

    bloom bug?

    package main
    
    import (
        "fmt"
        "github.com/spf13/cast"
    
        "github.com/tal-tech/go-zero/core/bloom"
        "github.com/tal-tech/go-zero/core/stores/redis"
    )
    
    func main() {
        store := redis.NewRedis("localhost:6379", "node")
        filter := bloom.New(store, "testbloom", 64)
    
        for i := 0; i < 10; i++ {
            filter.Add([]byte("kevin" + cast.ToString(i)))
        }
        fmt.Println(filter.Exists([]byte("kevin")))
        fmt.Println(filter.Exists([]byte("wan")))
        fmt.Println(filter.Exists([]byte("nothing")))
    }
    

    result:

    false <nil>
    false <nil>
    true <nil>
    

    This does not seem to meet expectations

    stale 
    opened by hr3685930 3
  • is:issue is:open 路径:zero-examples/shorturl/api/internal/handler/,expandhandler.go和shortenhandler.go两个文件中的func expandHandler改为func ExpandHandler,func shortenHandler改为func ShortenHandler

    is:issue is:open 路径:zero-examples/shorturl/api/internal/handler/,expandhandler.go和shortenhandler.go两个文件中的func expandHandler改为func ExpandHandler,func shortenHandler改为func ShortenHandler

    is:issue is:open 路径:zero-examples/shorturl/api/internal/handler/,expandhandler.go和shortenhandler.go两个文件中的func expandHandler改为func ExpandHandler,func shortenHandler改为func ShortenHandler

    stale 
    opened by lling1234 2
  • can not use $ goctl rpc proto -src transform.proto -dir .

    can not use $ goctl rpc proto -src transform.proto -dir .

    https://go-zero.dev/cn/shorturl.html ,the titile is 6. 编写 transform rpc 服务 when i use goctl rpc proto -src transform.proto -dir . get the result No help topic for 'proto' so i read the docs and found correct command

    $ goctl rpc protoc transform.proto --go_out=. --go-grpc_out=. --zrpc_out=.
    

    but i get

    ➜goctl rpc protoc  transform.proto --go-grpc_out . --go_out . --zrpc_out .
    protoc-gen-go-grpc: unable to determine Go import path for "transform.proto"
    
    Please specify either:
            • a "go_package" option in the .proto source file, or
            • a "M" argument on the command line.
    
    See https://developers.google.com/protocol-buffers/docs/reference/go-generated#package for more information.
    
    --go-grpc_out: protoc-gen-go-grpc: Plugin failed with status code 1.
    

    please teach me how to use goctl rpc protoc transform.proto thank you

    opened by xiao-jay 2
  • failed to get demo dependencies

    failed to get demo dependencies

    /go/src/github.com/zeromicro/zero-examples/shorturl/rpc/transform$ go get
    shorturl/rpc/transform imports
            github.com/tal-tech/go-zero/zrpc imports
            github.com/tal-tech/go-zero/zrpc/internal imports
            github.com/tal-tech/go-zero/zrpc/internal/resolver imports
            k8s.io/client-go/informers imports
            k8s.io/api/batch/v2alpha1: cannot find module providing package k8s.io/api/batch/v2alpha1
    shorturl/rpc/transform imports
            github.com/tal-tech/go-zero/zrpc imports
            github.com/tal-tech/go-zero/zrpc/internal imports
            github.com/tal-tech/go-zero/zrpc/internal/resolver imports
            k8s.io/client-go/informers imports
            k8s.io/api/discovery/v1alpha1: cannot find module providing package k8s.io/api/discovery/v1alpha1
    
    opened by zhlicen 0
Owner
null
RPC Framework abstraction layer. Provides foundation of the RonyDesc to generate RPC server/client codes.

RonyKit RonyKit provides the abstraction layer for creating a cluster aware API server. By defining separate components for each task, you are almost

Ronak Software Group 5 Dec 15, 2022
Microservice framework following best cloud practices with a focus on productivity.

patron Patron is a framework for creating microservices, originally created by Sotiris Mantzaris (https://github.com/mantzas). This fork is maintained

Beat Labs 102 Dec 22, 2022
Flamingo Framework and Core Library. Flamingo is a go based framework for pluggable web projects. It is used to build scalable and maintainable (web)applications.

Flamingo Framework Flamingo is a web framework based on Go. It is designed to build pluggable and maintainable web projects. It is production ready, f

Flamingo 343 Jan 5, 2023
Golanger Web Framework is a lightweight framework for writing web applications in Go.

/* Copyright 2013 Golanger.com. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except

golanger 298 Nov 14, 2022
The jin is a simplified version of the gin web framework that can help you quickly understand the core principles of a web framework.

jin About The jin is a simplified version of the gin web framework that can help you quickly understand the core principles of a web framework. If thi

null 8 Jul 14, 2022
An efficient, extensible and easy-to-use RPC framework.

eRPC eRPC is an efficient, extensible and easy-to-use RPC framework. Suitable for RPC, Microservice, Peer-to-Peer, IM, Game and other fields. 简体中文 Ins

henrylee2cn 2.4k Dec 29, 2022
⚡ Rux is an simple and fast web framework. support middleware, compatible http.Handler interface. 简单且快速的 Go web 框架,支持中间件,兼容 http.Handler 接口

Rux Simple and fast web framework for build golang HTTP applications. NOTICE: v1.3.x is not fully compatible with v1.2.x version Fast route match, sup

Gookit 84 Dec 8, 2022
Roche is a Code Generator and Web Framework, makes web development super concise with Go, CleanArch

It is still under development, so please do not use it. We plan to release v.1.0.0 in the summer. roche is a web framework optimized for microservice

Riita 14 Sep 19, 2022
A powerful go web framework for highly scalable and resource efficient web application

webfr A powerful go web framework for highly scalable and resource efficient web application Installation: go get -u github.com/krishpranav/webfr Exa

Krisna Pranav 13 Nov 28, 2021
A powerful go web framework for highly scalable and resource efficient web application

A powerful go web framework for highly scalable and resource efficient web application

null 22 Oct 3, 2022
A web app built using Go Buffalo web framework

Welcome to Buffalo Thank you for choosing Buffalo for your web development needs. Database Setup It looks like you chose to set up your application us

Mike Okoth 0 Feb 7, 2022
Gerasimos (Makis) Maropoulos 23.4k Dec 28, 2022
gnark is a fast, open-source library for zero-knowledge proof protocols written in Go

gnark gnark is a framework to execute (and verify) algorithms in zero-knowledge. It offers a high-level API to easily design circuits and fast impleme

ConsenSys Software 619 Jan 1, 2023
gnark is a fast, open-source library for zero-knowledge proof protocols written in Go

gnark gnark is a framework to execute (and verify) algorithms in zero-knowledge. It offers a high-level API to easily design circuits and fast impleme

ConsenSys Software 1.3k Jun 1, 2021
hiboot is a high performance web and cli application framework with dependency injection support

Hiboot - web/cli application framework About Hiboot is a cloud native web and cli application framework written in Go. Hiboot is not trying to reinven

hidevops.io 179 Nov 20, 2022
Package macaron is a high productive and modular web framework in Go.

Macaron Package macaron is a high productive and modular web framework in Go. Getting Started The minimum requirement of Go is 1.6. To install Macaron

Macaron 3.3k Jan 2, 2023
Mango is a modular web-application framework for Go, inspired by Rack, and PEP333.

Mango Mango is a modular web-application framework for Go, inspired by Rack and PEP333. Note: Not actively maintained. Overview Mango is most of all a

Paul Bellamy 371 Nov 17, 2022
Simple and lightweight Go web framework inspired by koa

VOX A golang web framework for humans, inspired by Koa heavily. Getting started Installation Using the go get power: $ go get -u github.com/aisk/vox B

An Long 79 Dec 14, 2022