An idiomatic Go REST API starter kit (boilerplate) following the SOLID principles and Clean Architecture

Overview

Go RESTful API Starter Kit (Boilerplate)

GoDoc Build Status Code Coverage Go Report

This starter kit is designed to get you up and running with a project structure optimized for developing RESTful API services in Go. It promotes the best practices that follow the SOLID principles and clean architecture. It encourages writing clean and idiomatic Go code.

The kit provides the following features right out of the box:

  • RESTful endpoints in the widely accepted format
  • Standard CRUD operations of a database table
  • JWT-based authentication
  • Environment dependent application configuration management
  • Structured logging with contextual information
  • Error handling with proper error response generation
  • Database migration
  • Data validation
  • Full test coverage
  • Live reloading during development

The kit uses the following Go packages which can be easily replaced with your own favorite ones since their usages are mostly localized and abstracted.

Getting Started

If this is your first time encountering Go, please follow the instructions to install Go on your computer. The kit requires Go 1.13 or above.

Docker is also needed if you want to try the kit without setting up your own database server. The kit requires Docker 17.05 or higher for the multi-stage build support.

After installing Go and Docker, run the following commands to start experiencing this starter kit:

# download the starter kit
git clone https://github.com/qiangxue/go-rest-api.git

cd go-rest-api

# start a PostgreSQL database server in a Docker container
make db-start

# seed the database with some test data
make testdata

# run the RESTful API server
make run

# or run the API server with live reloading, which is useful during development
# requires fswatch (https://github.com/emcrisostomo/fswatch)
make run-live

At this time, you have a RESTful API server running at http://127.0.0.1:8080. It provides the following endpoints:

  • GET /healthcheck: a healthcheck service provided for health checking purpose (needed when implementing a server cluster)
  • POST /v1/login: authenticates a user and generates a JWT
  • GET /v1/albums: returns a paginated list of the albums
  • GET /v1/albums/:id: returns the detailed information of an album
  • POST /v1/albums: creates a new album
  • PUT /v1/albums/:id: updates an existing album
  • DELETE /v1/albums/:id: deletes an album

Try the URL http://localhost:8080/healthcheck in a browser, and you should see something like "OK v1.0.0" displayed.

If you have cURL or some API client tools (e.g. Postman), you may try the following more complex scenarios:

# authenticate the user via: POST /v1/login
curl -X POST -H "Content-Type: application/json" -d '{"username": "demo", "password": "pass"}' http://localhost:8080/v1/login
# should return a JWT token like: {"token":"...JWT token here..."}

# with the above JWT token, access the album resources, such as: GET /v1/albums
curl -X GET -H "Authorization: Bearer ...JWT token here..." http://localhost:8080/v1/albums
# should return a list of album records in the JSON format

To use the starter kit as a starting point of a real project whose package name is github.com/abc/xyz, do a global replacement of the string github.com/qiangxue/go-rest-api in all of project files with the string github.com/abc/xyz.

Project Layout

The starter kit uses the following project layout:

.
├── cmd                  main applications of the project
│   └── server           the API server application
├── config               configuration files for different environments
├── internal             private application and library code
│   ├── album            album-related features
│   ├── auth             authentication feature
│   ├── config           configuration library
│   ├── entity           entity definitions and domain logic
│   ├── errors           error types and handling
│   ├── healthcheck      healthcheck feature
│   └── test             helpers for testing purpose
├── migrations           database migrations
├── pkg                  public library code
│   ├── accesslog        access log middleware
│   ├── graceful         graceful shutdown of HTTP server
│   ├── log              structured and context-aware logger
│   └── pagination       paginated list
└── testdata             test data scripts

The top level directories cmd, internal, pkg are commonly found in other popular Go projects, as explained in Standard Go Project Layout.

Within internal and pkg, packages are structured by features in order to achieve the so-called screaming architecture. For example, the album directory contains the application logic related with the album feature.

Within each feature package, code are organized in layers (API, service, repository), following the dependency guidelines as described in the clean architecture.

Common Development Tasks

This section describes some common development tasks using this starter kit.

Implementing a New Feature

Implementing a new feature typically involves the following steps:

  1. Develop the service that implements the business logic supporting the feature. Please refer to internal/album/service.go as an example.
  2. Develop the RESTful API exposing the service about the feature. Please refer to internal/album/api.go as an example.
  3. Develop the repository that persists the data entities needed by the service. Please refer to internal/album/repository.go as an example.
  4. Wire up the above components together by injecting their dependencies in the main function. Please refer to the album.RegisterHandlers() call in cmd/server/main.go.

Working with DB Transactions

It is the responsibility of the service layer to determine whether DB operations should be enclosed in a transaction. The DB operations implemented by the repository layer should work both with and without a transaction.

You can use dbcontext.DB.Transactional() in a service method to enclose multiple repository method calls in a transaction. For example,

func serviceMethod(ctx context.Context, repo Repository, transactional dbcontext.TransactionFunc) error {
    return transactional(ctx, func(ctx context.Context) error {
        repo.method1(...)
        repo.method2(...)
        return nil
    })
}

If needed, you can also enclose method calls of different repositories in a single transaction. The return value of the function in transactional above determines if the transaction should be committed or rolled back.

You can also use dbcontext.DB.TransactionHandler() as a middleware to enclose a whole API handler in a transaction. This is especially useful if an API handler needs to put method calls of multiple services in a transaction.

Updating Database Schema

The starter kit uses database migration to manage the changes of the database schema over the whole project development phase. The following commands are commonly used with regard to database schema changes:

# Execute new migrations made by you or other team members.
# Usually you should run this command each time after you pull new code from the code repo. 
make migrate

# Create a new database migration.
# In the generated `migrations/*.up.sql` file, write the SQL statements that implement the schema changes.
# In the `*.down.sql` file, write the SQL statements that revert the schema changes.
make migrate-new

# Revert the last database migration.
# This is often used when a migration has some issues and needs to be reverted.
make migrate-down

# Clean up the database and rerun the migrations from the very beginning.
# Note that this command will first erase all data and tables in the database, and then
# run all migrations. 
make migrate-reset

Managing Configurations

The application configuration is represented in internal/config/config.go. When the application starts, it loads the configuration from a configuration file as well as environment variables. The path to the configuration file is specified via the -config command line argument which defaults to ./config/local.yml. Configurations specified in environment variables should be named with the APP_ prefix and in upper case. When a configuration is specified in both a configuration file and an environment variable, the latter takes precedence.

The config directory contains the configuration files named after different environments. For example, config/local.yml corresponds to the local development environment and is used when running the application via make run.

Do not keep secrets in the configuration files. Provide them via environment variables instead. For example, you should provide Config.DSN using the APP_DSN environment variable. Secrets can be populated from a secret storage (e.g. HashiCorp Vault) into environment variables in a bootstrap script (e.g. cmd/server/entryscript.sh).

Deployment

The application can be run as a docker container. You can use make build-docker to build the application into a docker image. The docker container starts with the cmd/server/entryscript.sh script which reads the APP_ENV environment variable to determine which configuration file to use. For example, if APP_ENV is qa, the application will be started with the config/qa.yml configuration file.

You can also run make build to build an executable binary named server. Then start the API server using the following command,

./server -config=./config/prod.yml
Issues
  • Services communication

    Services communication

    Hi @qiangxue! I have one problem and one solution for your template, I'd want to know your comments.

    Problem: one service needs to interact with other services to handle the request in some scenarios, but if service A needs to service B and service B needs to the A, so we will have dependency-cycle in Go.

    example: We have two services: Subscription and Payment. now if the user wants to pay and subscribe, he should request to subscribe and then pay, after payment he redirects to the app and app need to do two jobs:

    • set payment status to verified (if payment is ok).
    • create a new subscription.

    these two services need to interact with each other, how do you handle it in your template?

    If we have microservices, so this is not a problem, because our subscription microservice interact with other microservices over a network. but in monolith apps, we should have a solution.

    Solution: we should implement the subscription and the order services as a subpackage of their packages and on initializing them, pass services interface to them.

    services directory structure relative to my solution:

    ├── order
    │   ├── orderservice
    │   │   └── service.go // the service implementation.
    │   └── service.go // the service interface.
    └── subscription
        ├── service.go // the service interface.
        └── subscriptionservice
            └── service.go // the service implementation.
    

    How do you think about it?

    opened by mehran-prs 6
  • how to add pictures related to albums

    how to add pictures related to albums

    First off, this is amazing groundwork you have put together. Thanks!

    My question is this: How would you (i.e. what is the recommended way to) implement relational data to an entity? E.g how would you add pictures as an entity related to an album?

    opened by kamilakis 4
  • Migrate testdata

    Migrate testdata

    $ make testdata

    make migrate-reset
    make[1]: Entering directory 'HOME_DIR/go/src/go-rest-api'
    Resetting database...
    Are you sure you want to drop the entire database schema? [y/N]
    Aborted dropping the entire database schema
    Makefile:112: recipe for target 'migrate-reset' failed
    make[1]: *** [migrate-reset] Error 1
    make[1]: Leaving directory 'HOME_DIR/go/src/go-rest-api'
    Makefile:83: recipe for target 'testdata' failed
    make: *** [testdata] Error 2
    

    what is the solution?

    opened by itgelo 2
  • token 创建 albums 接口返回401  signature is invalid

    token 创建 albums 接口返回401 signature is invalid

    之前一直觉得go的rest api轮子没有特别好用的,直到这几天发现强哥出了新的,甚是开心。 我是强哥的铁粉,早几年做PHP,跟随强哥的yii1,然后yii2,yii2 太好用了,至今没看的比yii2好用的框架。现在也跟随强哥的go轮子,在做golang了。

    在使用的途中,发现登录之后,用token去创建 albums,提示401 signature is invalid

    用jwt token的操作如下: curl -X POST -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1ODIzNjI4MjgsImlkIjoiMTAwIiwibmFtZSI6ImRlbW8ifQ.UBTz2RJ0l92JRwRYpqQpOiNoaTVx9kheRP7gt0Xv9XE" -d '{"name": "demo321"}' http://localhost:8080/v1/albums

    {"status":401,"message":"signature is invalid"}

    我自己尝试看懂代码,想自己解决,无奈golang还不是很熟练,看了很久也没能找到解决办法,期待强哥的回复。

    opened by wuchongde 1
  • Cannot user MySQL

    Cannot user MySQL

    Describe the bug I'd like to use my local mysql instead of Docker postgresql, so I used this in local.yml:

    dsn: "mysql://127.0.0.1:3306/mydb?sslmode=disable&user=myuser&password=mypassword" To Reproduce Steps to reproduce the behavior:

    1. At make run I get this error:
    make run
    go run -ldflags "-X main.Version=v1.0.1-5-gc310133-dirty" cmd/server/main.go
    {"level":"error","ts":1636005794.9442818,"caller":"server/main.go:47","msg":"sql: unknown driver \"mysql\" (forgotten import?)","version":"v1.0.1-5-gc310133-dirty","stacktrace":"main.main\n\t/home/pc/go/src/go-rest-api/cmd/server/main.go:47\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:203"}
    exit status 255
    Makefile:34: recipe for target 'run' failed
    make: *** [run] Error 1
    

    Expected behavior Expect a clear documenation or hint on how to switch DB

    Environment (please complete the following information):

    • OS: Ubuntu Linux
    opened by bcodus 0
  • routing.GracefulShutdown 无效

    routing.GracefulShutdown 无效

    Describe the bug 关于routing.GracefulShutdown 无效

    To Reproduce Steps to reproduce the behavior: 1.设置routing.GracefulShutdown 返回timeout = 5s 2.设置接口 /healthcheck 延迟 3s 3.make build 构建项目 并运行

    Expected behavior 请求/healthcheck 接口 马上通过control+c 关闭程序 浏览器显示无法访问

    Environment (please complete the following information):

    • OS: MAC(11.0.1)
    • GO: version go1.15.5 darwin/amd64
    • ozzo-routing: v2.3.0

    Additional context 替换 代码 后可以正常 `

    // go routing.GracefulShutdown(hs, 5*time.Second, logger.Infof)
    
    // make sure idle connections returned
    processed := make(chan struct{})
    go func() {
    	c := make(chan os.Signal, 1)
    	signal.Notify(c, os.Interrupt)
    	<-c
    
    	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    	defer cancel()
    	if err := hs.Shutdown(ctx); nil != err {
    		logger.Errorf("server shutdown failed, err: %v\n", err)
    	}
    	logger.Infof("server gracefully shutdown")
    
    	close(processed)
    }()
    
    logger.Infof("server %v is running at %v", Version, address)
    if err := hs.ListenAndServe(); err != nil && err != http.ErrServerClosed {
    	logger.Error(err)
    	os.Exit(-1)
    }
    
    <-processed
    

    `

    opened by skyfall 1
  • About clean architecture

    About clean architecture

    You are implementing the wrong clean architecture. Because each layer depends on. For example, you depend on ozzo-routing. Your repository depend library database. Logic and framework, library not independent.

    If you implement correctly then will easily change framework like echo, gin ... If I wrong please comment and improve together ! Thank you for your share

    opened by tranphuoctien 0
  • how to get started with mysql

    how to get started with mysql

    Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

    Describe the solution you'd like A clear and concise description of what you want to happen.

    Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

    Additional context Add any other context or screenshots about the feature request here.

    opened by xshnjau 0
  • Zap defer logger.Sync()

    Zap defer logger.Sync()

    Hi! Why we don't defer logger.Sync()?

    func ExampleNewStdLog() {
    	logger := zap.NewExample()
    	defer logger.Sync()
    
    	std := zap.NewStdLog(logger)
    	std.Print("standard logger wrapper")
    	// Output:
    	// {"level":"info","msg":"standard logger wrapper"}
    }
    
    opened by kolkov 0
Releases(v1.0.1)
Owner
Qiang Xue
Creator of @yiisoft and @go-ozzo, seasoned software engineer with a background in full-stack Web development, AWS cloud computing, and machine learning.
Qiang Xue
Clean architecture in Go.

go-employee Clean architecture in Go. Based on the learnings from the book Get Your Hands Dirty on Clean Architecture by Tom Hombergs testing requires

RubinThomas 19 Aug 3, 2022
Idiomatic HTTP Middleware for Golang

Negroni Notice: This is the library formerly known as github.com/codegangsta/negroni -- Github will automatically redirect requests to this repository

null 7.2k Aug 4, 2022
🍐 Elegant Golang REST API Framework

An Elegant Golang Web Framework Goyave is a progressive and accessible web application framework focused on REST APIs, aimed at making backend develop

SystemGlitch 1.1k Aug 4, 2022
A small and evil REST framework for Go

go-rest A small and evil REST framework for Go Reflection, Go structs, and JSON marshalling FTW! go get github.com/ungerik/go-rest import "github.com/

Erik Unger 125 Apr 18, 2022
Simple HTTP and REST client library for Go

Resty Simple HTTP and REST client library for Go (inspired by Ruby rest-client) Features section describes in detail about Resty capabilities Resty Co

Go Resty 6.5k Aug 9, 2022
package for building REST-style Web Services using Go

go-restful package for building REST-style Web Services using Google Go Code examples using v3 REST asks developers to use HTTP methods explicitly and

Ernest Micklei 4.5k Aug 1, 2022
Gin is a HTTP web framework written in Go (Golang). It features a Martini-like API with much better performance -- up to 40 times faster. If you need smashing performance, get yourself some Gin.

Gin Web Framework Gin is a web framework written in Go (Golang). It features a martini-like API with performance that is up to 40 times faster thanks

Gin-Gonic 61.7k Aug 8, 2022
Flamingo Framework and Core Library. Flamingo is a go based framework for pluggable web projects. It is used to build scalable and maintainable (web)applications.

Flamingo Framework Flamingo is a web framework based on Go. It is designed to build pluggable and maintainable web projects. It is production ready, f

Flamingo 306 Aug 4, 2022
A lightweight and fast http router from outer space

Alien Alien is a lightweight http router( multiplexer) for Go( Golang ), made for humans who don't like magic. Documentation docs Features fast ( see

Geofrey Ernest 122 Jul 21, 2022
:link: Generate HTML and CSS together, on the fly

On The Fly Package for generating HTML and CSS together, on the fly. Can also be used for generating HTML, XML or CSS (or templates). HTML and CSS can

Alexander F. Rødseth 42 Apr 21, 2022
A simple blog framework built with GO. Uses HTML files and a JSON dict to give you more control over your content.

Go-Blog A simple template based blog framework. Instructions Built for GO version: 1 See the Documentation or Getting Started pages in the wiki. Notes

Matt West 44 Jul 15, 2022
Fast and Reliable Golang Web Framework

Gramework The Good Framework Gramework long-term testing stand metrics screenshot made with Gramework Stats Dashboard and metrics middleware What is i

null 367 Jul 13, 2022
Mango is a modular web-application framework for Go, inspired by Rack, and PEP333.

Mango Mango is a modular web-application framework for Go, inspired by Rack and PEP333. Note: Not actively maintained. Overview Mango is most of all a

Paul Bellamy 368 Aug 7, 2022
This is only a mirror and Moved to https://gitea.com/lunny/tango

Tango 简体中文 Package tango is a micro & pluggable web framework for Go. Current version: v0.5.0 Version History Getting Started To install Tango: go get

Lunny Xiao 835 May 31, 2022
Golang CTF framework and exploit development module

Golang CTF framework and exploit development module

Frank Hübner 12 Aug 5, 2022
Community built data connectors and processors for Spice.ai

data-components-contrib Community built data connectors and processors for Spice.ai The vision for data-components-contrib is a community-driven libra

Spice.ai 8 Apr 5, 2022
A Go rest API project that is following solid and common principles and is connected to local MySQL database.

This is an intermediate-level go project that running with a project structure optimized RESTful API service in Go. API's of that project is designed based on solid and common principles and connected to the local MySQL database.

Kıvanç Aydoğmuş 21 Jun 6, 2022
This is a POC (Proof of Concept) using Hexagonal Architecture, SOLID, DDD, Clean Code, Clean Architecture

Projeto Planetas Star Wars: Esse projeto de trata de uma POC utilizando os conceitos de Clean Arch, Hexagonal Arch, Clean Code, DDD, e SOLID. O princi

Marcelo Martins 0 Feb 10, 2022
Gecho-starter - A starter/boilerplate for Golang Echo API

gecho-starter gecho-starter is a boilerplate for writing services in Golang quic

Jesse Pham 5 Jul 24, 2022
This is an example to demonstrate implementation golang microservices using domain driven design principles and sugestions from go-kit

go-kit DDD Domain Driven Design is prevelent and rising standard for organizing your microservice code. This design architecture emphasis on Code orga

Sourabh Mandal 1 Feb 9, 2022
REST API boilerplate built with go and clean architecture - Echo Framework

GO Boilerplate Prerequisite Install go-migrate for running migration https://github.com/golang-migrate/migrate App requires 2 database (postgreSQL an

Syahid Firdaus 4 Jun 8, 2022
Golang Clean Architecture based on Uncle Bob's Clean Architecture and Summer internship in 2021

clean-architecture-api Description This is an example of implemention of Clean Architecture in Golang projects. This project has 4 layer : Infrastruct

daiki sakuma 4 Feb 20, 2022
Go (Golang) Clean Architecture based on Reading Uncle Bob's Clean Architecture

go-clean-arch Changelog v1: checkout to the v1 branch Proposed on 2017, archived to v1 branch on 2018 Desc: Initial proposal by me. The story can be r

Iman Tumorang 6.3k Aug 5, 2022
A boilerplate/starter project for quickly building APIs using Golang with Clean Architechture

A boilerplate/starter project for quickly building APIs using Golang with Clean Architechture

Nyoman Frastyawan 1 Mar 2, 2022
A software architecture style example for APIs that utilizes the features of SOLID-Principle.

Engelbyte's Waterbyte Clean Architecture A software architecture style example for APIs that utilizes the features of SOLID-Principle. The example sho

Waterbyte 0 Feb 9, 2022
Go-kit-microservices - Example microservices implemented with Go Kit

Go Kit Microservices Example microservices implemented with go kit, a programmin

Hao-Ming, Hsu 1 Jan 18, 2022
⚔ Personal Golang starter kit with an engineer research perspective, expressjs developer friendly, and aims for rapid app development.

Goku (WIP; Author Only) ⚔ Personal Golang starter kit with an engineer research perspective, expressjs developer friendly, and aims for rapid app deve

Fauzan 1 Jan 6, 2022
Onion-Layer-Golang - A web application boilerplate built with go and clean architecture

go-clean-architecture-web-application-boilerplate A web application boilerplate

Endy Gigih Pratama 1 May 4, 2022
Gin-boilerplate - Gin boilerplate preconfigured with basic rest api features

Gin Boilerplate Build apis with gin faster with this template Features Validatio

Mohan barman 4 Jun 24, 2022