An production-ready microservice using Go and a few lightweight libraries

Overview

Go Micro Example

This small sample project was created as a collection of the various things I've learned about best practices building microservices using Go. I structured the project using a hexagonal style abstracting away business logic from dependencies like the RESTful API, the Postgres, and RabbitMQ.

Structure

The Go community generally likes application directory structures to be as simple as possible which is totally admirable and applicable for a small simple microservice. I could probably have kept everything for this project in a single directory and focused on making sure it met twelve factors. But I'm a big fan of Domain Driven Design, and how it gels so nicely with Hexagonal Architecture and I wanted to see how a Go microservice might look structured using them.

The starting point of the application is under the cmd/app directory. The "domain" core of the application where all business logic should reside is under the internal/core directory. The other directories listed there are each of the external dependencies for the project.

structure diagram

Running the Application Locally

This project requires that you have Docker, Go and Make installed locally. If you do, you can start the application first by starting the docker-compose file, then start the application using the supplied Makefile.

docker-compose -f ./scripts/docker-compose.yml up -d
make run

If you want to create a deployable executable and run it:

make build
./bin/inventory

Run Docker Compose

docker-compose up

Application Features

RESTful API

This application uses the wonderful go-chi for routing beautifuly documentation served as the main inspiration for how to structure the API. Seriously, I was so impressed.

In Java I like to generate the controller layer using Open API so that the contract and implementation always match exactly. I couldn't quite find an equivalent solution I liked.

Truth be told, if I were doing inter-microservice communication I would strongly consider using gRPC rather than a RESTful API.

Authentication

Many of the endpoints in this project are protected by using a simple authentication middleware. If you're interested in hitting them you can use basic auth admin:admin. Users are stored in the database along with their hashed password. Users are locally cached using golang-lru. In a production setting if I actually wanted caching I'd either use a remote cache like Redis, or a distributed local cache like groupcache to prevent stale or out of sync data.

Metrics

This application outputs prometheus metrics using middleware I plugged into the go-chi router. If you're running locally check them out at http://localhost:8080/metrics. Every URL automatically gets a hit count and a latency metric added. You can find the configurations here.

Logging

I ended up going with zerolog for logging in this project. I really like its API and their benchmarks look really great too! You can get structured logging or nice human readable logging by changing some configs

Configuration

I'm most comfortable with using Spring Cloud Config for externalized configurations, but I couldn't find any libraries written for Go that can connect to Spring Cloud Config servers so I wrote one.

Testing

I chose not to go with any of the test frameworks when putting this project together. I felt like using interfaces and injecting dependencies would be enough to allow me to mock what I need to. There's a fair bit of boilerplate code required to mock, say, the inventory repository but not having to pull in and learn yet another dependency for testing seemed like a fair tradeoff.

The testing in this project is pretty bare-bones and mostly just proof-of-concept. If you want to see some tests, though, they're in internal/api/invapi. I personally prefer more integration tests that test an application front-to-back for features rather than tons and tons of tightly-coupled unit tests.

Database Migrations

I'm using the migrate project to manage database migrations.

migrate create -ext sql -dir db/migrations -seq create_products_table

migrate -database postgres://postgres:[email protected]:5432/smfg-db?sslmode=disable -path db/migrations up

migrate -source file://db/migrations -database postgres://localhost:5432/database down

12 Factors

One of the goals of this service was to ensure all 12 principals of a 12-factor app are adhered to. This was a nice way to make sure the app I built offered most of what you need out of a Spring Boot application.

I. Codebase

The application is stored in my git repository.

II. Dependencies

Go handles this for us through its dependency management system (yay!)

III. Config

The app initially starts up using environment variables that define a URL and credentials for connecting to a Spring Cloud Config service. This allows for the externalization of configurations which are managed through a typical version control system. Pretty nice! I wrote a basic library for connecting to and reading configurations from Spring Cloud Config since I'm mostly writing Spring Boot microservices these days.

IV. Backing Services

The application connects to all external dependencies (in this case, RabbitMQ, and Postgres) via URLs which it gets from remote configuration.

V. Build, release, run

The application can easily be plugged into any CI/CD pipeline. This is mostly thanks to Go making this easy through great command line tools.

VI. Processes

This app is not strictly stateless. There is a cache in the user repository. This was a design choice I made in the interest of seeing what setting up a local cache in go might look like. In a more real-world application you would probably want an external cache (like Redis), or a distributed cache (like Group Cache - which is really cool!)

This app is otherwise stateless and threadsafe.

VII. Port Binding

The application binds to a supplied port on startup.

VIII. Concurrency

Other than maintaining an instance-based cache (see Process above), the application will scale horizontally without issue. The database dependency would need to scale vertically unless you started using sharding, or a distributed data store like Cosmos DB.

IX. Disposability

One of the wonderful things about Go is how fast it starts up. This application can start up and shut down in a fraction of the time that similar Spring Boot microservices. In addition they use a much smaller footprint. This is perfect for services that need to be highly elastic on demand.

X. Dev/Prod Parity

Docker makes standing up a prod-like environment on your local environment a breeze. This application has a docker-compose file that starts up a local instance of rabbit and postgres. This obviously doesn't account for ensuring your dev and stage environments are up to snuff but at least that's a good start for local development.

XI. Logs

Logs in the application are written to the stdout allowing for logscrapers like logstash to consume and parse the logs. Through configuration the logs can output as plain text for ease of reading during local development and then switched after deployment into json structured logs for automatic parsing.

XII. Admin Processes

Database migration is automated in the project using migrate.

You might also like...
Kitex byte-dance internal Golang microservice RPC framework with high performance and strong scalability, customized extensions for byte internal.
Kitex byte-dance internal Golang microservice RPC framework with high performance and strong scalability, customized extensions for byte internal.

Kitex 字节跳动内部的 Golang 微服务 RPC 框架,具有高性能、强可扩展的特点,针对字节内部做了定制扩展。

Trying to build an Ecommerce Microservice in Golang and Will try to make it Cloud Native - Learning Example extending the project of Nic Jackson

Golang Server Project Best Practices Dependency Injection :- In simple words, we want our functions and packages to receive the objects they depend on

Fast HTTP microservice written in Go for high-level image processing backed by bimg and libvips.

imaginary Fast HTTP microservice written in Go for high-level image processing backed by bimg and libvips. imaginary can be used as private or public

micro-draft-manager is a microservice that helps you to manage unstructured data in your application with sorting and full-text search

micro-draft-manager is a microservice that helps you to manage unstructured data in your application with sorting and full-text search. For example, y

Omh-users-management - A go microservice that can enables us to create, modify, fetch, and delete users

Users Management System A go microservice that can enables us to create, modify, fetch, and delete users Usage To Run the application $ go run main.

Golang Microservice making use of protobuf and gRPC as the underlying transport protocol.

Go-Microservices Golang Microservice making use of protobuf and gRPC as the underlying transport protocol. I will be building a generic microservice,

Cortex Gateway: a microservice which strives to help you administrating and operating your Cortex Cluster in multi tenant environments
Cortex Gateway: a microservice which strives to help you administrating and operating your Cortex Cluster in multi tenant environments

Cortex Gateway Cortex Gateway is a microservice which strives to help you administrating and operating your Cortex Cluster in multi tenant environment

Go gRPC RabbitMQ email microservice

Go, RabbitMQ and gRPC Clean Architecture microservice 👋 👨‍💻 Full list what has been used: GRPC - gRPC RabbitMQ - RabbitMQ sqlx - Extensions to data

A Microservice Toolkit from The New York Times
A Microservice Toolkit from The New York Times

Gizmo Microservice Toolkit This toolkit provides packages to put together server and pubsub daemons with the following features: Standardized configur

Comments
Owner
Sean K Smith
Java developer at work, Go developer for fun!
Sean K Smith
Authentication-microservice - Microservice for user authentication built with golang and gRPC

Authentication-microservice - Microservice for user authentication built with golang and gRPC

Isaque Veras 6 May 30, 2022
Customer-microservice - Microservice of customer built with golang and gRPC

?? Building microservices to manage customer data using Go and gRPC Command to g

Isaque Veras Labs 3 Sep 8, 2022
Microservice - A sample architecture of a microservice in go

#microservice Folder structure required. service certs config config.yaml loggin

null 0 Feb 3, 2022
Microservice - Microservice golang & nodejs

Microservice Gabungan service dari bahasa pemograman go, nodejs Demo API ms-auth

Didin Tri Anggoro 2 May 21, 2022
GCP Cloud Functions ready to Go starter with hot reload 🔥

GCP Cloud Functions - Go Starter Features: funcFramework already set up - ready for local development and testing. Hot Reload ready-to-go thanks to th

Eric Crescioni 0 Dec 16, 2021
An example microservice demo using kubernetes concepts like deployment, services, persistent volume and claims, secrets and helm chart

Docker vs Kubernetes Docker Kubernetes container tech, isolated env for apps infra management, multiple containers automated builds and deploy apps -

abhijit wakchaure 0 Dec 13, 2021
Go microservice tutorial project using Domain Driven Design and Hexagonal Architecture!

"ToDo API" Microservice Example Introduction Welcome! ?? This is an educational repository that includes a microservice written in Go. It is used as t

Mario Carrion 616 Jan 4, 2023
String Service. Microservice example using gokit library

Example of Microservices using go-kit Go kit is a collection of Go (golang) packages (libraries) that help you build robust, reliable, maintainable mi

Gabriel Camps 0 Dec 6, 2021
A microservice gateway developed based on golang.With a variety of plug-ins which can be expanded by itself, plug and play. what's more,it can quickly help enterprises manage API services and improve the stability and security of API services.

Goku API gateway is a microservice gateway developed based on golang. It can achieve the purposes of high-performance HTTP API forwarding, multi tenant management, API access control, etc. it has a powerful custom plug-in system, which can be expanded by itself, and can quickly help enterprises manage API services and improve the stability and security of API services.

Eolink 378 Dec 29, 2022
Microservice Boilerplate for Golang with gRPC and RESTful API. Multiple database and client supported

Go Microservice Starter A boilerplate for flexible Go microservice. Table of contents Features Installation Todo List Folder Structures Features: Mult

Ahmad Saugi 14 Jul 28, 2022