A Go rest API project that is following solid and common principles and is connected to local MySQL database.

Overview

RESTful API in GO

This is an intermediate-level go project that running with a project structure optimized RESTful API service in Go. API's of that project is designed based on solid and common principles and connected to the local MySQL database.

Highlights of that project are listed at below

  • The RESTful API presents standard CRUD operations of a database table
  • This project has clean architecture and it has been covered with tests.
  • Data validation
  • It uses JWT-based authentication and auth middleware.
  • Each token is expired in 15 minutes to prevent system bugs.
  • Error handling is done with clear responses.
  • It presents structured logging with the username, password for 15 minutes with help of a token that is created by JWT. After that, the user can create TODOs.
  • The project used the following packages during the development time

Getting Started

If you're have not encountered Go before, you should visit this website here

After installing Go , you should run the following commands to experience this project

# download the starter code
git clone https://github.com/Kivanc10/golang-rest-api-with-mysql.git

# open the code
cd golang-rest-api-with-mysql

# start the database server and run the code
go run ./operate/operate.go

After that, you have a RESTful API that is running at http://127.0.0.1:8080. It provides us following endpoints

  • GET /users : it provides us the list of all users logged-in
  • POST /signUp : it allows the user to sign up. It saves the user info into db and creates token with JWT.It accepts attached data like that:
    • PersonID can be anything because db arranged to auto increment
    • {
          "PersonID":0,
          "UserName":"sample user name",
          "Password":"12312321"
       }
  • POST /signIn : authenticates and login. It creates token again with JWT.It accepts attached data to , something like up above.
  • GET /users/me : It allows the user to access his information.To do this , user must be authenticated,otherwise system wil not let that happen
  • PUT /users/update/me : It updates the current authenticated user with accepted data.To do this user must be authenticated.It accepts attached data like:
    • {
            "PersonID":0,
            "UserName":"new user name",
            "Password":"new password"
      }
  • DELETE /user/me : It deletes the current authenticated user.To do this user must be authenticated.
  • GET /users/logout/me : It allows the user to logout from all tokens.To do this user must be authenticated.The user will be not deleted from db.
  • POST /todo : It allows the user to create todos. To do this user must be authenticated.It accepts attached data like that :
    •     {
              "Context" : "sample todo"
           }
  • GET /todos : It lists all todos that created by authenticated users.
  • GET /todos/me : It lists just the todos belong to the current user authenticated

If you have API client tools like Postman, you can handle complicated operations easily

If you create a new environment in Postman and declare variables you'll use , you make everything clear.

You also should add a set of codes in the test section of some requests arrange automatically with the bearer token of the header of the request.(signup,login,getme,update,logout) Before the code,you should define an environment token named authToken in section of Authorization of edit collection.

Then past the code in signup,login,getme,update,logout requests

if (pm.response.code == 200){
    pm.environment.set('authToken',pm.response.json()["Token"])
}

After that,your header of the request will updated by valid tokens automatically to authenticate.

You must ensure that you choose that environment

how to use endpoints

# sign up the user via POST /signUp
curl -X POST -H "Content-Type: application/json" -d `{ "PersonID":0,"UserName":"sample user name","Password":"12312321"}` http://localhost:8080/signUp
# it should return response.header with jwt and token

# sign in with user via POST /signIn
curl -X POST -H "Content-Type: application/json" -d `{ "PersonID":0,"UserName":"sample user name","Password":"12312321"}` http://localhost:8080/signIn
# it should return response.Header with jwt and token
# save token during the loggedin and inherit auth from postman environment,it handles itself
curl -X GET -H "Authorization: Bearer ...JWT token here..." http://localhost:8080/users/me

# to create todos for the user authenticated
curl -X POST -H "Authorization: Bearer ...JWT token here..." -d `{"Context" : "sample todo"}` http://localhost:8080/todo
# it returns the saved todo belong to the user authenticated

Project layout

.
├── dbOp                 main database operations of the project
├── middleware           auth middleware and tokens will create
├── operate              main application of the project
├── route                routes operations of the project related to RESTful api functions
└── static               static files belong to the project

Working with JWT based authenticaton and auth middleware

JWT
  • JSON Web Token (JWT) is a self-contained way for securely transmitting information between parties as a JSON object. If you will deal with authorization and information exchange, the most logical thing you will do is use JWT.
Auth middleware
  • We have the authentication service and its adapter and the login middleware in place, we can create middleware that checks for authenticated users, having it redirect to the /login page if the user is not authenticated

In that project,we create tokens if necessary such as signUp,signIn,... to make and save changes.Then we use auth middleware to access the current user if its token is valid. Auth middleware allows us to design a real secure system.

to create tokens

", atClaims) return token, nil // return token created and no error if succeed } ">
func CreateToken(userId uint64, name string) (string, error) { // it accepts userId and username
/* This function creates a token belong to the user with a set of information. The created token will be expired in 15 minutes */
	var err error
	//Creating Access Token
	os.Setenv("ACCESS_SECRET", mySignInKey) // define a global access key
	atClaims := jwt.MapClaims{} // create empty map to store keys-values belong to the user
  # store infos into the map
	atClaims["authorized"] = true
	atClaims["user_id"] = userId
	atClaims["user_name"] = name
	atClaims["exp"] = time.Now().Add(time.Minute * 15).Unix() // token is valid for 15 minutes
	at := jwt.NewWithClaims(jwt.SigningMethodHS256, atClaims) // use HS256 algorithm
	token, err := at.SignedString([]byte(os.Getenv("ACCESS_SECRET")))
	if err != nil { // error handling
		return "", errors.New("an error occured during the create token")
	}
	fmt.Println("jwt map --> ", atClaims)
	return token, nil // return token created and no error if succeed
}

to integrate auth middleware

%s and len -> %d\n", authHeader, len(authHeader)) if len(authHeader) != 2 || authHeader[0] == "null" { // ["Bearer ","Token..."],if it is not like that,there is an error there //fmt.Println("Malformed token") w.WriteHeader(http.StatusUnauthorized) w.Write([]byte("Malformed Token")) log.Fatal("Malformed token") } jwtToken := authHeader[1] // get the token token, err := jwt.Parse(jwtToken, func(token *jwt.Token) (interface{}, error) { // parse the token if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok { return nil, fmt.Errorf("Unexpected signing method: %v", token.Header["alg"]) } return []byte(mySignInKey), nil }) // type conversion with jwt.MapClaims and to check the token is valid if claims, ok := token.Claims.(jwt.MapClaims); ok && token.Valid { // if that token is valid ctx := context.WithValue(r.Context(), "props", claims) // props is context key // Access context values in handlers like this next.ServeHTTP(w, r.WithContext(ctx)) // if succeed serve http with context } else { // if there is an error fmt.Println("token err -> ", err) //r.Header.Set("ExpiredToken", jwtToken) //DelTokenIfExpired(jwtToken) // usernameInter := claims["user_name"] // if username, ok := usernameInter.(fmt.Stringer); ok { // person := dbop.GetPersonToDelToken(username.String()) // dbop.DeleteTokenIfExpired(person) // } w.WriteHeader(http.StatusUnauthorized) w.Write([]byte("you are Unauthorized or your token is expired")) } } }) } ">
func MiddleWare(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		
		 
			authHeader := strings.Split(r.Header.Get("Authorization"), "Bearer ") // split request header acc. to Bearer 
			fmt.Printf("authheader -> %s and len -> %d\n", authHeader, len(authHeader))
			if len(authHeader) != 2 || authHeader[0] == "null" { // ["Bearer ","Token..."],if it is not like that,there is an error there
				//fmt.Println("Malformed token")
				w.WriteHeader(http.StatusUnauthorized)
				w.Write([]byte("Malformed Token"))
				log.Fatal("Malformed token")
        }
			
				jwtToken := authHeader[1] // get the token
				token, err := jwt.Parse(jwtToken, func(token *jwt.Token) (interface{}, error) { // parse the token
					if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
						return nil, fmt.Errorf("Unexpected signing method: %v", token.Header["alg"])
					}
					return []byte(mySignInKey), nil
				})
        // type conversion with jwt.MapClaims and to check the token is valid
				if claims, ok := token.Claims.(jwt.MapClaims); ok && token.Valid { // if that token is valid
					ctx := context.WithValue(r.Context(), "props", claims) // props is context key
					// Access context values in handlers like this					
					next.ServeHTTP(w, r.WithContext(ctx)) // if succeed serve http with context

				} else { // if there is an error
					fmt.Println("token err -> ", err)
					//r.Header.Set("ExpiredToken", jwtToken)
					//DelTokenIfExpired(jwtToken)
					// usernameInter := claims["user_name"]
					// if username, ok := usernameInter.(fmt.Stringer); ok {
					// 	person := dbop.GetPersonToDelToken(username.String())
					// 	dbop.DeleteTokenIfExpired(person)
					// }

					w.WriteHeader(http.StatusUnauthorized)
					w.Write([]byte("you are Unauthorized or your token is expired"))
				}
			
		}

	})
}

Database scheme

Issues
  • dbOp package suggestions

    dbOp package suggestions

    Followed your Reddit link and took a look; your Reddit post there did not ask for suggestions but, having scanned your dbOp package I do have a few so thought raising an issue might be the best way to communicate (feel free to ignore/delete this). I've only looked at the dbOp package:

    • You are calling sql.Open with every call. Consider doing this once at startup (as the docs say "...the Open function should be called just once. It is rarely necessary to close a DB."). sql.DB will reuse connections (and establish new ones where needed) which avoids the cost that comes with establishing lots of connections.
    • Consider adding defer rows.Close() when running a query (this is called automatically when rows.Next() returns false but you may not always fully read the rows e.g. GetLastLoginToken). Also consider checking for errors - rows.Err()
    • sql.Prepare is really only adds value when you will be running the same statement multiple times (with different parameters).
    • Consider returning error more often; its nice to be able to let the end user know when something has gone wrong (and calling panic when, for example, the username is not found will not work in production).
    • Consider storing a password hash (e.g. bcrypt) rather than the password itself (OWASP cheatssheet).
    • A number of the functions take the request body (reqBody []byte) and parse it; this does not really seem to fit within the database package (a significant part of the rationale for having a separate package is that there may be other users and these might not accept JSON). A common approach is to have a model package.
    opened by MattBrittan 3
Owner
Kıvanç Aydoğmuş
3nd Grade In Computer Engineering
Kıvanç Aydoğmuş
Golang restAPI crud project with mySql database.

Golang RestAPI using gorilla/mux Golang restAPI crud project with mySql database. Test Api with Thunder Client vs code beautiful Extension. and use Be

Md Abu. Raihan 6 Mar 26, 2022
sqlcomment is an ent driver that adds SQL comments following sqlcommenter specification.

sqlcomment sqlcomment is an ent driver that adds SQL comments following sqlcommenter specification. sqlcomment includes support for OpenTelemetry and

ariga 49 Jun 28, 2022
mysql to mysql 轻量级多线程的库表数据同步

goMysqlSync golang mysql to mysql 轻量级多线程库表级数据同步 测试运行 设置当前binlog位置并且开始运行 go run main.go -position mysql-bin.000001 1 1619431429 查询当前binlog位置,参数n为秒数,查询结

null 13 Jun 14, 2022
CRUD API example is written in Go using net/http package and MySQL database.

GoCrudBook CRUD API example is written in Go using net/http package and MySQL database. Requirements Go MySQL Code Editor Project Structure GoCrudBook

Serhat Karabulut 3 May 15, 2022
Database - Example project of database realization using drivers and models

database Golang based database realization Description Example project of databa

Denis 1 Feb 10, 2022
REST based Redis client built on top of Upstash REST API

An HTTP/REST based Redis client built on top of Upstash REST API.

Andreas Thomas 4 Jun 2, 2022
Demo Go REST + MySQL

Demo REST API em Go com Integração MySQL Iniciando a aplicação Deve ser criado uma base de dados para que a aplicação consiga se conectar. Os dados ba

Thiago Cardoso Silva 0 Nov 28, 2021
Interactive terminal user interface and CLI for database connections. MySQL, PostgreSQL. More to come.

?? dbui dbui is the terminal user interface and CLI for database connections. It provides features like, Connect to multiple data sources and instance

Kanan Rahimov 91 Jun 30, 2022
A proxy is database proxy that de-identifies PII for PostgresDB and MySQL

Surf Surf is a database proxy that is capable of de-identifying PII and anonymizing sentive data fields. Supported databases include Postgres, MySQL,

null 1 Dec 14, 2021
Vitess is a database clustering system for horizontal scaling of MySQL.

Vitess Vitess is a database clustering system for horizontal scaling of MySQL through generalized sharding. By encapsulating shard-routing logic, Vite

Vitess 14.3k Jul 6, 2022
Vitess is a database clustering system for horizontal scaling of MySQL.

Vitess Vitess is a database clustering system for horizontal scaling of MySQL through generalized sharding. By encapsulating shard-routing logic, Vite

Vitess 14.3k Jun 29, 2022
A MySQL-compatible relational database with a storage agnostic query engine. Implemented in pure Go.

go-mysql-server go-mysql-server is a SQL engine which parses standard SQL (based on MySQL syntax) and executes queries on data sources of your choice.

DoltHub 817 Jun 28, 2022
A tool I made to quickly store bug bounty program scopes in a local sqlite3 database

GoScope A tool I made to quickly store bug bounty program scopes in a local sqlite3 database. Download or copy a Burpsuite configuration file from the

null 3 Nov 18, 2021
A go package to add support for data at rest encryption if you are using the database/sql.

go-lockset A go package to add support for data at rest encryption if you are using the database/sql to access your database. Installation In your Gol

Bartlomiej Mika 0 Jan 30, 2022
[mirror] the database client and tools for the Go vulnerability database

The Go Vulnerability Database golang.org/x/vulndb This repository is a prototype of the Go Vulnerability Database. Read the Draft Design. Neither the

Go 44 Jun 24, 2022
Приложение представляет собой API на языке Golang с функциями CRUD для MySQL.

golang-crud-mysql Приложение представляет собой API на языке Golang с функциями CRUD для MySQL. Также реализован UI при помощи HTML5, CSS3, немного JQ

Emil 0 Jan 18, 2022
WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

null 2.2k Jun 28, 2022
BQB is a lightweight and easy to use query builder that works with sqlite, mysql, mariadb, postgres, and others.

Basic Query Builder Why Simple, lightweight, and fast Supports any and all syntax by the nature of how it works Doesn't require learning special synta

Aaron M 37 Jun 11, 2022