Simple JSON type checking.

Overview

go-map-schema

Go Reference GitHub tag (latest SemVer)

Table of Contents

Overview

go-map-schema is a tiny library that's useful for comparing a map (usually from JSON) to a struct, and finding any fields that are missing or that have incompatible types.

Use Case

The most common usage would be for an API that accepts JSON from clients.

Before we can fulfill the request we need to know if the JSON matches what we expect. By verifying the JSON before we even try to json.Unmarshal it into a struct, we can be sure the JSON will be safely converted with no loss of data or swallowed errors.

As a result, you end up with an API that has

  1. strict type checking, and
  2. can give the client helpful error messages when the request is invalid

Do I Really Need This?

For something like an API, it's worth asking who is going to be the client that is using the API. Is this an API for your single page app, or are you providing a service and it will be used by other developers?

If the API is "internal" and only used within the context of your site, I don't think it's necessary. But if someone is trying to use your API, I think it's worth having. API documentation is rarely perfect, and giving the client a readable error can help them debug when a request throws a 400 Bad Request.

Examples

Read below for a quick look at how you can use go-map-schema.

For more examples, check out the examples/ directory.

Usage

Suppose we have a Person model that contains a nested Address

type Address struct {
    Country     string `json:"country"`
    City        string `json:"city"`
    AddressLine string `json:"address_line"`
}

type Person struct {
    FirstName string  `json:"first_name"`
    LastName  string  `json:"last_name"`
    Age       int     `json:"age"`
    Address   Address `json:"address"`
}

and a client comes along and makes a POST request with this JSON.

{
    "first_name": "Jessie",
    "age": "26",
    "address": {
      "country": "US",
      "city": null
    }
}

We can unmarshal the JSON into a map to make it easier to work with, and then compare it with the Person model.

src := make(map[string]interface{})
json.Unmarshal(payload, &src)

dst := Person{}
results, err := schema.CompareMapToStruct(&dst, src)

After comparing we now have a CompareResults instance stored in results.

type CompareResults struct {
	MismatchedFields []FieldMismatch
	MissingFields    []FieldMissing
}

With this, we can quickly see which fields have mismatched types, as well as any fields that are in the Person struct but not the JSON.

Check out examples/type-errors for the complete example.

Universal Type Names

By default, CompareMapToStruct will use the DetailedTypeName func when reporting a type mismatch. The detailed type name includes some extra information that you may not want a client to see.

For example, trying to convert a float64 into an int32 will report a mismatch between float64 and int32.

If you don't want to include bit size, you can use SimpleTypeName which converts floatX -> float, intX -> int, uintX -> uint.

opts := &schema.CompareOpts{
    TypeNameFunc: schema.SimpleTypeName,
}

schema.CompareMapToStruct(dst, src, opts)
Comments
  • Support nested structures

    Support nested structures

    If dst contains a struct which is not embedded, we should recursively compare it. FieldMismatch will need to be updated to include something that describes the field's "path". Having a list of strings that keeps track of the parent would probably be the easiest approach.

    enhancement good first issue 
    opened by Kangaroux 2
  • Add a check to prevent float -> int conversions

    Add a check to prevent float -> int conversions

    https://github.com/Kangaroux/go-map-schema/blob/482c9d8b8f692a9017ad5a4633589cf79ab4cd91/schema_test.go#L113

    json.Unmarshal doesn't discriminate between floats and ints. When unmarshaling into a interface{} it always stores it as a float64.

    We should check when converting to an int field if there would be loss of data (i.e. truncating everything after the decimal). Unfortunately we can't tell the difference between 3 and 3.0. Technically 3.0 is a type mismatch, but there is no loss of data if we truncate it.

    enhancement good first issue 
    opened by Kangaroux 0
  • Add a check for Required (with solution/code provided)

    Add a check for Required (with solution/code provided)

    Currently when applying this code, every property gets reported back as missing wether its marked as required or not. I could make my own filter to fix that, but adding a simple isRequired property it would make this a lot easier.

    I've come up with a backwards compatible way to still report all missing fields, but also giving the developer the option to decide wether or not to do something with the error:

    type FieldMissing struct {
    	/* Other fields here */ 	
    
    	// IsRequired indicates whether or not the field is marked as required in the structs tags.
    	IsRequired boolean
    }
    
    type FieldMismatch struct {
    	/* Other fields here */ 	
    
    	// IsRequired indicates whether or not the field is marked as required in the structs tags.
    	IsRequired boolean
    }
    
    func contains(s []string, str string) bool {
    	for _, v := range s {
    		if v == str {
    			return true
    		}
    	}
    
    	return false
    }
    
    func fieldIsRequired(f reflect.StructField) (bool) {
    	tag := f.Tag.Get("validate")
    	if tag == "" {
    		return false;
    	}
    	
    	return contains(strings.Split(tags, ","), "required")
    }
    

    And then apply them in schema.go, eg:

    missing := FieldMissing{Field: fieldName, IsRequired: fieldIsRequired(field)}
    mismatch := FieldMismatch{ IsRequired: fieldIsRequired(field), /* */}
    
    opened by tg-mvooges 1
  • Add a data validation example

    Add a data validation example

    By "data validation" I mean validating the actual values of the payload, not just the types. For example, we may want to check that a field is not blank.

    I'm hesitant to use the term "form validation" since this library isn't really for form validation. It can certainly be used to help with form validation, but it's a bit redundant. It's more appropriate to use this for an API meant for third party consumers.

    The straightforward approach to using this library for data validation would be:

    1. Input a JSON payload
    2. Use CompareMapToStruct() to check the types, and return the errors if there are any
    3. Unmarshal the JSON into dst and check the values using some validation logic, and return the errors if there are any

    There are plenty of data validation libraries out there, like ozzo-validation, which would work pretty seamlessly with this.

    documentation good first issue 
    opened by Kangaroux 0
  • [Needs discussion] Add support for arrays

    [Needs discussion] Add support for arrays

    We should support type checking arrays just like any other value.

    I'm not sure what the best way of reporting type mismatches would be. The two things that come to mind would be to report the entire array as a mismatch, or to report each individual index as a mismatch.

    Calling out the specific indexes that are mismatched would definitely be a bit too verbose in most cases. I'd also be willing to bet that if one of the indexes is mismatched then chances are the whole array is mismatched.

    On the other hand, I'd rather be overly verbose and provide a way to simplify the mismatch reporting.

    enhancement needs discussion 
    opened by Kangaroux 4
Releases(v0.6.1)
  • v0.6.1(Jun 7, 2021)

  • v0.6.0(Jun 6, 2021)

  • v0.5.0(May 25, 2021)

    • Replaced AsMap() with Errors() which now returns an error type, and also returns nil if there are no errors
    • Updated documentation and added examples
    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(May 21, 2021)

  • v0.3.0(May 21, 2021)

    • Added CompareResults.AsMap() method, which returns a map of field names to mismatch errors.
    • Updated FieldMismatch.String() to correctly say things like "is an int" or "is null" instead of "is a int" and "is a null".
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(May 20, 2021)

    • Added a type mismatch when converting from a float -> int if the float is not a whole number.
    • Added a CompareOpts type. This can be used to control how type names are reported for mismatches, as well as determining if two types are convertible.
    • Added a TypeNameSimple func, which returns type names without the bit size (for int, uint, and float types). This is used with CompareOpts.
    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(May 20, 2021)

Owner
Jessie
Who are you when no one is watching?
Jessie
Gin Middleware to extract json tag value from playground validator's errors validation

Json Tag Extractor for Go-Playground Validator This is Gin Middleware that aim to extract json tag and than store it to FieldError.Field() object. Ins

Muhamad Surya Iksanudin 0 Jan 14, 2022
Validate Golang request data with simple rules. Highly inspired by Laravel's request validation.

Validate golang request data with simple rules. Highly inspired by Laravel's request validation. Installation Install the package using $ go get githu

Saddam H 1.2k Dec 29, 2022
The Hyperscale InputFilter library provides a simple inputfilter chaining mechanism by which multiple filters and validator may be applied to a single datum in a user-defined order.

Hyperscale InputFilter Branch Status Coverage master The Hyperscale InputFilter library provides a simple inputfilter chaining mechanism by which mult

Hyperscale Stack 0 Oct 20, 2021
Simple module for validation inn control number

simple module for validation inn control number

Pavel 1 Sep 4, 2022
Curtis Milo 2 May 8, 2022
convert JSON of a specific format to a type structure(Typescript type or more)

json2type convert JSON of a specific format to a type structure(Typescript type or more) Quick Start CLI Install use go tool install go install github

Kim 3 Mar 28, 2022
Simple IPO Checking Utility.

Simple IPO Checking Utility Usage: share-check [--version] [--help] <command> [<args>] Available commands are: check Check multiple share by p

Mahesh C. Regmi 2 Feb 14, 2022
A simple, standalone, and lightWeight tool that can do health/status checking, written in Go.

EaseProbe EaseProbe is a simple, standalone, and lightWeight tool that can do health/status checking, written in Go. Table of Contents EaseProbe 1. Ov

MegaEase 1.4k Dec 24, 2022
[TOOL, CLI] - Filter and examine Go type structures, interfaces and their transitive dependencies and relationships. Export structural types as TypeScript value object or bare type representations.

typex Examine Go types and their transitive dependencies. Export results as TypeScript value objects (or types) declaration. Installation go get -u gi

Daniel T. Gorski 172 Dec 6, 2022
Go generator to copy values from type to type and fields from struct to struct. Copier without reflection.

Copygen is a command-line code generator that generates type-to-type and field-to-field struct code without adding any reflection or dependenc

SwitchUpCB 218 Dec 29, 2022
A drop-in replacement to any Writer type, which also calculates a hash using the provided hash type.

writehasher A drop-in replacement to any Writer type, which also calculates a hash using the provided hash type. Example package main import ( "fmt"

Max 0 Jan 10, 2022
GNU Aspell spell checking library bindings for Go (golang)

Aspell library bindings for Go GNU Aspell is a spell checking tool written in C/C++. This package provides simplified Aspell bindings for Go. It uses

Vladimir Sibirov 43 Nov 14, 2022
manage your mocks / run mockgen more quickly / mocks up-to-date checking

gomockhandler If you find any bugs or have feature requests, please feel free to create an issue. gomockhandler is handler of golang/mock, as the name

Kensei Nakada 80 Dec 30, 2022
Generate QR-Codes for checking into events using the official Corona Warn App.

Corona Warn App QR-Code Generator Generate QR-Codes for checking into events using the official Corona Warn App. Table of Contents Introduction Instal

Lukas Malkmus 8 Oct 5, 2022
Key-Checker - Go scripts for checking API key / access token validity

Key-Checker Go scripts for checking API key / access token validity Update V1.0.0 ?? Added 37 checkers! Screenshoot ?? How to Install go get github.co

Muhammad Daffa 190 Dec 19, 2022
A Go package for checking conditions for slices and maps.

check Go package The check package of Go helps one to check various conditions for slices: []int []float64 []string []bool maps: map[string]int map[st

null 4 Aug 26, 2022
A tool for checking the accessibility of your data by IPFS peers

ipfs-check Check if you can find your content on IPFS A tool for checking the accessibility of your data by IPFS peers Documentation Build go build wi

Adin Schmahmann 18 Dec 17, 2022
gostatement is an analyzer checking for occurrence of `go` statements

gostatement gostatement is an analyzer checking for occurrence of go statements. You may want to use a custom func wrapping the statement utilizing re

Felix 0 Nov 4, 2021
Checking if some websites of interest are online

Web site status checker Checking if some websites of interest are online. A little weekend project. Technologies used Go Gin Current sites supported G

null 0 Nov 28, 2021
PinGo is a standalone and feature-rich tool for common IP-based reachability checking tasks. Ping or Trace and Observe in real-time the statistics.

pingo As a network champion from designing and implementing to troubleshooting large scale networks - I know that is usually not easy for administrato

Jerome Amon 3 Sep 26, 2022