reasonable handling of nullable values

Related tags

Utilities go json sql
Overview

null GoDoc CircleCI

import "gopkg.in/guregu/null.v4"

null is a library with reasonable options for dealing with nullable SQL and JSON values

There are two packages: null and its subpackage zero.

Types in null will only be considered null on null input, and will JSON encode to null. If you need zero and null be considered separate values, use these.

Types in zero are treated like zero values in Go: blank string input will produce a null zero.String, and null Strings will JSON encode to "". Zero values of these types will be considered null to SQL. If you need zero and null treated the same, use these.

All types implement sql.Scanner and driver.Valuer, so you can use this library in place of sql.NullXXX. All types also implement: encoding.TextMarshaler, encoding.TextUnmarshaler, json.Marshaler, and json.Unmarshaler. A null object's MarshalText will return a blank string.

null package

import "gopkg.in/guregu/null.v4"

null.String

Nullable string.

Marshals to JSON null if SQL source data is null. Zero (blank) input will not produce a null String.

null.Int

Nullable int64.

Marshals to JSON null if SQL source data is null. Zero input will not produce a null Int.

null.Float

Nullable float64.

Marshals to JSON null if SQL source data is null. Zero input will not produce a null Float.

null.Bool

Nullable bool.

Marshals to JSON null if SQL source data is null. False input will not produce a null Bool.

null.Time

Marshals to JSON null if SQL source data is null. Zero input will not produce a null Time.

zero package

import "gopkg.in/guregu/null.v4/zero"

zero.String

Nullable string.

Will marshal to a blank string if null. Blank string input produces a null String. Null values and zero values are considered equivalent.

zero.Int

Nullable int64.

Will marshal to 0 if null. 0 produces a null Int. Null values and zero values are considered equivalent.

zero.Float

Nullable float64.

Will marshal to 0.0 if null. 0.0 produces a null Float. Null values and zero values are considered equivalent.

zero.Bool

Nullable bool.

Will marshal to false if null. false produces a null Float. Null values and zero values are considered equivalent.

zero.Time

Will marshal to the zero time if null. Uses time.Time's marshaler.

Can you add support for other types?

This package is intentionally limited in scope. It will only support the types that driver.Value supports. Feel free to fork this and add more types if you want.

Can you add a feature that ____?

This package isn't intended to be a catch-all data-wrangling package. It is essentially finished. If you have an idea for a new feature, feel free to open an issue to talk about it or fork this package, but don't expect this to do everything.

Package history

As of v4, unmarshaling from JSON sql.NullXXX JSON objects (ex. {"Int64": 123, "Valid": true}) is no longer supported. It's unlikely many people used this, but if you need it, use v3.

Bugs

json's ",omitempty" struct tag does not work correctly right now. It will never omit a null or empty String. This might be fixed eventually.

License

BSD

Comments
  • Not to use fmt.Errorf for UnmarshalJSON error

    Not to use fmt.Errorf for UnmarshalJSON error

    What version of Go are you using (go version)?

    $ go version
    go version go1.15.6 darwin/amd64
    

    Does this issue reproduce with the latest release?

    Yes

    What did you do?

    https://play.golang.org/p/7kgBsW80Mzf

    What is the problem?

    If we would have not empty decoder.errorContext.FieldStack and we received error of fmt.wrapError type from UnmarshalJSON then later addErrorContext wouldn't be able to set proper value for err.Field and err.Struct because error type won't match with *UnmarshalTypeError.

    https://github.com/golang/go/blob/a7e16abb22f1b249d2691b32a5d20206282898f2/src/encoding/json/decode.go#L181-L183 https://github.com/golang/go/blob/a7e16abb22f1b249d2691b32a5d20206282898f2/src/encoding/json/decode.go#L255-L258

    opened by DhZero 6
  • Add null.Time and testing

    Add null.Time and testing

    PR https://github.com/guregu/null/pull/6 seems to have stalled, so here's another take on it.

    Key points/thoughts/questions:

    • No dependency on pq.
    • One additional type: null.Time. If the null.Time version feels right, I'll add the zero.Time version.
    • Uses RFC3339 time format just as the native time package does here.
    • Unmarshals both string and time.Time type. I'm tempted to only unmarshal strings. Thoughts?
    opened by jedborovik 6
  • Add support for the time.Duration type

    Add support for the time.Duration type

    Like this library, very clean & nice to use.

    Would you be interested in adding support for / accepting a PR with an implementation of null.Duration ?

    opened by njern 5
  • Storing null values using this library in database

    Storing null values using this library in database

    Hello Greg (hope I got the name right :) )

    First of all, great library for marshalling and unmarshalling Golang's SQL null datatypes. Really saved me a lot of time and effort.

    However, I noticed that this library is missing one piece of functionality which is storing null values in database. I have gone through the library and could not find it anywhere. I had to implement this part in my own codebase. I would be happy to raise a Pull Request so that this functionality can be merged to master.

    Please do let me know if I can go ahead with raising a pull request.

    opened by padas2 3
  • Why does null.Time text-marshaling return null string?

    Why does null.Time text-marshaling return null string?

    Testing my models on GraphQL, I detected some issues regarding text-marshaling implementation in this library.

    null.Time returns "null"

    func (t Time) MarshalText() ([]byte, error) {
    	if !t.Valid {
    		return []byte("null"), nil
    	}
    	return t.Time.MarshalText()
    }
    

    Other null.X implementations

    // MarshalText implements encoding.TextMarshaler.
    // It will encode a blank string when this String is null.
    func (s String) MarshalText() ([]byte, error) {
    	if !s.Valid {
    		return []byte{}, nil
    	}
    	return []byte(s.String), nil
    }
    
    // MarshalText implements encoding.TextMarshaler.
    // It will encode a blank string if this Int is null.
    func (i Int) MarshalText() ([]byte, error) {
    	if !i.Valid {
    		return []byte{}, nil
    	}
    	return []byte(strconv.FormatInt(i.Int64, 10)), nil
    }
    

    My question/curiosity is: why does null.Time text-marshaling return a "null" string instead of an empty string like the others?

    opened by limoli 3
  • MySQL time scan

    MySQL time scan

    type JsonUser struct { ID null.Intjson:"id" db:"id"Email null.Stringjson:"email" db:"email"Password null.Stringjson:"-" db:"password"FirstName null.Stringjson:"first_name" db:"first_name"LastName null.Stringjson:"last_name" db:"last_name"CreatedAt null.Timejson:"created_at" db:"created_at"UpdatedAt null.Timejson:"updated_at" db:"updated_at"` }

    func UserByID(w http.ResponseWriter, req *http.Request) { vars := mux.Vars(req) if userID, err := strconv.Atoi(vars["id"]); err != nil { fmt.Fprint(w, response.Success("No record found.", http.StatusNotFound, nil).JSON()) return } else { jsonUser := &JsonUser{} err = DB.QueryRow("SELECT id, email, first_name, last_name, created_at, updated_at FROM users WHERE id = ? LIMIT 1;", userID).Scan(&jsonUser.ID, &jsonUser.Email, &jsonUser.FirstName, &jsonUser.LastName, &jsonUser.CreatedAt, &jsonUser.UpdatedAt) if err != nil { fmt.Fprint(w, response.Error(err.Error(), http.StatusInternalServerError).JSON()) return } fmt.Fprint(w, response.Success("Request successful!", http.StatusOK, jsonUser).JSON()) }

    return
    

    }`

    CreatedAt is not null, but UpdatedAt is for the particular record I'm retrieving.

    The error that is being thrown is: "sql: Scan error on column index 4: null: cannot scan type []uint8 into null.Time: [50 48 49 56 45 48 56 45 50 50 32 48 48 58 53 54 58 50 50]"

    It doesn't seem that this package's Time is not handling []uint8 but I'm not sure how to get this working. Thanks!

    opened by ecarter202 3
  • Implement BindUnmarshaler

    Implement BindUnmarshaler

    This adds the UnmarshalParam() function from the BindUnmarshaler interface from echo: https://github.com/labstack/echo/blob/60072188355b8972c738a041666bd4b05957c135/bind.go#L24

    This is useful for unmarshaling form and query values, and since it's (mostly) a proxy to UnmarshalText() I don't see any downsides in adding it. The only exception is the Bool; I had to add on and off. I could add these just for UnmarshalParam, but it seemed useful to me to add it to UnmarshalText as well.

    opened by arp242 3
  • Add all int and float null types

    Add all int and float null types

    • Add Scan and Value methods for non sql.Int64 and sql.Float64 types
    • Change NullInt to use int in the code opposed to int64
    • Rename Float to Float64
    • Add support for all missing key int and float types: float32, int8, int16, int32, int64, uint8, uint16, uint32, uint64
    • Update README to include new additions/changes
    opened by nullbio 3
  • How can one make null.JsonRawMessage

    How can one make null.JsonRawMessage

    Purpose: to allow cases where field it's value does not exist inside an wrapping object to be sent to the server as NULL JSON/JSONB Value (where structure type is json.RawMessage)

    If someone could implement it it would be even better.

    help wanted 
    opened by hllhll 3
  • null unmarshalling

    null unmarshalling

    Unmarshalling and marshalling types in the null package isn't symmetric. For example:

    type T struct {
      Name null.String
    }
    
    in := []byte(`{"Name":""}`)
    var t T
    json.Unmarshal(in, &t)
    fmt.Printf("%+v\n", t) // => {Name:{NullString:{String: Valid:false}}}
    out, _ := json.Marshal(t)
    fmt.Println(string(out)) // => {"Name":null}
    

    Why not have in unmarshal to {Name:{NullString:{String: Valid:true}}}? In addition to clearing up the issue shown above it would make the null package incredibly useful for the common issue of how to determine if a RESTful request intends to set a value to the zero value or if it just wasn't included in the request (a PATCH-like request).

    Right now the common recommendation is to use pointers for the fields in your structs as described here, but I'd much prefer using the null package for this if I could.

    Let me know what you think. I'm happy to create a PR for this if you think this idea has legs.

    opened by jedborovik 3
  • Decoding large integers.

    Decoding large integers.

    Hi guregu, My teammates and I really like your library, as it has helped us a lot when going from JSON to go structs to databases (thank you!). One small issue we found is unmarshalling large numbers from JSON. Right now, because there is only one number type in Javascript, integers in JSON that are large enough are decoded in scientific notation. Because of this, these numbers cannot be unmarshalled into integer types, in particular your null.Int and zero.Int types. I was wondering if you had any thoughts on the issue.

    If you're interested, I've already written an implementation that passes all of the tests, if you want to take a look.

    opened by juandiegopalomino 3
  • how use json null set sql time field to null?

    how use json null set sql time field to null?

    1. defind web api dto struct
    type Dto struct {
       Start null.Time
    }
    
    1. send this json to web api
    { 
      "Start": null
    }
    
    1. service: use gorm update entry
    gorm.Model(&MyTable).Where("id = 1").Update(&dto)
    
    1. Hope to generate sql
    update my_table set start = null  where id = 1
    

    The start field is now ignored when used in this way

    opened by ystyle 1
  • Add check for zero or blank

    Add check for zero or blank

    Hi @guregu,

    I added a new method which checks if a string is zero or consists only of spaces. This is a helpful check especially for mandatory fields.

    Would it be possible to integrate this into the null library? Otherwise I would have to release my own version. I think this could be interesting for others as well.

    Regards. Nils

    opened by nilskasseckert 0
  • Generics support

    Generics support

    Any ideas or opinions on how to handle generics in this library? I played around with creating a generic Null[T] type, but I wonder if it's even useful at that point, because we are essentially recreating the Optional type you might find in other languages. At what point is it going too far? Maybe ideally we could make generic Null/Zero types and use the materialized version of it for each pre-existing type.

    I'm also curious if we'll see a generic sql.Null[T] type in the standard library, which might obsolete a good chunk of this library, especially if this change lands: https://github.com/golang/go/issues/5901 (which lets you easily specify custom marshalers for JSON, moving the responsibility from the object itself to the encoding site).

    See also: https://github.com/golang/go/issues/48702

    opened by guregu 2
  • Issue with omitempty

    Issue with omitempty

    Less of an issue, and more of an FYI. I have been keeping tabs on the (un)marshaling/(de)encoding issue with omitempty, and it looks like they have a adjacent proposal that may provide a workable solution:

    https://github.com/golang/go/issues/5901

    Didn't see anywhere else to leave a comment about this, and I figured it is relevant to this project in particular (I know its relevant to my usage of this project! Thanks by the way!)

    opened by ERobsham 2
  • Not to use sql.Null*** for storing value?

    Not to use sql.Null*** for storing value?

    Each struct of this library uses sql package (persistence or infrastructure layer), while it can be used for JSON marshaling (presentation layer). It seems to be against some software architecture patterns based on layered architecture or etc.

    opened by ypresto 1
  • Int from string

    Int from string

    Adds Int from sql.NullString (super helpful when dealing with id's on databases where the id might not be in a struct used to receive/pass to client). It's helpful for my purposes, though there may be a better way you've found.

    opened by MikeCongdon1 0
Releases(v4.0.0)
  • v4.0.0(Apr 25, 2020)

    This is a new major release that improves performance and fixes an inconsistency that existed in v3.

    • null.Time and zero.Time now embed sql.NullTime.
    • null.Time and zero.Time's UnmarshalText returns a blank string if invalid, making them consistent with the rest of the package (see #47).
    • Errors returned by this package are now wrapped using Go 1.13-style errors.
    • Support for unmarshaling from the JSON representation of sql.NullXXX objects (ex. {"Int64": 123, "Valid": true}) has been dropped, significantly improving unmarshaling time and reducing allocations.
    • Unmarshaling errors no longer affect Valid. Always check for errors.

    Support for Equal was added in the previous release.

    Source code(tar.gz)
    Source code(zip)
  • v3.5.0(Apr 25, 2020)

    This release adds an Equal method to all types, for easy equality checks. It also adds ValueOrZero to the zero package. Types in the null package are considered Equal if they're either both invalid, or both valid and have the same value. Types on the zero package are considered Equal if their ValueOrZero is the same, so an invalid string and a valid "" are equal.

    This will be the last release before v4.

    Source code(tar.gz)
    Source code(zip)
  • v3.4.0(Aug 1, 2018)

  • v3.3.0(Dec 14, 2017)

    Added a new method ValueOrZero to every type in the null package. This method returns the inner value if valid, or the zero value if invalid (similar to the zero package). See #17.

    Source code(tar.gz)
    Source code(zip)
  • v3.2.1(Dec 14, 2017)

    If you try to JSON marshal Infinity or NaN, you'll get *encoding/json.UnsupportedValueError. This brings us in line with the standard library.

    Source code(tar.gz)
    Source code(zip)
  • v3.2.0(Aug 20, 2017)

  • v3.1(Feb 28, 2016)

  • v3.0.1(Feb 28, 2016)

  • v3(Oct 2, 2015)

    Blank string input now produces a null null.String. Now the null package is consistent. Please use the zero.String type or v2 if you would like the old behavior.

    Fixes #10.

    Source code(tar.gz)
    Source code(zip)
  • v2.2(Sep 13, 2015)

  • v2.1.2(Sep 10, 2015)

  • v2.1.1(Sep 17, 2014)

  • v2.1(Sep 9, 2014)

  • v2.0.1(Sep 2, 2014)

    v2 Changes

    Change package names of nuller to null and the previous null to zero.

    This way zeroing types will be called zero.Type and nulling types will be called null.Type, so they are easier to understand at a glance.

    v2.0.1 Changes

    Update the docs

    Source code(tar.gz)
    Source code(zip)
  • v2(Sep 1, 2014)

    Change package names of nuller to null and the previous null to zero.

    This way zeroing types will be called zero.Type and nulling types will be called null.Type, so they are easier to understand at a glance.

    Source code(tar.gz)
    Source code(zip)
Owner
Greg
Greg
Go library for decoding generic map values into native Go structures and vice versa.

mapstructure mapstructure is a Go library for decoding generic map values to structures and vice versa, while providing helpful error handling. This l

Mitchell Hashimoto 6.4k Dec 28, 2022
Hex dump and read values of files quickly and swiftly with Go-Hex a program designed to dump any file in a hexadecimal format

Go-Hex Hex dump and read values of files quickly and swiftly with Go-Hex a program designed to dump any file in a hexadecimal format Dump Hashes ----

RE43P3R 0 Oct 10, 2021
DO NOT use `named return values` in Go

namedreturn DO NOT use named return values in Go namedreturn finds a named return value, you must remove it.

Tomochika Hara 3 Dec 17, 2021
Access and modify property values in deeply nested maps, using dot-separated paths

Dig lets you access and modify property values in deeply nested, unstructured maps, using dot-separated paths: source := make(map[string]interface{})

Preslav Rachev 12 May 7, 2022
Grpc-gateway-map-null - gRPC Gateway test using nullable values in map

Demonstrate gRPC gateway behavior with nullable values in maps Using grpc-gatewa

null 1 Jan 6, 2022
Optional-go - Library for conveniently and safely dealing with optional (nullable) values

Optional Go Library for conveniently and safely dealing with optional (nullable)

Adolfo Martinelli 3 Oct 24, 2022
Proxima is the only reasonable proxy manager

Proxima: a Proxy Manager built in Go, configured in Prolog. What is this? Proxima is the only reasonable proxy manager. You can fine tune how your sys

Yutaka Ichibangase 0 Jan 22, 2022
Zero allocation Nullable structures in one library with handy conversion functions, marshallers and unmarshallers

nan - No Allocations Nevermore Package nan - Zero allocation Nullable structures in one library with handy conversion functions, marshallers and unmar

Andrey Kuzmin 63 Dec 20, 2022
Nullable Go types that can be marshalled/unmarshalled to/from JSON.

Nullable Go types Description This package provides nullable Go types for bool, float64, int64, int32, string and time.Time replacing sql.NullString,

Emvi 30 Dec 12, 2022
Go package that adds marshal and unmarshal features to nullable sql types.

#Nullable Very simple Go module to handle nullable fields. Basically, it adds to sql package types the JSON marshal and unmarshal features. It has 100

Diego Hordi 1 Jan 20, 2022
:steam_locomotive: Decodes url.Values into Go value(s) and Encodes Go value(s) into url.Values. Dual Array and Full map support.

Package form Package form Decodes url.Values into Go value(s) and Encodes Go value(s) into url.Values. It has the following features: Supports map of

Go Playgound 573 Dec 26, 2022
Composable, observable and performant config handling for Go for the distributed processing era

Konfig Composable, observable and performant config handling for Go. Written for larger distributed systems where you may have plenty of configuration

Lalamove 634 Dec 11, 2022
Simple error handling primitives

errors Package errors provides simple error handling primitives. go get github.com/pkg/errors The traditional error handling idiom in Go is roughly ak

null 8k Dec 26, 2022
A comprehensive error handling library for Go

Highlights The errorx library provides error implementation and error-related utilities. Library features include (but are not limited to): Stack trac

Joom 922 Jan 6, 2023
a tool for handling file uploads simple

baraka a tool for handling file uploads for http servers makes it easier to make operations with files from the http request. Contents Install Simple

Enes Furkan Olcay 46 Nov 30, 2022
Currency handling for Go.

currency Handles currency amounts, provides currency information and formatting. Powered by CLDR v38, in just ~30kb of data. Backstory: https://bojanz

Bojan Zivanovic 360 Jan 3, 2023
A sync.WaitGroup with error handling and concurrency control

go-waitgroup How to use An package that allows you to use the constructs of a sync.WaitGroup to create a pool of goroutines and control the concurrenc

Pieter Claerhout 36 Dec 31, 2022
:speedboat: a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation

Package pool Package pool implements a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation. Features

Go Playgound 707 Jan 1, 2023
A Go package for handling common HTTP JSON responses.

go-respond A Go package for handling common HTTP JSON responses. Installation go get github.com/nicklaw5/go-respond Usage The goal of go-respond is to

Nick Law 50 Sep 26, 2022
Address handling for Go.

address Handles address representation, validation and formatting. Inspired by Google's libaddressinput. Backstory: https://bojanz.github.io/address-h

Bojan Zivanovic 58 Dec 28, 2022