reflect api without runtime reflect.Value cost

Overview

reflect2

Sourcegraph GoDoc Build Status codecov rcard License

reflect api that avoids runtime reflect.Value cost

  • reflect get/set interface{}, with type checking
  • reflect get/set unsafe.Pointer, without type checking
  • reflect2.TypeByName works like Class.forName found in java

json-iterator use this package to save runtime dispatching cost. This package is designed for low level libraries to optimize reflection performance. General application should still use reflect standard library.

reflect2.TypeByName

// given package is github.com/your/awesome-package
type MyStruct struct {
	// ...
}

// will return the type
reflect2.TypeByName("awesome-package.MyStruct")
// however, if the type has not been used
// it will be eliminated by compiler, so we can not get it in runtime

reflect2 get/set interface{}

valType := reflect2.TypeOf(1)
i := 1
j := 10
valType.Set(&i, &j)
// i will be 10

to get set type, always use its pointer *type

reflect2 get/set unsafe.Pointer

valType := reflect2.TypeOf(1)
i := 1
j := 10
valType.UnsafeSet(unsafe.Pointer(&i), unsafe.Pointer(&j))
// i will be 10

to get set type, always use its pointer *type

benchmark

Benchmark is not necessary for this package. It does nothing actually. As it is just a thin wrapper to make go runtime public. Both reflect2 and reflect call same function provided by runtime package exposed by go language.

unsafe safety

Instead of casting []byte to sliceHeader in your application using unsafe. We can use reflect2 instead. This way, if sliceHeader changes in the future, only reflect2 need to be upgraded.

reflect2 tries its best to keep the implementation same as reflect (by testing).

Comments
  • fix possible memory confusion in unsafe slice cast

    fix possible memory confusion in unsafe slice cast

    I found an incorrect cast from string to []byte in reflect2.go. The problem is that when reflect.SliceHeader is created as a composite literal (instead of deriving it from an actual slice by cast), then the Go garbage collector will not treat its Data field as a reference. If the GC runs just between creating the SliceHeader and casting it into the final, real []byte slice, then the underlying data might have been collected already, effectively making the returned []byte slice a dangling pointer.

    This has a low probability to occur, but projects that import this library might still use it in a code path that gets executed a lot, thus increasing the probability to happen. Depending on the memory layout at the time of the GC run, this could potentially create an information leak vulnerability.

    This PR changes the function to create the reflect.SliceHeader from an actual slice by first instantiating the return value.

    opened by jlauinger 5
  • drop legacy Go support

    drop legacy Go support

    Go 1.9 was released in August 2017 so it's safe to make a claim that projects are already using new enough Go version.

    Tested with:

    $ go version
    go version go1.13.8 linux/amd64
    $ GO111MODULE=off ./test.sh
    ok      github.com/modern-go/reflect2-tests     0.005s  coverage: 12.3% of statements in github.com/modern-go/reflect2
    ok      github.com/modern-go/reflect2-tests/test15      0.004s  coverage: 6.4% of statements in github.com/modern-go/reflect2
    ok      github.com/modern-go/reflect2-tests/tests       0.010s  coverage: 65.4% of statements in github.com/modern-go/reflect2
    

    Signed-off-by: Mikko Ylinen [email protected]

    opened by mythi 3
  • unsafe_map.UnsafeSetIndex cost much memory

    unsafe_map.UnsafeSetIndex cost much memory

    When i deserialized a json string, it size is about 40KB. and in benchmark case , I found the UnsafeSetIndex method cost much memory, it's about 40% memory, and the gc can't collection immediately. and this problem trouble me a lot

    opened by 995270418L 3
  • Can not use gccgo to compile .go file.

    Can not use gccgo to compile .go file.

    Hi All,

    I found if I use any reflect2 function in my app, I can't use go build --compiler=gccgo to compile it. It will return error like that:

    go build --compiler=gccgo map_1.go

    command-line-arguments

    /tmp/go-build467403193/b023/pkg.a(go.o): In function github_com_modern_go_reflect2.loadGo15Types': /usr/lib/go-1.10/src/github.com/modern-go/reflect2/type_map.go:42: undefined reference toreflect.typelinks' /tmp/go-build467403193/b023/pkg.a(go.o): In function github_com_modern_go_reflect2.loadGo17Types': /usr/lib/go-1.10/src/github.com/modern-go/reflect2/type_map.go:74: undefined reference toreflect.typelinks' /usr/lib/go-1.10/src/github.com/modern-go/reflect2/type_map.go:78: undefined reference to `reflect.resolveTypeOff' collect2: error: ld returned 1 exit status

    Any idea to fix it?

    Thanks Zhang Chen

    opened by zhangckid 2
  • Change reflect2 to rely on sync.Map (go 1.9+) rather than concurrent.Map

    Change reflect2 to rely on sync.Map (go 1.9+) rather than concurrent.Map

    Reduces dependencies of reflect2 and makes it cleaner to import.

    Tested by importing gls,test,reflect2-tests and running the tests:

    $ ./test.sh 
    ok      github.com/modern-go/reflect2-tests     0.023s  coverage: 12.3% of statements in github.com/modern-go/reflect2
    ok      github.com/modern-go/reflect2-tests/test15      0.024s  coverage: 6.0% of statements in github.com/modern-go/reflect2
    ok      github.com/modern-go/reflect2-tests/tests       0.055s  coverage: 62.6% of statements in github.com/modern-go/reflect2
    
    opened by artemispax 2
  • Update mapiterinit to reflect changes in go1.18 reflect

    Update mapiterinit to reflect changes in go1.18 reflect

    The commit: https://github.com/golang/go/commit/1b2d794ca3ba60c2dbc958a271662784a7122739

    was landed, so reflect.mapiterinit will change in go1.18.

    See also: https://github.com/golang/go/issues/48238

    opened by cuonglm 1
  • gccgo compatibility

    gccgo compatibility

    Lots of projects use reflect2.TypeOf, but they can't be built with gccgo because of the way reflect2.TypeByName is implemented. Here's a slight modification to have gccgo skip type_map.go.

    Projects that use reflect2.TypeByName will still be unbuildable by gccgo, but all the rest of reflect2 should work.

    This fixes #6 and #9

    opened by paleozogt 1
  • unsafe_map crash

    unsafe_map crash

    fatal error: fault [signal SIGSEGV: segmentation violation code=0x80 addr=0x0 pc=0x4616bf]

    goroutine 40 [running]: runtime.throw({0x90159d?, 0x8?}) /usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc0000bb2f0 sp=0xc0000bb2c0 pc=0x435051 runtime.sigpanic() /usr/local/go/src/runtime/signal_unix.go:825 +0x305 fp=0xc0000bb340 sp=0xc0000bb2f0 pc=0x44a545 aeshashbody() /usr/local/go/src/runtime/asm_amd64.s:1343 +0x39f fp=0xc0000bb348 sp=0xc0000bb340 pc=0x4616bf runtime.mapiternext(0xc0002ace40) /usr/local/go/src/runtime/map.go:934 +0x2cb fp=0xc0000bb3b8 sp=0xc0000bb348 pc=0x410f0b runtime.mapiterinit(0x1?, 0x1b?, 0x10?) /usr/local/go/src/runtime/map.go:861 +0x228 fp=0xc0000bb3d8 sp=0xc0000bb3b8 pc=0x410be8 reflect.mapiterinit(0x861b00?, 0x40b9b5?, 0x87ca98?) /usr/local/go/src/runtime/map.go:1373 +0x19 fp=0xc0000bb400 sp=0xc0000bb3d8 pc=0x45e539 github.com/modern-go/reflect2.(*UnsafeMapType).UnsafeIterate(...) /root/go/pkg/mod/github.com/modern-go/[email protected]/unsafe_map.go:112 github.com/json-iterator/go.(*mapEncoder).Encode(0xc0003cca50, 0xc0002acbe0, 0xc000281680) /root/go/pkg/mod/github.com/json-iterator/[email protected]/reflect_map.go:257 +0x21c fp=0xc0000bb4b0 sp=0xc0000bb400 pc=0x58cddc github.com/json-iterator/go.(*structFieldEncoder).Encode(0xc0003ccbd0, 0x839d14?, 0xc000281680) /root/go/pkg/mod/github.com/json-iterator/[email protected]/reflect_struct_encoder.go:110 +0x56 fp=0xc0000bb528 sp=0xc0000bb4b0 pc=0x59a6f6 github.com/json-iterator/go.(*structEncoder).Encode(0xc0003ccc30, 0x0?, 0xc000281680) /root/go/pkg/mod/github.com/json-iterator/[email protected]/reflect_struct_encoder.go:158 +0x652 fp=0xc0000bb600 sp=0xc0000bb528 pc=0x59aff2 github.com/json-iterator/go.(*OptionalEncoder).Encode(0xc00011e280?, 0x0?, 0x0?) /root/go/pkg/mod/github.com/json-iterator/[email protected]/reflect_optional.go:74 +0xa4 fp=0xc0000bb650 sp=0xc0000bb600 pc=0x592724 github.com/json-iterator/go.(*onePtrEncoder).Encode(0xc0003b7330, 0xc0002acbc0, 0xc0003cc3f0?) /root/go/pkg/mod/github.com/json-iterator/[email protected]/reflect.go:219 +0x82 fp=0xc0000bb688 sp=0xc0000bb650 pc=0x585e02 github.com/json-iterator/go.(*Stream).WriteVal(0xc000281680, {0x895760, 0xc0002acbc0}) /root/go/pkg/mod/github.com/json-iterator/[email protected]/reflect.go:98 +0x158 fp=0xc0000bb6f8 sp=0xc0000bb688 pc=0x585118 github.com/json-iterator/go.(*frozenConfig).Marshal(0xc00011e280, {0x895760, 0xc0002acbc0}) /root/go/pkg/mod/github.com/json-iterator/[email protected]/config.go:299 +0xc9 fp=0xc0000bb790 sp=0xc0000bb6f8 pc=0x57d809 github.com/json-iterator/go.Marshal(...) /root/go/pkg/mod/github.com/json-iterator/[email protected]/adapter.go:34 github.asiainfo-sec.com/ZSP-SDP/aisdp-common/aisdp-rocketmq/rocketmq-client-go/v2/internal/remote.(*jsonCodec).encodeHeader(0xc0000bb838?, 0x4104fd?) /root/go/pkg/mod/github.asiainfo-sec.com/!z!s!p-!s!d!p/aisdp-common/[email protected]/rocketmq-client-go/v2/internal/remote/codec.go:281 +0x33 fp=0xc0000bb7b8 sp=0xc0000bb790 pc=0x7bda93

    opened by cuisj 4
  • nil pointer panic in reflect2 Type2

    nil pointer panic in reflect2 Type2

    go version is 1.15.5 I got panic when I use jsoniter.Unmarshal,the input object type is

    struct {
    	Data  json.RawMessage     `json:"data"`
    	Extra map[string]interface{} `json:"extra"`
    }{}
    
    recover from panic: err=runtime error: invalid memory address or nil pointer dereference stack=goroutine 24838623 [running]:
    runtime/debug.Stack(0xc0002283f0, 0x0, 0xdf23865)
    /usr/local/go/src/runtime/debug/stack.go:24 +0x9f
    xxxxxxxxx(0xdf2b129, 0x6, 0xdf21b8d, 0x4, 0xc0c5217400)
    xxxxx:244 +0x30a
    panic(0xd39d2a0, 0x14396cd0)
    /usr/local/go/src/runtime/panic.go:969 +0x1b9
    sync.(*Map).Store(0xc0ca8046c0, 0xd1a7520, 0xc0c53a5298, 0xdd28c00, 0xc0c53ec700)
    /usr/local/go/src/sync/map.go:151 +0x365
    github.com/modern-go/reflect2.(*frozenConfig).Type2(0xc000206ba0, 0xecd8b40, 0xd0a0f20, 0x0, 0x0)
    /opt/tiger/compile_path/pkg/mod/github.com/modern-go/[email protected]/reflect2.go:162 +0x10e
    github.com/modern-go/reflect2.(*frozenConfig).TypeOf(0xc000206ba0, 0xd0a0f20, 0x0, 0xc0c3cee4b8, 0x4f3614d)
    /opt/tiger/compile_path/pkg/mod/github.com/modern-go/[email protected]/reflect2.go:149 +0xb7
    github.com/modern-go/reflect2.TypeOf(...)
    /opt/tiger/compile_path/pkg/mod/github.com/modern-go/[email protected]/reflect2.go:211
    github.com/modern-go/reflect2.TypeOfPtr(...)
    /opt/tiger/compile_path/pkg/mod/github.com/modern-go/[email protected]/reflect2.go:215
    github.com/json-iterator/go.(*frozenConfig).escapeHTML(0xc0c53ad180, 0xc0c53f3470)
    /opt/tiger/compile_path/pkg/mod/github.com/json-iterator/[email protected]/config.go:271 +0x46
    github.com/json-iterator/go.Config.Froze(0x0, 0x1010100, 0x0, 0x0, 0x100, 0xc0c53f3300, 0x30)
    /opt/tiger/compile_path/pkg/mod/github.com/json-iterator/[email protected]/config.go:155 +0x2e7
    github.com/json-iterator/go.Config.frozeWithCacheReuse(0x0, 0x1010100, 0x0, 0x0, 0x100, 0x0, 0x0, 0x0, 0xe944ea0)
    /opt/tiger/compile_path/pkg/mod/github.com/json-iterator/[email protected]/config.go:174 +0x114
    github.com/json-iterator/go.(*Decoder).UseNumber(...)
    /opt/tiger/compile_path/pkg/mod/github.com/json-iterator/[email protected]/adapter.go:103
    
    opened by cabuda 0
  • undefined reference to `reflect.unsafe_New'

    undefined reference to `reflect.unsafe_New'

    Hello! I have tried to compile my own project which has a dependency of "reflect2" package with the most recent gccgo but failed with a message of "undefined reference". Is it possible to make it compile using gccgo?

    Details shown below:

    What version of "reflect2" is used to compile?

    The most recent commit (commit 333559e1834b0c1840d283d79220bf121d704022).

    What version of Go are you using (go version)?

    go version go1.16.3 gccgo (GCC) 11.0.1 20210423 (prerelease) linux/amd64
    

    Additionally, gccgo --version tells:

    gccgo (GCC) 11.0.1 20210423 (prerelease)
    Copyright (C) 2021 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    

    What operating system and processor architecture are you using?

    $ cat /etc/issue
    Arch Linux \r (\l)
    $ uname -a
    Linux 5.11.16-arch1-1 #1 SMP PREEMPT Wed, 21 Apr 2021 17:22:13 +0000 x86_64 GNU/Linux
    

    What did you do?

    Trying to build some code that depends on this package.

    What is expected to see?

    The code successfully compiles.

    What you see instead?

    /usr/bin/ld: /home/lam/.cache/go-build/fd/fd4891df5167b3e9b000124e5b53c9fccc0f3aa55647fda5583b97f1c087b5d9-d(_go_.o): in function `github_0com_1modern_x2dgo_1reflect2.UnsafeSliceType.UnsafeMakeSlice':
    /home/lam/go/pkg/mod/github.com/modern-go/[email protected]/unsafe_slice.go:78: undefined reference to `reflect.unsafe_NewArray'
    /usr/bin/ld: /home/lam/.cache/go-build/fd/fd4891df5167b3e9b000124e5b53c9fccc0f3aa55647fda5583b97f1c087b5d9-d(_go_.o): in function `github_0com_1modern_x2dgo_1reflect2.unsafeType.UnsafeNew':
    /home/lam/go/pkg/mod/github.com/modern-go/[email protected]/unsafe_type.go:48: undefined reference to `reflect.unsafe_New'
    /usr/bin/ld: /home/lam/go/pkg/mod/github.com/modern-go/[email protected]/unsafe_type.go:48: undefined reference to `reflect.unsafe_New'
    /usr/bin/ld: /home/lam/go/pkg/mod/github.com/modern-go/[email protected]/unsafe_type.go:48: undefined reference to `reflect.unsafe_New'
    /usr/bin/ld: /home/lam/go/pkg/mod/github.com/modern-go/[email protected]/unsafe_type.go:48: undefined reference to `reflect.unsafe_New'
    /usr/bin/ld: /home/lam/go/pkg/mod/github.com/modern-go/[email protected]/unsafe_type.go:48: undefined reference to `reflect.unsafe_New'
    /usr/bin/ld: /home/lam/.cache/go-build/fd/fd4891df5167b3e9b000124e5b53c9fccc0f3aa55647fda5583b97f1c087b5d9-d(_go_.o):/home/lam/go/pkg/mod/github.com/modern-go/[email protected]/unsafe_type.go:48: more undefined references to `reflect.unsafe_New' follow
    collect2: error: ld returned 1 exit status
    

    Thanks for your patient and effort.

    opened by eat-swap 2
  • codecov in .travis.yml - security issue

    codecov in .travis.yml - security issue

    Good day, https://codecov.io/bash recently outputted a security issue. Details can be found at https://about.codecov.io/security-update/

    The following file and code are impacted. Please follow the recommended actions in the link above to secure your env and code integrity. file: .travis.yml Code: after_success:

    bash <(curl -s https://codecov.io/bash)

    opened by windsparks33 0
  • heads up: codecov.io security incident - https://about.codecov.io/security-update/

    heads up: codecov.io security incident - https://about.codecov.io/security-update/

    Hi there.

    This might be an unusual "issue" beeing reported.

    There has been a security incident in codecov.io with the bash-uploader script (see [1] for details) which potentially exposed secrets to 3rd parties.

    It seems you are using the referenced bash uploader in your .travis.yml file. I wanted to draw your attention to this incident in case you missed it.

    It would be great if you could verify that no code has been altered and check the impact of this security incident on your repository.

    Regards, Robert

    [1] https://about.codecov.io/security-update/

    opened by robert-heinzmann-logmein 0
Releases(v1.0.2)
Owner
Modern Go Programming
Better developer experience
Modern Go Programming
Continuous profiling for analysis of CPU, memory usage over time, and down to the line number. Saving infrastructure cost, improving performance, and increasing reliability.

Continuous profiling for analysis of CPU, memory usage over time, and down to the line number. Saving infrastructure cost, improving performance, and increasing reliability.

Parca 2.6k Sep 28, 2022
Golang source code parsing, usage like reflect package

gotype Golang source code parsing, usage like reflect package English 简体中文 Usage API Documentation Examples License Pouch is licensed under the MIT Li

null 47 Aug 31, 2022
Calling functions by name and getting outputs by using reflect package.

Invoker A library to call (invoke) functions by taking names and sample inputs of those functions as parameters. And returns the types and values of o

null 3 Dec 20, 2021
Golang library to act on structure fields at runtime. Similar to Python getattr(), setattr(), hasattr() APIs.

go-attr Golang library to act on structure fields at runtime. Similar to Python getattr(), setattr(), hasattr() APIs. This package provides user frien

Shyamsunder Rathi 27 Jul 5, 2022
Minict is a minimal container runtime written in Go.

Minict Minict is a minimal container runtime written in Go. It was made mainly for learning purposes and is intended to be as simple as possible.

null 158 Sep 29, 2022
A Runtime Struct Builder for Go

A Runtime Struct Builder for Go

null 31 Jul 8, 2022
efaceconv - Code generation tool for high performance conversion from interface{} to immutable type without allocations.

efaceconv High performance conversion from interface{} to immutable types without additional allocations This is tool for go generate and common lib (

Ivan 50 May 14, 2022
Hotswap provides a solution for reloading your go code without restarting your server, interrupting or blocking any ongoing procedure.

Hotswap provides a solution for reloading your go code without restarting your server, interrupting or blocking any ongoing procedure. Hotswap is built upon the plugin mechanism.

Edwin 115 Sep 21, 2022
A program to create assembly 8086 strings to print without using any printing/strings related function but only mov-xchg-int and loops

Assembly String builder tool A program to create assembly 8086 strings to print without using any printing/strings related function but only mov-xchg-

Reg 2 Feb 1, 2022
Execute a binary from memory, without touching the disk. Linux only.

Memit Execute a binary from memory, without touching the disk. Linux only. Available as both a Go module and a binary. Using the Go module The Command

Liam Galvin 235 Sep 12, 2022
Create deep copies (clones) of your maps and slices without using reflection.

DeepCopy DeepCopy helps you create deep copies (clones) of your maps and slices. Create deep copies (clones) of your objects The package is based on t

null 5 Sep 16, 2022
Copier for golang, copy value from struct to struct and more

Copier I am a copier, I copy everything from one to another Features Copy from field to field with same name Copy from method to field with same name

Jinzhu 3.6k Sep 30, 2022
Helm plugin to reference value files packaged in dependency charts

Helm Octopus Plugin This Helm plugin allows to reference packaged value files (other than the default values.yaml). Install helm plugin install https:

Softonic 3 Sep 23, 2021
Go API backed by the native Dart Sass Embedded executable.

This is a Go API backed by the native Dart Sass Embedded executable. The primary motivation for this project is to provide SCSS support to Hugo. I wel

Bjørn Erik Pedersen 26 Sep 12, 2022
A pluggable backend API that enforces the Event Sourcing Pattern for persisting & broadcasting application state changes

A pluggable "Application State Gateway" that enforces the Event Sourcing Pattern for securely persisting & broadcasting application state ch

Coleman Word 28 May 4, 2022
The easiest way to make API documents for GraphQL

Document Generator for GraphQL gqldoc is now alpha gqldoc is command line tool to generate documents from GraphQL schema or your GraphQL endpoint. the

Kei Kamikawa 158 Sep 25, 2022
Cross-platform Bluetooth API for Go and TinyGo.

Go Bluetooth is a cross-platform package for using Bluetooth Low Energy hardware from the Go programming language.

TinyGo 436 Sep 15, 2022
this is an api that execute your deno code and send you the output

this a simple api that execute your deno code and send you the output, has not limit per request example request: in deno: const rawResponse = await f

pai 13 May 28, 2022
Go scripts for finding an API key / some keywords in repository

Git-Secret Go scripts for finding an API key / some keywords in repository Update V1.0 ?? Added some API Key checker Screenshoot ?? How to Install go

Muhammad Daffa 144 Sep 13, 2022