Expression evaluation in golang

Overview

Gval

Go Reference Build Status Coverage Status Go Report Card

Gval (Go eVALuate) provides support for evaluating arbitrary expressions, in particular Go-like expressions.

gopher

Evaluate

Gval can evaluate expressions with parameters, arimethetic, logical, and string operations:

It can easily be extended with custom functions or operators:

You can parse gval.Expressions once and re-use them multiple times. Parsing is the compute-intensive phase of the process, so if you intend to use the same expression with different parameters, just parse it once:

The normal Go-standard order of operators is respected. When writing an expression, be sure that you either order the operators correctly, or use parentheses to clarify which portions of an expression should be run first.

Strings, numbers, and booleans can be used like in Go:

Parameter

Variables can be accessed via string literals. They can be used for values with string keys if the parameter is a map[string]interface{} or map[interface{}]interface{} and for fields or methods if the parameter is a struct.

Bracket Selector

Map and array elements and Struct Field can be accessed via [].

Dot Selector

A nested variable with a name containing only letters and underscores can be accessed via a dot selector.

Custom Selector

Parameter names like response-time will be interpreted as response minus time. While gval doesn't support these parameter names directly, you can easily access them via a custom extension like JSON Path:

Jsonpath is also suitable for accessing array elements.

Fields and Methods

If you have structs in your parameters, you can access their fields and methods in the usual way:

It also works if the parameter is a struct directly Hello + World() or if the fields are nested foo.Hello + foo.World()

This may be convenient but note that using accessors on strucs makes the expression about four times slower than just using a parameter (consult the benchmarks for more precise measurements on your system). If there are functions you want to use, it's faster (and probably cleaner) to define them as functions (see the Evaluate section). These approaches use no reflection, and are designed to be fast and clean.

Default Language

The default language is in serveral sub languages like text, arithmetic or propositional logic defined. See Godoc for details. All sub languages are merged into gval.Full which contains the following elements:

  • Modifiers: + - / * & | ^ ** % >> <<
  • Comparators: > >= < <= == != =~ !~
  • Logical ops: || &&
  • Numeric constants, as 64-bit floating point (12345.678)
  • String constants (double quotes: "foobar")
  • Date function 'Date(x)', using any permutation of RFC3339, ISO8601, ruby date, or unix date
  • Boolean constants: true false
  • Parentheses to control order of evaluation ( )
  • Json Arrays : [1, 2, "foo"]
  • Json Objects : {"a":1, "b":2, "c":"foo"}
  • Prefixes: ! - ~
  • Ternary conditional: ? :
  • Null coalescence: ??

Customize

Gval is completly customizable. Every constant, function or operator can be defined separately and existing expression languages can be reused:

For details see Godoc.

Implementing custom selector

In a case you want to provide custom logic for selectors you can implement SelectGVal(ctx context.Context, k string) (interface{}, error) on your struct. Function receives next part of the path and can return any type of var that is again evaluated through standard gval procedures.

Example Custom Selector

External gval Languages

A list of external libraries for gval. Feel free to add your own library.

Performance

The library is built with the intention of being quick but has not been aggressively profiled and optimized. For most applications, though, it is completely fine. If performance is an issue, make sure to create your expression language with all functions, constants and operators only once. Evaluating an expression like gval.Evaluate("expression, const1, func1, func2, ...) creates a new gval.Language everytime it is called and slows execution.

The library comes with a bunch of benchmarks to measure the performance of parsing and evaluating expressions. You can run them with go test -bench=..

For a very rough idea of performance, here are the results from a benchmark run on a Dell Latitude E7470 Win 10 i5-6300U.

BenchmarkGval/const_evaluation-4                               500000000                 3.57 ns/op
BenchmarkGval/const_parsing-4                                    1000000              1144 ns/op
BenchmarkGval/single_parameter_evaluation-4                     10000000               165 ns/op
BenchmarkGval/single_parameter_parsing-4                         1000000              1648 ns/op
BenchmarkGval/parameter_evaluation-4                             5000000               352 ns/op
BenchmarkGval/parameter_parsing-4                                 500000              2773 ns/op
BenchmarkGval/common_evaluation-4                                3000000               434 ns/op
BenchmarkGval/common_parsing-4                                    300000              4419 ns/op
BenchmarkGval/complex_evaluation-4                             100000000                11.6 ns/op
BenchmarkGval/complex_parsing-4                                   100000             17936 ns/op
BenchmarkGval/literal_evaluation-4                             300000000                 3.84 ns/op
BenchmarkGval/literal_parsing-4                                   500000              2559 ns/op
BenchmarkGval/modifier_evaluation-4                            500000000                 3.54 ns/op
BenchmarkGval/modifier_parsing-4                                  500000              3755 ns/op
BenchmarkGval/regex_evaluation-4                                   50000             21347 ns/op
BenchmarkGval/regex_parsing-4                                     200000              6480 ns/op
BenchmarkGval/constant_regex_evaluation-4                        1000000              1000 ns/op
BenchmarkGval/constant_regex_parsing-4                            200000              9417 ns/op
BenchmarkGval/accessors_evaluation-4                             3000000               417 ns/op
BenchmarkGval/accessors_parsing-4                                1000000              1778 ns/op
BenchmarkGval/accessors_method_evaluation-4                      1000000              1931 ns/op
BenchmarkGval/accessors_method_parsing-4                         1000000              1729 ns/op
BenchmarkGval/accessors_method_parameter_evaluation-4            1000000              2162 ns/op
BenchmarkGval/accessors_method_parameter_parsing-4                500000              2618 ns/op
BenchmarkGval/nested_accessors_evaluation-4                      2000000               681 ns/op
BenchmarkGval/nested_accessors_parsing-4                         1000000              2115 ns/op
BenchmarkRandom-4                                                 500000              3631 ns/op
ok

API Breaks

Gval is designed with easy expandability in mind and API breaks will be avoided if possible. If API breaks are unavoidable they wil be explicitly stated via an increased major version number.


Credits to Reene French for the gophers.

Comments
  • Date evaluation

    Date evaluation

    Hello,

    First your library looks great. Thank you for sharing. I would like to evaluate date + duration: date(myObject.date) + 3d I was wondering what was the best approach: adding a new language? Cheers, JC

    opened by jcantonio 8
  • Without space after && leading error

    Without space after && leading error

    package main
    
    import (
    	"fmt"
    	"github.com/PaesslerAG/gval"
    )
    
    func main() {
    	vars := map[string]interface{}{"name": true}
    
    	value, err := gval.Evaluate("true&&name", vars)
    	if err != nil {
    		fmt.Println(err)
    	}
    	fmt.Println(value)
    
    	value, err = gval.Evaluate("true&& name", vars)
    	if err != nil {
    		fmt.Println(err)
    	}
    
    	fmt.Println(value)
    }
    

    runing result:

    parsing error: true&&name	 - 1:8 unknown operator &&n
    <nil>
    true
    

    My expected result is the first expression also working fine.

    opened by Ovi3 4
  • Error callback instead of terminating error in JSON

    Error callback instead of terminating error in JSON

    Hi and thanks for this greate piece of software! :)

    We're using it to do sensor mapping:

    • i.e fetch sensor data as a single HTTP request
    • map those sensors into our own sensor model.

    This works great, except when for some reason we do not get one or more needed sensor data points. In this case it will error out.

    Could it be possible to do a callback to an error function instead, and if that error function do return an error, it will fail, otherwise it will remove the json key or set the value to nil (or a value returned by the error function?

    func OnError(c context.Context, key, expr string, e error) (val interface{], err error) {}
    

    Example mapping document that we use (partial)

    { "tz": TZ,
    	"sensors": { 
    			"IDT_O3": float64(datapoint["1!6WHH6DKH3CCVW-"].Value),
    			"IDT_O5": float64(datapoint["1!6WHH6DKHADPZ1N"].Value), 
                             "AEUPS": float64(datapoint["1!6WHH6DKHICVDSS"].Value * 1000) }
    }
    

    The error function may act as a OnErrorResumeNext or a circuit-breaker (the latter is how it currently works).

    What do you think about this? Is is hard to implement in your library, would it fit?

    To maybe do other than current error handling when this or this returns an error?

    p.Camouflage("object", ',', '}')
    key, err := p.ParseExpression(c)
    if err != nil {
      return nil, err
    }
    

    Cheers, Mario :)

    opened by mariotoffia 4
  • unable to get value

    unable to get value

    i have run the following with jsonPath

    package main
    
    func main() {
    	v := interface{}(nil)
    	json.Unmarshal([]byte(`{
    		"foo": {
    			"bar": {
    				"val": 3
    			}
    		}
    	}`), &v)
    
    	value, err := jsonpath.Get("$.*.*.val", v)
    	if err != nil {
    		fmt.Println(err)
    		os.Exit(1)
    	}
    
    	for _, value := range value.([]interface{}) {
    		fmt.Println(value)
    	}
    }
    
    

    and this is my output $.*.*.valmap[foo:map[bar:map[val:3]]]3

    however if i run

    v := interface{}(nil)
    	json.Unmarshal([]byte(`{
    		"foo": {
    			"bar": {
    				"val": 3
    			}
    		}
    	}`), &v)
    value, err := gval.Evaluate("$.*.*.val", v, jsonpath.Language())
    

    it returns

    [] am i using this wrong?

    i would expect it to return

    3 and if i run

    v := interface{}(nil)
    	json.Unmarshal([]byte(`{
    		"foo": {
    			"bar": {
    				"val": 3
    			}
    		}
    	}`), &v)
    value, err := gval.Evaluate("$.*.*.val == 3", v, jsonpath.Language())
    

    i would expect

    true 
    
    opened by Gigaclank 4
  • Case Mismatch: Able to create Evaluable but actual evaluate fails

    Case Mismatch: Able to create Evaluable but actual evaluate fails

    Hi,

    I am using gval to build an excel formula parser. Repo name is efp. Check efp_test.go. You can change case in expression to see behavior.

    In excel, all function names are capitalized so, I have created Functions with Capital names. Language allows creation of non-null Evaluable even when case does not match. But, when Eval* function is called on Evaluable the results are incorrect.

    The function mapped to keyword is called correctly and the value returned from the function is accurate but, it seems this value is lost after evaluation. Any thoughts?

    Ideally Parse should fail in this scenario so that users get feedback when expression is built improperly.

    opened by praveentiru 4
  • Guidance; abusing gval to set keys

    Guidance; abusing gval to set keys

    Hey folks; first up thanks for gval... it's a seriously awesome little library. Hard to believe it doesn't have more adoption!

    To the point... I'm using gval in a little tool I wrote to smash up kubernetes manifests. https://github.com/mikesimons/kpatch. Please excuse the nasty code; it's prototype grade at the moment.

    I've implemented an = infix operator and it works pretty well to overwrite values that already exist.

    The problem is that because I can't get the verbatim key from the left expression I'm having to set values based on a walk of the input structure and matching on reflection values. It works but it means that if the key doesn't exist no value matches and the value is not set.

    To get around it I implemented a set function but from a user experience perspective I'd really like to be able to do some.key = "value" even if some.key wasn't previously set.

    I had a go with an InfixEvalExpression but due to the input being wrapped in an Evaluable func I couldn't get the value out.

    Any advice on a way to do this (even if it means patching gval)?

    opened by mikesimons 4
  • Undefined attributes key returns error

    Undefined attributes key returns error

    Hi, I'm trying to create a conditional expression if a variable is nil, but it throws an Error instead.

    parameters := make(map[string]interface)
    parameters["foo"] = "bar"
    
    expression := "fooz ?? foo"
    result, err := gval.Evaluate(expression, parameters)
    if err != nil {
      panic(err)
    }
    

    Throw error unknown parameter fooz when it should be bar. I've tried to solve this problem by change return error value to a nil value from private function func variable(path ...Evaluable)

    opened by ramarahmanda 4
  • Extending variable() to support custom selectors on value

    Extending variable() to support custom selectors on value

    I'm struggling to get a bit more control over how path selection is done.

    I would like to expose one of my structs with prop that's internally slice of structs ({ name, value, position }) to be accessed in more "fluent" way (record.values.foo or record.values.bar[0]). More info here: https://github.com/cortezaproject/corteza-server/blob/develop/compose/types/record.go#L38

    Currently, I'm doing this by casting into map[string]interface{} but I would like to avoid this because it requires keeping copies of the original data and constantly syncing and casting.

    I've tried with adding an operator but I have to admit I'm lacking know-how of the internal workings of gval to make even a proof of concept. A custom function (getValue(record, name, position)) would work but that ruins the fluentness :)

    After digging more into gval, I'm thniking that a) variable fn could be modified to get additional case that would cover a special interface interface { GvalSelect(key string) (interface{}, error) } (I'm not 100% happy with the fn name here... ).

    If you think this would be a good approach I can open a PR in next couple of days.

    opened by darh 3
  • Make ident and parentheses parsers available separately from Base()

    Make ident and parentheses parsers available separately from Base()

    When implementing a language with custom operators, only a part of the Base() is useful, yet you can't easily get that part without the rest of the features.

    An example of these parts are private parse*() functions. While things like parseString() can be easily re-implemented, the parseParentheses() and parseIdent() are somewhat more complicated, yet they're not available for re-using externally.

    This change keeps the Base() as is, but allows one to re-use at least the complex parts of it.

    opened by edigaryev 3
  • Publish new version for DecimalArithmetic

    Publish new version for DecimalArithmetic

    Hi Guys,

    I have a requirement for the recently added DecimalArithmetic functionality added here by @machship-mm.

    However, this change is not yet published in the latest release. @generikvault Would it be possible for you or someone else to release a new version?

    Thanks!

    opened by shaswatk 2
  • Handle decimal/money arithmetic

    Handle decimal/money arithmetic

    This library is fantastic!

    The one issue I've found is that the Arithmetic is done as floating point numbers (which is fine for that use case) but this falls short when doing math involving decimals.

    An example:

    func TestDecimalArithmetic(t *testing.T) {
    	input := ArithmeticVariables{
    		NumberMap: map[string]float64{
    			"x": 12.5,
    			"y": -5,
    		},
    		Expression: "(x * 12.146) - y",
    	}
    
    	result, _ := gval.Arithmetic().Evaluate(input.Expression, input.NumberMap)
    
    	want := 156.825
    	if result != want {
    		t.Errorf("wanted %f; got %f", want, result)
    		t.FailNow()
    	}
    
    	t.Log(result)
    }
    

    This results in this output:

    === RUN   TestDecimalArithmetic
        arithmetic_test.go:46: wanted 156.825000; got 156.825000
    --- FAIL: TestDecimalArithmetic (0.00s)
    

    When inspecting the variable values, I can see that result has the actual value of 156.82500000000002.

    Do you have any suggestions for what I should do, or is this even a solvable problem?

    I was thinking that there could be an EvaluateDecimal method, which instead of treating numbers as floats, would treat them as a decimal type? For decimals, I use github.com/shopspring/decimal.

    opened by machship-mm 2
  • fix negation in decimal language

    fix negation in decimal language

    Happened to notice this by inspection.

    I was even getting some weird errors with just a plain "-1", since I suspect this gets parsed as a negate on the literal "1", but I didn't check too deep into it. But my initial concern was about precision, so I've added a test to that effect as well.

    opened by imirkin 0
  • Evaluating strings with backslashes raises parsing errors

    Evaluating strings with backslashes raises parsing errors

    Env

    This is on github.com/PaesslerAG/gval v1.2.1

    Issue

    We have a custom handler for regular expressions (match(value, regex)) where the regex can be configured from the front-end (web application).

    For example, inputting match(variable, "[a-z]\d") would be presented as (when dumping the value) "match(variable, \"[a-z]\\d\")" (which seems correct to me), but when passed to gval, it raises parsing error: match(variable, "[a-z]\d") :1:11 - 1:20 could not parse string: invalid syntax.

    If I bypass all of the surrounding code and run it directly as so x, err := gval.Evaluate("match(variable, \"[a-z]\\d\")", map[string]any{"a": "b"}) the error is the same.

    Another example would be running this x, err := gval.Evaluate("\"\\\"", map[string]any{"a": "b"}) -- same error in regards to the invalid string (parsing error: "\" :1:1 - 1:4 could not parse string: invalid syntax).

    But then, if I do this x, err := gval.Evaluate("\"apple\\banana\"", map[string]any{"a": "b"}), it passes and outputs (when running spew.Dump(x, err))

    (string) (len=11) "apple\banana"
    (interface {}) <nil>
    

    Conclusion

    Am I doing something wrong with my strings, or is this a (un)intentional edge case?

    If this needs more investigation I can take a look through the code and propose a fix as well, but I'd appreciate some notes on where would be a good place to start/your suspicions about what's wrong

    opened by tjerman 1
  • Add support for typed map and slices with method calls

    Add support for typed map and slices with method calls

    I noticed that gval didn't support expressions for typed maps and slice method call.

    Example map:

    type MyMap map[string][]int
    
    func (m MyMap) Sum(key string) int {
        values, ok := m[key]
        if !ok {
            return -1
        }
        sum := 0
        for _, v := range values {
            sum += v
        }
        return sum
    }
    
    opened by tw1nk 0
  • Feature Request: high order functions (collection)

    Feature Request: high order functions (collection)

    What is it?

    Some special functions like filter, map, exists, all.

    For example:

    ["project1/text.txt", "project2/text.txt"].exists(t => t.startsWith("project1"))
    ; => true
    

    Why?

    This would simplify a lot of some operations that would require very specific custom functions.


    Let me know if that's something you have thought about it.

    opened by RafaeLeal 0
  • Allow dashes in identfiers

    Allow dashes in identfiers

    I have the case that one of my identifiers contains a -, like type-id. The parser fails because - is not an allowed rune in an identifier.

    Example:

    func main() {
        vars := map[string]interface{}{"type-id": "4"}
        expr := `type-id == "4"`
        e, err := gval.Full().NewEvaluable(expr)
        if err != nil {
            return
        }
        value, err := e.EvalBool(context.Background(), vars)
        if err != nil {
            return
        }
        fmt.Println(value)
    }
    

    I could open a PR, but the fix is so simple: Add the condition (pos > 0 && r == '-') to https://github.com/PaesslerAG/gval/blob/master/parser.go#L33 . The example works, I tested it even for multiple dashes. But I'm new to gval, so I might miss some cases.

    opened by TomTheBear 0
Releases(v1.2.1)
Owner
null
Fast, portable, non-Turing complete expression evaluation with gradual typing (Go)

Common Expression Language The Common Expression Language (CEL) is a non-Turing complete language designed for simplicity, speed, safety, and portabil

Google 1.4k Nov 22, 2022
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing

Expr Expr package provides an engine that can compile and evaluate expressions. An expression is a one-liner that returns a value (mostly, but not lim

Anton Medvedev 3.2k Nov 27, 2022
Logexp - Logical expression compiler for golang

Logical Expression Compiler Functions: - Compile(exp string) - Match(text string

Jinglever 1 Jan 24, 2022
Mathematical expression parsing and calculation engine library. 数学表达式解析计算引擎库

Math-Engine 使用 Go 实现的数学表达式解析计算引擎库,它小巧,无任何依赖,具有扩展性(比如可以注册自己的函数到引擎中),比较完整的完成了数学表达式解析执行,包括词法分析、语法分析、构建AST、运行。 go get -u github.com/dengsgo/math-engine 能够

Deng.Liu 251 Nov 28, 2022
Suan - Mathematical expression calculation tool

suan Suan( 算 ) is a CLI tool to calculate given mathematical expression. Current

null 1 Feb 14, 2022
Transpiling fortran code to golang code

f4go Example of use > # Install golang > # Compile f4go > go get -u github.com/Konstantin8105/f4go > cd $GOPATH/src/github.com/Konstantin8105/f4go > g

Konstantin 34 Sep 26, 2022
Golang->Haxe->CPP/CSharp/Java/JavaScript transpiler

TARDIS Go -> Haxe transpiler Haxe -> C++ / C# / Java / JavaScript Project status: a non-working curiosity, development currently on-ice The advent of

TARDIS Go 423 Nov 27, 2022
A JavaScript interpreter in Go (golang)

otto -- import "github.com/robertkrimen/otto" Package otto is a JavaScript parser and interpreter written natively in Go. http://godoc.org/github.com/

Robert Krimen 6.9k Nov 22, 2022
A BASIC interpreter written in golang.

05 PRINT "Index" 10 PRINT "GOBASIC!" 20 PRINT "Limitations" Arrays Line Numbers IF Statement DATA / READ Statements Builtin Functions Types 30 PRINT "

Steve Kemp 289 Nov 23, 2022
PHP bindings for the Go programming language (Golang)

PHP bindings for Go This package implements support for executing PHP scripts, exporting Go variables for use in PHP contexts, attaching Go method rec

Alex Palaistras 878 Nov 22, 2022
High-performance PHP-to-Golang IPC bridge

High-performance PHP-to-Golang IPC bridge Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/r

Spiral Scout 1.1k Nov 23, 2022
High-performance PHP application server, load-balancer and process manager written in Golang

RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports running as a serv

Spiral Scout 6.9k Nov 27, 2022
golang AST matcher

goastch (GO AST matCH) Introduction Inspired by ast matcher. There are four different basic categories of matchers: Node Matchers: Matchers that match

Helloyi He 13 Nov 11, 2022
Scriptable interpreter written in golang

Anko Anko is a scriptable interpreter written in Go. (Picture licensed under CC BY-SA 3.0, photo by Ocdp) Usage Example - Embedded package main impor

mattn 1.3k Nov 21, 2022
hotbuild - a cross platform hot compilation tool for golang

hotbuild A cross platform hot compilation tool By monitoring the modification of the project directory file, the recompilation and running are automat

wander 188 Nov 22, 2022
The golang tool of the zig compiler automatically compiles different targets according to the GOOS GOARCH environment variable. You need to install zig.

The golang tool of the zig compiler automatically compiles different targets according to the GOOS GOARCH environment variable. You need to install zig.

dosgo 30 Nov 18, 2022
Tgo - Test Helpers for Standard Golang Packages

Test Helpers for Standard Golang Packages see example_test.go go test --- FAIL:

krhubert 1 Apr 26, 2022
Runcmd - just golang binary that runs commands from url or local file and logs output

runcmd just golang binary that runs commands from url or local file and logs out

boredhackerblog 0 Feb 2, 2022
A compiler for the ReCT programming language written in Golang

ReCT-Go-Compiler A compiler for the ReCT programming language written in Golang

null 7 Nov 20, 2022