GopherLua: VM and compiler for Lua in Go

Related tags

go lua gopher-lua
Overview

GopherLua: VM and compiler for Lua in Go.

Join the chat at https://gitter.im/yuin/gopher-lua

GopherLua is a Lua5.1 VM and compiler written in Go. GopherLua has a same goal with Lua: Be a scripting language with extensible semantics . It provides Go APIs that allow you to easily embed a scripting language to your Go host programs.

Design principle

  • Be a scripting language with extensible semantics.
  • User-friendly Go API
    • The stack based API like the one used in the original Lua implementation will cause a performance improvements in GopherLua (It will reduce memory allocations and concrete type <-> interface conversions). GopherLua API is not the stack based API. GopherLua give preference to the user-friendliness over the performance.

How about performance?

GopherLua is not fast but not too slow, I think.

GopherLua has almost equivalent ( or little bit better ) performance as Python3 on micro benchmarks.

There are some benchmarks on the wiki page .

Installation

go get github.com/yuin/gopher-lua

GopherLua supports >= Go1.9.

Usage

GopherLua APIs perform in much the same way as Lua, but the stack is used only for passing arguments and receiving returned values.

GopherLua supports channel operations. See "Goroutines" section.

Import a package.

import (
    "github.com/yuin/gopher-lua"
)

Run scripts in the VM.

L := lua.NewState()
defer L.Close()
if err := L.DoString(`print("hello")`); err != nil {
    panic(err)
}
L := lua.NewState()
defer L.Close()
if err := L.DoFile("hello.lua"); err != nil {
    panic(err)
}

Refer to Lua Reference Manual and Go doc for further information.

Note that elements that are not commented in Go doc equivalent to Lua Reference Manual , except GopherLua uses objects instead of Lua stack indices.

Data model

All data in a GopherLua program is an LValue . LValue is an interface type that has following methods.

  • String() string
  • Type() LValueType

Objects implement an LValue interface are

Type name Go type Type() value Constants
LNilType (constants) LTNil LNil
LBool (constants) LTBool LTrue, LFalse
LNumber float64 LTNumber -
LString string LTString -
LFunction struct pointer LTFunction -
LUserData struct pointer LTUserData -
LState struct pointer LTThread -
LTable struct pointer LTTable -
LChannel chan LValue LTChannel -

You can test an object type in Go way(type assertion) or using a Type() value.

lv := L.Get(-1) // get the value at the top of the stack
if str, ok := lv.(lua.LString); ok {
    // lv is LString
    fmt.Println(string(str))
}
if lv.Type() != lua.LTString {
    panic("string required.")
}
lv := L.Get(-1) // get the value at the top of the stack
if tbl, ok := lv.(*lua.LTable); ok {
    // lv is LTable
    fmt.Println(L.ObjLen(tbl))
}

Note that LBool , LNumber , LString is not a pointer.

To test LNilType and LBool, You must use pre-defined constants.

lv := L.Get(-1) // get the value at the top of the stack

if lv == lua.LTrue { // correct
}

if bl, ok := lv.(lua.LBool); ok && bool(bl) { // wrong
}

In Lua, both nil and false make a condition false. LVIsFalse and LVAsBool implement this specification.

lv := L.Get(-1) // get the value at the top of the stack
if lua.LVIsFalse(lv) { // lv is nil or false
}

if lua.LVAsBool(lv) { // lv is neither nil nor false
}

Objects that based on go structs(LFunction. LUserData, LTable) have some public methods and fields. You can use these methods and fields for performance and debugging, but there are some limitations.

  • Metatable does not work.
  • No error handlings.

Callstack & Registry size

The size of an LState's callstack controls the maximum call depth for Lua functions within a script (Go function calls do not count).

The registry of an LState implements stack storage for calling functions (both Lua and Go functions) and also for temporary variables in expressions. Its storage requirements will increase with callstack usage and also with code complexity.

Both the registry and the callstack can be set to either a fixed size or to auto size.

When you have a large number of LStates instantiated in a process, it's worth taking the time to tune the registry and callstack options.

Registry

The registry can have an initial size, a maximum size and a step size configured on a per LState basis. This will allow the registry to grow as needed. It will not shrink again after growing.

 L := lua.NewState(lua.Options{
    RegistrySize: 1024 * 20,         // this is the initial size of the registry
    RegistryMaxSize: 1024 * 80,      // this is the maximum size that the registry can grow to. If set to `0` (the default) then the registry will not auto grow
    RegistryGrowStep: 32,            // this is how much to step up the registry by each time it runs out of space. The default is `32`.
 })
defer L.Close()

A registry which is too small for a given script will ultimately result in a panic. A registry which is too big will waste memory (which can be significant if many LStates are instantiated). Auto growing registries incur a small performance hit at the point they are resized but will not otherwise affect performance.

Callstack

The callstack can operate in two different modes, fixed or auto size. A fixed size callstack has the highest performance and has a fixed memory overhead. An auto sizing callstack will allocate and release callstack pages on demand which will ensure the minimum amount of memory is in use at any time. The downside is it will incur a small performance impact every time a new page of callframes is allocated. By default an LState will allocate and free callstack frames in pages of 8, so the allocation overhead is not incurred on every function call. It is very likely that the performance impact of an auto resizing callstack will be negligible for most use cases.

 L := lua.NewState(lua.Options{
     CallStackSize: 120,                 // this is the maximum callstack size of this LState
     MinimizeStackMemory: true,          // Defaults to `false` if not specified. If set, the callstack will auto grow and shrink as needed up to a max of `CallStackSize`. If not set, the callstack will be fixed at `CallStackSize`.
 })
defer L.Close()

Option defaults

The above examples show how to customize the callstack and registry size on a per LState basis. You can also adjust some defaults for when options are not specified by altering the values of lua.RegistrySize, lua.RegistryGrowStep and lua.CallStackSize.

An LState object that has been created by *LState#NewThread() inherits the callstack & registry size from the parent LState object.

Miscellaneous lua.NewState options

  • Options.SkipOpenLibs bool(default false)
    • By default, GopherLua opens all built-in libraries when new LState is created.
    • You can skip this behaviour by setting this to true .
    • Using the various OpenXXX(L *LState) int functions you can open only those libraries that you require, for an example see below.
  • Options.IncludeGoStackTrace bool(default false)
    • By default, GopherLua does not show Go stack traces when panics occur.
    • You can get Go stack traces by setting this to true .

API

Refer to Lua Reference Manual and Go doc(LState methods) for further information.

Calling Go from Lua

func Double(L *lua.LState) int {
    lv := L.ToInt(1)             /* get argument */
    L.Push(lua.LNumber(lv * 2)) /* push result */
    return 1                     /* number of results */
}

func main() {
    L := lua.NewState()
    defer L.Close()
    L.SetGlobal("double", L.NewFunction(Double)) /* Original lua_setglobal uses stack... */
}
print(double(20)) -- > "40"

Any function registered with GopherLua is a lua.LGFunction, defined in value.go

type LGFunction func(*LState) int

Working with coroutines.

co, _ := L.NewThread() /* create a new thread */
fn := L.GetGlobal("coro").(*lua.LFunction) /* get function from lua */
for {
    st, err, values := L.Resume(co, fn)
    if st == lua.ResumeError {
        fmt.Println("yield break(error)")
        fmt.Println(err.Error())
        break
    }

    for i, lv := range values {
        fmt.Printf("%v : %v\n", i, lv)
    }

    if st == lua.ResumeOK {
        fmt.Println("yield break(ok)")
        break
    }
}

Opening a subset of builtin modules

The following demonstrates how to open a subset of the built-in modules in Lua, say for example to avoid enabling modules with access to local files or system calls.

main.go

func main() {
    L := lua.NewState(lua.Options{SkipOpenLibs: true})
    defer L.Close()
    for _, pair := range []struct {
        n string
        f lua.LGFunction
    }{
        {lua.LoadLibName, lua.OpenPackage}, // Must be first
        {lua.BaseLibName, lua.OpenBase},
        {lua.TabLibName, lua.OpenTable},
    } {
        if err := L.CallByParam(lua.P{
            Fn:      L.NewFunction(pair.f),
            NRet:    0,
            Protect: true,
        }, lua.LString(pair.n)); err != nil {
            panic(err)
        }
    }
    if err := L.DoFile("main.lua"); err != nil {
        panic(err)
    }
}

Creating a module by Go

mymodule.go

package mymodule

import (
    "github.com/yuin/gopher-lua"
)

func Loader(L *lua.LState) int {
    // register functions to the table
    mod := L.SetFuncs(L.NewTable(), exports)
    // register other stuff
    L.SetField(mod, "name", lua.LString("value"))

    // returns the module
    L.Push(mod)
    return 1
}

var exports = map[string]lua.LGFunction{
    "myfunc": myfunc,
}

func myfunc(L *lua.LState) int {
    return 0
}

mymain.go

package main

import (
    "./mymodule"
    "github.com/yuin/gopher-lua"
)

func main() {
    L := lua.NewState()
    defer L.Close()
    L.PreloadModule("mymodule", mymodule.Loader)
    if err := L.DoFile("main.lua"); err != nil {
        panic(err)
    }
}

main.lua

local m = require("mymodule")
m.myfunc()
print(m.name)

Calling Lua from Go

L := lua.NewState()
defer L.Close()
if err := L.DoFile("double.lua"); err != nil {
    panic(err)
}
if err := L.CallByParam(lua.P{
    Fn: L.GetGlobal("double"),
    NRet: 1,
    Protect: true,
    }, lua.LNumber(10)); err != nil {
    panic(err)
}
ret := L.Get(-1) // returned value
L.Pop(1)  // remove received value

If Protect is false, GopherLua will panic instead of returning an error value.

User-Defined types

You can extend GopherLua with new types written in Go. LUserData is provided for this purpose.

type Person struct {
    Name string
}

const luaPersonTypeName = "person"

// Registers my person type to given L.
func registerPersonType(L *lua.LState) {
    mt := L.NewTypeMetatable(luaPersonTypeName)
    L.SetGlobal("person", mt)
    // static attributes
    L.SetField(mt, "new", L.NewFunction(newPerson))
    // methods
    L.SetField(mt, "__index", L.SetFuncs(L.NewTable(), personMethods))
}

// Constructor
func newPerson(L *lua.LState) int {
    person := &Person{L.CheckString(1)}
    ud := L.NewUserData()
    ud.Value = person
    L.SetMetatable(ud, L.GetTypeMetatable(luaPersonTypeName))
    L.Push(ud)
    return 1
}

// Checks whether the first lua argument is a *LUserData with *Person and returns this *Person.
func checkPerson(L *lua.LState) *Person {
    ud := L.CheckUserData(1)
    if v, ok := ud.Value.(*Person); ok {
        return v
    }
    L.ArgError(1, "person expected")
    return nil
}

var personMethods = map[string]lua.LGFunction{
    "name": personGetSetName,
}

// Getter and setter for the Person#Name
func personGetSetName(L *lua.LState) int {
    p := checkPerson(L)
    if L.GetTop() == 2 {
        p.Name = L.CheckString(2)
        return 0
    }
    L.Push(lua.LString(p.Name))
    return 1
}

func main() {
    L := lua.NewState()
    defer L.Close()
    registerPersonType(L)
    if err := L.DoString(`
        p = person.new("Steeve")
        print(p:name()) -- "Steeve"
        p:name("Alice")
        print(p:name()) -- "Alice"
    `); err != nil {
        panic(err)
    }
}

Terminating a running LState

GopherLua supports the Go Concurrency Patterns: Context .

L := lua.NewState()
defer L.Close()
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
defer cancel()
// set the context to our LState
L.SetContext(ctx)
err := L.DoString(`
  local clock = os.clock
  function sleep(n)  -- seconds
    local t0 = clock()
    while clock() - t0 <= n do end
  end
  sleep(3)
`)
// err.Error() contains "context deadline exceeded"

With coroutines

L := lua.NewState()
defer L.Close()
ctx, cancel := context.WithCancel(context.Background())
L.SetContext(ctx)
defer cancel()
L.DoString(`
    function coro()
          local i = 0
          while true do
            coroutine.yield(i)
                i = i+1
          end
          return i
    end
`)
co, cocancel := L.NewThread()
defer cocancel()
fn := L.GetGlobal("coro").(*LFunction)

_, err, values := L.Resume(co, fn) // err is nil

cancel() // cancel the parent context

_, err, values = L.Resume(co, fn) // err is NOT nil : child context was canceled

Note that using a context causes performance degradation.

time ./glua-with-context.exe fib.lua
9227465
0.01s user 0.11s system 1% cpu 7.505 total

time ./glua-without-context.exe fib.lua
9227465
0.01s user 0.01s system 0% cpu 5.306 total

Sharing Lua byte code between LStates

Calling DoFile will load a Lua script, compile it to byte code and run the byte code in a LState.

If you have multiple LStates which are all required to run the same script, you can share the byte code between them, which will save on memory. Sharing byte code is safe as it is read only and cannot be altered by lua scripts.

// CompileLua reads the passed lua file from disk and compiles it.
func CompileLua(filePath string) (*lua.FunctionProto, error) {
    file, err := os.Open(filePath)
    defer file.Close()
    if err != nil {
        return nil, err
    }
    reader := bufio.NewReader(file)
    chunk, err := parse.Parse(reader, filePath)
    if err != nil {
        return nil, err
    }
    proto, err := lua.Compile(chunk, filePath)
    if err != nil {
        return nil, err
    }
    return proto, nil
}

// DoCompiledFile takes a FunctionProto, as returned by CompileLua, and runs it in the LState. It is equivalent
// to calling DoFile on the LState with the original source file.
func DoCompiledFile(L *lua.LState, proto *lua.FunctionProto) error {
    lfunc := L.NewFunctionFromProto(proto)
    L.Push(lfunc)
    return L.PCall(0, lua.MultRet, nil)
}

// Example shows how to share the compiled byte code from a lua script between multiple VMs.
func Example() {
    codeToShare := CompileLua("mylua.lua")
    a := lua.NewState()
    b := lua.NewState()
    c := lua.NewState()
    DoCompiledFile(a, codeToShare)
    DoCompiledFile(b, codeToShare)
    DoCompiledFile(c, codeToShare)
}

Goroutines

The LState is not goroutine-safe. It is recommended to use one LState per goroutine and communicate between goroutines by using channels.

Channels are represented by channel objects in GopherLua. And a channel table provides functions for performing channel operations.

Some objects can not be sent over channels due to having non-goroutine-safe objects inside itself.

  • a thread(state)
  • a function
  • an userdata
  • a table with a metatable

You must not send these objects from Go APIs to channels.

func receiver(ch, quit chan lua.LValue) {
    L := lua.NewState()
    defer L.Close()
    L.SetGlobal("ch", lua.LChannel(ch))
    L.SetGlobal("quit", lua.LChannel(quit))
    if err := L.DoString(`
    local exit = false
    while not exit do
      channel.select(
        {"|<-", ch, function(ok, v)
          if not ok then
            print("channel closed")
            exit = true
          else
            print("received:", v)
          end
        end},
        {"|<-", quit, function(ok, v)
            print("quit")
            exit = true
        end}
      )
    end
  `); err != nil {
        panic(err)
    }
}

func sender(ch, quit chan lua.LValue) {
    L := lua.NewState()
    defer L.Close()
    L.SetGlobal("ch", lua.LChannel(ch))
    L.SetGlobal("quit", lua.LChannel(quit))
    if err := L.DoString(`
    ch:send("1")
    ch:send("2")
  `); err != nil {
        panic(err)
    }
    ch <- lua.LString("3")
    quit <- lua.LTrue
}

func main() {
    ch := make(chan lua.LValue)
    quit := make(chan lua.LValue)
    go receiver(ch, quit)
    go sender(ch, quit)
    time.Sleep(3 * time.Second)
}
Go API

ToChannel, CheckChannel, OptChannel are available.

Refer to Go doc(LState methods) for further information.

Lua API
  • channel.make([buf:int]) -> ch:channel
    • Create new channel that has a buffer size of buf. By default, buf is 0.
  • channel.select(case:table [, case:table, case:table ...]) -> {index:int, recv:any, ok}
    • Same as the select statement in Go. It returns the index of the chosen case and, if that case was a receive operation, the value received and a boolean indicating whether the channel has been closed.
    • case is a table that outlined below.
      • receiving: {"|<-", ch:channel [, handler:func(ok, data:any)]}
      • sending: {"<-|", ch:channel, data:any [, handler:func(data:any)]}
      • default: {"default" [, handler:func()]}

channel.select examples:

local idx, recv, ok = channel.select(
  {"|<-", ch1},
  {"|<-", ch2}
)
if not ok then
    print("closed")
elseif idx == 1 then -- received from ch1
    print(recv)
elseif idx == 2 then -- received from ch2
    print(recv)
end
channel.select(
  {"|<-", ch1, function(ok, data)
    print(ok, data)
  end},
  {"<-|", ch2, "value", function(data)
    print(data)
  end},
  {"default", function()
    print("default action")
  end}
)
  • channel:send(data:any)
    • Send data over the channel.
  • channel:receive() -> ok:bool, data:any
    • Receive some data over the channel.
  • channel:close()
    • Close the channel.
The LState pool pattern

To create per-thread LState instances, You can use the sync.Pool like mechanism.

type lStatePool struct {
    m     sync.Mutex
    saved []*lua.LState
}

func (pl *lStatePool) Get() *lua.LState {
    pl.m.Lock()
    defer pl.m.Unlock()
    n := len(pl.saved)
    if n == 0 {
        return pl.New()
    }
    x := pl.saved[n-1]
    pl.saved = pl.saved[0 : n-1]
    return x
}

func (pl *lStatePool) New() *lua.LState {
    L := lua.NewState()
    // setting the L up here.
    // load scripts, set global variables, share channels, etc...
    return L
}

func (pl *lStatePool) Put(L *lua.LState) {
    pl.m.Lock()
    defer pl.m.Unlock()
    pl.saved = append(pl.saved, L)
}

func (pl *lStatePool) Shutdown() {
    for _, L := range pl.saved {
        L.Close()
    }
}

// Global LState pool
var luaPool = &lStatePool{
    saved: make([]*lua.LState, 0, 4),
}

Now, you can get per-thread LState objects from the luaPool .

func MyWorker() {
   L := luaPool.Get()
   defer luaPool.Put(L)
   /* your code here */
}

func main() {
    defer luaPool.Shutdown()
    go MyWorker()
    go MyWorker()
    /* etc... */
}

Differences between Lua and GopherLua

Goroutines

  • GopherLua supports channel operations.
    • GopherLua has a type named channel.
    • The channel table provides functions for performing channel operations.

Unsupported functions

  • string.dump
  • os.setlocale
  • lua_Debug.namewhat
  • package.loadlib
  • debug hooks

Miscellaneous notes

  • collectgarbage does not take any arguments and runs the garbage collector for the entire Go program.
  • file:setvbuf does not support a line buffering.
  • Daylight saving time is not supported.
  • GopherLua has a function to set an environment variable : os.setenv(name, value)

Standalone interpreter

Lua has an interpreter called lua . GopherLua has an interpreter called glua .

go get github.com/yuin/gopher-lua/cmd/glua

glua has same options as lua .

How to Contribute

See Guidlines for contributors .

Libraries for GopherLua

  • gopher-luar : Simplifies data passing to and from gopher-lua
  • gluamapper : Mapping a Lua table to a Go struct
  • gluare : Regular expressions for gopher-lua
  • gluahttp : HTTP request module for gopher-lua
  • gopher-json : A simple JSON encoder/decoder for gopher-lua
  • gluayaml : Yaml parser for gopher-lua
  • glua-lfs : Partially implements the luafilesystem module for gopher-lua
  • gluaurl : A url parser/builder module for gopher-lua
  • gluahttpscrape : A simple HTML scraper module for gopher-lua
  • gluaxmlpath : An xmlpath module for gopher-lua
  • gmoonscript : Moonscript Compiler for the Gopher Lua VM
  • loguago : Zerolog wrapper for Gopher-Lua
  • gluacrypto : A native Go implementation of crypto library for the GopherLua VM.
  • gluasql : A native Go implementation of SQL client for the GopherLua VM.
  • purr : A http mock testing tool.
  • vadv/gopher-lua-libs : Some usefull libraries for GopherLua VM.
  • gluaperiphery : A periphery library for the GopherLua VM (GPIO, SPI, I2C, MMIO, and Serial peripheral I/O for Linux).
  • glua-async : An async/await implement for gopher-lua.
  • gopherlua-debugger : A debugger for gopher-lua

Donation

BTC: 1NEDSyUmo4SMTDP83JJQSWi1MvQUGGNMZB

License

MIT

Author

Yusuke Inuzuka

Issues
  • Allow installing some but not all of the standard modules.

    Allow installing some but not all of the standard modules.

    Currently you can have all, or no standard modules. This produces problems when, for example, you need everything but os and io to be available.

    The fix for this is trivial, just export the openXYZ functions so that users can install modules piecemeal if they wish.

    enhancement 
    opened by milochristiansen 16
  • Implemented OpenXXX LGFunctions for each built-in module, refactored LState.OpenLibs

    Implemented OpenXXX LGFunctions for each built-in module, refactored LState.OpenLibs

    WRT #55.

    Apologies for the whitespace removal diff-noise, that was my editor..

    opened by cathalgarvey 13
  • Passing LTable to CallByParam

    Passing LTable to CallByParam

    • [x] GopherLua is a Lua5.1 implementation. You should be familiar with Lua programming language. Have you read Lua 5.1 reference manual carefully?
    • [x] GopherLua is a Lua5.1 implementation. In Lua, to keep it simple, it is more important to remove functionalities rather than to add functionalities unlike other languages . If you are going to introduce some new cool functionalities into the GopherLua code base and the functionalities can be implemented by existing APIs, It should be implemented as a library.

    Please answer the following before submitting your issue:

    1. What version of GopherLua are you using? : https://github.com/yuin/gopher-lua/commit/1cd887cd7036
    2. What version of Go are you using? : go version go1.13.3
    3. What operating system and processor architecture are you using? : linux/amd64
    4. What did you do? : Exactly like this issue : https://github.com/yuin/gopher-lua/issues/178
    lTable.RawSetString("varTest", "test")
    L.CallByParam(lua.P{
    	[...],
    }, &lTable)
    

    When I print my lTable in Go, I get something like that : {<nil> [] map[] map[varTest:test] [varTest] map[varTest:0]}

    And in my test function in LUA :

    
    function Test(tableTest)
    	print("tableTest:", tableTest)
    end
    

    (I tried .Metatable() too, in case)

    1. What did you expect to see? : The keys and values in my lTable
    2. What did you see instead? : I get a nil
    opened by Julien2313 11
  • how do you think of supporting closure serialization of lua?

    how do you think of supporting closure serialization of lua?

    Closure serialization, which means code and it's context partly evaluated. In my business, If a closure can be serialized, it can be passed to remote service to apply with the input provided by the remote service.

    a closure(args1...N) -----> remote service( apply(closure,input) )

    This feature is quite useful in distributed system, for example:

    1. Data broker
    2. Remote data warehouse analyze

    Without this, I can only implement Context(some arguments provided) + Code Snippet String (not evaluated) to remote service and Apply(Context, Args, Code)

    How do you think of this?

    PS: https://github.com/jeremeamia/super_closure

    question 
    opened by xtaci 9
  • Accessing comments through AST

    Accessing comments through AST

    I would like to annotate Lua code through comments, much like Go to generate documentation. However, I don't see the comments when I call parse.Dump.

    How should I access the comments through the AST?

    -- relase <var> [optionalVar]
    function target.relase(var, optionalVar)
    
    end
    
    question 
    opened by otm 9
  • Auto growing stack and registry

    Auto growing stack and registry

    Fixes #197 .

    Changes proposed in this pull request:

    • Memory savings:
      • Can optionally set the LState's callstack to grow / shrink automatically (up to a max size)
      • Can optionally set the LState's registry to grow automatically (up to a max size)

    It's been a long time since I opened #197 , sorry for the delay. Please feel free to feedback on this PR or ask for changes.

    There are two parts to this PR - first the registry can be set to auto grow as needed. Originally, the registry was implemented as a fixed size slice. This PR simply allows this slice to be reallocated if it runs out of space. A maximum upper size to be set in the Options struct, as well as a grow step size. By default the RegistryMaxSize will be 0, which disables the auto grow behaviour and makes the code behaviour exactly the same as previously. As there is no penalty in this case, other than an inlined if statement, I have not abstracted the registry growth functionality via an interface.

    The second part of this PR is the more complicated, it allows the callstack to be automatically resized as needed. This has been implemented now via an interface with the LState being configured on construction to either use the previous fixed size callstack, or the new auto growing callstack from this PR. The Options struct dictates which one should be used.

    Abstracting the callstack into an interface is good in that it allows us to switch between implementations depending on the requirements : the auto growing one will use minimal memory, but has a slight performance penalty, whereas the fixed one will always use "worst case" memory, but will have predicable and fast performance.

    Abstracting the callstack to be behind an interface has meant disabling the inlining which was in place for manipulating the stack from within the LState. I have added benchmarks which just do stack manipulation and I did not think the performance hit was so bad, but it might be worth benchmaking the new code (in both the fixed and auto growing configurations) against some actual lua benchmarking scripts, to see how it fares in actual usage.

    opened by tul 7
  • inspect.lua doesn't work

    inspect.lua doesn't work

    Hi, I've just played with inspect.lua (https://github.com/kikito/inspect.lua) but it seems it doesn't work correctly.

    for example)

    macbook% ../bin/glua bug.lua
    
    inspect.lua:300: attempt to index a non-table object(nil)
    stack traceback:
    inspect.lua:300: in inspect.lua:297
    (tailcall): ?
    bug.lua:2: in main chunk
    [G]: ?
    macbook% lua bug.lua
    "hello"
    

    https://gist.github.com/chobie/bcff5c74c915422a3075

    also, I've tested above script with lua 5.1.5 and lua 5.2.3 and it works fine. can you check this if you have a chance?

    Thanks,

    bug 
    opened by chobie 7
  • Allow for optional disabling of library opening and stdin/stdout control

    Allow for optional disabling of library opening and stdin/stdout control

    These two changes are to make https://github.com/jtolds/go-manhole work.

    I'd be okay if we just merged the second change (the optional skipping of library opening), as it makes the first change (control of stdout/stdin/stderr) unnecessary for my purposes.

    Thanks!

    opened by jtolio 7
  • thread-safety and convenience of `basePrint`

    thread-safety and convenience of `basePrint`

    The basePrint function in baselib.go currently writes its output directly to os.Stdout through fmt.Print and friends. This will cause problems when calling print from multiple goroutines/Lua threads. Notably, that multiple Print calls can have their output interwoven on the console when they both run in parallel. For example:

    fmt.Println("foo and bar")
    go fmt.Println("bar and foo")
    

    Can, under certain conditions, give output like this:

    foo abar and foo
    nd bar
    

    Additionally, it would be nice if the host application could specify a different output target, aside from the default os.Stdout. For this reason, I'm wondering if it is not more convenient and safer to have it use Go's log package instead. This package takes care of both problems at once.

    The changes to this package can be as simple as:

    func basePrint(L *LState) int {
        var buf bytes.Buffer
    
        top := L.GetTop()
        for i := 1; i <= top; i++ {
            fmt.Fprint(&buf, L.Get(i).String(), " ")
        }
    
        log.Println(buf.String())
        return 0
    }
    

    The host now has the guarantee that logging stuff is thread safe and the output can be redirected to any target by calling log.SetOutput(io.Writer).

    question wontfix 
    opened by ghost 6
  • gmatch unit test failing

    gmatch unit test failing

    • [X] GopherLua is a Lua5.1 implementation. You should be familiar with Lua programming language. Have you read Lua 5.1 reference manual carefully?
    • [X] GopherLua is a Lua5.1 implementation. In Lua, to keep it simple, it is more important to remove functionalities rather than to add functionalities unlike other languages . If you are going to introduce some new cool functionalities into the GopherLua code base and the functionalities can be implemented by existing APIs, It should be implemented as a library.

    Please answer the following before submitting your issue:

    1. What version of GopherLua are you using? : 8bfc7677f583b35a5663a9dd934c08f3b5774bbb
    2. What version of Go are you using? : go version go1.12.1 darwin/amd64
    3. What operating system and processor architecture are you using? : macOS
    4. What did you do? : make test
    5. What did you expect to see? : All tests pass
    6. What did you see instead? :
    testing _glua-tests/strings.lua
    --- FAIL: TestGlua (0.04s)
        script_test.go:76: issues.lua:180: '[' can not be nested at 0
            stack traceback:
            	[G]: in function 'gmatch'
            	issues.lua:180: in main chunk
            	[G]: ?
    

    The failing code is in issues.lua

    local s = [=[["a"]['b'][9] - ["a"]['b'][8] > ]=]
    local result = {}
    for i in s:gmatch([=[[[][^%s,]*[]]]=]) do 
      table.insert(result, i)
    end
    assert(result[1] == [=[["a"]['b'][9]]=])
    assert(result[2] == [=[["a"]['b'][8]]=])
    

    There is something wrong with the pattern being passed to gmatch, but I don't know enough about what this is trying to test to fix it.

    help wanted 
    opened by tul 6
  • how can I get the variable which function is now calling

    how can I get the variable which function is now calling

    main.go

    func Module(L *lua.LState) int {
    	t := L.NewTable()
    	L.SetFuncs(t, test)
    	L.Push(t)
    	return 1
    }
    
    var test = map[string]lua.LGFunction{
    	"begin": Begin,
            "commit" : Commit,
    }
    
    func Begin(L *lua.LState) int {
    	tx := L.NewTable()
            L.SetTable(tx, lua.string("key"), lua.string("key"))
    	L.SetFuncs(tx, test)
    	L.Push(tx) 
    
    	return 1
    }
    
    func Commit(L *lua.LState) int {
    	//I want get the tx variable which pushed in begin function
            tx := get_tx_who_has_the_commit_method()
            //the tx do some other logic
            key := L.GetTable(tx, lua.string("key"))
            blablabla
    	return 1
    }
    
    
    

    main.lua

    local ops = require("xxx") 
    
    tx = ops.begin() 
    tx.commit()  -- when run here , how can i get the tx variable in golang func Commit
    
    
    opened by domyway 0
  • [BUG] Found an error in state.go (line1491)

    [BUG] Found an error in state.go (line1491)

    [BUG] Found an error in state.go (line1491)

    image

    opened by wwhai 0
  • [Suggestion] Do not throw error on string.dump

    [Suggestion] Do not throw error on string.dump

    • [x] GopherLua is a Lua5.1 implementation. You should be familiar with Lua programming language. Have you read Lua 5.1 reference manual carefully?
    • [x] GopherLua is a Lua5.1 implementation. In Lua, to keep it simple, it is more important to remove functionalities rather than to add functionalities unlike other languages . If you are going to introduce some new cool functionalities into the GopherLua code base and the functionalities can be implemented by existing APIs, It should be implemented as a library.

    Please answer the following before submitting your issue:

    1. What version of GopherLua are you using? : Roughly https://github.com/yuin/gopher-lua/commit/8bfc7677f583b35a5663a9dd934c08f3b5774bbb
    2. What version of Go are you using? : 1.14 I believe
    3. What operating system and processor architecture are you using? : Windows x86
    4. What did you do? : Tried to load compat52 or compat53 in gopher-lua
    5. What did you expect to see? : Success
    6. What did you see instead? : The error about gopher-lua not supporting string.dump

    The compat52 and compat53 libraries test for LuaJIT by checking the output of string.dump, as you can see here. By throwing an error on the use of string.dump, you effectively bar the use of (unmodified) compat libraries. Perhaps it would be better to return nil?

    opened by tooolbox 0
  • Question: LState pooling

    Question: LState pooling

    Thanks for the great project!

    Please answer the following before submitting your issue:

    1. What version of GopherLua are you using? : v0.0.0-20200603152657-dc2b0ca8b37e
    2. What version of Go are you using? : go version go1.16.5
    3. What operating system and processor architecture are you using? : linux/amd64

    I have a question to validate, if my assumption and tests are correct and are fine to use.

    To summarize I am maintainer of https://github.com/zalando/skipper proxy and we have a lua() filter that uses this project to implement it. A filter is an instance that is part of a route. It's not shared between routes. A user just needs to define code like this to execute lua code in request and response path:

    function request(ctx, params) 
        print(c.request.url); 
    end
    function response(ctx, params) 
        print(c.response.status_code); 
    end
    

    Right now we use for every filter instance a separate lua statepool, but if you think about having 40000 routes and maybe 4000 routes with lua() filters we would have a huge amount of wasted memory or we would have to create LStates all the time.

    Right now I am testing it more in depth and tried to share the LState pool. Basically I create at startup 10000 LState and put these into a buffered channel of size 10000. The function to create the LState:

     func newState() (*lua.LState, error) {
        L := lua.NewState()
        L.PreloadModule("base64", base64.Loader)
        L.PreloadModule("http", gluahttp.NewHttpModule(&http.Client{}).Loader)
        L.PreloadModule("url", gluaurl.Loader)
        L.PreloadModule("json", gjson.Loader)
        L.SetGlobal("print", L.NewFunction(printToLog))
        L.SetGlobal("sleep", L.NewFunction(sleep))
        return L, nil
       }
    

    When the filter is called, we get a LState from the pool and pass it and the compiled lua code from the filter to execute:

    func createScript(L *lua.LState, proto *lua.FunctionProto) (*lua.LState, error) {
        L.Push(L.NewFunctionFromProto(proto))
        err := L.PCall(0, lua.MultRet, nil)
        if err != nil {
            L.Close()
            return nil, err
        }
        return L, nil
    }
    

    As far as I see it's not a documented behavior that we can reuse LState and overwrite the Request and Response functions with L.Push(), but it seems to work (tested locally with vegeta and some 10000s of requests).

    Is it a safe assumption that overwriting a Function is fine? Maybe there is a better or safer way to do this. Do we leak resources that we need to cleanup?

    Thanks, sandor

    opened by szuecs 0
  • Detect invalid argument indexing.

    Detect invalid argument indexing.

    Fixes #331

    Changes proposed in this pull request:

    Fix argument indexing. Compared behavior with Lua 5.1 and LuaJIT.

    opened by tongson 1
  • Index out of range error in state.go's where function

    Index out of range error in state.go's where function

    1. What version of GopherLua are you using? : This is the version written in go.mod: github.com/yuin/gopher-lua v0.0.0-20210529063254-f4c35e4016d9 // indirect
    2. What version of Go are you using? : go1.16.3 linux/amd64
    3. What operating system and processor architecture are you using? : Ubuntu 20.04.1 LTS
    4. What did you do? : While I was fuzzing gopher-lua, I am encountered with panic. This is the piece of code that I use.
    package main
    
    import (
        "github.com/yuin/gopher-lua"
        "fmt"
    )
    
    func main(){
    
    data := "return  select(0)(select(1/9))"
    
    L := lua.NewState()
    defer L.Close()
    if err := L.DoString(data); err != nil {
     fmt.Println(data)
    }
    
    }
    
    1. What did you expect to see? : I think, it shouldn't be panicked.
    2. What did you see instead? : I think, it should handle index out of range error. It might control index, which can not be negative number, before giving it as an argument.

    This is my output:

    panic: runtime error: index out of range [-1] [recovered]
     panic: runtime error: index out of range [-1]
    
    goroutine 1 [running]:
    github.com/yuin/gopher-lua.(*LState).where(0xc0001b4000, 0x0, 0xc00047ef00, 0xc000118001, 0x8)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:735 +0x27e
    github.com/yuin/gopher-lua.(*LState).Where(...)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/auxlib.go:252
    github.com/yuin/gopher-lua.(*LState).stackTrace(0xc0001b4000, 0x0, 0xc00018bd90, 0xc000180090)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:747 +0x1d3
    github.com/yuin/gopher-lua.(*LState).PCall.func1(0x581df0, 0xc0001b4000, 0xc000061ef0, 0x0, 0x0, 0x0)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:2018 +0x277
    panic(0x5683a0, 0xc0000160d8)
     /usr/local/go/src/runtime/panic.go:965 +0x1b9
    github.com/yuin/gopher-lua.(*LState).where(0xc0001b4000, 0x0, 0x1, 0x40376b, 0xc0004becc0)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:735 +0x27e
    github.com/yuin/gopher-lua.(*LState).raiseError(0xc0001b4000, 0x1, 0x579c16, 0x11, 0x0, 0x0, 0x0)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:689 +0x1b7
    github.com/yuin/gopher-lua.(*LState).RaiseError(...)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:1671
    github.com/yuin/gopher-lua.(*LState).registryOverflow(0xc0001b4000)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:1666 +0x5a
    github.com/yuin/gopher-lua.(*registry).resize(0xc0001b20a0, 0x1401)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:394 +0x102
    github.com/yuin/gopher-lua.(*registry).SetTop(0xc0001b20a0, 0x1401)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:411 +0xf5
    github.com/yuin/gopher-lua.init.3.func27(0xc0001b4000, 0xc080000000, 0xc0001b6000, 0x0)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/vm.go:997 +0x7d7
    github.com/yuin/gopher-lua.mainLoop(0xc0001b4000, 0xc0001b6000)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/vm.go:31 +0xea
    github.com/yuin/gopher-lua.(*LState).callR(0xc0001b4000, 0x0, 0xffffffffffffffff, 0x0)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:1211 +0x256
    github.com/yuin/gopher-lua.(*LState).Call(...)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:1967
    github.com/yuin/gopher-lua.(*LState).PCall(0xc0001b4000, 0x0, 0xffffffffffffffff, 0x0, 0x5b00d8, 0xc0001e6380)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/state.go:2030 +0x114
    github.com/yuin/gopher-lua.(*LState).DoString(0xc0001b4000, 0x57d180, 0x1e, 0xc0001b4000, 0xc000050778)
     /home/fuzz/go/pkg/mod/github.com/yuin/[email protected]/auxlib.go:405 +0xbd
    main.main()
     /home/fuzz/fuzz-go/deneme.go:14 +0x7e
    exit status 2
    
    opened by CT-Zer0 1
  • Start at 0x3a so we skip numbers.

    Start at 0x3a so we skip numbers.

    Fixes #279

    Changes proposed in this pull request:

    Avoid numbers from matching pattern character class.

    opened by tongson 1
  • Iterate from start of array to match Lua 5.1 `#` operator behavior.

    Iterate from start of array to match Lua 5.1 `#` operator behavior.

    Fixes #304

    Changes proposed in this pull request:

    Follow behavior of Lua 5.1 # operator.

    opened by tongson 2
  • Allow tables with __call metamethod.

    Allow tables with __call metamethod.

    Fixes #326

    Changes proposed in this pull request:

    Allow calling functions within __call metamethod.

    opened by tongson 1
  • pcall() not allowing tables with __call metamethod

    pcall() not allowing tables with __call metamethod

    1. What version of GopherLua are you using? : HEAD
    2. What version of Go are you using? : 1.16
    3. What operating system and processor architecture are you using? : linux amd64
    4. What did you do? : pcall() on table with __call metamethod.
    5. What did you expect to see? : Call the function assigned to __call.
    6. What did you see instead? : attempt to call a table value error.
    opened by tongson 0
Owner
Yusuke Inuzuka
Software developer from Japan. Explicit is better than implicit.
Yusuke Inuzuka
A Lua VM in Go

A Lua VM in pure Go go-lua is a port of the Lua 5.2 VM to pure Go. It is compatible with binary files dumped by luac, from the Lua reference implement

Shopify 2.1k Jul 21, 2021
Go bindings for Lua C API - in progress

Go Bindings for the lua C API Simplest way to install: # go get github.com/aarzilli/golua/lua You can then try to run the examples: $ cd golua/_examp

Alessandro Arzilli 549 Jul 18, 2021
High level go to Lua binder. Write less, do more.

Binder High level go to Lua binder. Write less, do more. Package binder allows to easily bind to Lua. Based on gopher-lua. Write less, do more! Killer

Alexey Popov 51 Jun 2, 2021
A fast script language for Go

The Tengo Language Tengo is a small, dynamic, fast, secure script language for Go. Tengo is fast and secure because it's compiled/executed as bytecode

daniel 2.3k Jul 16, 2021
Fast, portable, non-Turing complete expression evaluation with gradual typing (Go)

Common Expression Language The Common Expression Language (CEL) is a non-Turing complete language designed for simplicity, speed, safety, and portabil

Google 836 Jul 19, 2021
Duktape JavaScript engine bindings for Go

Duktape bindings for Go(Golang) Duktape is a thin, embeddable javascript engine. Most of the api is implemented. The exceptions are listed here. Usage

Oleg Lebedev 767 Jul 8, 2021
Gentee - script programming language for automation. It uses VM and compiler written in Go (Golang).

Gentee script programming language Gentee is a free open source script programming language. The Gentee programming language is designed to create scr

Alexey Krivonogov 79 Jul 19, 2021
ECMAScript/JavaScript engine in pure Go

Goja is an implementation of ECMAScript 5.1 in pure Go with emphasis on standard compliance and performance.

Dmitry Panov 2.3k Jul 25, 2021
Expression evaluation engine for Go: fast, non-Turing complete, dynamic typing, static typing

Expr Expr package provides an engine that can compile and evaluate expressions. An expression is a one-liner that returns a value (mostly, but not lim

Anton Medvedev 1.9k Jul 18, 2021
Simple LISP in Go

gisp Simple (non standard) compiler of Lisp/Scheme to Go. Includes Lexer based on Rob Pike's Lexical Scanning in Go Simple recursive parser, supportin

Joseph Adams 462 Jul 20, 2021
Ale is a Lisp Environment for Go applications

Ale is a Lisp Environment Ale is a Lisp Environment for Go applications How To Install Make sure your GOPATH is set, then run go get to retrieve the p

Thomas Bradford 137 Jul 19, 2021
Expression evaluation in golang

Gval Gval (Go eVALuate) provides support for evaluating arbitrary expressions, in particular Go-like expressions. Evaluate Gval can evaluate expressio

null 354 Jul 16, 2021
PHP bindings for the Go programming language (Golang)

PHP bindings for Go This package implements support for executing PHP scripts, exporting Go variables for use in PHP contexts, attaching Go method rec

Alex Palaistras 804 Jul 23, 2021
An embeddable implementation of the Ngaro Virtual Machine for Go programs

Ngaro Go Overview This is an embeddable Go implementation of the Ngaro Virtual Machine. This repository contains the embeddable virtual machine, a rud

Denis Bernard 20 Apr 6, 2020