A Lua VM in Go

Overview

Build Status GoDoc

A Lua VM in pure Go

go-lua is a port of the Lua 5.2 VM to pure Go. It is compatible with binary files dumped by luac, from the Lua reference implementation.

The motivation is to enable simple scripting of Go applications. For example, it is used to describe flows in Shopify's load generation tool, Genghis.

Usage

go-lua is intended to be used as a Go package. It does not include a command to run the interpreter. To start using the library, run:

go get github.com/Shopify/go-lua

To develop & test go-lua, you'll also need the lua-tests submodule checked out:

git submodule update --init

You can then develop with the usual Go commands, e.g.:

go build
go test -cover

A simple example that loads & runs a Lua script is:

package main

import "github.com/Shopify/go-lua"

func main() {
  l := lua.NewState()
  lua.OpenLibraries(l)
  if err := lua.DoFile(l, "hello.lua"); err != nil {
    panic(err)
  }
}

Status

go-lua has been used in production in Shopify's load generation tool, Genghis, since May 2014, and is also part of Shopify's resiliency tooling.

The core VM and compiler has been ported and tested. The compiler is able to correctly process all Lua source files from the Lua test suite. The VM has been tested to correctly execute over a third of the Lua test cases.

Most core Lua libraries are at least partially implemented. Prominent exceptions are regular expressions, coroutines and string.dump.

Weak reference tables are not and will not be supported. go-lua uses the Go heap for Lua objects, and Go does not support weak references.

Benchmarks

Benchmark results shown here are taken from a Mid 2012 MacBook Pro Retina with a 2.6 GHz Core i7 CPU running OS X 10.10.2, go 1.4.2 and Lua 5.2.2.

The Fibonacci function can be written a few different ways to evaluate different performance characteristics of a language interpreter. The simplest way is as a recursive function:

  function fib(n)
    if n == 0 then
      return 0
    elseif n == 1 then
      return 1
    end
    return fib(n-1) + fib(n-2)
  end

This exercises the call stack implementation. When computing fib(35), go-lua is about 6x slower than the C Lua interpreter. Gopher-lua is about 20% faster than go-lua. Much of the performance difference between go-lua and gopher-lua comes from the inclusion of debug hooks in go-lua. The remainder is due to the call stack implementation - go-lua heap-allocates Lua stack frames with a separately allocated variant struct, as outlined above. Although it caches recently used stack frames, it is outperformed by the simpler statically allocated call stacks in gopher-lua.

  $ time lua fibr.lua
  real  0m2.807s
  user  0m2.795s
  sys   0m0.006s
  
  $ time glua fibr.lua
  real  0m14.528s
  user  0m14.513s
  sys   0m0.031s
  
  $ time go-lua fibr.lua
  real  0m17.411s
  user  0m17.514s
  sys   0m1.287s

The recursive Fibonacci function can be transformed into a tail-recursive variant:

  function fibt(n0, n1, c)
    if c == 0 then
      return n0
    else if c == 1 then
      return n1
    end
    return fibt(n1, n0+n1, c-1)
  end
  
  function fib(n)
    fibt(0, 1, n)
  end

The Lua interpreter detects and optimizes tail calls. This exhibits similar relative performance between the 3 interpreters, though gopher-lua edges ahead a little due to its simpler stack model and reduced bookkeeping.

  $ time lua fibt.lua
  real  0m0.099s
  user  0m0.096s
  sys   0m0.002s

  $ time glua fibt.lua
  real  0m0.489s
  user  0m0.484s
  sys   0m0.005s

  $ time go-lua fibt.lua
  real  0m0.607s
  user  0m0.610s
  sys   0m0.068s

Finally, we can write an explicitly iterative implementation:

  function fib(n)
    if n == 0 then
      return 0
    else if n == 1 then
      return 1
    end
    local n0, n1 = 0, 1
    for i = n, 2, -1 do
      local tmp = n0 + n1
      n0 = n1
      n1 = tmp
    end
    return n1
  end

This exercises more of the bytecode interpreter’s inner loop. Here we see the performance impact of Go’s switch implementation. Both go-lua and gopher-lua are an order of magnitude slower than the C Lua interpreter.

  $ time lua fibi.lua
  real  0m0.023s
  user  0m0.020s
  sys   0m0.003s

  $ time glua fibi.lua
  real  0m0.242s
  user  0m0.235s
  sys   0m0.005s

  $ time go-lua fibi.lua
  real  0m0.242s
  user  0m0.240s
  sys   0m0.028s

License

go-lua is licensed under the MIT license.

Issues
  • go get fails on Windows 7 (go version go1.4 windows/amd64)

    go get fails on Windows 7 (go version go1.4 windows/amd64)

    I couldn't install the library by go getting it. I got the following errors:

    C:\Users\hugo>go get github.com/Shopify/go-lua

    github.com/Shopify/go-lua

    c:\users\hugo\go\src\github.com\Shopify\go-lua\base.go:12: undefined: CheckType c:\users\hugo\go\src\github.com\Shopify\go-lua\base.go:23: undefined: MetaField c:\users\hugo\go\src\github.com\Shopify\go-lua\base.go:24: undefined: CheckType c:\users\hugo\go\src\github.com\Shopify\go-lua\base.go:41: undefined: CheckInteger c:\users\hugo\go\src\github.com\Shopify\go-lua\base.go:42: undefined: CheckType c:\users\hugo\go\src\github.com\Shopify\go-lua\base.go:95: undefined: CheckStackWithMessage c:\users\hugo\go\src\github.com\Shopify\go-lua\base.go:101: undefined: Errorf c:\users\hugo\go\src\github.com\Shopify\go-lua\base.go:326: undefined: SetFunctions c:\users\hugo\go\src\github.com\Shopify\go-lua\bit32.go:32: undefined: CheckUnsigned c:\users\hugo\go\src\github.com\Shopify\go-lua\bit32.go:43: undefined: CheckUnsigned c:\users\hugo\go\src\github.com\Shopify\go-lua\bit32.go:43: too many errors

    opened by hugows 13
  • Rename func Type(*State,int) to TypeOf. typedef Type enumerations.

    Rename func Type(*State,int) to TypeOf. typedef Type enumerations.

    Rename the Type func to TypeOf to avoid colliding with the new Type typedef for the type enumeration. Fix bug in missuse of TypeName.

    r: @fbogsany cc: @Shopify/performance

    opened by aybabtme 8
  • Translate section 4.8 of Lua C reference manual to GoDocs.

    Translate section 4.8 of Lua C reference manual to GoDocs.

    Also:

    • Add some typedefs (Mode, Operator, CmpOperator)
    • replace Status by error (to make it obvious that some functions return errors)
    • restrict range of some integers (max of 255 upvalues)
    • provide error message to RawEqual (useful for user).

    r: @fbogsany cc: @Shopify/performance

    opened by aybabtme 5
  • Fix PushFString %p format token.

    Fix PushFString %p format token.

    Fixes a nasty bug where using a %p in a format string does not advance the argument index, which causes the next format token to look at the same argument again and all subsequent tokens will be off-by-one.

    opened by AlexMax 5
  • Add information to panic messages.

    Add information to panic messages.

    Add some context to the panic messages.

    r: @fbogsany cc: @Shopify/performance

    opened by aybabtme 4
  • Handle runtime panic where comparison operand is nil

    Handle runtime panic where comparison operand is nil

    All table/UD comparisons were assuming the other operand wasn't nil. This doesn't mesh with Lua, since it results in a panic where the result should be false. Related to issue #45 and pull request #57, but addresses the issue for userdata as well.

    The issue's pretty simple: if one side of the comparison is a table or userdata and the other side is nil, the other side is assumed to be a userdata or table and a type assertion is made. If the assertion fails, the program has a runtime panic and dies.

    I don't know if this issue exists for metamethods on other types as well.

    opened by nilium 4
  • Use the string on the stack for the runtime error thrown.

    Use the string on the stack for the runtime error thrown.

    Makes for nicer error messages.

    r: @fbogsany cc: @Shopify/performance

    opened by aybabtme 3
  • Panic when returning from a lua function directly, but not if I store the results in a variable first

    Panic when returning from a lua function directly, but not if I store the results in a variable first

    package main
    
    import (
        "bytes"
        "log"
    
        "github.com/Shopify/go-lua"
    )
    
    func main() {
        l := lua.NewState()
        lua.OpenLibraries(l)
    
        loadInline := func(code string) {
            if err := l.Load(bytes.NewBufferString(code), "", "bt"); err != nil {
                log.Fatal(err)
            }
        }
    
        // Load in a global function
        loadInline(`
            function doingStuff(a, b)
                return a + b
            end
        `)
        l.Call(0, 0)
    
        // Call the function, first storing the result in a local variable
        loadInline(`
            local res = doingStuff(1, 2)
            return res
        `)
        l.Call(0, 1)
        i, _ := l.ToInteger(-1)
        log.Printf("i: %d", i)
        l.Remove(-1)
    
        // Call the function, directly returning the result from it
        loadInline(`
            return doingStuff(1, 2)
        `)
        l.Call(0, 1)
        i, _ = l.ToInteger(-1)
        log.Printf("i: %d", i)
        l.Remove(-1)
    }
    

    So first I'm loading a more or less useless function into the global namespace. Then I go to call it. If I Store the return value as a local variable It works fine and prints out 3 like it should. However, if I just directly return the result without first storing it I get a panic:

    panic: runtime error: index out of range
    
    goroutine 1 [running]:
    github.com/Shopify/go-lua.func·125(0xc2080485f0, 0xc200400006, 0x5d6ca8, 0xc200400006)
            /home/mediocregopher/src/go/src/github.com/Shopify/go-lua/vm.go:358 +0x3fc
    github.com/Shopify/go-lua.(*State).executeFunctionTable(0xc208038000)
            /home/mediocregopher/src/go/src/github.com/Shopify/go-lua/vm.go:942 +0x22e
    github.com/Shopify/go-lua.(*State).execute(0xc208038000)
            /home/mediocregopher/src/go/src/github.com/Shopify/go-lua/vm.go:928 +0x28
    github.com/Shopify/go-lua.(*State).call(0xc208038000, 0x1, 0x1, 0xc208010000)
            /home/mediocregopher/src/go/src/github.com/Shopify/go-lua/stack.go:381 +0xa6
    github.com/Shopify/go-lua.(*State).CallWithContinuation(0xc208038000, 0x0, 0x1, 0x0, 0x0)
            /home/mediocregopher/src/go/src/github.com/Shopify/go-lua/lua.go:349 +0x126
    github.com/Shopify/go-lua.(*State).Call(0xc208038000, 0x0, 0x1)
            /home/mediocregopher/src/go/src/github.com/Shopify/go-lua/lua.go:1345 +0x4e
    main.main()
            /tmp/wat.go:42 +0x240
    

    Could this be a bug with the shopify implementation, or is what I'm doing fundamentally incorrect in some way? Thanks!

    opened by mediocregopher 3
  • ToBoolean returns true on too-large stack index

    ToBoolean returns true on too-large stack index

    package main
    
    import (
        "fmt"
    
        "github.com/Shopify/go-lua"
    )
    
    func main() {
        l := lua.NewState()
        l.PushBoolean(true)
        l.PushBoolean(false)
        fmt.Printf("Top: %d\n", l.Top())
        for i := 0; i < 4; i++ {
            fmt.Printf("ToBoolean(%d) %t\n", i, l.ToBoolean(i))
        }
    }
    

    Results in:

    Top: 2
    ToBoolean(0) false
    ToBoolean(1) true
    ToBoolean(2) false
    ToBoolean(3) true
    

    That last result is puzzling.

    Here's the same sort of thing written in C against PUC-Rio Lua 5.2:

    #include <stdio.h>
    
    #include "lua.h"
    #include "lauxlib.h"
    
    int main() {
        lua_State *L;
        L = luaL_newstate();
        lua_pushboolean(L, 1);
        lua_pushboolean(L, 0);
        printf("Top: %d\n", lua_gettop(L));
        for (int i = 0; i < 4; i++) {
            printf("ToBoolean(%d) %d\n", i, lua_toboolean(L, i));
        }
    }
    

    Results in:

    Top: 2
    ToBoolean(0) 0
    ToBoolean(1) 1
    ToBoolean(2) 0
    ToBoolean(3) 0
    

    Other ToWhatever API's in go-lua appear to have a value, ok multi-valued return. Is the correct fix here to add a second API value, or to simply return false? The former would be consistent with other go-lua API's, the latter would be consistent with the C API and on the surface would not require changing existing code - although this might actually be a downside if there is any code that relies on this peculiarity of out-of-bounds truthiness.

    opened by AlexMax 3
  • Open Source go-lua

    Open Source go-lua

    Steps to release Shopify/go-lua as an open source project:

    • [x] #25 move test fixtures (originally Lua test code) to a separate repo
    • [x] sanity check documentation
    • [x] cleanup README.md, include status & limitations
    • [x] add license?
    • [x] ask "someone" to open the repo
    opened by fbogsany 3
  • Author ? I want to know go-lua how to support lua-socket??

    Author ? I want to know go-lua how to support lua-socket??

    I rewrite a project from c++ to golang. Issue : it use lua-socket in cpp and i use go-lua in go...

    I want to know go-lua is or not support lua-socket??

    C++ code:

    static int base_open(lua_State *L) {
        if (socket_open()) {
            /* export functions (and leave namespace table on top of stack) */
            lua_newtable(L);
            luaL_setfuncs(L, func, 0);
    #ifdef LUASOCKET_DEBUG
            lua_pushstring(L, "_DEBUG");
            lua_pushboolean(L, 1);
            lua_rawset(L, -3);
    #endif
            /* make version string available to scripts */
            lua_pushstring(L, "_VERSION");
            lua_pushstring(L, LUASOCKET_VERSION);
            lua_rawset(L, -3);
            return 1;
        } else {
            lua_pushstring(L, "unable to initialize library");
            lua_error(L);
            return 0;
        }
    }
    
    opened by zld126126 0
  • Description Lua table was misjudged as function

    Description Lua table was misjudged as function

    lua code: local some = {} sometime, vm miscalculation 'some' is function.

    opened by cjackie200 0
  • implement the read function

    implement the read function

    Adding all the features for the io.read and file.read methods.

    opened by jessicaxiejw 0
  •  bad argument #2 to '__newindex' (not a number in proper range)

    bad argument #2 to '__newindex' (not a number in proper range)

    is go-lua can only run in 64bit machine? when i run my lua script on ubuntu 64 bit, it works ok( lua test.lua), and i run my golang code using go run main.go,int works ok too, but when i just cross compile the same golang code using GOOS=linux GOARCH=arm GOARM=7 go build main.go error occured, i hope to use go-lua on my 32bit arm board, [email protected]_lcd:~# uname -a Linux b503_lcd 4.1.15-2.1.0+g37e48c8 #90 SMP PREEMPT Fri Oct 26 11:41:29 CST 2018 armv7l GNU/Linux [email protected]_lcd:~# ./main test go lua... this is CONNECT,127.0.0.15050 panic: runtime error: test.lua:34: bad argument #2 to '__newindex' (not a number in proper range) goroutine 1 [running]: main.main() /mnt/hgfs/b503/gopath/src/golua/main.go:11 +0xa0

    the 34 line is string.format("%012d",34567890121111) why? is go-lua can only run in 64bit machine?

    Here's my test: package main import "github.com/Shopify/go-lua" import "fmt" var arg1 = 1 var arg2 = 2 func add(x, y int) int { fmt.Printf("this is go func add\n") return x + y }

    var luatestGofunc0 = func(l *lua.State) int { fmt.Printf("this is a go func-0 called by lua script!\n") top := l.Top() fmt.Printf("top:%d\n", top) a0 := l.ToUserData(0) a1 := l.ToUserData(1) a2 := l.ToUserData(2) fmt.Printf("ao,a1,a2=%#v,%#v,%#v\n", a0, a1, a2) return 2 }

    func main() { fmt.Printf("hello golang,test lua script...\n") l := lua.NewState() l.Register("LtestGofunc0", luatestGofunc0) l.Register("LtestGofunc1", func(l *lua.State) int { fmt.Printf("this is a go func 1...\n") return 0 }) lua.OpenLibraries(l) if err := lua.DoFile(l, "test1.lua"); err != nil { panic(err) } }

    bellow is test1.lua: this is a lua script demo:

    ip= "127.0.0.1" port = 5050

    function CONNECT( ip,port ) print("this is CONNECT,"..ip..port) end

    function TxData( tx ) print("this is TxData,"..tx) end

    function DISCONNECT( ... ) print("this is DISCONNECT") end

    ret = CONNECT(ip,port)

    MTI = 'B001' -- STI = '52' -- DBL = '0000001C' -- DATA = '' --

    shanghu = string.format("%012d",34567890121111) ----------------------the 34 line poscode = string.format("%016d",34567890112233) samcode = '313233343536' optcode = '3132333435363738'

    DATA = shanghu..poscode..samcode..optcode DBL = string.format("%08x",string.len(DATA)/2) TX = MTI .. STI .. DBL .. DATA

    ret,rcv = TxData(TX)
    print(rcv)

    MTI = 'B004' blackver = '00000000' DATA = blackver .. poscode DBL = string.format("%02x",string.len(DATA)/2) TX = MTI .. STI .. DBL .. DATA

    ret,rcv = TxData(TX)
    print(rcv)

    DISCONNECT()

    opened by yangyongzhen 0
  • Status of this project?

    Status of this project?

    👋

    The readme says "not currently maintained" ... but I'm curious how many active contributors there are? I see a number of MR's sitting there. Is anyone reviewing these? Will they ever get merged?

    Abandonware? Active forks?

    opened by SandyWalsh 0
  • Use SVG badges in README.

    Use SVG badges in README.

    cla-needed 
    opened by AlekSi 1
  • Avoid undefined behaviour when doing range checks for integers

    Avoid undefined behaviour when doing range checks for integers

    Casting from floating point to integer is undefined for values that exceed the valid range of the target integer type and this results in failure of test assert(not pcall(string.format, "%d", 2^63)) in lua-tests for aarch64.

    Replace with a safer test for value ranges that does not rely on the cast from float to integer.

    opened by siddhesh 0
  • Port to arm64?

    Port to arm64?

    Anyone working on an arm64 port of this?

    Lua is often tricky on arm64 (48-bit lightuserdata, for example) and wonder if anyone has been down this particular path before I jump in.

    opened by vielmetti 2
  • Fix Status.Next() get lua array panic

    Fix Status.Next() get lua array panic

    #102

    opened by cchu1988 0
  • can not use State.Next() func read lua array

    can not use State.Next() func read lua array

           // balabala....
    
    	if l.IsTable(-1) {
    		var m = map[string]string{}
    
    		l.PushNil()
    		for l.Next(-2) {
    			key, _ := l.ToString(-2)
    			value, _ := l.ToString(-1)
    			l.Pop(1)
    
    			m[key] = value
    		}
    
    		l.Pop(2)
    
    		return m
    	}
    
            // balabala....
    

    table is ok, but array will panic.

    add some debug log tables.go:215

    func arrayIndex(k value) int {
            if n, ok := k.(float64); ok {
               fmt.Println("array_index", k, n)
            } else {
               t := reflect.TypeOf(k)
    	   fmt.Println("array_index", t, k)
            }
    
    	if n, ok := k.(float64); ok {
    		if i := int(n); float64(i) == n {
    			return i
    		}
    	}
    	return -1
    }
    

    it is a string type.

    array_index <nil> <nil>
    array_index string 1
    array_index string 1
    
    opened by Fruneng 0
Owner
Shopify
Shopify
Go bindings for Lua C API - in progress

Go Bindings for the lua C API Simplest way to install: # go get github.com/aarzilli/golua/lua You can then try to run the examples: $ cd golua/_examp

Alessandro Arzilli 569 Nov 24, 2021
GopherLua: VM and compiler for Lua in Go

GopherLua: VM and compiler for Lua in Go. GopherLua is a Lua5.1 VM and compiler written in Go. GopherLua has a same goal with Lua: Be a scripting lang

Yusuke Inuzuka 4.4k Nov 29, 2021
LuaHelper is a High-performance lua plugin, Language Server Protocol for lua.

LuaHelper is a High-performance lua plugin, Language Server Protocol for lua.

Tencent 87 Dec 4, 2021
High level go to Lua binder. Write less, do more.

Binder High level go to Lua binder. Write less, do more. Package binder allows to easily bind to Lua. Based on gopher-lua. Write less, do more! Killer

Alexey Popov 55 Nov 14, 2021
A Lua VM in Go

A Lua VM in pure Go go-lua is a port of the Lua 5.2 VM to pure Go. It is compatible with binary files dumped by luac, from the Lua reference implement

Shopify 2.2k Nov 28, 2021
Go bindings for Lua C API - in progress

Go Bindings for the lua C API Simplest way to install: # go get github.com/aarzilli/golua/lua You can then try to run the examples: $ cd golua/_examp

Alessandro Arzilli 569 Nov 24, 2021
GopherLua: VM and compiler for Lua in Go

GopherLua: VM and compiler for Lua in Go. GopherLua is a Lua5.1 VM and compiler written in Go. GopherLua has a same goal with Lua: Be a scripting lang

Yusuke Inuzuka 4.4k Nov 29, 2021
:tophat: Small self-contained pure-Go web server with Lua, Markdown, HTTP/2, QUIC, Redis and PostgreSQL support

Web server with built-in support for QUIC, HTTP/2, Lua, Markdown, Pongo2, HyperApp, Amber, Sass(SCSS), GCSS, JSX, BoltDB (built-in, stores the databas

Alexander F. Rødseth 1.9k Dec 6, 2021
Go bindings for Lua C API - in progress

Go Bindings for the lua C API Simplest way to install: # go get github.com/aarzilli/golua/lua You can then try to run the examples: $ cd golua/_examp

Alessandro Arzilli 569 Nov 24, 2021
A Lua VM in Go

A Lua VM in pure Go go-lua is a port of the Lua 5.2 VM to pure Go. It is compatible with binary files dumped by luac, from the Lua reference implement

Shopify 2.2k Dec 3, 2021
GopherLua: VM and compiler for Lua in Go

GopherLua: VM and compiler for Lua in Go. GopherLua is a Lua5.1 VM and compiler written in Go. GopherLua has a same goal with Lua: Be a scripting lang

Yusuke Inuzuka 4.4k Dec 2, 2021
Heart 💜A high performance Lua web server with a simple, powerful API

Heart ?? A high performance Lua web server with a simple, powerful API. See the full documentation here. Overview Heart combines Go's fasthttp with Lu

Hyperspace Logistics 66 Nov 2, 2021
GopherLua: VM and compiler for Lua in Go

GopherLua: VM and compiler for Lua in Go. GopherLua is a Lua5.1 VM and compiler written in Go. GopherLua has a same goal with Lua: Be a scripting lang

Yusuke Inuzuka 4.4k Dec 4, 2021
A Lua VM in Go

A Lua VM in pure Go go-lua is a port of the Lua 5.2 VM to pure Go. It is compatible with binary files dumped by luac, from the Lua reference implement

Shopify 2.2k Nov 23, 2021
A Lua 5.3 VM and compiler written in Go.

DCLua - Go Lua Compiler and VM: This is a Lua 5.3 VM and compiler written in Go. This is intended to allow easy embedding into Go programs, with minim

Milo Christiansen 882 Nov 26, 2021
The Humboldt Web Framework and Toolkit. Using this as an interpeter and server, build webistes in an MVC pattern using Lua.

Humboldt Web Framework Humboldt is a framework written in Go using Lua files to build web applications. What is this framework for? People who want to

Selene Software 0 Dec 5, 2021