Go binding for TensorFlow Lite

Overview

go-tflite

Go binding for TensorFlow Lite

Usage

model := tflite.NewModelFromFile("sin_model.tflite")
if model == nil {
	log.Fatal("cannot load model")
}
defer model.Delete()

options := tflite.NewInterpreterOptions()
defer options.Delete()

interpreter := tflite.NewInterpreter(model, options)
defer interpreter.Delete()

interpreter.AllocateTensors()

v := float64(1.2) * math.Pi / 180.0
input := interpreter.GetInputTensor(0)
input.Float32s()[0] = float32(v)
interpreter.Invoke()
got := float64(interpreter.GetOutputTensor(0).Float32s()[0])

See _example for more examples

Requirements

  • TensorFlow Lite - This release requires 2.2.0-rc3

Tensorflow Installation

You must install Tensorflow Lite C API. Assuming the source is under /source/directory/tensorflow

$ cd /source/directory/tensorflow
$ bazel build --config opt --config monolithic tensorflow:libtensorflow_c.so

Or to just compile the tensorflow lite libraries:

$ cd /some/path/tensorflow
$ bazel build --config opt --config monolithic //tensorflow/lite:libtensorflowlite.so
$ bazel build --config opt --config monolithic //tensorflow/lite/c:libtensorflowlite_c.so

In order for go to find the headers you must set the CGO_CFLAGS environment variable for the source and libraries of tensorflow. If your libraries are not installed in a standard location, you must also give the go linker the path to the shared librares with the CGO_LDFLAGS environment variable.

$ export CGO_CFLAGS=-I/source/directory/tensorflow
$ export CGO_LDFLAGS=-L/path/to/tensorflow/libaries

If you don't love bazel, you can try Makefile.tflite. Put this file as Makefile in tensorflow/lite/c, and run make. Sorry, this has not been test for Linux or Mac

Then run go build on some of the examples.

Edge TPU

To be able to compile and use the EdgeTPU delegate, you need to install the libraries from here: https://github.com/google-coral/edgetpu

There is also a deb package here: https://coral.withgoogle.com/docs/accelerator/get-started/#1-install-the-edge-tpu-runtime

The libraries from should be installed in a system wide library path like /usr/local/lib The include files should be installed somewhere that is accesable from your CGO include path

For x86:

cd /tmp && git clone https://github.com/google-coral/edgetpu.git && \
cp edgetpu/libedgetpu/direct/k8/libedgetpu.so.1.0 /usr/local/lib/libedgetpu.so.1.0 && \
ln -rs /usr/local/lib/libedgetpu.so.1.0 /usr/local/lib/libedgetpu.so.1 && \
ln -rs /usr/local/lib/libedgetpu.so.1.0 /usr/local/lib/libedgetpu.so && \
mkdir -p /usr/local/include/libedgetpu && \
cp edgetpu/libedgetpu/edgetpu.h /usr/local/include/edgetpu.h && \
cp edgetpu/libedgetpu/edgetpu_c.h /usr/local/include/edgetpu_c.h && \
rm -Rf edgetpu

License

MIT

Author

Yasuhrio Matsumoto (a.k.a. mattn)

Comments
  • Raspberry Pi 4 + edgetpu

    Raspberry Pi 4 + edgetpu "Internal: Unsupported data type in custom op handler: 0"

    Hello

    I tried to run example with edgetpu but it gives me

    EdgeTPU Version: BuildLabel(COMPILER=6.3.0 20170516,DATE=redacted,TIME=redacted,CL_NUMBER=291256449), RuntimeVersion(13)
    Internal: Unsupported data type in custom op handler: 0
    Node number -1095957936 (8ɶ<­¾üÍ϶D) @2.
    
    2020/06/15 23:44:53 allocate failed
    

    What would be the cause?

    opened by KyungWonPark 23
  • status = interpreter.Invoke() takes one second or more

    status = interpreter.Invoke() takes one second or more

    Hi, I have created a Docker builder image for arm64v8 so I can compile binary from my code using your framework.

    All is ok on the local ubuntu amd64 machine, but when I start the binary (compiled for arm64) on raspberry arm64v8, Invoke is slow, about a second or more.

    status = interpreter.Invoke() //takes more than 1 second

    Do you know about this issue and where the problem might be?

    Raspberry is using arm64v8 architecture For building the builder image I was using an ec2 arm machine (m6g.8xlarge)

    opened by zmajew 7
  • Cannot open shared object file: No such file or directory

    Cannot open shared object file: No such file or directory

    Hey, I wanted to try your promising library so I went through all the steps to produce my libtensorflowlite_c.so :smiley:.

    I then copy/pasted your label_image example and ran it using go run main.go producing this error:

    /tmp/go-build075675325/b001/exe/main: error while loading shared libraries: libtensorflowlite_c.so: cannot open shared object file: No such file or directory
    exit status 127
    

    I understand my libtensorflowlite_c.so may not be placed in the correct location. However, in https://github.com/mattn/go-tflite/blob/master/tflite.go I see...

    #cgo LDFLAGS: -L${SRCDIR}/../../tensorflow/tensorflow/tensorflow/lite/experimental/c -ltensorflowlite_c
    

    ...which seems to refer to the actual path my file sits in. So I don't really get why it cannot find it. Did I miss something ? Is there another step necessary to run go-tflite a part from placing the c library in tensorflow repo ?

    P.S: just to make it clear, here is an extract of my tree:

    $GOPATH/src/
        |_ github.com/
            |_ mattn/
            |       |_ go-tflite/
            |       |_ go-pointer/
            |_ Jonarod/
            |       |_ tests-labels-tflite/
            |           |_ main.go
            |_ tensorflow/
                    |_ tensorflow/
                        |_ tensorflow/
                            |_ lite/
                                |_ experimental / c / libtensorflowlite_c.so
    
    opened by Jonarod 7
  • Add cgo flags for android

    Add cgo flags for android

    The pull request remove -lrt for android build #37 solved some of my issues. I used to change the cgo LDFLAGS every time I switch testing amd64 / arm32 / arm64. These two flags cgo android and cgo linux,!android work like a charm!

    Commit changes

    • Adding two other library flags -lm -llog for android, which is needed when building android aar library using gomobile bind.
    • Using -ltensorflowlite_c instead of -ltensorflowlite-delegate_xnnpack -lXNNPACK in delegates/xnnpack/xnnpack.go. I'm not sure if I'm right about this. In my practice, compiling lib using
      $ bazel build --config opt --config monolithic  --define tflite_with_xnnpack=true //tensorflow/lite:libtensorflowlite.so
      $ bazel build --config opt --config monolithic  --define tflite_with_xnnpack=true //tensorflow/lite/c:libtensorflowlite_c.so
      

      will include xnnpack in libtensorflowlite_c.so. This seems easier in using of xnnpack.

    • In my basic tests, -ldl -lrt is not required for both linux and android, maybe they are not for my testing scenarios.

    Testing Android flags

    • Project folder structure
      $ tree testTflite/
      # testTflite/
      # ├── main.go
      # └── runTest
      #     └── runTest.go
      
    • runTest/runTest.go
      package runTest
      
      import (
      	"fmt"
      	"github.com/mattn/go-tflite"
      	"github.com/mattn/go-tflite/delegates/xnnpack"
      )
      
      func Run(model_path string) {
      	model := tflite.NewModelFromFile(model_path)
      	options := tflite.NewInterpreterOptions()
      	options.AddDelegate(xnnpack.New(xnnpack.DelegateOptions{NumThreads: 2}))
      	defer options.Delete()
      
      	interp := tflite.NewInterpreter(model, options)
      	status := interp.AllocateTensors()
      	input := interp.GetInputTensor(0)
      	in := input.Float32s()
      	fmt.Println(status, len(in))
      	status = interp.Invoke()
      	output := interp.GetOutputTensor(0).Float32s()
      	fmt.Println(output)
      }
      
    • main.go
      package main
      
      import "testTflite/runTest"
      
      func main() {
      	runTest.Run("model.tflite")
      }
      
    • Test
      cd testTflite
      go mod init testTflite
      # Use my custom go-tflite
      go mod edit -replace=github.com/mattn/go-tflite=$HOME/workspace/go-tflite
      go mod tidy
      
      # Build test for amd64
      export CGO_LDFLAGS=-L$HOME/workspace/tensorflow/bazel-bin/tensorflow/lite/c
      export CGO_CFLAGS=-I$HOME/workspace/tensorflow/
      go build main.go
      
      # Build test for arm64
      export CGO_LDFLAGS="-L$HOME/workspace/tensorflow.arm64/bazel-bin/tensorflow/lite/c"
      export CGO_CFLAGS="-I$HOME/workspace/tensorflow.arm64/"
      SDK_HOME="$HOME/Android/Sdk/ndk/21.0.6113669/toolchains/llvm/prebuilt/linux-x86_64"
      ANDROID_CC="$SDK_HOME/bin/aarch64-linux-android29-clang -Wl,-rpath-link,$SDK_HOME/sysroot/usr/lib/aarch64-linux-android/29"
      ANDROID_CXX="$SDK_HOME/bin/aarch64-linux-android29-clang++ -Wl,-rpath-link,$SDK_HOME/sysroot/usr/lib/aarch64-linux-android/29"
      ANDROID_ARCH="arm64"
      CGO_ENABLED=1 GOOS=android GOARCH=$ANDROID_ARCH CC=$ANDROID_CC CXX=$ANDROID_CXX go build main.go
      
      # gomobile bind test for arm64, will throw error without `-llog -lm`
      go get golang.org/x/mobile/bind
      gomobile bind -v -o test.aar -target=android/arm64 testTflite/runTest
      
    opened by leondgarse 5
  • ARM support

    ARM support

    Hey, I'm curious to know if this has go-tflite has been tried on devices like the rasberry pi?

    I'm currently rebuilding the libtensorflow_c.so file for ARM devices to use along on my rasberry pi, if this has been done before, would love to hear how that turned out to be.

    If not, once I make any headway, I can share what I did to get things right. Let me know :)

    opened by rushabhnagda11 5
  • Can you add a license?

    Can you add a license?

    This is great work. I actually got this to work with r14 and have been playing around with it.

    I would like to open source my code but would feel better if I either got your permission to release my code (which includes much of yours) or for you to add a license. (preferably BSD) Feel free to copy this and put your name in it's place: https://github.com/snowzach/gorestapi/blob/master/LICENSE.md

    I have added support for the Coral edgetpu hardware accelerator.

    Thanks!

    opened by snowzach 5
  • How does one install Tensorflow Lite C API

    How does one install Tensorflow Lite C API

    Hi, I must be pretty ignorant, but I cannot figure out how to install Tensorflow Lite C API. I just got my coral TPU, and the sales from Google told me that they worked with you @mattn to allow tensorflow lite to work in go. But everywhere on the tensorflow website, it shows installation with pip in python. I got bazel working, but I don't know where /source/directory/tensorflow is supposed to be. I git clone https://github.com/tensorflow/tensorflow and checked out v2.2.0-rc3 but no dice.

    Could you enlighten me for the initial setup?

    My goal is to develop in go on a ubuntu intel 64 machine and then, once everything is running, install everything on a raspberry pi and compile it over there. I'm saying this because the only instructions to make Tensorflow for C is for arm based systems on their website https://www.tensorflow.org/lite/guide/build_arm64

    opened by willmendil 4
  • Guide for Raspberry Pi? (cannot find -ltensorflow-lite)

    Guide for Raspberry Pi? (cannot find -ltensorflow-lite)

    Hello

    Could you write tutorial for setting this up in raspberry pi?

    I'm trying to build it but I'm stuck at

    /usr/bin/ld: cannot find -ltensorflow-lite
    

    Thank you

    opened by KyungWonPark 3
  • GPU delegate

    GPU delegate

    added a gpu delegate with high level API. Simply add

    delegate := tflite.NewGPUDelegate(nil)
    if delegate == nil {
    	log.Fatal("cannot create delegate")
    }
    defer delegate.Delete()
    

    and

    options := tflite.NewInterpreterOptions()
    ...
    options.AddDelegate(delegate)
    

    However, at the moment of writing, my GPU delegate throws

    INFO: Created TensorFlow Lite delegate for GPU. Next operations are not supported by GPU delegate: ADD: AVERAGE_POOL_2D: CONV_2D: DEPTHWISE_CONV_2D: RESHAPE: First 0 operations will run on the GPU, and the remaining 65 on the CPU.

    for a simple mobilenet_v2 model

    opened by nibeh 3
  • Can I build tflite cpp using

    Can I build tflite cpp using "offical" scripts?

    I found that Tensorflow provides the build scripts for the tflite cpp lib in: https://www.tensorflow.org/lite/guide/build_rpi

    So I'm wondering if I can use it instead? Or if not, can I simply modify something to make it compatible? Because I got the following error running the example:

    [email protected]:~/go/src/github.com/mattn/go-tflite/_example/ssd $ go run main.go 
    # github.com/mattn/go-tflite
    ../../tflite_experimental.go: In function 'writeToTensorAsVector':
    ../../tflite_experimental.go:64:13: error: 'TfLiteTensor {aka struct <anonymous>}' has no member named 'quantization'; did you mean 'allocation'?
       if (tensor->quantization.type == kTfLiteAffineQuantization) {
                 ^~
    ../../tflite_experimental.go:64:36: error: 'kTfLiteAffineQuantization' undeclared (first use in this function)
       if (tensor->quantization.type == kTfLiteAffineQuantization) {
                                        ^~~~~~~~~~~~~~~~~~~~~~~~~
    ../../tflite_experimental.go:64:36: note: each undeclared identifier is reported only once for each function it appears in
    ../../tflite_experimental.go:65:5: error: unknown type name 'TfLiteAffineQuantization'
         TfLiteAffineQuantization* q_params =
         ^~~~~~~~~~~~~~~~~~~~~~~~
    ../../tflite_experimental.go:66:10: error: 'TfLiteAffineQuantization' undeclared (first use in this function)
             (TfLiteAffineQuantization*)(tensor->quantization.params);
              ^~~~~~~~~~~~~~~~~~~~~~~~
    ../../tflite_experimental.go:66:35: error: expected expression before ')' token
             (TfLiteAffineQuantization*)(tensor->quantization.params);
                                       ^
    ../../tflite_experimental.go:66:43: error: 'TfLiteTensor {aka struct <anonymous>}' has no member named 'quantization'; did you mean 'allocation'?
             (TfLiteAffineQuantization*)(tensor->quantization.params);
                                               ^~
    ../../tflite_experimental.go:67:17: error: request for member 'scale' in something not a structure or union
         if (q_params->scale) {
                     ^~
    ../../tflite_experimental.go:68:20: error: request for member 'scale' in something not a structure or union
           free(q_params->scale);
                        ^~
    ../../tflite_experimental.go:69:15: error: request for member 'scale' in something not a structure or union
           q_params->scale = NULL;
                   ^~
    ../../tflite_experimental.go:71:17: error: request for member 'zero_point' in something not a structure or union
         if (q_params->zero_point) {
                     ^~
    ../../tflite_experimental.go:72:20: error: request for member 'zero_point' in something not a structure or union
           free(q_params->zero_point);
                        ^~
    ../../tflite_experimental.go:73:15: error: request for member 'zero_point' in something not a structure or union
           q_params->zero_point = NULL;
                   ^~
    ../../tflite_experimental.go:82:9: error: 'TfLiteTensor {aka struct <anonymous>}' has no member named 'quantization'; did you mean 'allocation'?
       tensor->quantization.type = kTfLiteNoQuantization;
             ^~
    ../../tflite_experimental.go:82:31: error: 'kTfLiteNoQuantization' undeclared (first use in this function)
       tensor->quantization.type = kTfLiteNoQuantization;
                                   ^~~~~~~~~~~~~~~~~~~~~
    ../../tflite_experimental.go:83:9: error: 'TfLiteTensor {aka struct <anonymous>}' has no member named 'quantization'; did you mean 'allocation'?
       tensor->quantization.params = NULL;
             ^~
    

    Sorry that I have no ideas about those C stuffs! Could you please give me some advices?

    opened by imdatngo 3
  • How to 2D input inference

    How to 2D input inference

    Hello Everyone,

    Thanks to your work, I could use the GO in my project. while I am using go-tflite, I have a question for input/output. My tflite input shape [4][3]float32 output shape [2][1]float32. How am I supposed to input and output properly?

    Thanks for the reply in advance.

    
    predict := []float32
    for i := 0; i < num; i++ {
    		inputs := [][]float32{}
    		for j := 0; j < steps; j++ {
    			inputs = append(inputs, data[j])
    		}
    		interpreter.GetInputTensor(0).CopyFromBuffer(inputs) // <<-- Error occurred here
    		interpreter.Invoke()
    		interpreter.GetOutputTensor(0).CopyToBuffer(predict)
    		fmt.Println(predict)
    	}
    

    but it gave an error like below

    panic: runtime error: cgo argument has Go pointer to Go pointer

    opened by meteoricfarm 2
  • Add support for newer tf light

    Add support for newer tf light

    The docs says only 2.2.0-rc3 is supported ... to build this requires an old version of bazel which requires old version of jdk. In short pretty hard to build on recent linux since dependencies arn't easily available.

    Will a more recent release of tensorflow work ?

    opened by plord12 1
  • Cannot open shared object file: No such file or directory... part 2

    Cannot open shared object file: No such file or directory... part 2

    I have followed the instructions from issue and still have: /tmp/go-build1465283250/b001/exe/main: error while loading shared libraries: libtensorflowlite_c.so: cannot open shared object file: No such file or directory exit status 127

    I put the libtensorflowlite_c.so into the /usr/local/lib and export LD_LIBRARY_PATH=/usr/local/lib and in /etc/environment but does not help. I am trying to run esrgan example.

    tflite.go has: /* #ifndef GO_TFLITE_H #include "tflite.go.h" #endif #cgo LDFLAGS: -ltensorflowlite_c #cgo android LDFLAGS: -ldl #cgo linux,!android LDFLAGS: -ldl -lrt */` I am using Go 1.17

    opened by zmajew 4
  • Any thoughts about how to possibly use tflite-micro?

    Any thoughts about how to possibly use tflite-micro?

    Hello, there

    I was looking at https://github.com/tensorflow/tflite-micro and wondering if it would be possible to use go-tflite with that and TinyGo.

    Any thoughts on this would be appreciated. Thank you.

    opened by deadprogram 3
  • use static library

    use static library

    tensorflow provides scripts to build static tf lite libraries: tflite make scripts

    Are there plans to support .a files additionally to the .so files build with bazel?

    opened by rschwarz-sotec 1
  • Include path changed for tensorflow files

    Include path changed for tensorflow files

    You may or may not have seen the git issue where I was trying to get the edgetpu to work for DOODS.

    With the tensorflow commit that supports the edgetpu they have changed the import path to tensorflow/lite/c/c_api.h from tensorflow/lite/experimental/c/c_api.h

    Not sure how you want to handle this. I was trying to base my DOODS code off of your repo.

    opened by snowzach 2
Owner
mattn
Long-time Golang user&contributor, Google Dev Expert for Go, and author of many Go tools, Vim plugin author. Windows hacker C#/Java/C/C++
mattn
TFKG - A Tensorflow and Keras Golang port

TFKG - A Tensorflow and Keras Golang port This is experimental and quite nasty under the hood* Support macOS: running docker container, no GPU acceler

Tim Marshall 40 Oct 18, 2022
face detction/recognization golang lib using tensorflow facenet

Golang lib for detect/recognize by tensorflow facenet Prerequest libtensorfow 1.x Follow the instruction Install TensorFlow for C facenet tenorflow sa

Prof Syd Xu 1 Sep 23, 2022
A Go idiomatic binding to the C++ core of PyTorch

GoTorch GoTorch reimplements PyTorch high-level APIs, including modules and functionals, in idiomatic Go. Thus enables deep learning programming in Go

Yi Wang 177 Nov 14, 2022
lite version of gobuster. Only subdomain brute. 内网轻量化子域名爆破工具

gobusterdns lite version of gobuster. Only subdomain brute. 内网轻量化子域名爆破工具 适合指定dns跑内网子域名 与原版的修改 精简功能,仅支持子域名扫描 可导入domain list文件扫描

TimWhite 39 Nov 9, 2022
Go implement D*Lite with SDL2

dstarlite-gosdl Go implement D*Lite with SDL2 使用Go SDL2库实现的D*Lite算法 注意,尽可能使用 go mod ,自动下载依赖库(确保已经安装了SDL2库) 推荐使用Linux或者macOS运行 git clone https://github

joxrays 2 Apr 2, 2022
A utility library to make use of the X Go Binding easier. (Implements EWMH and ICCCM specs, key binding support, etc.)

xgbutil is a utility library designed to work with the X Go Binding. This project's main goal is to make various X related tasks easier. For example,

Andrew Gallant 185 Oct 13, 2022
A utility library to make use of the X Go Binding easier. (Implements EWMH and ICCCM specs, key binding support, etc.)

xgbutil is a utility library designed to work with the X Go Binding. This project's main goal is to make various X related tasks easier. For example,

Andrew Gallant 185 Oct 13, 2022
mass-binding-target is a command line tool for generating binding target list by search plot files from disk.

mass-binding-target mass-binding-target is a command line tool for generating binding target list by search plot files from disk. Build Go 1.13 or new

null 0 Nov 5, 2021
Tensorflow + Go, the gopher way

tfgo: TensorFlow in Go tfgo: TensorFlow in Go Dependencies Installation Getting started Computer Vision using data flow graph Train in Python, Serve i

Paolo Galeone 2.1k Nov 23, 2022
Personal Photo Management powered by Go and Google TensorFlow

PhotoPrism® is a privately hosted app for browsing, organizing, and sharing your photo collection. It makes use of the latest technologies to tag and find pictures automatically without getting in your way.

PhotoPrism 23.4k Nov 28, 2022
TFKG - A Tensorflow and Keras Golang port

TFKG - A Tensorflow and Keras Golang port This is experimental and quite nasty under the hood* Support macOS: running docker container, no GPU acceler

Tim Marshall 40 Oct 18, 2022
face detction/recognization golang lib using tensorflow facenet

Golang lib for detect/recognize by tensorflow facenet Prerequest libtensorfow 1.x Follow the instruction Install TensorFlow for C facenet tenorflow sa

Prof Syd Xu 1 Sep 23, 2022
A Go language binding for encodeing and decoding data in the bencode format that is used by the BitTorrent peer-to-peer file sharing protocol.

bencode-go A Go language binding for encoding and decoding data in the bencode format that is used by the BitTorrent peer-to-peer file sharing protoco

Jack Palevich 204 Nov 15, 2022
SDL2 binding for Go

SDL2 binding for Go go-sdl2 is SDL2 wrapped for Go users. It enables interoperability between Go and the SDL2 library which is written in C. That mean

Ve & Co. 1.9k Nov 21, 2022
Reflectionless data binding for Go's net/http (not actively maintained)

binding Reflectionless data binding for Go's net/http Features HTTP request data binding Data validation (custom and built-in) Error handling Benefits

Matt Holt 793 Nov 18, 2022
Go binding for the cairo graphics library

go-cairo Go binding for the cairo graphics library Based on Dethe Elza's version https://bitbucket.org/dethe/gocairo but significantly extended and up

Erik Unger 125 Sep 30, 2022
Go OpenCL (GOCL) Binding

gocl Go OpenCL (GOCL) Binding (http://www.gocl.org) Library documentation: http://www.khronos.org/registry/cl/sdk/1.1/docs/man/xhtml/ http://www.khron

Rain Liu 85 Jan 25, 2022
OpenGL binding generator for Go

GoGL GoGL is an OpenGL binding generator for Go. No external dependencies like GLEW are needed. Install the OpenGL bindings For example, OpenGL 2.1 bi

Christoph Schunk 140 Jul 24, 2022
Go binding for GTK

go-gtk WHATS Go bindings for GTK SCREENSHOT INSTALL You can experiment with go-gtk by running the various example programs: git clone https://github.c

mattn 1.9k Nov 23, 2022
The X Go Binding is a low-level API to communicate with the X server. It is modeled on XCB and supports many X extensions.

Note that this project is largely unmaintained as I don't have the time to do or support more development. Please consider using this fork instead: ht

Andrew Gallant 457 Nov 28, 2022