Hello

I tried to run example with edgetpu but it gives me

EdgeTPU Version: BuildLabel(COMPILER=6.3.0 20170516,DATE=redacted,TIME=redacted,CL_NUMBER=291256449), RuntimeVersion(13)
Internal: Unsupported data type in custom op handler: 0
Node number -1095957936 (8ɶ<­¾üÍ϶D) @2.

2020/06/15 23:44:53 allocate failed

What would be the cause?

Hi, I have created a Docker builder image for arm64v8 so I can compile binary from my code using your framework.

All is ok on the local ubuntu amd64 machine, but when I start the binary (compiled for arm64) on raspberry arm64v8, Invoke is slow, about a second or more.

status = interpreter.Invoke() //takes more than 1 second

Do you know about this issue and where the problem might be?

Raspberry is using arm64v8 architecture For building the builder image I was using an ec2 arm machine (m6g.8xlarge)

Hey, I wanted to try your promising library so I went through all the steps to produce my libtensorflowlite_c.so :smiley:.

I then copy/pasted your label_image example and ran it using go run main.go producing this error:

/tmp/go-build075675325/b001/exe/main: error while loading shared libraries: libtensorflowlite_c.so: cannot open shared object file: No such file or directory
exit status 127

I understand my libtensorflowlite_c.so may not be placed in the correct location. However, in https://github.com/mattn/go-tflite/blob/master/tflite.go I see...

#cgo LDFLAGS: -L${SRCDIR}/../../tensorflow/tensorflow/tensorflow/lite/experimental/c -ltensorflowlite_c

...which seems to refer to the actual path my file sits in. So I don't really get why it cannot find it. Did I miss something ? Is there another step necessary to run go-tflite a part from placing the c library in tensorflow repo ?

P.S: just to make it clear, here is an extract of my tree:

$GOPATH/src/
    |_ github.com/
        |_ mattn/
        |       |_ go-tflite/
        |       |_ go-pointer/
        |_ Jonarod/
        |       |_ tests-labels-tflite/
        |           |_ main.go
        |_ tensorflow/
                |_ tensorflow/
                    |_ tensorflow/
                        |_ lite/
                            |_ experimental / c / libtensorflowlite_c.so

The pull request remove -lrt for android build #37 solved some of my issues. I used to change the cgo LDFLAGS every time I switch testing amd64 / arm32 / arm64. These two flags cgo android and cgo linux,!android work like a charm!

Commit changes

• Adding two other library flags -lm -llog for android, which is needed when building android aar library using gomobile bind.
• Using -ltensorflowlite_c instead of -ltensorflowlite-delegate_xnnpack -lXNNPACK in delegates/xnnpack/xnnpack.go. I'm not sure if I'm right about this. In my practice, compiling lib using

  $ bazel build --config opt --config monolithic  --define tflite_with_xnnpack=true //tensorflow/lite:libtensorflowlite.so
  $ bazel build --config opt --config monolithic  --define tflite_with_xnnpack=true //tensorflow/lite/c:libtensorflowlite_c.so
  

  will include xnnpack in libtensorflowlite_c.so. This seems easier in using of xnnpack.

• In my basic tests, -ldl -lrt is not required for both linux and android, maybe they are not for my testing scenarios.

Testing Android flags

• Project folder structure

  $ tree testTflite/
  # testTflite/
  # ├── main.go
  # └── runTest
  #     └── runTest.go
  

• runTest/runTest.go

  package runTest
  
  import (
  	"fmt"
  	"github.com/mattn/go-tflite"
  	"github.com/mattn/go-tflite/delegates/xnnpack"
  )
  
  func Run(model_path string) {
  	model := tflite.NewModelFromFile(model_path)
  	options := tflite.NewInterpreterOptions()
  	options.AddDelegate(xnnpack.New(xnnpack.DelegateOptions{NumThreads: 2}))
  	defer options.Delete()
  
  	interp := tflite.NewInterpreter(model, options)
  	status := interp.AllocateTensors()
  	input := interp.GetInputTensor(0)
  	in := input.Float32s()
  	fmt.Println(status, len(in))
  	status = interp.Invoke()
  	output := interp.GetOutputTensor(0).Float32s()
  	fmt.Println(output)
  }
  

• main.go

  package main
  
  import "testTflite/runTest"
  
  func main() {
  	runTest.Run("model.tflite")
  }
  

• Test

  cd testTflite
  go mod init testTflite
  # Use my custom go-tflite
  go mod edit -replace=github.com/mattn/go-tflite=$HOME/workspace/go-tflite
  go mod tidy
  
  # Build test for amd64
  export CGO_LDFLAGS=-L$HOME/workspace/tensorflow/bazel-bin/tensorflow/lite/c
  export CGO_CFLAGS=-I$HOME/workspace/tensorflow/
  go build main.go
  

  # Build test for arm64
  export CGO_LDFLAGS="-L$HOME/workspace/tensorflow.arm64/bazel-bin/tensorflow/lite/c"
  export CGO_CFLAGS="-I$HOME/workspace/tensorflow.arm64/"
  SDK_HOME="$HOME/Android/Sdk/ndk/21.0.6113669/toolchains/llvm/prebuilt/linux-x86_64"
  ANDROID_CC="$SDK_HOME/bin/aarch64-linux-android29-clang -Wl,-rpath-link,$SDK_HOME/sysroot/usr/lib/aarch64-linux-android/29"
  ANDROID_CXX="$SDK_HOME/bin/aarch64-linux-android29-clang++ -Wl,-rpath-link,$SDK_HOME/sysroot/usr/lib/aarch64-linux-android/29"
  ANDROID_ARCH="arm64"
  CGO_ENABLED=1 GOOS=android GOARCH=$ANDROID_ARCH CC=$ANDROID_CC CXX=$ANDROID_CXX go build main.go
  

  # gomobile bind test for arm64, will throw error without `-llog -lm`
  go get golang.org/x/mobile/bind
  gomobile bind -v -o test.aar -target=android/arm64 testTflite/runTest
  

Hey, I'm curious to know if this has go-tflite has been tried on devices like the rasberry pi?

I'm currently rebuilding the libtensorflow_c.so file for ARM devices to use along on my rasberry pi, if this has been done before, would love to hear how that turned out to be.

If not, once I make any headway, I can share what I did to get things right. Let me know :)

This is great work. I actually got this to work with r14 and have been playing around with it.

I would like to open source my code but would feel better if I either got your permission to release my code (which includes much of yours) or for you to add a license. (preferably BSD) Feel free to copy this and put your name in it's place: https://github.com/snowzach/gorestapi/blob/master/LICENSE.md

I have added support for the Coral edgetpu hardware accelerator.

Thanks!

Hi, I must be pretty ignorant, but I cannot figure out how to install Tensorflow Lite C API. I just got my coral TPU, and the sales from Google told me that they worked with you @mattn to allow tensorflow lite to work in go. But everywhere on the tensorflow website, it shows installation with pip in python. I got bazel working, but I don't know where /source/directory/tensorflow is supposed to be. I git clone https://github.com/tensorflow/tensorflow and checked out v2.2.0-rc3 but no dice.

Could you enlighten me for the initial setup?

My goal is to develop in go on a ubuntu intel 64 machine and then, once everything is running, install everything on a raspberry pi and compile it over there. I'm saying this because the only instructions to make Tensorflow for C is for arm based systems on their website https://www.tensorflow.org/lite/guide/build_arm64

Hello

Could you write tutorial for setting this up in raspberry pi?

I'm trying to build it but I'm stuck at

/usr/bin/ld: cannot find -ltensorflow-lite

Thank you

added a gpu delegate with high level API. Simply add

delegate := tflite.NewGPUDelegate(nil)
if delegate == nil {
	log.Fatal("cannot create delegate")
}
defer delegate.Delete()

and

options := tflite.NewInterpreterOptions()
...
options.AddDelegate(delegate)

However, at the moment of writing, my GPU delegate throws

INFO: Created TensorFlow Lite delegate for GPU. Next operations are not supported by GPU delegate: ADD: AVERAGE_POOL_2D: CONV_2D: DEPTHWISE_CONV_2D: RESHAPE: First 0 operations will run on the GPU, and the remaining 65 on the CPU.

for a simple mobilenet_v2 model

I found that Tensorflow provides the build scripts for the tflite cpp lib in: https://www.tensorflow.org/lite/guide/build_rpi

So I'm wondering if I can use it instead? Or if not, can I simply modify something to make it compatible? Because I got the following error running the example:

pi@raspberrypi:~/go/src/github.com/mattn/go-tflite/_example/ssd $ go run main.go 
# github.com/mattn/go-tflite
../../tflite_experimental.go: In function 'writeToTensorAsVector':
../../tflite_experimental.go:64:13: error: 'TfLiteTensor {aka struct <anonymous>}' has no member named 'quantization'; did you mean 'allocation'?
   if (tensor->quantization.type == kTfLiteAffineQuantization) {
             ^~
../../tflite_experimental.go:64:36: error: 'kTfLiteAffineQuantization' undeclared (first use in this function)
   if (tensor->quantization.type == kTfLiteAffineQuantization) {
                                    ^~~~~~~~~~~~~~~~~~~~~~~~~
../../tflite_experimental.go:64:36: note: each undeclared identifier is reported only once for each function it appears in
../../tflite_experimental.go:65:5: error: unknown type name 'TfLiteAffineQuantization'
     TfLiteAffineQuantization* q_params =
     ^~~~~~~~~~~~~~~~~~~~~~~~
../../tflite_experimental.go:66:10: error: 'TfLiteAffineQuantization' undeclared (first use in this function)
         (TfLiteAffineQuantization*)(tensor->quantization.params);
          ^~~~~~~~~~~~~~~~~~~~~~~~
../../tflite_experimental.go:66:35: error: expected expression before ')' token
         (TfLiteAffineQuantization*)(tensor->quantization.params);
                                   ^
../../tflite_experimental.go:66:43: error: 'TfLiteTensor {aka struct <anonymous>}' has no member named 'quantization'; did you mean 'allocation'?
         (TfLiteAffineQuantization*)(tensor->quantization.params);
                                           ^~
../../tflite_experimental.go:67:17: error: request for member 'scale' in something not a structure or union
     if (q_params->scale) {
                 ^~
../../tflite_experimental.go:68:20: error: request for member 'scale' in something not a structure or union
       free(q_params->scale);
                    ^~
../../tflite_experimental.go:69:15: error: request for member 'scale' in something not a structure or union
       q_params->scale = NULL;
               ^~
../../tflite_experimental.go:71:17: error: request for member 'zero_point' in something not a structure or union
     if (q_params->zero_point) {
                 ^~
../../tflite_experimental.go:72:20: error: request for member 'zero_point' in something not a structure or union
       free(q_params->zero_point);
                    ^~
../../tflite_experimental.go:73:15: error: request for member 'zero_point' in something not a structure or union
       q_params->zero_point = NULL;
               ^~
../../tflite_experimental.go:82:9: error: 'TfLiteTensor {aka struct <anonymous>}' has no member named 'quantization'; did you mean 'allocation'?
   tensor->quantization.type = kTfLiteNoQuantization;
         ^~
../../tflite_experimental.go:82:31: error: 'kTfLiteNoQuantization' undeclared (first use in this function)
   tensor->quantization.type = kTfLiteNoQuantization;
                               ^~~~~~~~~~~~~~~~~~~~~
../../tflite_experimental.go:83:9: error: 'TfLiteTensor {aka struct <anonymous>}' has no member named 'quantization'; did you mean 'allocation'?
   tensor->quantization.params = NULL;
         ^~

Sorry that I have no ideas about those C stuffs! Could you please give me some advices?

Hello Everyone,

Thanks to your work, I could use the GO in my project. while I am using go-tflite, I have a question for input/output. My tflite input shape [4][3]float32 output shape [2][1]float32. How am I supposed to input and output properly?

Thanks for the reply in advance.

predict := []float32
for i := 0; i < num; i++ {
		inputs := [][]float32{}
		for j := 0; j < steps; j++ {
			inputs = append(inputs, data[j])
		}
		interpreter.GetInputTensor(0).CopyFromBuffer(inputs) // <<-- Error occurred here
		interpreter.Invoke()
		interpreter.GetOutputTensor(0).CopyToBuffer(predict)
		fmt.Println(predict)
	}

but it gave an error like below

panic: runtime error: cgo argument has Go pointer to Go pointer

The docs says only 2.2.0-rc3 is supported ... to build this requires an old version of bazel which requires old version of jdk. In short pretty hard to build on recent linux since dependencies arn't easily available.

Will a more recent release of tensorflow work ?

I have followed the instructions from issue and still have: /tmp/go-build1465283250/b001/exe/main: error while loading shared libraries: libtensorflowlite_c.so: cannot open shared object file: No such file or directory exit status 127

I put the libtensorflowlite_c.so into the /usr/local/lib and export LD_LIBRARY_PATH=/usr/local/lib and in /etc/environment but does not help. I am trying to run esrgan example.

tflite.go has: /* #ifndef GO_TFLITE_H #include "tflite.go.h" #endif #cgo LDFLAGS: -ltensorflowlite_c #cgo android LDFLAGS: -ldl #cgo linux,!android LDFLAGS: -ldl -lrt */` I am using Go 1.17

Hello, there

I was looking at https://github.com/tensorflow/tflite-micro and wondering if it would be possible to use go-tflite with that and TinyGo.

Any thoughts on this would be appreciated. Thank you.

tensorflow provides scripts to build static tf lite libraries: tflite make scripts

Are there plans to support .a files additionally to the .so files build with bazel?

You may or may not have seen the git issue where I was trying to get the edgetpu to work for DOODS.

With the tensorflow commit that supports the edgetpu they have changed the import path to tensorflow/lite/c/c_api.h from tensorflow/lite/experimental/c/c_api.h

Not sure how you want to handle this. I was trying to base my DOODS code off of your repo.