TFKG - A Tensorflow and Keras Golang port

Overview

TFKG - A Tensorflow and Keras Golang port

This is experimental and quite nasty under the hood*

Support

  • macOS: running docker container, no GPU acceleration
  • Ubuntu 18.04: binary execution on linux ubuntu with CUDA 11.2 and Python 3.8, with GPU acceleration
  • Windows: ?

Find your version

Versions starting with v0 are liable to change radically.

  • Tensorflow 2.6 experimental support: go get github.com/codingbeard/tfkg v0.2.6.4

Requirements

If not using the container: Make sure to install the correct versions to match the version of this library

Features

  • Nvidia CUDA support on applicable platforms during Golang training/evaluation due to using the Tensorflow C library
  • Define, Train, evaluate, and save Tensorflow compatible models in Golang
  • Load models created with this library in Golang
  • Load, shuffle, and preprocess csv datasets efficiently, even very large ones (tested on 330GB csv file)
    • String Tokenizer
    • Float/Int normalization to between 0-1

Keras model types supported

  • tensorflow.keras.Sequential
  • Functional coming soon

Keras Layers supported

  • tensorflow.keras.layers.Input
  • tensorflow.keras.layers.Dense
  • More coming soon
  • Interfaced - you can define custom layers

Metrics

  • Sparse Categorical Cross Entropy:
    • Accuracy
    • False positive rate at true positive rate (Specificity at Sensitivity)
    • True positive rate at false positive rate (Sensitivity at Specificity)

Limitations

  • Python Tensorflow Libraries are still required to use this library, though the docker container has it all
  • This is an incomplete port of Tensorflow/Keras: There are many layers, metrics, and optimisers not yet ported
  • There is no community support or documentation. You must understand Tensorflow/Keras and Golang well to have a chance of getting this working on a new project
  • Loading/Training models saved by vanilla python Tensorflow/Keras is not supported, and may never be
  • Image datasets and preprocessing is not yet supported
  • Class weighting only works for datasets with two categories currently: negative/positive. It /will/ cause unintended side effects if you have more than two categories with imbalanced classes

Examples:

See the full example with comments in ./examples/iris

To test it out run:

docker-compose up -d
make examples-iris

Define a model:

m := model.NewSequentialModel(
    errorHandler,
    logger,
    layer.NewInput(tf.MakeShape(-1, 4), layer.Float32),
    layer.NewDense(100, layer.Float32, layer.DenseConfig{Activation: "swish"}),
    layer.NewDense(100, layer.Float32, layer.DenseConfig{Activation: "swish"}),
    layer.NewDense(3, layer.Float32, layer.DenseConfig{Activation: "softmax"}),
)
e = m.CompileAndLoad(
    3, // batchSize
)

Load a dataset:

dataset, e := data.NewSingleFileDataset(
    logger,
    errorHandler,
    "examples/iris/data/iris.data", // filePath
    cacheDir,
    4, // categoryOffset
    0.8, // trainPercentage
    0.1, // valPercentage
    0.1, // testPercentage
    preprocessor.NewProcessor(
        errorHandler,
        cacheDir,
        "petal_sizes", // name
        0, // offset
        4, // dataLength
        true, // requiresFit
        preprocessor.NewDivisor(errorHandler),
        nil, // tokenizer
        preprocessor.ReadCsvFloat32s,
        preprocessor.ConvertDivisorToFloat32SliceTensor,
    ),
)

Train a model:

m.Fit(
    dataset,
    model.FitConfig{
        Epochs:     10,
        Validation: true,
        BatchSize:  3,
        PreFetch:   10,
        Verbose:    1,
        Metrics: []metric.Metric{
            &metric.SparseCategoricalAccuracy{
                Name:       "acc",
                Confidence: 0.5,
                Average:    true,
            },
        },
        Callbacks: []callback.Callback{
            &callback.Logger{
                FileLogger: logger,
            },
            &callback.Checkpoint{
                OnEvent:    callback.EventEnd,
                OnMode:     callback.ModeVal,
                MetricName: "val_acc",
                Compare:    callback.CheckpointCompareMax,
                SaveDir:    saveDir,
            },
        },
    },
)

Load and predict using a saved TFKG model:

divisor := preprocessor.NewDivisor(errorHandler)
e = divisor.Load(filepath.Join(cacheDir, "petal_sizes-divisor.json"))
if e != nil {
    errorHandler.Error(e)
    return
}

processedInput, e := divisor.Divide([]float32{6.0, 3.0, 4.8, 1.8})
if e != nil {
    return
}

inputTensor, e := tf.NewTensor([][]float32{processedInput})
if e != nil {
    errorHandler.Error(e)
    return
}

m, e = model.LoadModel(errorHandler, logger, saveDir)
if e != nil {
    errorHandler.Error(e)
    return
}

outputTensor, e := m.Predict(inputTensor)
if e != nil {
    return
}

outputValues := outputTensor.Value().([][]float32)

logger.InfoF(
    "main",
    "Predicted classes: %s: %f, %s: %f, %s: %f",
    "Iris-setosa",
    outputValues[0][0],
    "Iris-versicolor",
    outputValues[0][1],
    "Iris-virginica",
    outputValues[0][2],
)

*Nasty under the hood

The Tensorflow/Keras python package saves a Graph (see more: https://www.tensorflow.org/guide/intro_to_graphs) which can be executed in other languages using their C library as long as there are C bindings.

The C library does not contain all the functionality of the python library when it comes to defining and saving models, it can only execute Graphs.

The Graph is calculated in python based on your model configuration, and a lot of clever code on the part of the developers in optimising the graph.

While possible, it is not currently feasible for me to generate the Graph in Golang, so I am relying on python to do so.

This means while the model is technically defined and trained in Golang, it just generates a json config string which static python code uses to configure the model and then saves it ready for loading in Golang for training. For the moment this is a needed evil.

If some kind soul wants to replicate Keras and Autograph to generate the Graph in Golang, feel free to make a pull request. I may eventually do it, but it is not likely. There is a branch origin/scratch which allows you to investigate the graph of a saved model.

Acknowledgements

Big shout out to github.com/galeone for their Tensorflow Golang fork for 2.6 and again for their article on how to train a model in golang which helped me figure out how to then save the trained variables: https://pgaleone.eu/tensorflow/go/2020/11/27/deploy-train-tesorflow-models-in-go-human-activity-recognition/

Releases(v0.2.6.5)
  • v0.2.6.5(Dec 8, 2021)

    libtensorflow.tar.gz contains compiled linux amd64 libraries for Tensorflow 2.6.0 with AVX disabled.

    tensorflow-2.6.0-cp38-cp38-linux_x86_64.whl is a compiled python wheel for linux amd64 Tensorflow 2.6.0 with AVX disabled.

    Disabling AVX allows the libraries to be used in a docker container on an Apple Silicon M1 Macbook running macOS and docker for macOS M1.

    These are the steps I took to compile the library and wheel from sources:

    // On a linux amd64 machine with docker installed:
    git clone https://github.com/tensorflow/tensorflow
    cd tensorflow
    git checkout v2.6.0
    docker run -it -w /tensorflow_src -v $PWD:/mnt -v $PWD:/tensorflow_src -e HOST_PERMS="$(id -u):$(id -g)" tensorflow/tensorflow:devel-gpu bash
    > apt update && apt install apt-transport-https curl gnupg
    > curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg && \
        mv bazel.gpg /etc/apt/trusted.gpg.d/ && \
        echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list
    > apt update && apt install bazel-3.7.2 nano
    > nano .bazelrc
    // add the lines after the existing build:cuda lines:
    build:cuda --linkopt=-lm
    build:cuda --linkopt=-ldl
    build:cuda --host_linkopt=-lm
    build:cuda --host_linkopt=-ldl
    > ./configure 
    // take the defaults EXCEPT :
    // ... "--config=opt" is specified [Default is -Wno-sign-compare]: -mno-avx
    // The below will compile it for a specific GPU, find your gpu's compute capability and enter it twice separated by a comma (3000 series is 8.6)
    // ... TensorFlow only supports compute capabilities >= 3.5 [Default is: 3.5,7.0]: 8.6,8.6
    > bazel-3.7.2 build --config=cuda --config=opt //tensorflow/tools/lib_package:libtensorflow
    > mkdir output
    > cp bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz ./output/
    > cp bazel-bin/tensorflow/tools/lib_package/clicenses.tar ./output/
    > rm -r bazel-*
    > bazel-3.7.2 build --config=cuda --config=opt //tensorflow/tools/pip_package:build_pip_package
    > ./bazel-bin/tensorflow/tools/pip_package/build_pip_package ./output/tf-2.6.0-gpu-noavx
    > quit
    // copy the libs and wheel from ./output into the TFKG project under ./docker/tf-jupyter-golang-m1
    ...
    
    Source code(tar.gz)
    Source code(zip)
    libtensorflow.tar.gz(176.16 MB)
    LICENSE(15.13 KB)
    tensorflow-2.6.0-cp38-cp38-linux_x86_64.whl(252.54 MB)
    THIRD_PARTY_TF_C_LICENSES(353.09 KB)
Owner
Tim Marshall
Tim Marshall
Tensorflow + Go, the gopher way

tfgo: TensorFlow in Go tfgo: TensorFlow in Go Dependencies Installation Getting started Computer Vision using data flow graph Train in Python, Serve i

Paolo Galeone 2k Aug 5, 2022
Go binding for TensorFlow Lite

go-tflite Go binding for TensorFlow Lite Usage model := tflite.NewModelFromFile("sin_model.tflite") if model == nil { log.Fatal("cannot load model")

mattn 242 Jul 27, 2022
Gota: DataFrames and data wrangling in Go (Golang)

Gota: DataFrames, Series and Data Wrangling for Go This is an implementation of DataFrames, Series and data wrangling methods for the Go programming l

null 2.3k Aug 1, 2022
The open source, end-to-end computer vision platform. Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises.

End-to-end computer vision platform Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises. onepa

Onepanel, Inc. 611 Aug 9, 2022
Go types, funcs, and utilities for working with cards, decks, and evaluating poker hands (Holdem, Omaha, Stud, more)

cardrank.io/cardrank Package cardrank.io/cardrank provides a library of types, funcs, and utilities for working with playing cards, decks, and evaluat

null 56 Jul 30, 2022
Naive Bayesian Classification for Golang.

Naive Bayesian Classification Perform naive Bayesian classification into an arbitrary number of classes on sets of strings. bayesian also supports ter

Jake Brukhman 737 Jul 20, 2022
Ensembles of decision trees in go/golang.

CloudForest Google Group Fast, flexible, multi-threaded ensembles of decision trees for machine learning in pure Go (golang). CloudForest allows for a

Ryan Bressler 713 Jul 21, 2022
Genetic Algorithms library written in Go / golang

Description Genetic Algorithms for Go/Golang Install $ go install git://github.com/thoj/go-galib.git Compiling examples: $ git clone git://github.com

Thomas Jager 192 May 26, 2022
Golang Genetic Algorithm

goga Golang implementation of a genetic algorithm. See ./examples for info on how to use the library. Overview Goga is a genetic algorithm solution wr

null 164 Jul 27, 2022
Golang Neural Network

Varis Neural Networks with GO About Package Some time ago I decided to learn Go language and neural networks. So it's my variation of Neural Networks

Artem Filippov 44 Mar 20, 2022
Golang implementation of the Paice/Husk Stemming Algorithm

##Golang Implementation of the Paice/Husk stemming algorithm This project was created for the QUT course INB344. Details on the algorithm can be found

Aaron Groves 28 Jan 23, 2022
Golang HTML to PDF Converter

Golang HTML to PDF Converter For reading any document, one prefers PDF format over any other formats as it is considered as a standard format for any

MindInventory 244 Jul 29, 2022
A high-performance timeline tracing library for Golang, used by TiDB

Minitrace-Go A high-performance, ergonomic timeline tracing library for Golang. Basic Usage package main import ( "context" "fmt" "strcon

TiKV Project 43 May 5, 2022
Golang k-d tree implementation with duplicate coordinate support

Golang k-d tree implementation with duplicate coordinate support

DownFlux 45 Apr 13, 2022
Another AOC repo (this time in golang!)

advent-of-code Now with 100% more golang! (It's going to be a long advent of code...) To run: Get your data for a given year/day and copy paste it to

Jon Schwartz 0 Dec 14, 2021
Go (Golang) encrypted deep learning library; Fully homomorphic encryption over neural network graphs

DC DarkLantern A lantern is a portable case that protects light, A dark lantern is one who's light can be hidden at will. DC DarkLantern is a golang i

Raven 1 Dec 2, 2021
Clean Architecture With Golang

Clean Architecture With Golang When init a new project go mod init github.com/samuelterra22/clean-architecture-go Run testes go test ./... Generate a

Samuel Terra 1 Dec 14, 2021
Genetic algorithms using Golang Generics

Package genetic Package genetic implements genetic algorithms using Golang's Gen

Konnor Klashinsky 6 Jul 22, 2022
Genetic Algorithm and Particle Swarm Optimization

evoli Genetic Algorithm and Particle Swarm Optimization written in Go Example Problem Given f(x,y) = cos(x^2 * y^2) * 1/(x^2 * y^2 + 1) Find (x,y) suc

Guillaume Simonneau 24 Jul 4, 2022