Golang RServe client. Use R from Go

Overview

Roger

GoDoc Build Status Join the chat at https://gitter.im/senseyeio/roger

Roger is a Go RServe client, allowing the capabilities of R to be used from Go applications.

The communication between Go and R is via TCP. It is thread safe and supports long running R operations synchronously or asynchronously (using channels).

package main

import (
	"fmt"

	"github.com/senseyeio/roger"
)

func main() {
	rClient, err := roger.NewRClient("127.0.0.1", 6311)
	if err != nil {
		fmt.Println("Failed to connect")
		return
	}

	value, err := rClient.Eval("pi")
	if err != nil {
		fmt.Println("Command failed: " + err.Error())
	} else {
		fmt.Println(value) // 3.141592653589793
	}

	helloWorld, _ := rClient.Eval("as.character('Hello World')")
	fmt.Println(helloWorld) // Hello World

	arrChan := rClient.Evaluate("Sys.sleep(5); c(1,1)")
	arrResponse := <-arrChan
	arr, _ := arrResponse.GetResultObject()
	fmt.Println(arr) // [1, 1]
}

Response Type Support

Roger currently supports the following response types from R:

  • string and string arrays
  • booleans and boolean arrays
  • doubles and double arrays
  • ints and int arrays
  • complex and complex arrays
  • lists
  • raw byte arrays

With the use of JSON, this capability can be used to transfer any serializable object. For examples see sexp_parsing_test.go.

Assignment Support

Roger allows variables to be defined within an R session from Go. Currently the following types are supported for variable assignment:

  • string and string arrays
  • byte arrays
  • doubles and double arrays
  • ints and int arrays

For examples see assignment_test.go.

Setup

Rserve should be installed and started from R:

install.packages("Rserve")
require('Rserve')
Rserve()

More information is available on RServe's website.

If you would like to exploit the current R environment from go, start RServe using the following command:

install.packages("Rserve")
require('Rserve')
run.Rserve()

Install Roger using:

go get github.com/senseyeio/roger

Testing

To ensure the library functions correctly, the end to end functionality must be tested. This is achieved using Docker and Docker Compose. To run tests, ensure you have both Docker and Docker Compose installed, then run docker-compose build && docker-compose up -d from within the test directory. This command will build and start a docker container containing multiple RServe servers. These servers will be utilized when running go test from the project's base directory. To stop the docker container call docker-compose stop from the test directory.

Contributing

Issues, pull requests and questions are welcomed. If required, assistance can be found in the project's gitter chat room.

Pull Requests

  • Fork the repository
  • Make changes
  • Ensure tests pass
  • Raise pull request
Comments
  • keep session open for several command

    keep session open for several command

    Hi Daniel,

    Now, in roger struct, defined address *net.TCPAddr, there is no session (or rconnection). Each Eval function will be closed and the variables evaluated will be released. It make the code below not possible r.Eval("x <- 1") x, err := r.Eval(x) // since it is a new connection, x will not exit in R.

    Do you think we can define sess *session in roger struct And each r.Eval will use the same r.sess. Of course define a r.Close() to close the connection when no r scripts will be evaluated.

    Regards, Jia

    enhancement 
    opened by lujiacn 7
  • Added public GetError method to the packet.

    Added public GetError method to the packet.

    Before the only way for the user to get the error was to call GetResultObject. However, the parse would also be called in this case and then the user wouldn't know (programatically) where the error came from. Off course, the librarie's user could make some string matching to try to deduce, but this is error prone and non scalable.

    opened by cesarsalgado 3
  • assign function

    assign function

    Hi Daniel,

    Do you plan implement some basic assignment functions? Which can assign go variable values to R variable. May be start from string array -> string vector or others. If do so, any idea how to programming. May be convert go values to strings with R eval format and evaluate the string?

    Regards, Jia

    opened by lujiacn 3
  • Unsupported expression type: 23 (XT_LANG_TAG)

    Unsupported expression type: 23 (XT_LANG_TAG)

    roger seems unable to handle this type. If I try:

      rClient.EvaluateSync("try(source('does not exist'))")
    

    Based on this header it seems like the missing sexp type is XT_LANG_TAG.

    opened by bfallik 2
  • panic with echo disabled

    panic with echo disabled

    Hi,

    I can reliably generate a panic by disabling echo within a session:

        session, err := rClient.GetSession()
        if err != nil {
            log.Fatal(err)
        }
        defer session.Close()
    
        pkt := session.SendCommand(`options(echo = FALSE)`)
        v, err := pkt.GetResultObject()
    

    results in

    panic: runtime error: slice bounds out of range
    
    goroutine 1 [running]:
    _vendor/github.com/senseyeio/roger/sexp.parseReturningOffset(0xc82000e7e4, 0x2c, 0x2c, 0x29, 0x0, 0x0, 0x0, 0x0, 0x0)
        /Users/bfallik/sandbox/gillnet/go/src/_vendor/github.com/senseyeio/roger/sexp/factory.go:26 +0xc41
    _vendor/github.com/senseyeio/roger/sexp.parseVector(0x13d6c0, 0xc82000e840, 0xc82000e7e4, 0x2c, 0x2c, 0x29, 0x2c, 0x0, 0x0, 0xc, ...)
        /Users/bfallik/sandbox/go/src/_vendor/github.com/senseyeio/roger/sexp/xt-vector.go:33 +0xbd
    _vendor/github.com/senseyeio/roger/sexp.parseReturningOffset(0xc82000e7e4, 0x2c, 0x2c, 0x20, 0x0, 0x0, 0x7a15d, 0x0, 0x0)
        /Users/bfallik/sandbox/go/src/_vendor/github.com/senseyeio/roger/sexp/factory.go:68 +0x78e
    _vendor/github.com/senseyeio/roger/sexp.Parse(0xc82000e7e4, 0x2c, 0x2c, 0x0, 0x0, 0x0, 0x0, 0x0)
        /Users/bfallik/sandbox/go/src/_vendor/github.com/senseyeio/roger/sexp/factory.go:13 +0x61
    _vendor/github.com/senseyeio/roger.(*packet).GetResultObject(0xc82000e810, 0x0, 0x0, 0x0, 0x0)
        /Users/bfallik/sandbox/go/src/_vendor/github.com/senseyeio/roger/packet.go:77 +0x31e
    main.main()
        /Users/bfallik/sandbox/go/src/tools/rogercmd/main.go:49 +0x4ed
    
    goroutine 17 [syscall, locked to thread]:
    runtime.goexit()
        /usr/local/Cellar/go/1.5.1/libexec/src/runtime/asm_amd64.s:1696 +0x1
    exit status 2
    
    opened by bfallik 2
  • Large string assignment

    Large string assignment

    Adding the following test in assignment_test.go fails due to the string length:

    func TestLargeStringAssignment(t *testing.T) {
    	checkAssignment(t, strings.Repeat("a", 20000000))
    }
    
    opened by dareid 1
  • Error on common.go

    Error on common.go

    I'm having this issue:

    $ go build

    github.com/senseyeio/roger/assign

    ......\github.com\senseyeio\roger\assign\common.go:27: constant 4278190080 overflows int ......\github.com\senseyeio\roger\assign\common.go:45: constant 4278190080 overflows int

    I believe this is because of data type int which can't handle the shift >> 24.

    opened by predixdeveloperACN 1
  • isLong in sexp causes index out of range error

    isLong in sexp causes index out of range error

    This line throws an error in case of big outputs:

    https://github.com/senseyeio/roger/blob/master/sexp/factory.go#L19

    I am trying to export ~30 Mbytes JSON output from R.

    Commenting this block out solves the issue.

    Could you spare some info?

    opened by vamitrou 1
  • added voidEval functions

    added voidEval functions

    Just added a voidEval function which do not need return content, just the error status. No content returning expected.

    PS: I want to say thank you very much program the Rserve client for go. I am looking for it for a long time and created a very ugly client which only can do voidEval :(

    feel free to reject this pull request :p

    opened by lujiacn 1
  • Extremely long strings

    Extremely long strings

    Adding the following test to SEXP parsing test suite fails:

    func TestReallyLongResponse(t *testing.T) {
    	obj, err := getResultObject("paste(rep(\"a\", 16780000), sep=\"\", collapse = \"\")")
    	assert.Nil(t, err)
    	str, ok := obj.(string)
    	assert.Equal(t, ok, true, "Return obj should be a string")
    	assert.Equal(t, len(str), 16780000, "String length expected to be 16780000 characters")
    }
    
    opened by dareid 0
  • Reporting unsupport RServe version for an unused but open port

    Reporting unsupport RServe version for an unused but open port

    Discovered during exploration for #25. If an connection is attempted to an incorrect port, the lib reports an unsupported RServe version. Should really be reporting an incorrect handshake.

    opened by dareid 0
  • Proposal: Please start using Semantic Versioning

    Proposal: Please start using Semantic Versioning

    I found that this project already supports Go modules. But sadly, the tags doesn't follow Semantic Versioning, which means that all tags of this project will be ignored by Go modules and replaced by pseudo-versions, go get acts weirdly when tags are not in that form. It would be great to have the tagged release be named in the format vX.X.X format so that go mod can read it.

    	github.com/senseyeio/roger v0.0.0-20191009211040-43e330bee47f
    

    Else the mod file shows something like github.com/senseyeio/roger v0.0.0-20191009211040-43e330bee47f which is not very readable and difficult to upgrade. It’s hard to verify which version is in use. This is not conducive to version control.

    So, I propose this project to follow Semantic Versioning in future versions. For example, v1.0.1, v2.0.0, v3.1.0-alpha, v3.1.0-beta.2etc.

    opened by KateGo520 1
  • Improve error messages

    Improve error messages

    The status code - error message map in packet.go does not provide detailed enough error messages. Sometimes error messages are misleading, as we receive the message "Command error with status: Unknown variable/method" even when the command failed due to other reasons.

    A better approach could be to query the R session and return the message of last error such that the following test passes.

    func TestErrorReturn(t *testing.T) {
        con, err := NewRClient("localhost", 6311)
        if err != nil {
            t.Error("Could not connect to RServe: " + err.Error())
            return
        }
    
        errorText := "this is a test"
        _, err = con.Eval("stop('" + errorText + "')")
    
        if err == nil {
            t.Error("No error was returned")
        }
    
        expectedMessage := "Command error with status: " + errorText
        if err.Error() != expectedMessage {
            t.Error("Expected '" + err.Error() + "' to equal '" + expectedMessage + "'")
        }
    }
    
    enhancement 
    opened by harryrose 5
  • panic when creating a list using bool or NA values

    panic when creating a list using bool or NA values

    When sending a command to create a list I'm getting a panic: runtime error: index out of range with certain combinations of values. What I've observed is the following.

    1. Creating a list of strings, ints, and floats in any combination works. Ex: list(int=1,string='s',float=0.5)

    2. If the list has a bool or NA as the last value, the correct result is returned but there is a Warning: Error whilst constructing vector: Abruptly reached end of buffer outputted. Ex: list(int=1,string='s',float=0.5,bool=TRUE) Ex: list(var=NA)

    3. If the list contains a bool or NA value with anything else following it, a panic happens. Ex: list(int=1,string='s',float=0.5,bool=TRUE,anything='this causes a panic') Ex: list(var1=NA,var2=1)

    Here is the full stack trace for the panic:

    panic: runtime error: index out of range
    
    goroutine 1 [running]:
    roger/sexp.getLength(0xc820431484, 0x38, 0x38, 0x29, 0x1, 0x420460, 0x0, 0x0)
        roger/sexp/factory.go:25 +0x12f
    roger/sexp.parseReturningOffset(0xc820431484, 0x38, 0x38, 0x29, 0x0, 0x0, 0x0, 0x0, 0x0)
        roger/sexp/factory.go:35 +0xa0
    roger/sexp.parseVector(0x437840, 0xc820436960, 0xc820431484, 0x38, 0x38, 0x29, 0x38, 0x0, 0x0, 0x0, ...)
        roger/sexp/xt-vector.go:37 +0xba
    roger/sexp.parseReturningOffset(0xc820431484, 0x38, 0x38, 0x20, 0x0, 0x0, 0x8591d, 0x0, 0x0)
        roger/sexp/factory.go:80 +0x752
    roger/sexp.Parse(0xc820431484, 0x38, 0x38, 0x0, 0x0, 0x0, 0x0, 0x0)
        roger/sexp/factory.go:13 +0x61
    roger.(*packet).GetResultObject(0xc820436930, 0x0, 0x0, 0x0, 0x0)
        roger/packet.go:77 +0x31e
    
    opened by APILLSBURY 5
  • unsupported rserve version

    unsupported rserve version

    Hi,

    I recently upgraded roger to address https://github.com/senseyeio/roger/issues/23 but now when I try to connect to our Rserve daemon roger reports:

    The version of RServe installed is not officially supported. Please consider upgrading 
    to the latest version of RServe.
    

    In terms of the Rserve package version we're using a relatively recent install from cran.rstudio.com. sessionInfo() reports Rserve_1.7-3. I'm relatively new to R but is there some way to install a newer version? What version is required? We don't install a specific version of Rserve so I'm wondering why our recent install is behind Roger's minimum version?

    Thanks, brian

    opened by bfallik 7
Releases(0.3.2)
  • 0.3.2(Sep 4, 2018)

  • 0.3.1(Jul 13, 2017)

  • 0.3.0(Jun 20, 2017)

    • Added support for DT_LARGE responses, these are used for extremely large responses from R
    • Fixed bug where handshake issues were reported as incompatible RServe versions
    • Assignment capabilities no longer beta
    Source code(tar.gz)
    Source code(zip)
  • 0.2.0(Feb 8, 2016)

    • Added variable assignment (beta) (thanks @qingdimeng)
    • Fixed boolean array parsing in lists (thanks @harryrose)
    • Fixed long text response parsing
    • Improved list parsing
    • Added support for complex numbers and raw responses
    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Aug 24, 2015)

Owner
Senseye
Analytics for the Internet of Things
Senseye
Implements a simple floating point arithmetic expression evaluator in Go (golang).

evaler https://github.com/soniah/evaler Package evaler implements a simple floating point arithmetic expression evaluator. Evaler uses Dijkstra's Shun

Sonia Hamilton 50 Sep 27, 2022
Parses the Graphviz DOT language in golang

Parses the Graphviz DOT language and creates an interface, in golang, with which to easily create new and manipulate existing graphs which can be writ

Walter Schulze 504 Nov 23, 2022
A well tested and comprehensive Golang statistics library package with no dependencies.

Stats - Golang Statistics Package A well tested and comprehensive Golang statistics library / package / module with no dependencies. If you have any s

Montana Flynn 2.6k Dec 7, 2022
Clusterpedia-client - clusterpedia-client supports the use of native client-go mode to call the clusterpedia API

clusterpedia-client supports the use of native client-go mode to call the cluste

Calvin Chen 4 Jan 7, 2022
Client-go - Clusterpedia-client supports the use of native client-go mode to call the clusterpedia API

clusterpedia-client supports the use of native client-go mode to call the cluste

clusterpedia.io 9 Oct 25, 2022
Use Consul to do service discovery, use gRPC +kafka to do message produce and consume. Use redis to store result.

目录 gRPC/consul/kafka简介 gRPC+kafka的Demo gRPC+kafka整体示意图 限流器 基于redis计数器生成唯一ID kafka生产消费 kafka生产消费示意图 本文kafka生产消费过程 基于pprof的性能分析Demo 使用pprof统计CPU/HEAP数据的

null 43 Jul 9, 2022
A Go client implementing a client-side distributed consumer group client for Amazon Kinesis.

Kinesumer is a Go client implementing a client-side distributed consumer group client for Amazon Kinesis.

당근마켓 69 Nov 28, 2022
Distributed lock manager. Warning: very hard to use it properly. Not because it's broken, but because distributed systems are hard. If in doubt, do not use this.

What Dlock is a distributed lock manager [1]. It is designed after flock utility but for multiple machines. When client disconnects, all his locks are

Sergey Shepelev 25 Dec 24, 2019
A test repo to demonstrate the current (go1.17.2) issue when trying to use retractA test repo to demonstrate the current (go1.17.2) issue when trying to use retract

test-go-mod-retract This is a test repo to demonstrate the current (go1.17.2) issue when trying to use retract in go.mod to retract a version in a non

Yuxuan 'fishy' Wang 0 Oct 16, 2021
Blog-mongodb - this repository for educational purpose, learn how to use mongodb and use mongodb with go

ENDPOINT ENDPOINT METHOD ACCESS /register POST all /login POST all /articles GET all /articles POST all /articles/{articleId} GET all /articles/{artic

Muhammad Al Farizzi 0 Jan 4, 2022
A golang library about socks5, supports all socks5 commands. That Provides server and client and easy to use. Compatible with socks4 and socks4a.

socks5 This is a Golang implementation of the Socks5 protocol library. To see in this SOCKS Protocol Version 5. This library is also compatible with S

chenhao zhang 40 Nov 22, 2022
Client-server-golang-sqs - Client Server with SQS and golang

Client Server with SQS and golang Multi-threaded client-server demo with Go What

null 0 Feb 14, 2022
Simple and easy to use client for stock market, forex and crypto data from finnhub.io written in Go. Access real-time financial market data from 60+ stock exchanges, 10 forex brokers, and 15+ crypto exchanges

go-finnhub Simple and easy to use client for stock, forex and crpyto data from finnhub.io written in Go. Access real-time market data from 60+ stock e

Miles Croxford 77 Oct 14, 2022
An easy-to-use CLI client for RabbitMQ.

buneary, pronounced bun-ear-y, is an easy-to-use RabbitMQ command line client for managing exchanges, managing queues and publishing messages to exchanges.

Dominik Braun 56 Sep 3, 2022
Native ZooKeeper client for Go. This project is no longer maintained. Please use https://github.com/go-zookeeper/zk instead.

Native Go Zookeeper Client Library License 3-clause BSD. See LICENSE file. This Repository is No Longer Maintained Please use https://github.com/go-zo

Samuel Stauffer 1.6k Dec 2, 2022
GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right away.

GoBigdis GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right

Riccardo 5 Apr 27, 2022
Self-hostable , easy-to-use , lightweight and feature-rich torrent client written in Go

Self-hostable , easy-to-use , lightweight and feature-rich torrent client written in Go . It comes with beautiful Web UI and Optional Multi-User Support . Run Locally or Host in Server . Open/Download/Stream Torrents in Browser Right Away!

Var Bhat 1.5k Nov 28, 2022
An Easy to use Go framework for Kubernetes based on kubernetes/client-go

k8devel An Easy to use Go framework for Kubernetes based on kubernetes/client-go, see examples dir for a quick start. How to test it ? Download the mo

null 10 Mar 25, 2022
This bot require you to run the GETH client + use ethers framework.

Mad Liquidity Sniper This bot require you to run the GETH client + use ethers framework. All addresses and private keys contained have been changed fo

MAD CRYPTO 0 Oct 19, 2021
Use pingser to create client and server based on ICMP Protocol to send and receive custom message content.

pingser Use pingser to create client and server based on ICMP Protocol to send and receive custom message content. examples source code: ./examples Us

zznQ 13 Nov 9, 2022