A toy project to stream from a Remarkable2

Overview

goMarkableStream

I use this toy project to stream my remarkable 2 (firmware 2.5) on my laptop using the local wifi.

video/demo here

Quick start

You need ssh access to your remarkable

Download two files from the release page:

  • the server "Linux/Armv7" for your remarkable
  • the client for your laptop according to the couple OS/arch

or build it yourself if you have the go toolchain installed on your machine.

The server

Copy the server on the remarkable and start it.

scp goMarkableStreamServer.arm remarkable:
ssh remarkable './goMarkableStreamServer.arm /proc/$(pidof xochitl)/mem'

The client

Configuration

It is possible to tweak the configuration via environment variables:

Server

Env var Default Descri[ption
RK_SERVER_BIND_ADDR :2000 the TCP listen address
RK_FB_ADDRESS 4387048 the location of the pointer to the framebuffer in the xochitl process. Default works for firmware 2.5

Client

Env var Default Descri[ption
RK_CLIENT_BIND_ADDR :8080 the TCP listen address
RK_SERVER_ADDR remarkabke:2000 the address of the remarkable

How it works?

The server loop

  • The server gets the address of the framebuffer in the memory space of the xochitl
  • The server launches a "ticketing system" to avoid congestion. The ticketing system is a channel that gets an event every 200ms.
  • Then it exposes a gRPC function (with TLS and mutual authentication).
  • The gRPC function waits for a "ticket" on the channel, and then grabs the data from the framebuffer.
  • It packs the data into an image message encoded in protobuf and sends it to the consumer

The client loop

  • The client creates an MJPEG stream and serves it over HTTP on the provided address
  • The client dial server and sends its certificate, and add the compression header.
  • Then it triggers a goroutine to get the image in a for loop.
  • The image is then encoded into JPEG format and added to the MJPEG stream.

Security

The communication is using TLS. The client and the server owns an embedded certificate chain (with the CA). There are performing mutual authentication. A new certificate chain is generated per build. Therefore, if you want restrict the access to your server to your client only, you must rebuild the tool yourself.

Manual build

Note: you need go > 1.16beta to build the tool because of the embedding mechanism for the certificate.

To build the tool manually, the easiest way is to use goreleaser:

goreleaser --snapshot --skip-publish --rm-dist

To build the services manually:

go generate ./... # This generates the certificates
cd server && GOOS=linux GOARCH=arm GOARM=7 go build -o goStreamServer.arm
cd client && go build -o goStreamClient

Inside the remarkable

Most of the information on how to hack the remarkable 2 comes from the reStream project See #28 for more info. All I did was to plumb the information to suit my own need.

Here is the recap:

  • To get the remarkable version:
reMarkable: ~/ cat /usr/share/remarkable/update.conf
[General]
#REMARKABLE_RELEASE_APPID={98DA7DF2-4E3E-4744-9DE6-EC931886ABAB}
#SERVER=https://get-updates.cloud.remarkable.engineering/service/update2
#GROUP=Prod
#PLATFORM=reMarkable2
REMARKABLE_RELEASE_VERSION=2.5.0.27
  • To find the location of the framebuffer pointer:
strace xochitl

...
563 openat(AT_FDCWD, "/dev/fb0", O_RDWR)    = 5
564 ioctl(5, FBIOGET_FSCREENINFO, 0x7ee9d5f4) = 0
565 ioctl(5, FBIOGET_VSCREENINFO, 0x42f0ec) = 0
566 ioctl(5, FBIOPUT_VSCREENINFO, 0x42f0ec) = 0

Global framebuffer is located at 0x42f0ec-4 =0x42f0e8 (4387048 in decimal)

  • To extract a picture:
#!/bin/sh
pid=`pidof xochitl`
addr=`dd if=/proc/$pid/mem bs=1 count=4 skip=4387048  2>/dev/null | hexdump | awk '{print $3$2}'`
skipbytes=`printf "%d" $((16#$addr))`
dd if=/proc/$pid/mem bs=1 count=2628288 skip=$skipbytes > out.data

Note: 1404*1872 =2628288 is the size of the binary data to get

Acknowledgement

All the people in the reStream projet and specially @ddvk and @raisjn

Issues
  • White screen only despite connection established

    White screen only despite connection established

    Hi, wondering if something in 2.10 broke streaming. Here is what I got on a 2.10.3.379 rM2 :

    reMarkable: ~/ ./goMarkableStreamServer.arm 
    2021/12/04 15:56:07 listening on tcp :2000
    2021/12/04 15:56:37 New connection from:  192.168.0.129:48916
    

    and on desktop

    [email protected]:~/Downloads$ RK_SERVER_ADDR=remarkable2:2000 goMarkableClient_0.5.2_Linux_x86_64/goMarkableClient
    2021/12/04 16:56:37 listening on :8080
    2021/12/04 16:56:37 Connection established
    

    so it looks all fine but in the browser, a white screen. If I click on it, I can download the screenshot but then it's all white, empty to be precise as I end up with a transparent png.

    Might be related to https://github.com/rien/reStream/issues/84 hence my reMarkable breaking update suspicion.

    opened by Utopiah 7
  • Issue with running client script

    Issue with running client script

    Hi thanks for your work!

    I tried to run the instructions (I also use the shortcut remarkable for the ssh so it was straightforward). But when I run the client script I get the following error:

    > ssh remarkable './goMarkableStreamServer.arm $(pidof xochtil)'
    2021/04/29 10:42:10 Usage: ./goMarkableStreamServer.arm $(pidof xochitl)
    >  RK_SERVER_ADDR=remarkable:2000 ./goMarkableClient
    2021/04/29 12:42:22 listening on :8080, registered /video
    press enter to take screenshot -> 2021/04/29 12:42:22 Error when calling GetImage: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup remarkable on 127.0.0.53:53: server misbehaving"
    

    Am I missing something?

    Forgot to mention: Remarkable 2 - Version 2.6.2.75 Linux x64 - Ubuntu 20.04

    opened by theogf 3
  • not working after update 2.6.2.75

    not working after update 2.6.2.75

    Olivier, thank you for your code, made my work and my teaching a lot easier over zoom :-)

    A couple of days ago I updated my remarkable2 (firmware 2.6.2.75) and ... panic ! goMarkableStream is not working... on the client I get the following

    ./goMarkableClient listening on :8080, registered /video Error when calling GetImage: rpc error: code = Unknown desc = read /proc/218/mem: input/output error

    On the remarkable I tried this reMarkable: ~/ pid=pidof xochitl reMarkable: ~/ addr=dd if=/proc/$pid/mem bs=1 count=4 skip=4387048 2>/dev/null | hexdump | awk '{print $3$2}' reMarkable: ~/ skipbytes=printf "%d" $((16#$addr)) reMarkable: ~/ dd if=/proc/$pid/mem bs=1 count=2628288 skip=$skipbytes > out.data

    --> dd: /proc/218/mem: Input/output error and I get the message above...

    there are more remarkable2's in the family, all with the previous system -I told them to reject this update- and they all work fine with the same goMarkableStream installation. Is anyone having the same issue ?

    Thank you very much Joao Costeira

    opened by jpcosteira 3
  • client cannot start: input/output error

    client cannot start: input/output error

    After starting the client, I get

    # RK_SERVER_ADDR=10.11.99.1:2000 ./goMarkableClient
    2021/03/25 14:15:31 listening on :8080, registered /video
    2021/03/25 14:15:31 Error when calling GetImage: rpc error: code = Unknown desc = read /proc/3053/mem: input/output error
    

    On the server side, there is no more details

    2021/03/25 13:12:48 listening on tcp :2000
    2021/03/25 13:14:17 New connection from:  10.11.99.2:48004
    
    opened by sveatlo 3
  • Input/output error on rM1, Firmware 2.8.0.98

    Input/output error on rM1, Firmware 2.8.0.98

    Thanks for writing this software, it is exactly what I was looking for!

    Unfortunately I'm having trouble getting it to work with my rM1 on firmware 2.8.0.98:

    2021/06/22 14:11:55 Connection established
    2021/06/22 14:11:56 cannot grab picture rpc error: code = Unknown desc = read /proc/216/mem: input/output error
    2021/06/22 14:11:56 Connection established
    2021/06/22 14:11:56 cannot grab picture rpc error: code = Unknown desc = read /proc/216/mem: input/output error
    2021/06/22 14:11:56 Connection established
    2021/06/22 14:11:56 cannot grab picture rpc error: code = Unknown desc = read /proc/216/mem: input/output error
    2021/06/22 14:11:56 Connection established
    2021/06/22 14:11:56 cannot grab picture rpc error: code = Unknown desc = read /proc/216/mem: input/output error
    2021/06/22 14:11:56 Connection established
    2021/06/22 14:11:56 cannot grab picture rpc error: code = Unknown desc = read /proc/216/mem: input/output error
    

    and so on, forever. The server prints

    2021/06/22 12:11:57 New connection from:  10.11.99.4:37978
    2021/06/22 12:11:58 New connection from:  10.11.99.4:37980
    2021/06/22 12:11:58 New connection from:  10.11.99.4:37982
    2021/06/22 12:11:58 New connection from:  10.11.99.4:37984
    2021/06/22 12:11:58 New connection from:  10.11.99.4:37986
    

    also forever.

    opened by cobalamin 1
  • Single script for starting client and server

    Single script for starting client and server

    Pretty nice work. I really like it. I developed a small script which allows me to start the client and the server simply from the terminal by just issuing rmshare. It uses mpv for showing the video feed.

    Here is the script. Feel free to add it to your repo as well.

    https://gist.github.com/breiting/6cd7d55f829bcfc88d2c46eb3a089d14

    opened by breiting 1
  • address already in use

    address already in use

    The tool works perfectly (thanks a lot for it!). I'm just a bit confused by a message which I get when running the server:

    [...] listen tcp :2000: bind: address already in use

    My OS: ubuntu 18.04.

    opened by guhraman 1
  • Orientation

    Orientation

    This implements an endpoint to change the orientation.

    sending a GET request to /orientation?orientation=[landscape|portrait] should change the orientation.

    This will implement #24

    ex:

    $ curl http://localhost:9090/orientation\?orientation\=portrait
    $ curl http://localhost:9090/orientation\?orientation\=landscape
    
    opened by owulveryck 0
  • Features: orientation and screenshot

    Features: orientation and screenshot

    • Add a (not fully bullet proof) feature that automatically rotates the screen
    • Add a screenshot capability (with transparent background) to the client
    opened by owulveryck 0
  • Feature request: manual rotate

    Feature request: manual rotate

    For some reason the autorotate functionality doesn't work for me. Might be due to the ddvk binary patch of xochitl. A workaround might be a simple manual rotate, say pressing r in the streaming endpoint.

    opened by StaffanBetner 3
  • Improve the service file to run the server as a service on the tablet

    Improve the service file to run the server as a service on the tablet

    I made a script that triggers the server as a daemon on the tablet:

    https://github.com/owulveryck/goMarkableStream/blob/main/service/goMarkableStream.service

    I had to make an ugly sleep because the server needs to xochitl process to be ready and I could not figure out how to do it ?

    It needs an update (if you know how systemctl works, it would be helpful).

    Hacktoberfest 
    opened by owulveryck 0
  • Implement support for RM1

    Implement support for RM1

    As discussed in #19 it should be straightforward to implement a support for RM1.

    I will not do it as I do not have a RM1 for testing, but I will eventually provide some help if someone wants to do it.

    Hacktoberfest 
    opened by owulveryck 0
  • Button to start/stop service on remarkable

    Button to start/stop service on remarkable

    Thank you for your hard work. Do you think, would it be possible to have some button in remarkable to start or stop service? Or it has to be implemented on remarkable side? Thanks again

    opened by mvala 2
Releases(v0.6.4)
Owner
Olivier Wulveryck
Here I am a geek and a gopher. In real life, I am a consultant. It means that most of the contribution to the OSS projects is out of office hours.
Olivier Wulveryck
Toy gRPC Tunnel over CloudFlare (Proof of Concept)

gun You know what it means. Guide Server Go to your domain in CloudFlare. In "Network" tab, turn on gRPC.

Qv2ray Workgroup 141 Jun 20, 2022
Toy TLS certificate viewer

veilig Toy tls certificate viewer that I built because openssl s_client confuses me Source available at: https://github.com/noqqe/veilig/ Please repor

Florian Baumann 9 Aug 25, 2021
A toy MMO example built using Ebiten and WebRTC DataChannels (UDP)

Ebiten WebRTC Toy MMO ⚠️ This is a piece of incomplete hobby work and not robust. Please read the "Why does this project exist?" section. What is this

Jae Bentvelzen 13 Apr 18, 2022
A simple toy example for running Graphsync + libp2p.

graphsync-example Here we outline a simple toy example (main.go) where two local peers transfer Interplanetary Linked Data (IPLD) graphs using Graphsy

Myel 1 Dec 8, 2021
Toy application level encryption protocol

noodle A toy application level encryption module. A a high level, it provides co

null 1 Jan 3, 2022
A library to simplify writing applications using TCP sockets to stream protobuff messages

BuffStreams Streaming Protocol Buffers messages over TCP in Golang What is BuffStreams? BuffStreams is a set of abstraction over TCPConns for streamin

Sean Kelly 249 Jun 6, 2022
A lightweight stream processing library for Go

go-streams A lightweight stream processing library for Go. go-streams provides a simple and concise DSL to build data pipelines. Wiki In computing, a

Eugene R. 973 Jun 28, 2022
V3IO Frames ("Frames") is a Golang based remote data frames access (over gRPC or HTTP stream)

V3IO Frames ("Frames") is a multi-model open-source data-access library that provides a unified high-performance DataFrame API for working with different types of data sources (backends). The library was developed by Iguazio to simplify working with data in the Iguazio Data Science Platform ("the platform"), but it can be extended to support additional backend types.

null 23 Mar 3, 2022
Stream Camera based on TCP

streamera Term Project of Computer Networking streamera is a Stream Camera based on TCP, which contains client mode and server mode. Features Client M

null 4 Jun 11, 2022
Stream processing stuff for Go

GoStream Type safe Stream processing library inspired in the Java Streams API. Table of contents Requirements Usage examples Limitations Performance C

Mario Macias 44 Jun 16, 2022
Reflex stream client for redis streams

rredis A reflex stream client for a redis streams using the radix client implementation. It provides an API for inserting data into a stream and for c

null 0 Oct 5, 2021
Totem - A Go library that can turn a single gRPC stream into bidirectional unary gRPC servers

Totem is a Go library that can turn a single gRPC stream into bidirectional unar

Joe Kralicky 2 Jan 10, 2022
A simple Go server that broadcasts any data/stream

broadcast A simple Go server that broadcasts any data/stream usage data You can POST data. curl -X POST --data-binary "@111.png" localhost:9222/test.p

Zack 10 Feb 21, 2022
Reads MAWS formatted data and converts it into JSON output stream.

maws2json Usage examples Over serial line (stdin pipe) Lets assume that Vaisala weather station is connected via RS232 to USB serial dongle in /dev/tt

Sääsivu 0 Feb 6, 2022
Reads JSON object (stream) from file/stdin and routes it/them to GCP Pub/Sub topics.

json2pubsub Publish JSON object (stream) into GCP Pub/Sub topic based on a field value. Usage: json2pubsub --project=STRING <mapping> ... Reads JSON

Sääsivu 1 Feb 16, 2022
Broadcast-server - A simple Go server that broadcasts any data/stream

broadcast A simple Go server that broadcasts any data/stream usage data You can

Zack 50 Jun 24, 2022
A go module supply Java-Like generic stream programming (while do type check at runtime)

gostream A go module supplying Java-Like generic stream programming (while do type check at runtime) Using Get a Stream To get a Stream, using SliceSt

Sad_Dog 0 Jan 16, 2022
Library for directly interacting and controlling an Elgato Stream Deck on Linux.

Stream Deck Library for directly interacting and controlling an Elgato Stream Deck on Linux. This library is designed to take exclusive control over a

Matthew Penner 3 Jan 23, 2022
Rabbitio - Rabbit stream cipher package RFC 4503 for Go

rabbitio rabbitio is a rabbit stream cipher packge based on RFC 4503 for golang

Sina Ghaderi 7 Feb 8, 2022