Web-based Cloud Gaming service for Retro Game

Overview

CloudRetro

Build Latest release

Open-source Cloud Gaming Service For Retro Games
Video demo: https://www.youtube.com/watch?v=GUBrJGAxZZg
Technical wrapup: https://webrtchacks.com/open-source-cloud-gaming-with-webrtc/
CloudMorph: https://github.com/giongto35/cloud-morph: My current focus on generic solution for cloudgaming

Introduction

CloudRetro provides an open-source cloud gaming platform for retro games. It started as an experiment for testing cloud gaming performance with WebRTC and libretro, and now it aims to deliver the most modern and convenient gaming experience through the technology.

Theoretically, in cloud gaming, games are run on remote servers and media are streamed to the player optimally to ensure the most comfortable user interaction. It opens the ability to play any retro games on web-browser directly, which are fully compatible with multi-platform like Desktop, Android, IOS.

Announcement

(Currently, I'm working on CloudMorph: It offers more generic solution to run any offline games/application on browser in Cloud Gaming approach: https://github.com/giongto35/cloud-morph)

Try the service at

Single play: http://cloudretro.io
Direct play an existing game: Pokemon Emerald

*In ideal network condition and less resource contention on servers, the game will run smoothly as in the video demo. Because I only hosted the platform on limited servers in US East, US West, Eu, Singapore, you may experience some latency issues + connection problem. You can try hosting the service following the instruction the next section to have a better sense of performance.

Screenshot Screenshot
screenshot screenshot
screenshot screenshot

Feature

  1. Cloud gaming: Game logic and storage is hosted on cloud service. It reduces the cumbersome of game initialization. Images and audio are streamed to user in the most optimal way using advanced encoding technology.
  2. Cross-platform compatibility: The game is run on web browser, the most universal built-in app. No console, plugin, external app or devices are needed.
  3. Emulator agnostic: The game can be played directly without any extra effort to set up the gaming emulator or platform.
  4. Collaborate gameplay: Follow the idea of crowdplay(TwitchPlaysPokemon), multiple players can play the same game together by addressing the same deeplink. The game experience is powered by cloud-gaming, so the game is much smoother. Check CrowdPlay section
  5. Online multiplayer: The first time in history, you can play multiplayer on Retro games online. You can try Samurai Showndown with 2 players for fighting game example.
  6. Horizontally scaled: The infrastructure is designed to be able to scale under high traffic by adding more instances.
  7. Cloud storage: Game state is storing on online storage, so you can come back and continue playing your incomplete game later.

Development environment

Install Golang https://golang.org/doc/install . Because the project uses GoModule, so it requires Go1.11 version.

Install Dependencies

# Ubuntu / Windows (WSL2)
apt-get install -y make gcc pkg-config libvpx-dev libopus-dev libopusfile-dev libsdl2-dev

# MacOS
brew install libvpx pkg-config opus opusfile sdl2

# Windows (MSYS2)
pacman -Sy --noconfirm --needed git make mingw-w64-x86_64-{gcc,pkg-config,dlfcn,libvpx,opusfile,SDL2}

Because the coordinator and workers need to run simultaneously. Workers connect to the coordinator.

  1. Script
  • make dev.run
  • The scripts spawns 2 processes one in the background and one in foreground
  1. Manual
  • Need to run coordinator and worker separately in two session
  • go run cmd/coordinator/main.go - spawn coordinator
  • go run cmd/worker/main.go --coordinatorhost localhost:8000 - spawn workers connecting to coordinator

Additionally, you may install and configure an X Server display in order to be able to run OpenGL cores. See the docker-compose.yml file for Xvfb example config.

Run with Docker

Use makefile script: make dev.run-docker or Docker Compose directly: docker-compose up --build (CLOUD_GAME_GAMES_PATH is env variable for games on your host). It will spawn a docker environment and you can access the service on localhost:8000.

Configuration

The configuration parameters are stored in the configs/config.yaml file which is shared for all application instances on the same host system. It is possible to specify individual configuration files for each instance as well as override some parameters, for that purpose, please refer to the list of command-line options of the apps.

Technical Document

High level Worker internal
screenshot screenshot

FAQ

Crowd Play, play game together

By clicking these deep link, you can join the game directly and play it together with other people.

And you can host the new game by yourself by accessing cloudretro.io and click "share" button to generate a deeplink to your current game.


Synchronize a game session on multiple devices

Contribution

We are very much thankful to everyone who contributes to the project:

Credits

Art

Main Contributor

Sergey Stepanov
https://github.com/sergystepanov/

Author

Nguyen Huu Thanh
https://www.linkedin.com/in/huuthanhnguyen/

Tri Dang Minh
https://trich.im

Issues
  • How to get the game file?

    How to get the game file?

    I started a docker image, but there is no game. Please tell me how to get the game file.

    opened by lizhiqian-chandler 34
  • Add aspect ratio support

    Add aspect ratio support

    Added proper nearest neighbour up/downscaling support with aspect ratio calculation. Now it sets the initial viewport (output image) resolution to the base resolution from the loaded core upscaled by 2 to remove resampling artefacts (as on pictures below). So the manual resolution config for each core now removed completely. A Sony BIOS intro now is being shown properly. Need some testing.

    Before image

    After image

    opened by sergystepanov 21
  • WebRTC won't start in Firefox (Windows 10)

    WebRTC won't start in Firefox (Windows 10)

    Is it just me or the app still isn't working in Firefox? It says in the browser console: ICE failed, your TURN server appears to be broken, see about:webrtc for more details Is this suppose to work now without additional configuration?

    FF: 75.0 Windows

    I0427 23:52:40.728461    8564 handlers.go:161] Coordinator: A user is connecting...
    I0427 23:52:40.729464    8564 browser.go:35] Browser 5a74aa09-9b6d-4c7c-9bff-941b63c24e05] Generated worker ID
    I0427 23:52:40.731461    8564 browser.go:30] Browser 5a74aa09-9b6d-4c7c-9bff-941b63c24e05] Get Room  Zone  From URL /ws?room_id=&zone=
    I0427 23:52:40.731461    8564 browser.go:35] Browser 5a74aa09-9b6d-4c7c-9bff-941b63c24e05] Send sync[http://localhost:9000/echo]http://localhost:9000/echo
    I0427 23:52:40.755463    4088 worker.go:75] &{0xc000068aa0 0xc000574000 {} 0x4a3240 false false false false 0xc0000608c0 {0xc0000d0700 map[] false false} map[Access-Control-Allow-Origin:[*]] true 0 -1 0 false false [] 0 [0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] [0 0 0 0 0 0 0 0 0 0] [0 0 0] 0xc0004a0070 0} echo
    I0427 23:52:40.789460    8564 browser.go:35] Browser 5a74aa09-9b6d-4c7c-9bff-941b63c24e05] Latency mapmap[0xc000180480:41]
    I0427 23:52:40.826462    8564 browser.go:30] Browser 5a74aa09-9b6d-4c7c-9bff-941b63c24e05] Received initwebrtc request -> relay to worker: 6a1a220d-5c62-4f5b-a8b4-eea0c246d828
    I0427 23:52:40.828462    4088 coordinator.go:57] Received a request to createOffer from browser via coordinator
    I0427 23:52:40.828462    4088 webrtc.go:111] === StartClient ===
    I0427 23:52:40.933462    4088 webrtc.go:131] Add video track
    I0427 23:52:40.933462    4088 webrtc.go:142] Add audio track
    I0427 23:52:40.934460    4088 webrtc.go:200] Created Offer
    I0427 23:52:40.934460    4088 webrtc.go:181] OnIceCandidate: candidate:foundation 1 udp 2130706431 192.168.234.33 63191
    typ host
    I0427 23:52:40.935466    4088 coordinator.go:89] Start peerconnection 5a74aa09-9b6d-4c7c-9bff-941b63c24e05
    I0427 23:52:40.935466    8564 worker.go:41] Worker 6a1a220d-5c62-4f5b-a8b4-eea0c246d828] Received IceCandidate from worker -> relay to browser
    I0427 23:52:40.935466    4088 webrtc.go:181] OnIceCandidate: candidate:foundation 1 udp 2130706431 192.168.1.121 63192 typ host
    I0427 23:52:40.935466    8564 browser.go:35] Browser 5a74aa09-9b6d-4c7c-9bff-941b63c24e05] Received SDP from worker -> sending back to browser
    I0427 23:52:40.936462    4088 webrtc.go:181] OnIceCandidate: candidate:foundation 1 udp 2130706431 192.168.56.1 63193 typ host
    I0427 23:52:40.936462    8564 worker.go:41] Worker 6a1a220d-5c62-4f5b-a8b4-eea0c246d828] Received IceCandidate from worker -> relay to browser
    I0427 23:52:40.936462    4088 webrtc.go:181] OnIceCandidate: candidate:foundation 1 udp 1694498815 95.30.103.36 63194 typ srflx raddr 0.0.0.0 rport 63194
    I0427 23:52:40.937463    8564 worker.go:41] Worker 6a1a220d-5c62-4f5b-a8b4-eea0c246d828] Received IceCandidate from worker -> relay to browser
    I0427 23:52:40.938462    8564 worker.go:41] Worker 6a1a220d-5c62-4f5b-a8b4-eea0c246d828] Received IceCandidate from worker -> relay to browser
    I0427 23:52:40.939464    8564 worker.go:41] Worker 6a1a220d-5c62-4f5b-a8b4-eea0c246d828] Received IceCandidate from worker -> relay to browser
    I0427 23:52:40.975462    8564 browser.go:35] Browser 5a74aa09-9b6d-4c7c-9bff-941b63c24e05] Received browser answered SDP -> relay to worker
    I0427 23:52:40.982463    4088 coordinator.go:106] Received answer SDP from browser
    I0427 23:52:40.984465    4088 webrtc.go:233] Set Remote Description
    I0427 23:52:40.985462    4088 webrtc.go:165] ICE Connection State has changed: checking
    I0427 23:52:50.985852    4088 webrtc.go:165] ICE Connection State has changed: failed
    
    FF log
    (registry/INFO) insert 'ice' (registry) succeeded: ice
    (registry/INFO) insert 'ice.pref' (registry) succeeded: ice.pref
    (registry/INFO) insert 'ice.pref.type' (registry) succeeded: ice.pref.type
    (registry/INFO) insert 'ice.pref.type.srv_rflx' (UCHAR) succeeded: 0x64
    (registry/INFO) insert 'ice.pref.type.peer_rflx' (UCHAR) succeeded: 0x6e
    (registry/INFO) insert 'ice.pref.type.host' (UCHAR) succeeded: 0x7e
    (registry/INFO) insert 'ice.pref.type.relayed' (UCHAR) succeeded: 0x05
    (registry/INFO) insert 'ice.pref.type.srv_rflx_tcp' (UCHAR) succeeded: 0x63
    (registry/INFO) insert 'ice.pref.type.peer_rflx_tcp' (UCHAR) succeeded: 0x6d
    (registry/INFO) insert 'ice.pref.type.host_tcp' (UCHAR) succeeded: 0x7d
    (registry/INFO) insert 'ice.pref.type.relayed_tcp' (UCHAR) succeeded: 0x00
    (registry/INFO) insert 'stun' (registry) succeeded: stun
    (registry/INFO) insert 'stun.client' (registry) succeeded: stun.client
    (registry/INFO) insert 'stun.client.maximum_transmits' (UINT4) succeeded: 7
    (registry/INFO) insert 'ice.trickle_grace_period' (UINT4) succeeded: 5000
    (registry/INFO) insert 'ice.tcp' (registry) succeeded: ice.tcp
    (registry/INFO) insert 'ice.tcp.so_sock_count' (INT4) succeeded: 0
    (registry/INFO) insert 'ice.tcp.listen_backlog' (INT4) succeeded: 10
    (registry/INFO) insert 'ice.tcp.disable' (char) succeeded: \000
    (generic/ERR) UDP socket error:Internal error at Z:/task_1585933993/build/src/dom/network/UDPSocketParent.cpp:268 this=00000270B834C000
    (ice/ERR) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): failed to find default addresses
    (generic/ERR) UDP socket error:Internal error at Z:/task_1585933993/build/src/dom/network/UDPSocketParent.cpp:268 this=00000270B834E000
    (ice/ERR) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): failed to find default addresses
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_0 - 01f5805e:45385da4adf81024c3fcd93ae27bfec0
    (ice/INFO) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) no such component for candidate candidate:foundation 2 udp 2130706431 192.168.234.33 63191 typ host generation 0
    (ice/INFO) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) no such component for candidate candidate:foundation 2 udp 2130706431 192.168.1.121 63192 typ host generation 0
    (ice/INFO) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) no such component for candidate candidate:foundation 2 udp 2130706431 192.168.56.1 63193 typ host generation 0
    (ice/INFO) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) no such component for candidate candidate:foundation 2 udp 1694498815 95.30.103.36 63194 typ srflx raddr 0.0.0.0 rport 63194 generation 0
    (generic/ERR) UDP socket error:Internal error at Z:/task_1585933993/build/src/dom/network/UDPSocketParent.cpp:268 this=00000270B834E000
    (ice/ERR) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): failed to find default addresses
    (generic/ERR) UDP socket error:Internal error at Z:/task_1585933993/build/src/dom/network/UDPSocketParent.cpp:268 this=00000270B8350C00
    (ice/ERR) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): failed to find default addresses
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_0 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_3 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_4 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_0 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_3 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_4 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_0 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_3 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_4 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_0 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_3 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    (ice/WARNING) ICE(PC:1588020760802000 (id=10737418241 url=http://localhost:8000/)): peer (PC:1588020760802000 (id=10737418241 url=http://localhost:8000/):default) has no stream matching stream PC:1588020760802000 (id=10737418241 url=http://localhost:8000/) transport-id=transport_4 - 9d8a2bf7:e0e1a833f36d9e0fe0012984ec6dd249
    +++++++ END ++++++++
    
    SDP
    SDP
    Local SDP (Offer)
    v=0
    o=mozilla...THIS_IS_SDPARTA-75.0 2487803064638739471 1 IN IP4 0.0.0.0
    s=-
    t=0 0
    a=sendrecv
    a=fingerprint:sha-256 91:36:AF:63:EF:86:2D:1F:DB:C1:7E:E4:7A:C9:16:A4:AD:0F:D0:A5:4B:F1:08:65:36:3B:32:8A:19:B8:FD:58
    a=group:BUNDLE 0 1 2 3 4
    a=ice-options:trickle
    a=msid-semantic:WMS *
    m=video 9 UDP/TLS/RTP/SAVPF 96 98 126 97
    c=IN IP4 0.0.0.0
    a=recvonly
    a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
    a=extmap:4 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
    a=extmap:5 urn:ietf:params:rtp-hdrext:toffset
    a=extmap:6/recvonly http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
    a=fmtp:126 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1
    a=fmtp:97 profile-level-id=42e01f;level-asymmetry-allowed=1
    a=fmtp:96 max-fs=12288;max-fr=60
    a=fmtp:98 max-fs=12288;max-fr=60
    a=ice-pwd:e0e1a833f36d9e0fe0012984ec6dd249
    a=ice-ufrag:9d8a2bf7
    a=mid:0
    a=rtcp-fb:96 nack
    a=rtcp-fb:96 nack pli
    a=rtcp-fb:96 ccm fir
    a=rtcp-fb:96 goog-remb
    a=rtcp-fb:98 nack
    a=rtcp-fb:98 nack pli
    a=rtcp-fb:98 ccm fir
    a=rtcp-fb:98 goog-remb
    a=rtcp-fb:126 nack
    a=rtcp-fb:126 nack pli
    a=rtcp-fb:126 ccm fir
    a=rtcp-fb:126 goog-remb
    a=rtcp-fb:97 nack
    a=rtcp-fb:97 nack pli
    a=rtcp-fb:97 ccm fir
    a=rtcp-fb:97 goog-remb
    a=rtcp-mux
    a=rtpmap:96 VP8/90000
    a=rtpmap:98 VP9/90000
    a=rtpmap:126 H264/90000
    a=rtpmap:97 H264/90000
    a=setup:actpass
    a=ssrc:3368318818 cname:{569eb366-f0a8-49c2-ae9b-5093168654e2}
    m=audio 9 UDP/TLS/RTP/SAVPF 111 0 8 9 101
    c=IN IP4 0.0.0.0
    a=recvonly
    a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
    a=extmap:2/recvonly urn:ietf:params:rtp-hdrext:csrc-audio-level
    a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
    a=fmtp:111 maxplaybackrate=48000;stereo=1;useinbandfec=1
    a=fmtp:101 0-15
    a=ice-pwd:e0e1a833f36d9e0fe0012984ec6dd249
    a=ice-ufrag:9d8a2bf7
    a=mid:1
    a=rtcp-mux
    a=rtpmap:111 opus/48000/2
    a=rtpmap:0 PCMU/8000
    a=rtpmap:8 PCMA/8000
    a=rtpmap:9 G722/8000/1
    a=rtpmap:101 telephone-event/8000
    a=setup:actpass
    a=ssrc:2076023069 cname:{569eb366-f0a8-49c2-ae9b-5093168654e2}
    m=application 9 DTLS/SCTP 5000
    c=IN IP4 0.0.0.0
    a=sendrecv
    a=ice-pwd:e0e1a833f36d9e0fe0012984ec6dd249
    a=ice-ufrag:9d8a2bf7
    a=mid:2
    a=sctpmap:5000 webrtc-datachannel 256
    a=setup:actpass
    a=max-message-size:1073741823
    m=video 0 UDP/TLS/RTP/SAVPF 120 121 126 97
    c=IN IP4 0.0.0.0
    a=bundle-only
    a=sendrecv
    a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
    a=extmap:4 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
    a=extmap:5 urn:ietf:params:rtp-hdrext:toffset
    a=extmap:6/recvonly http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
    a=fmtp:126 profile-level-id=42e01f;level-asymmetry-allowed=1;packetization-mode=1
    a=fmtp:97 profile-level-id=42e01f;level-asymmetry-allowed=1
    a=fmtp:120 max-fs=12288;max-fr=60
    a=fmtp:121 max-fs=12288;max-fr=60
    a=ice-pwd:e0e1a833f36d9e0fe0012984ec6dd249
    a=ice-ufrag:9d8a2bf7
    a=mid:3
    a=msid:- {db5d210c-451c-4174-bbae-da94bd1c1626}
    a=rtcp-fb:120 nack
    a=rtcp-fb:120 nack pli
    a=rtcp-fb:120 ccm fir
    a=rtcp-fb:120 goog-remb
    a=rtcp-fb:121 nack
    a=rtcp-fb:121 nack pli
    a=rtcp-fb:121 ccm fir
    a=rtcp-fb:121 goog-remb
    a=rtcp-fb:126 nack
    a=rtcp-fb:126 nack pli
    a=rtcp-fb:126 ccm fir
    a=rtcp-fb:126 goog-remb
    a=rtcp-fb:97 nack
    a=rtcp-fb:97 nack pli
    a=rtcp-fb:97 ccm fir
    a=rtcp-fb:97 goog-remb
    a=rtcp-mux
    a=rtpmap:120 VP8/90000
    a=rtpmap:121 VP9/90000
    a=rtpmap:126 H264/90000
    a=rtpmap:97 H264/90000
    a=setup:actpass
    a=ssrc:1210881150 cname:{569eb366-f0a8-49c2-ae9b-5093168654e2}
    m=audio 0 UDP/TLS/RTP/SAVPF 109 9 0 8 101
    c=IN IP4 0.0.0.0
    a=bundle-only
    a=sendrecv
    a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
    a=extmap:2/recvonly urn:ietf:params:rtp-hdrext:csrc-audio-level
    a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:mid
    a=fmtp:109 maxplaybackrate=48000;stereo=1;useinbandfec=1
    a=fmtp:101 0-15
    a=ice-pwd:e0e1a833f36d9e0fe0012984ec6dd249
    a=ice-ufrag:9d8a2bf7
    a=mid:4
    a=msid:- {e0e11a51-926c-4e84-803f-8a1f46be31b7}
    a=rtcp-mux
    a=rtpmap:109 opus/48000/2
    a=rtpmap:9 G722/8000/1
    a=rtpmap:0 PCMU/8000
    a=rtpmap:8 PCMA/8000
    a=rtpmap:101 telephone-event/8000
    a=setup:actpass
    a=ssrc:1103474835 cname:{569eb366-f0a8-49c2-ae9b-5093168654e2}
    
    
    Remote SDP (Answer)
    v=0
    o=- 821757822 1588020760 IN IP4 0.0.0.0
    s=-
    t=0 0
    a=sendrecv
    a=fingerprint:sha-256 56:52:08:AD:85:F0:F2:04:BE:05:17:F8:9A:75:B5:65:31:1A:C1:B2:68:90:23:5F:AA:87:05:2F:DA:F9:3D:09
    a=group:BUNDLE 0 1 2
    m=video 9 UDP/TLS/RTP/SAVPF 96 98 102
    c=IN IP4 0.0.0.0
    a=candidate:foundation 1 udp 2130706431 192.168.234.33 63191 typ host generation 0
    a=candidate:foundation 2 udp 2130706431 192.168.234.33 63191 typ host generation 0
    a=candidate:foundation 1 udp 2130706431 192.168.1.121 63192 typ host generation 0
    a=candidate:foundation 2 udp 2130706431 192.168.1.121 63192 typ host generation 0
    a=candidate:foundation 1 udp 2130706431 192.168.56.1 63193 typ host generation 0
    a=candidate:foundation 2 udp 2130706431 192.168.56.1 63193 typ host generation 0
    a=candidate:foundation 1 udp 1694498815 95.30.103.36 63194 typ srflx raddr 0.0.0.0 rport 63194 generation 0
    a=candidate:foundation 2 udp 1694498815 95.30.103.36 63194 typ srflx raddr 0.0.0.0 rport 63194 generation 0
    a=candidate:foundation 1 udp 2130706431 192.168.234.33 63191 typ host
    a=candidate:foundation 1 udp 2130706431 192.168.1.121 63192 typ host
    a=candidate:foundation 1 udp 2130706431 192.168.56.1 63193 typ host
    a=candidate:foundation 1 udp 1694498815 95.30.103.36 63194 typ srflx raddr 0.0.0.0 rport 63194
    a=sendrecv
    a=end-of-candidates
    a=fmtp:102 profile-level-id=42001f;level-asymmetry-allowed=1;packetization-mode=1
    a=ice-pwd:vRStOTenOAAheuTypEDtRIewUaZaLmkQ
    a=ice-ufrag:vRStOTenOAAheuTy
    a=mid:0
    a=msid:game-video video
    a=rtcp-mux
    a=rtpmap:96 VP8/90000
    a=rtpmap:98 VP9/90000
    a=rtpmap:102 H264/90000
    a=setup:actpass
    a=ssrc:271976615 cname:game-video
    a=ssrc:271976615 msid:game-video video
    a=ssrc:271976615 mslabel:game-video
    a=ssrc:271976615 label:video
    m=audio 9 UDP/TLS/RTP/SAVPF 111 0 8 9
    c=IN IP4 0.0.0.0
    a=candidate:foundation 1 udp 2130706431 192.168.234.33 63191 typ host generation 0
    a=candidate:foundation 2 udp 2130706431 192.168.234.33 63191 typ host generation 0
    a=candidate:foundation 1 udp 2130706431 192.168.1.121 63192 typ host generation 0
    a=candidate:foundation 2 udp 2130706431 192.168.1.121 63192 typ host generation 0
    a=candidate:foundation 1 udp 2130706431 192.168.56.1 63193 typ host generation 0
    a=candidate:foundation 2 udp 2130706431 192.168.56.1 63193 typ host generation 0
    a=candidate:foundation 1 udp 1694498815 95.30.103.36 63194 typ srflx raddr 0.0.0.0 rport 63194 generation 0
    a=candidate:foundation 2 udp 1694498815 95.30.103.36 63194 typ srflx raddr 0.0.0.0 rport 63194 generation 0
    a=sendrecv
    a=end-of-candidates
    a=fmtp:111 maxplaybackrate=0;stereo=0;useinbandfec=1
    a=ice-pwd:vRStOTenOAAheuTypEDtRIewUaZaLmkQ
    a=ice-ufrag:vRStOTenOAAheuTy
    a=mid:1
    a=msid:game-audio audio
    a=rtcp-mux
    a=rtpmap:111 OPUS/48000/2
    a=rtpmap:0 PCMU/8000
    a=rtpmap:8 PCMA/8000
    a=rtpmap:9 G722/8000/1
    a=setup:actpass
    a=ssrc:1333919895 cname:game-audio
    a=ssrc:1333919895 msid:game-audio audio
    a=ssrc:1333919895 mslabel:game-audio
    a=ssrc:1333919895 label:audio
    m=application 9 DTLS/SCTP 5000
    c=IN IP4 0.0.0.0
    a=candidate:foundation 1 udp 2130706431 192.168.234.33 63191 typ host generation 0
    a=candidate:foundation 2 udp 2130706431 192.168.234.33 63191 typ host generation 0
    a=candidate:foundation 1 udp 2130706431 192.168.1.121 63192 typ host generation 0
    a=candidate:foundation 2 udp 2130706431 192.168.1.121 63192 typ host generation 0
    a=candidate:foundation 1 udp 2130706431 192.168.56.1 63193 typ host generation 0
    a=candidate:foundation 2 udp 2130706431 192.168.56.1 63193 typ host generation 0
    a=candidate:foundation 1 udp 1694498815 95.30.103.36 63194 typ srflx raddr 0.0.0.0 rport 63194 generation 0
    a=candidate:foundation 2 udp 1694498815 95.30.103.36 63194 typ srflx raddr 0.0.0.0 rport 63194 generation 0
    a=sendrecv
    a=end-of-candidates
    a=ice-pwd:vRStOTenOAAheuTypEDtRIewUaZaLmkQ
    a=ice-ufrag:vRStOTenOAAheuTy
    a=mid:2
    a=sctpmap:5000 webrtc-datachannel 1024
    a=setup:actpass
    
    opened by sergystepanov 19
  • YCbCr corrections

    YCbCr corrections

    Summary

    To try to correct conversion artifacts of YUV 4:2:0 images used in compressed video.

    Info

    In both video encoders we use YUV 4:2:0 encoded frames. YUV is a class of pixel formats used in video applications. YUV colors are represented with one "luminance" component called Y (equivalent to grey scale) and two "chrominance" components, called U (blue projection) and V (red projection) respectively. YUV can also be referred to as YCbCr, although the terms mean slightly different things.

    Planar formats Planar formats use separate matrices for each of the 3 color components. In other words, there is one table of luminance pixel values, and two separate tables for the chrominance components. This segregated representation in memory of pixels is more convenient for video coding.

    YUV 4:2:0 (I420/J420/YV12)

    I420 has the luma "luminance" plane Y first, then the U chroma plane and last the V chroma plane. The two chroma planes (blue and red projections) are sub-sampled in both the horizontal and vertical dimensions by a factor of 2. That is to say, for a 2×2 square of pixels, there are 4 Y samples but only 1 U sample and 1 V sample. This format requires 4×8+8+8=48 bits per 4 pixels, so its depth is 12 bits per pixel. I420 is by far the most common format.

    A graphical illustration: Each letter represents one bit.

    For a single I420 pixel: YYYYYYYY UU VV For a 50-pixel I420 frame: YYYYYYYY×50 UU×50 VV×50 (or Y×8×50 U×2×50 V×2×50 for short) For an n-pixel I420 frame: Y×8×n U×2×n V×2×n

    J420 J420 is exactly like I420, but with a full range ("digital", 0-255) luma (Y) component instead of limited range ("analog", 16-240). The chroma planes are exactly the same as in I420.

    YV12 YV12 is exactly like I420, but the order of the U and V planes is reversed. In the name, "YV" refers to the plane order: Y, then V (then U). "12" refers to the pixel depth: 12-bits per pixel as for I420.

    Color ranges BT.601 studio (TV), full (PC) swing

    Prior to the development of fast SIMD floating-point processors, most digital implementations of RGB → Y′UV used integer math, in particular fixed-point approximations. Approximation means that the precision of the used numbers is limited in favor of improved computation speeds.

    Y′ values are conventionally shifted and scaled to the range [16, 235] (referred to as studio swing or "TV levels") rather than using the full range of [0, 255] (referred to as full swing or "PC levels"). The value 235 accommodates a maximal black-to-white overshoot of 255 − 235 = 20, or 20 / (235 − 16) = 9.1%, which is slightly larger than the theoretical maximal overshoot (Gibbs phenomenon) of about 8.9% of the maximal step. The toe-room is smaller, allowing only 16 / 219 = 7.3% overshoot, which is less than the theoretical maximal overshoot of 8.9%. This is why 16 is added to Y′ and why the Y′ coefficients in the basic transform sum to 220 instead of 255. U and V values, which may be positive or negative, are summed with 128 to make them always positive, giving a studio range of 16–240 for U and V. (These ranges are important in video editing and production since using the wrong range will result either in an image with "clipped" blacks and whites, or a low-contrast image.)

    BT.601 (studio/full) swing. For getting the traditional "studio-swing" 8-bit representation of Y′UV for SDTV/BT.601 the following operations can be used:

    1. Basic transform from 8-bit RGB to 16-bit values (Y′: unsigned, U/V: signed, matrix values got rounded so that the later-on desired Y′ range of [16..235] and U/V range of [16..240] is reached):
    Y'  |  66 129  25 | R |
    U = | -38 -74 112 | G |
    V   | 112 -94 -18 | B |
    
    1. Scale down (">>8") to 8 bits with rounding ("+128") (Y′: unsigned, U/V: signed):
    Yt'= (Y'+ 128) >> 8
    Ut = (U + 128) >> 8
    Vt = (V + 128) >> 8
    
    1. Add an offset to the values to eliminate any negative values (all results are 8-bit unsigned):
    Yu'= Yt' + 16
    Uu = Ut + 128
    Vu = Vt + 128
    

    Full swing for BT.601. For getting a "full-swing" 8-bit representation of Y′UV for SDTV/BT.601 can be used with the same steps except for use of different weights on the first step.

    Our implementation uses studio swing integer math without SIMD. Generally speaking, full swing has the advantage of more picture information with less color banding or quantization errors but in our case, it's easier to stick with the studio range because VP8 codec doesn't support the full range (VP9 does) and we would have to implement both algorithms with some conditional switch on each codec. It is now accepted that SD content uses the BT.601 colorspace and HD uses BT.709.

    The problem

    We have YUV artifacts after RGB image conversion. So either YUV 4:2:0 converter is trash by itself or 4:2:0 is not meant for small images.

    RGB/YUV (edited) image comparison...

    image

    RGB/YUV raw image comparison...

    download (7) download (5)

    VP8 encoded clip...

    feature007

    Direct RGB/YUV UV (4:4:4/4:2:0) comparison...

    image image

    image image

    Solutions

    1. Since blue and red channels are very crippled it's possible just to upscale original frames by a factor of 2 or more.

    2. Rework the existing YUV-converter in a way that it could interpolate chrominance values more precisely. In the current algorithm, we calculate a single chroma value for all 2x2 RGB/luma values just based on the first pixel:

    XO  X   XO  X
    
    X   X   X   X
    

    This will lead to aliasing and color artifacts, instead, it is better to calculate a new chroma value for the average color between these 4 RGB pixels:

    X   X   X   X
      O       O
    X   X   X   X
    
    1. Try to use YUV 422 or 444 if codecs can support them. These formats should have fewer or no chrominance conversion artifacts but will reduce compression effectiveness as well as limit WebRTC client support.

    Implementation notes

    The first solution can be implemented just by changing one configuration param: https://github.com/giongto35/cloud-game/blob/90943f0cc4b916ccb959b20b32f755d231b64a24/configs/config.yaml#L64-L66 One disadvantage of this method is that change will scale output images for every emulator which could be heavy on the performance side.

    The second one is implemented in the new PR. Added optimized multithreaded YUV converter with 2x2 chroma color estimation. Try sqrt -- slow.

    The third solution seems to be hard to implement for both encoders because of difficulties in support of these image compression types.

    Literature

    001 / VideoLan Wiki (site) 002 / Wikipedia YCbCr (site)

    Tools used

    001 / yuvtoolkit (site) 002 / rawpixels.net (site)

    Originally posted by @sergystepanov in https://github.com/giongto35/cloud-game/issues/285#issuecomment-784655254

    opened by sergystepanov 19
  • Can't start any game

    Can't start any game

    I got some error when starting the server and can't start any game on localhost:8000.

    2019/09/10 10:30:56 Listening at port: localhost:8000
    2019/09/10 10:30:56 Running as worker
    2019/09/10 10:30:56 Warn: Failed to create client: dialing: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
    2019/09/10 10:30:56 Listening at port: localhost: 9000
    .....
    .....
    2019/09/10 10:31:04 === StartClient ===
    2019/09/10 10:31:04 Start peerconnection 1b9e405b-1c5f-4e67-84a8-26d76d84d7de
    2019/09/10 10:31:04 Error: Cannot create new webrtc session mDNS: failed to join multicast group
    2019/09/10 10:31:04 Overlord: Received sdp request from a worker
    2019/09/10 10:31:04 Overlord: Sending back sdp to browser
    Connected
    
    opened by namkazt 19
  • Initial Docker rewrite

    Initial Docker rewrite

    Related to #164.

    opened by sergystepanov 17
  • using cpu capabilities: none!   --- H264 performance issue

    using cpu capabilities: none! --- H264 performance issue

    hi, when I was using h264 to encode the video, I can see the video begin to stuck.

    the logs shows like this: I0202 16:25:33.807307 1926 media.go:134] Video Encoder: H264 x264 [info]: using cpu capabilities: none! x264 [info]: profile Constrained Baseline, level 5.0

    is the performence issue caused by this: using cpu capabilities: none!

    opened by hwwxj 16
  • Add initial support of portrait games

    Add initial support of portrait games

    See #158.

    Needs testing & code review.

    opened by sergystepanov 15
  • Use modified x264 lib

    Use modified x264 lib

    x264 info

    Settings

    A list of useful settings extracted from the source x264c.

    • profile Force the limits of an H.264 profile. Overrides all settings. (baseline, main, high, ...)
    for X264_CSP_I420 image encoding
    
       - baseline:
            --no-8x8dct --bframes 0 --no-cabac --cqm flat --weightp 0
            No interlaced.
            No lossless.
       - main:
            --no-8x8dct --cqm flat
            No lossless.
       - high:
            No lossless.
    
    • preset Use a preset to select encoding settings [medium]. Overridden by user settings. (ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow, placebo)
        - ultrafast:
            --no-8x8dct --aq-mode 0 --b-adapt 0 --bframes 0 --no-cabac --no-deblock
            --no-mbtree --me dia --no-mixed-refs --partitions none --rc-lookahead 0 --ref 1
            --scenecut 0 --subme 0 --trellis 0 --no-weightb --weightp 0
    
        - superfast:
            --no-mbtree --me dia --no-mixed-refs --partitions i8x8,i4x4 --rc-lookahead 0
            --ref 1 --subme 1 --trellis 0 --weightp 1
    
        - veryfast:
            --no-mixed-refs --rc-lookahead 10 --ref 1 --subme 2 --trellis 0 --weightp 1
    
        - faster:
            --no-mixed-refs --rc-lookahead 20 --ref 2 --subme 4 --weightp 1
    
        - fast:
            --rc-lookahead 30 --ref 2 --subme 6 --weightp 1
    
        - medium (Default settings apply)
     
        - slow:
            --direct auto --rc-lookahead 50 --ref 5 --subme 8 --trellis 2
    
        - slower:
            --b-adapt 2 --direct auto --me umh --partitions all --rc-lookahead 60 --ref 8 --subme 9 --trellis 2
    
        - veryslow:
            --b-adapt 2 --bframes 8 --direct auto --me umh --merange 24 --partitions all --ref 16 
            --subme 10 --trellis 2 --rc-lookahead 60
    
         - placebo:
            --bframes 16 --b-adapt 2 --direct auto --slow-firstpass --no-fast-pskip
            --me tesa --merange 24 --partitions all --rc-lookahead 60 --ref 16 --subme 11 --trellis 2
    
    • tune Tune the settings for a particular type of source or situation. Overridden by user settings. Multiple tunings are separated by commas. Only one psy tuning can be used at a time. (psy tunings: film, animation, grain, stillimage, psnr, ssim / other tunings: fastdecode, zerolatency)
        - film (psy tuning):
            --deblock -1:-1 --psy-rd <unset>:0.15
    
        - animation (psy tuning):
            --bframes {+2} --deblock 1:1 --psy-rd 0.4:<unset> --aq-strength 0.6 --ref {Double if >1 else 1}
    
        - grain (psy tuning):
            --aq-strength 0.5 --no-dct-decimate --deadzone-inter 6 --deadzone-intra 6
            --deblock -2:-2 --ipratio 1.1--pbratio 1.1 --psy-rd <unset>:0.25 --qcomp 0.8
    
        - stillimage (psy tuning):
            --aq-strength 1.2 --deblock -3:-3 --psy-rd 2.0:0.7 
    
        - psnr (psy tuning):
            --aq-mode 0 --no-psy
    
        - ssim (psy tuning):
            --aq-mode 2 --no-psy
    
        - fastdecode:
            --no-cabac --no-deblock --no-weightb --weightp 0
    
        - zerolatency:
            --bframes 0 --force-cfr --no-mbtree --sync-lookahead 0 --sliced-threads --rc-lookahead 0
    

    Changes

    • The old h264 encoder is rewritten in order to be able to make use of the external libx264 system library with SIMD instructions.
    • Huge memory leaks are fixed.
    • Added use of C YUV-converter.
    • Added ability to change video codec and some (h264) config params in the settings file.

    Issues

    • It is possible that libx264 support in the app will be broken again after another API (C header) change by its authors.
    • h264 encoder is harder to tweak for real-time encoding compared to more advanced and new vpx encoders.
    opened by sergystepanov 15
  • Fix errors/misuse with OpenGL-based core API

    Fix errors/misuse with OpenGL-based core API

    This PR is dedicated to fixing some errors and misuses in the OpenGL HW-accelerated Libretro cores renderer.

    How Libretro cores utilize OpenGL.

    If a Libretro core needs to use OpenGL, for example, to emulate 3D-accelerated games, in that case, your Libretro frontend has to own and share the OpenGL context with the core. The only job for the core is to render graphics for the next frame and put the final result into a shared FrameBufferObject (FBO) [3], all that done with a special render-to-texture OpenGL extension that should be supported by the video driver (OpenGL 3+). After that, your frontend can read the image from the FBO and show it in any convenient way.

    A simplified list of steps to deal with OpenGL Libretro cores:

    • Initialize the OpenGL context.
      • Make a window with the help of SDL. And we will need a window because OpenGL context initialization needs to gather data from the windowing system even though the rendering is going to be off-screen. (perhaps it is possible to reuse an existing window in the OS or even destroy new window right after context creation but this idea requires verification) ¯\_(ツ)_/¯
      • Using some OpenGL Loading Library (e.g. go-gl [Glow]), load pointers to OpenGL functions at runtime, core, and extensions.
      • Call the OpenGL init function and that will be enough.
      • Don't forget that before calling any GL functions in a thread, context should be made current (thread-local variable). That way each thread gonna have its own copy of the OpenGL context state to change and should flush these changes during your next context-make-current calls or will keep them inside the thread otherwise [4].
    • Initialize FrameBufferObject and a texture to draw on.
      • By default windowing system provide you with its own framebuffer but we need to create one explicitly in order to minimize data copy and increase the performance for core off-screen rendering.
      • To let OpenGL perform rendering into a texture, an image of a texture object must be attached to a framebuffer (GL_COLOR_ATTACHMENT).
      • Additionally, you can specify the depth and stencil components of the framebuffer (all three also are called attachments) if the core returns info about the usage of those.
      • Bind FBO's name to the current context for reading/writing operation allowance with GL_FRAMEBUFFER param (I'm not sure about the need for this step since it makes no sense here without any data to read, I yet to know how to properly bind/unbind that buffer between frontend/core, especially with an amazing ability of go-routines to jump between threads).
    • Notify the core that you are done (call context_reset callback).
    • Grab the image from the FBO in the video_refresh callback when it's ready.
      • Check the RETRO_HW_FRAMEBUFFER_VALID flag as an indicator for HW rendering.
      • Bind the framebuffer for reading op.
      • Read pixels from it (glReadPixels or glTexSubImage2D).

    Sources:

    1. Hans-Kristian Arntzen Implementing a Hardware Accelerated Libretro Core (pdf)
    2. Interactive Computer Graphics. 6.1 OpenGL Version, Context, and Profile (link)
    3. songho.ca OpenGL (link)
    4. WinAPI wglMakeCurrent (link)
    5. The Red Book (:<

    Issues:

    • SDL2 is a pretty bloated framework just for window creation, GLFW is much lighter
    • [x] to check OpenGL Cores Libretro doc
    Video driver should support render-to-texture extension (OpenGL 3.0+ Core).
    
    • [x] unnecessary OpenGL 4.1 Core profile use, it's better to follow 2.1 profile
    Switched to OpenGL 2.1 bindings.
    
    • [x] OpenGL coordinate system is flipped, use of slow and unnecessary third-party dependency just for the image vertical flip was unjustified

    https://github.com/giongto35/cloud-game/blob/a42aebec886f479be43ad33661c43077085b1c6b/pkg/emulator/libretro/nanoarch/nanoarch.go#L153-L166

    1. OpenGL's Lower Left Origin

    Given a sheet of paper, people write from the top of the page to the bottom. The origin for writing text is at the upper left-hand margin of the page (at least in European languages). However, if you were to ask any decent math student to plot a few points on an X-Y graph, the origin would certainly be at the lower left-hand corner of the graph. Most 2D rendering APIs mimic writers and use a 2D coordinate system where the origin is in the upper left-hand corner of the screen or window (at least by default). On the other hand, 3D rendering APIs adopt the mathematically minded convention and assume a lower left-hand origin for their 3D coordinate systems.

    If you are used to 2D graphics APIs, this difference of origin location can trip you up. When you specify 2D coordinates in OpenGL, they are generally based on a lower left-hand coordinate system. Keep this in mind when using glViewport, glScissor, glRasterPos2i, glBitmap, glTexCoord2f, glReadPixels, glCopyPixels, glCopyTexImage2D, glCopyTexSubImage2D, gluOrtho2D, and related routines.

    Another common pitfall related to 2D rendering APIs having an upper left-hand coordinate system is that 2D image file formats start the image at the top scan line, not the bottom scan line. OpenGL assumes images start at the bottom scan line by default. If you do need to flip an image when rendering, you can use glPixelZoom(1,-1) to flip the image in the Y direction. Note that you can also flip the image in the X direction. Figure 4 demonstrates using glPixelZoom to flip an image.

    Note that glPixelZoom only works when rasterizing image rectangles with glDrawPixels or glCopyPixels. It does not work with glBitmap or glReadPixels. Unfortunately, OpenGL does not provide an efficient way to read an image from the frame buffer into memory starting with the top scan line.

    Rewrite it either similarly to rotationFn aka 0-cost frame slice reader (+ combine with rotations somehow) or use GL matrix transform (which is slower):

    Libretro treats top-left as origin. OpenGL treats bottom-left as origin. To be compatible with the libretro model, top-left semantics are preserved. Rendering normally will cause the image to be flipped vertically. To avoid this, simply scale the final projection matrix by [1,− 1 , 1 ,1].

    Added a function that flips Y coordinate for each pixel after rotation and before RGBA image draw.
    
    • [x] Pixel formats can change from core to core, misused for textures and the frame buffer (BRG/RGB)
    Used corresponding OpenGL pixel formats and types based on core callback values.
    
    • FBO read speed can be increased with PBO.

    • [x] macOS "main thread problem" solution causes a crash with the native Windows build of the app (https://github.com/golang/go/wiki/LockOSThread)

    From go-sdl2 FAQ:

    Why does my program crash randomly or hang? Putting runtime.LockOSThread() at the start of your main() usually solves the problem (see SDL2 FAQ about multi-threading).

    UPDATE: Recent update added a call queue system where you can put thread-sensitive code and have it called synchronously on the same OS thread. See the render_queue or render_goroutines examples from https://github.com/veandco/go-sdl2-examples to see how it works.

    Added SDL/GL functions into main.thread wrapper. (rewrite later: remove mainthread lib 
    in favor of go-sdl2's included render-queue)
    

    image

    Seems that some problem with retro_load_game / retro_unload_game function in the worker.
    SDL/OpenGL deinitialization broken at least in Windows, a bunch of zombie? threads remain.
    Postponed until backporting normal test runners for Libretro cores.
    
    [won't fix] RetroArch with the same core has this issue. 
    
    • [x] to try to check strangely deterministic audio stutter with the native Windows build :( WebRTC audio jitter all over the place image
    A possible cause is in the Mesa drivers (slow software llvmpipe renderer).
    Disabling OpenGL drawing in the core removes audio stuttering.
    
    opened by sergystepanov 15
  • Change range UDP ports

    Change range UDP ports

    Hi, is there a way to restrict the range of udp ports used by the streaming?

    opened by silviacesari95 2
  • Test Google Colab

    Test Google Colab

    I have managed to get it to work in Google Colab, as far as Cloud-Game is running, but since it stays running, I can't run ngrok to make localhost public.

    image

    image

    Can you help me finish the cell?

    https://gist.github.com/ImanCol/2928cc0870efcca8cd0041cd1fa4975f

    Is there another way to access localhost?

    Google Colab: Distributor ID: Ubuntu Description: Ubuntu 18.04.5 LTS Release: 18.04 Codename: bionic

    opened by ImanCol 19
  • Kubernetes support

    Kubernetes support

    It is time to get your hands dirty and try to run this app as a k8s cluster.

    TBD...

    Goals

    • You are not Google
    • Easy deployment
    • Scaleability (scale-out, as well as downscale)
    • CDN

    Literature

    1. Orchestrating GPU-accelerated streaming apps using WebRTC (link)
    2. GPU-accelerated streaming using WebRTC (link)
    3. Designing Distributed Systems: Patterns and Paradigms for Scalable, Reliable Services. — Brendan Burns, 2018 (Amazon)
    4. Kubernetes in Action (1st Edition) — Marko Luksa, 2018 (Manning)
    Low priority 
    opened by sergystepanov 1
  • Create a CD pipeline to sync prod with master

    Create a CD pipeline to sync prod with master

    @sergystepanov I'm thinking to have CD pipeline to ensure master with prod, so it can ensure changes can be landed to master immediately as well. Is it possible with Github Action in this open-source environment while still keeping my digital ocean key safe

    opened by giongto35 4
Releases(v2.5.4)
  • v2.5.0(Apr 7, 2021)

    Experimental build with auto-generated executables for Linux, macOS, Windows.

    Changelog [:

    65dcaf7 : Enable release builds with v-tags (#310)
      Re-enable release builds push into Github releases page when new v-prefixed tags are pushed (i.e. v2.4.2)
    602b9ea : Make executables ready for static linking (#307)
      Disable CGO (C libs) for coordinator
      Remove unused code in the webrtc package
      Fix ST1005 "error strings should not be capitalized"
      Fix SA1015 Using time. Tick leaks the underlying ticker, consider using it only in endless functions, 
      tests, and the main package and use time.NewTicker here.
      Fix SA9004 Only the first constant in this group has an explicit type
      Fix S1000 Should use a simple channel send/receive instead of select with a single case
      Force dir creation/check for core downloader
      Update Makefile release script
      Disable ASLR for worker builds
      Remove WORKER_BUILD_PARAMS flag from the CI
      Don't force recompilation in release
      Add Debian packages installer script
      Build and test worker app without libopusfile
      Add opus wrapper without opusfile
      Make install.sh executable
      Add opus lib in Win builds
      Make insecure HTTPS requests when downloading libs
      Add ca-certificates for Docker build stage (Go mod download works only with SSL certificates installed)
      Move libx264 wrapper into the repo (that way we can get rid of a lot of not needed external Go dependencies)
      Update the dependencies
    d778a7e : Allow to disable default Pion interceptors (#306)
      For performance reasons.
    cbab5a1 : Add deprecation note for libx264 < v160 (#301)
      Deprecation due to libx264 version below v160 shouldn't work properly, and it's hard to support both v155, v160 in Golang.
    7ebd945 : Remove jQuery (#299)
      Remove jQuery
      Remove browser (vendor) prefixes for some CSS properties
      Use simple device orientation test
      Realign D-pad pointers
      Make GameBoy text unselectable
      Cleanup console.log
      Keep 90% size on mobile browsers due to gh-ribbon overlap
      Remove legacy `unselectable="on"` attributes
      Align UI buttons
      Remove not used UI elements
      Change Options button
      Don't show player change message when not in a game
      Add click/touch handler for circle-pad
    c7e67d5 : Use just one video/audio RTP codec (#298)
      Negate exact matches with multiple video codecs when a negotiated list of codecs will be stripped of partially matched ones.
      As an example, if we want to use the h264 codec along with vp8 then in some browsers we'll have the exact match on vp8 and 
      some partial matches on h264 (different browsers have different SDP Fmtp) which leads to the forced use of vp8 when the app 
      is streaming h264.
      As a workaround, we should add only one codec (vp8 or h264) into the list.
    f1613ea : Use identical core images for build/target stages in the main Docker file. (#296)
      It is possible that the official Go Docker image used in the build stage may produce an executable file dynamically linked 
      with some old libs that may not be present in the target OS image.
      Therefore use of identical Docker images is the better option.
    ccf082e : Set controller types explicitly (#295)
    a779353 : Reduce coordinator dependencies tree (#292)
      Reduce coordinator dependencies tree
      Parallelize build/check
      Update dependencies
    d5780ac : Fix Pion WebRTC codecs (#289)
      Since Pion's webrtc v3.0.13 and this fix if both peers have some fmtp set then codecs should match exactly.
      By default Pion defines a list of supported codecs with fmtp set but different browsers set their own fmtp values by default
      as well therefore they may not match.
      To fix this we can define our own list of codecs on the server with empty fmtp so it will match any client codecs.
      See: https://github.com/pion/webrtc/blob/e5c8c659caa915041dd6dbcaacbb85fe28733820/rtpcodec.go#L99
    f4c3a8c : Add new YUV converter with avg color calc (#287)
      Add new YUV converter with 2x2 pixel matrix YUV color estimation.
      The new YUV color converter, as opposed to the original one, uses more precise color calculations based on 
      four neighboring pixels' average color values which helps a great deal with image aliasing / shimmering artifacts.
      By default, it runs in the multithreaded mode with the game frames sliced between 2*(CPU cores) goroutines.
      In case if this estimation mode doesn't work as expected it is possible to switch back to the original mode.
      The default encoder is switched to x264 since it's faster now.
    fa28197 : Refactor and update VPX codec (#285)
      Refactor and update VPX codec
      Add VPX codec options support
      Add generic video encoding pipe
      Tweak in/out channels for the pipe
      Update YUV encoder
      Clean VP8 encoder
    4c883cf : Fix x264 build trash (#282)
      Check x264 lib version with some conditional hacks for versions < 159
    98ca6e7 : Skip normal ErrServerClosed errors
    0f54e0f : Fix copy-by-value of sync.Lock in the monitoring server struct
    bd701f1 : Use modified x264 lib (#275)
      Use modified x264 lib
      Add x264 system lib
      Set x264 version 155 for Debian (Buster)
      Add h264 config params
    b56069c : Fix stream freeze (#279)
      Fix D-pad key and overlay
      Add Video element error reporting
      Terminate clientside ws ping on the connection terminate
    c37c2c3 : Fix D-pad key and overlay
    92d0dd7 : Refactor input for safe concurrency (#278)
      Extract input stuff
      Move D-pad option into the main screen
      Add mutex lock on the whole input user sessions struct
    55208e5 : Update the Go version to 1.16 in the CI
    5de666d : Fix small bug, to not respond on Empty Packet
    c01ddc8 : Don't hide save/load after the help show
    7bc9661 : Add experimental core's SRAM save/load (#273)
      Add experimental core's SRAM save/load
      Clean room.go
      Update Save RAM states
      Update dependencies
      Extract emulator states into a separate module
      Google Cloud saves don't support save RAM persistence, but save state (.dat) files only.
      Add emulator storage path into the configuration file
      Verify offline storage path
    6584d82 : Disable PIE (ASLR) build mode for Windows (#272)
      Since Go version 1.15 ASLR security mode is enabled by default for Windows builds. 
      In order to make it work without random NPE crashes in the cores (for example in PSX games during load),
      you have to (re)compile every core with ASLR support or just disable this mode.
      See: https://github.com/golang/go/issues/35192
    816b210 : Client game screen configuration support (#269)
      Move game screen to a separate [stream] module
      Add a simple canvas video mirror for post-processing
      Add video mirroring to the options
    69bc3cb : Use new OPUS buffer every audio frame for less garbage
    76b7caf : Refactor OPUS encoder for quality and speed (#262)
      Refactor OPUS encoder for quality and speed.
      Use OPUS low delay mode:
      (https://www.opus-codec.org/docs/opus_api-.1.3/group__opus__ctlvalues.html#ga592232fb39db60c1369989c5c5d19a07)
      Slightly tweak linear resample function and disable it on 48kHz input
      Rewrite OPUS encoder as a struct with some internal buffer
      Use OPUS 192Kbps preset by default
      Add encoder callback function on a full buffer
      Reuse OPUS output buffer
      Add write limiter for samples
    f7152c7 : Fix control buttons width
    28f0409 : Use host network by default in docker-compose
    f05e149 : Remove unnecessary logging
    949f713 : Remove unnecessary rotation
    9df10f5 : Add explicit file close when reading game size
    328236e : Add explicit file close when reading game size
    70170a6 : Use embedded mutex for emulator structs
    fa46267 : Fix wording
    d141898 : Remove old game save files name separator
    646e0d6 : Change wording for worker rooms and sessions inside
    7984534 : Remove unused in workers ws upgrade function
    6b8cad6 : Clean room run check condition
    8a41c83 : Update GitHub CI build
    07b7479 : Update Docker configs
    072f7b5 : Remove NeoGeo from the default cores list
    6f4e27d : Optimize images
    924d6ad : Fix Docker run for bridged Docker network in Windows, macOS (#259) 
      Add port and IP config and env params for bridged Docker network in Windows, macOS
      - CLOUD_GAME_WEBRTC_ICEIPMAP -- replaces IP of any ICE candidates on the server
      - CLOUD_GAME_WEBRTC_ICEPORTS_MIN/MAX -- limits WebRTC random port range on the server
    a77985f : Add fallback repo for not found cores (#258)
      Add fallback repo for not found cores
    162c4b9 : Update README.md Add disclaimer about Docker.
    9c984aa : Add SSL certificates for HTTPS downloads
    348d853 : Use an interceptor for changing video timestamps (#255)
    fe1c5dc : Remove useless lock in the custom track sample struct
    44a8b24 : Show server modules info properly
    0898fb0 : Migrate to pion/webrtc v3 (#254)
      Add initial pion/webrtc v3 migration
      Use custom timestamps when sending video packets
    
    Source code(tar.gz)
    Source code(zip)
    cloud-game-v2.5.0-linux-x86_64.tar.gz(23.23 MB)
    cloud-game-v2.5.0-macos-x86_64.tar.gz(14.63 MB)
    cloud-game-v2.5.0-windows-x86_64.zip(15.32 MB)
  • v2.0.0(Sep 9, 2019)

    Integrate with libretro to be flexible on retro games. Nearest service finding. There is only single entrypoint: cloudretro.io. Don't need to pick region.

    Source code(tar.gz)
    Source code(zip)
chess package for go

chess Introduction chess is a set of go packages which provide common chess utilities such as move generation, turn management, checkmate detection, P

Logan Spears 295 Jul 3, 2021
🕹️ A basic gameboy emulator with terminal "Cloud Gaming" support

Gameboy.Live ??️ Gameboy.Live is a Gameboy emulator written in go for learning purposes. You can simply play Gameboy games on your desktop: Or, "Cloud

AaronLiu 4.2k Jul 26, 2021
A small fantasy game engine in WASM using GoLang

The GoLang Fantasy Engine (GoLF Engine) is a retro game engine. It draws inspiration from fantasy console projects like pico-8, tic-80, and pyxle. Like those projects it is designed to be a retro-feeling game creation/playing tool. Unlike those projects GoLF is more minimal in scope and only provides an API and a small set of tools to help you create your games. Tools like an image editor and code editor are not built in. Despite this minimalism creating games in GoLF is still easy and should still maintain the retro game feel.

Brandon Atkinson 75 Jun 7, 2021
This is a "simple" game server. Main functionalities are matching and establishing a connection between players

Game Server This is a "simple" game server. Main functionalities are matching and establishing a connection between players How to Run? run the server

eco 4 Jul 7, 2021
Snake game implemented in golang

little_pineapple(Snake game implemented in golang) 贪吃蛇golang实现 Snake game implemented in golang 数据结构:链表&数组 Data structures used: linked list&array 使用方

null 7 Aug 17, 2020
Snake game made in Go! 🐍

Snake This is a Terminal based snake game made by tristangoossens. Please star this repository to help my first big project grow! Documentation can be

Tristan Goossens 294 Jul 18, 2021
Scalable Distributed Game Server Engine with Hot Swapping in Golang

GoWorld Scalable Distributed Game Server Engine with Hot Reload in Golang Features Architecture Introduction Get GoWorld Manage GoWorld Servers Demos

Nan Lin 1.9k Jul 25, 2021
♛♔ Play chess against UCI engines in your terminal.

uchess ♛♔ Play chess in your terminal. Introduction uchess is an interactive terminal chess client designed to allow gameplay and move analysis in con

Travis Whitton 30 Jul 11, 2021
Terminal-based game engine for Go, built on top of Termbox

Termloop Termloop is a pure Go game engine for the terminal, built on top of the excellent Termbox. It provides a simple render loop for building game

Joel Auterson 1.2k Jul 18, 2021
Go 3D Game Engine

G3N - Go 3D Game Engine G3N (pronounced "gen") is an OpenGL 3D Game Engine written in Go. It can be used to write cross-platform Go applications that

G3N - Go 3D Game Engine Repositories 1.5k Jul 23, 2021
Rust game update notification

Rust Updater This repository is for update notifications from the rust game which helps server admins to get official game updates How to build Native

Aldiwildan 5 Jul 10, 2021
Engo is an open-source 2D game engine written in Go.

Engo A cross-platform game engine written in Go following an interpretation of the Entity Component System paradigm. Engo is currently compilable for

Engo 1.4k Jul 26, 2021
Dedicated Game Server Hosting and Scaling for Multiplayer Games on Kubernetes

Agones is a library for hosting, running and scaling dedicated game servers on Kubernetes. Agones, is derived from the Greek word agōn which roughly t

GoogleForGames 3.9k Jul 17, 2021
An open source re-implementation of Diablo 2

OpenDiablo2 Join us on Discord! Development Live stream Support us on Patreon We are also working on a toolset: https://github.com/OpenDiablo2/HellSpa

OpenDiablo2 9.6k Jul 26, 2021