A lightweight load balancer used to create big Selenium clusters

Overview

Go Grid Router

Build Status Coverage GoReport Release Docker Pulls

Go Grid Router (aka Ggr) is a lightweight active load balancer used to create scalable and highly-available Selenium clusters. Ggr Animation

Articles

Quick Start Guide

To use Go Grid Router do the following:

  1. Install Docker to host
  2. Create configuration directory:
$ mkdir -p /etc/grid-router/quota
  1. Create users.htpasswd file:
$ htpasswd -bc /etc/grid-router/users.htpasswd test test-password
  1. Start Selenium standalone server on port 4445:
$ java -jar selenium-server-standalone.jar -port 4445

You can also start Selenoid instead.

  1. Create quota file (use correct browser name and version):
$ cat /etc/grid-router/quota/test.xml
<qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru">
<browser name="firefox" defaultVersion="59.0">
    <version number="59.0">
        <region name="1">
            <host name="localhost" port="4445" count="1"/>
        </region>
    </version>
</browser>
</qa:browsers>

Note: file name should correspond to user name you added to htpasswd file. For user test we added on previous steps you should create test.xml.

  1. Start Ggr container:
# docker run -d --name ggr -v /etc/grid-router/:/etc/grid-router:ro --net host aerokube/ggr:latest-release
  1. Access Ggr on port 4444 in the same way you do for Selenium Hub but using the following url:
http://test:[email protected]:4444/wd/hub

Complete Guide & Build Instructions

Complete reference guide (including build instructions) can be found at: http://aerokube.com/ggr/latest/

Issues
  • Not able to set up GGR in mac with Cloud Service

    Not able to set up GGR in mac with Cloud Service

    @vania-pooh I have installed the docker/ selenoid and added cloud service. I have not changed the test.xml file name, can you help us to install in right way to execute the test in cloud service.

    Step 1 : Install docker Step 2 : $ mkdir -p /etc/grid-router/quota Step 3 : htpasswd -bc /etc/grid-router/users.htpasswd test test-password Step 4 : curl -s https://aerokube.com/cm/bash | bash Step 5 : ./cm selenoid start --vnc --port 4445 Step 6 : docker run -d --name ggr -v /etc/grid-router/:/etc/grid-router:ro --net host aerokube/ggr:latest-release

    Step 7 : http://test:[email protected]:4444/wd/hub

    /etc/grid-router/quota/test.xml <qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru"> <browser name="chrome" defaultVersion="70.0"> <version number="70.0"> <region name="1"> <host name="ondemand.saucelabs.com" port="443" count="1" username="saucelabs-id" password="saucelabs-password" scheme="https" /> </region> </version> </browser> </qa:browsers>

    opened by DeChrish 26
  • ggr docker start issue

    ggr docker start issue

    I have created selenium hub and node .. i have also created quotes directory as well as test.xml file in proper location .. also given 777 permissions .. but still below router creation failed.

    docker run -d --name ggr -v /etc/grid-router/:/etc/grid-router:ro --net host aerokube/ggr:latest-release

    [[email protected] quota]# docker run -d --name ggr -v /etc/grid-router/:/etc/grid-router:ro --net host aerokube/ggr:latest-release 524a34207a46657de84183fb94c5705e64bc5ce8d13524348d8a5b7794e37427 [[email protected] quota]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 524a34207a46 aerokube/ggr:latest-release "/usr/bin/ggr -lis..." 3 seconds ago Exited (1) 2

    [email protected] quota]# docker run -it aerokube/ggr:latest-release 2018/03/26 21:31:27 [-] [-] [INIT] [-] [-] [-] [-] [-] [-] [Users file is "/etc/grid-router/users.htpasswd"] 2018/03/26 21:31:27 [-] [-] [INIT] [-] [-] [-] [-] [-] [-] [Users file "/etc/grid-router/users.htpasswd" does not exist]

    opened by narayannewworld 25
  • Issue in setting up GGR on windows machine.

    Issue in setting up GGR on windows machine.

    I have done the setup with docker +GGR facing issue while invoking following url. http://test:[email protected]:4444/wd/hub Result : localhost refused to connect. Here is the command to start ggr container docker run -d -it --name ggr --mount src=%CD%,target=/etc/grid-router,type=bind aerokube/ggr -listen :4444

    stale 
    opened by tbhatt-conga 22
  • 404 page not found

    404 page not found

    I followed all the steps to setup grid router . When I hit http://test:[email protected]:4444/wd/hub , I get 404 page not found , is this expected or will it work only with selenium tests? When I Run with selenium tests I get ,

    [Stream 1] Connecting to selenium grid: http://test:[email protected]:4444/wd/hub [Stream 1] Desired capabilities: Capabilities [{browserName=chrome, version=, platform=ANY}] [Stream 1] org.openqa.selenium.remote.UnreachableBrowserException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure. [Stream 1] Build info: version: '3.4.0', revision: 'unknown', time: 'unknown' [Stream 1] System info: host: 'mave-host', ip: '10.116.31.40', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-514.21.2.el7.x86_64', java.version: '1.8.0_121' [Stream 1] Driver info: driver.version: RemoteWebDriver [Stream 1] at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:658)

    opened by kvkchaitanya88 21
  • Hub selection logic algorithm not correct?

    Hub selection logic algorithm not correct?

    Hi

    First off all thanks for the create job!. We can now successful use selenium testcases for loadtesting. I don't know if this is the correct way to submit an issue/question. If not please let me know.

    The issue: We have configured 10 hubs in the quota xml file. These 10 hubs can build max 5 nodes. They are configured for the same region using the same chrome browser version. In the quota file we did configure the count="5" for all hubs.

    When we start 40 parallel testcases against the Go Gridrouter we would expect the router to divide/allocate the 40 testcases over the hubs available in the quota xml file. We have set the count="5" for all 10 hubs. So we would expect if there are enough hubs there could not be more then 5 parallel session per hub.

    What we now see is that the Go Gridrouter divide/allocate more then 5 testcases per hub. Some hubs are getting 10, others are getting 3 etc. In this way we cant guarantee for example 40 concurrent / parallel testcases.

    We have checked the hub selection logic: 2. Randomly selects a host from all hosts and tries to obtain browser on that host. Our algorithm also considers browser counts specified in XML for each host so that hosts with more browsers get more connections.

    Is this a bug or as designed? Can we make this work?

    Thanks in advanced Best Regards Michel Langras

    opened by langrasm 20
  • ggr docker start issue in windows 10

    ggr docker start issue in windows 10

    Hi, I am very new to docker with selenoid. I am trying to use selenoid instead of selenium grid. I am trying to configuring ggr(go grid router). Can you please anyone explain about it. also, I have seen some ggr configuration in GitHub. but I am unable to find the exact setup. facing some issues while creating multiple selenoid hubs. can you please anybody help me. GGR setup. for load balancing. using multiple selenoid hubs. and also we tried couple of video also which was given what are the links you shared to me we alredy gone through it. But i am unable to find exact setup.While configuring GGR setup i mean the way of using this feature. its not clear for me. Can you please , if you have any example base setup. its very helpful . Followed steps in power shell windows 10:- 1.Install Docker to host 2.mkdir -p /etc/grid-router/quota 3. C:\Windows\system32> htpasswd -bc /etc/grid-router/users.htpasswd test test-password Adding password for user test 4.C:\etc\grid-router\quota> docker run -d --name ggr -v /etc/grid-router/:/etc/grid-router:rw --net host aerokube/ggr:l atest-release bc5af4b44631e500c9f5115d27c77192d724b0f14bc3f8b738060b22607b491c logs:- 2020/03/16 07:33:15 [-] [-] [INIT] [-] [-] [-] [-] [-] [-] [Users file is "/etc/grid-router/users.htpasswd"]

    2020/03/16 07:33:15 [-] [-] [INIT] [-] [-] [-] [-] [-] [-] [Users file "/etc/grid-router/users.htpasswd" does not exist or is a directory]

    can you please help me

    stale 
    opened by Revanth2020 19
  • GGR Configuration Issue in windows

    GGR Configuration Issue in windows

    Hi,

    I am very new to docker with selenoid. I am trying to use selenoid instead of selenium grid.

    I am trying to configuring ggr(go grid router). Can you please anyone explain about it.

    also, I have seen some ggr configuration in GitHub. but I am unable to find the exact setup. facing some issues while creating multiple selenoid hubs. can you please anybody help me. GGR setup. for load balancing. using multiple selenoid hubs.

    and also we tried couple of video also which was given

    what are the links you shared to me we alredy gone through it. But i am unable to find exact setup.While configuring GGR setup i mean the way of using this feature. its not clear for me. Can you please , if you have any example base setup. its very helpful .

    opened by ramqtp92 19
  • Unable to ping ggr from host machine

    Unable to ping ggr from host machine

    First off, thank you for all the hard work on ggr/selenoid/-ui.

    My setup: Host Machine: Windows Server 2012 Hypervisor: Hyper-V running Ubuntu 16.04 LTS Docker (client/server) Version: 1.12.0, API Version: 1.24, Go version: go1.6.3

    I have no issues executing tests on selenoid and viewing them on selenoid-ui from my host machine.

    Followed all the instructions to create quota directory and create users.htpasswd file etc.

    To make sure it's working properly, I only started the ggr container using:

    docker run -d --name ggr -v /etc/grid-router/:/etc/grid-router:ro --net host aerokube/ggr:latest-release

    When I try to ping it from inside of ubuntu, everything works fine: {"uptime":"16.624757336s","lastReloadTime":"2018-05-15 18:23:18.079808378 +0000 UTC m=+0.001698442","numRequests":0,"version":"1.5.3"}

    Issue is when I try to ping it from my host machine: ERR_CONNECTION_TIMED_OUT..

    I can access selenoid/selenoid-ui / run tests without any issues from my host machine, but only having issues when trying to ping/access ggr. I can see the container status running without any issues either.

    I start the selenoid container and captured the netstat info to compare vs when running only ggr on using: docker run -d --name selenoid
    -p 4444:4444 \ (just fyi, when running it with ggr, I tried -p 4445:4444) -v /etc/selenoid:/etc/selenoid:ro
    -v /var/run/docker.sock:/var/run/docker.sock
    aerokube/selenoid:1.6.0
    -conf /etc/selenoid/browsers.json -limit 16

    netstat info when only running ggr: https://pasteboard.co/HlkAjFK.jpg

    netstat info when only running selenoid: https://pasteboard.co/HlkAvVm.jpg

    any additional info please advise

    opened by ed84 16
  • org.openqa.selenium.WebDriverException: cannot create session chrome-80.0 on any hosts after 2 attempt(s)

    org.openqa.selenium.WebDriverException: cannot create session chrome-80.0 on any hosts after 2 attempt(s)

    I'm trying to create a selenium cluster using ggr.

    It's throwing below error: org.openqa.selenium.WebDriverException: cannot create session chrome-80.0 on any hosts after 2 attempt(s)

    Will get same error when I set browse name = firefox& version 74.0: can not create session firefox-74.0

    Test code:- (DesiredCapabilities capabilities = new DesiredCapabilities(); capabilities.setBrowserName("chrome"); capabilities.setVersion("80.0"); capabilities.setCapability("enableVNC", true); capabilities.setCapability("enableVideo", false);

        RemoteWebDriver driver = new RemoteWebDriver(
    	    URI.create("http://test:[email protected]:4444/wd/hub").toURL(), 
    	    capabilities
    	);)
    

    ggr ec2 instance :- image

    when is use ping : image

    quota loaded by ggr

    image

    HUB -1 ec2 instance image]

    HUB-2 ec2 instance image

    test.xml file image

    I have created password using this command:- $ htpasswd -bc /etc/grid-router/users.htpasswd test test-password

    Please let me know what I'm doing wrong.

    When I try directly to connect to selenoid hub. it works fine

    (Test code ) --> (HUB-1)Selenoid hub ---> Works

    (Test code)-- ggr(HUB-1,HUB-2) ---->(HUB-1/HUB-2) ---> not works

    stale 
    opened by ankur1233 13
  • Error with creating webdriver instance while using GGR with external selenium service

    Error with creating webdriver instance while using GGR with external selenium service

    Trying a use case to run this GGR in a RHEL box with instructions mentioned in Dockerfile.

    Do I really need to install GO lang as a prerequisite to run the command:

    ["/usr/bin/ggr", "-listen", ":4444", "-users", "/etc/grid-router/users.htpasswd", "-quotaDir", "/etc/grid-router/quota"]

    stale 
    opened by harshagv 13
  • Balancer only works with two hubs

    Balancer only works with two hubs

    MY ENV

    5cbfd8521449        aerokube/ggr:latest        "/usr/bin/ggr -lis..."   33 minutes ago      Up 33 minutes                                ggr
    e60db9cb01c9        aerokube/selenoid:latest   "/usr/bin/selenoid..."   About an hour ago   Up About an hour    0.0.0.0:4456->4444/tcp   selenoid2
    67177dac9cd7        aerokube/selenoid:latest   "/usr/bin/selenoid..."   About an hour ago   Up About an hour    0.0.0.0:4455->4444/tcp   selenoid1
    2e104a5cfe62        aerokube/selenoid:latest   "/usr/bin/selenoid..."   About an hour ago   Up About an hour    0.0.0.0:4454->4444/tcp   selenoid
    
    
    <qa:browsers xmlns:qa="urn:config.gridrouter.qatools.ru">
    <browser name="firefox" defaultVersion="52.0">
        <version number="52.0">
            <region name="1">
                <host name="localhost" port="4454" count="1"/>
                <host name="localhost" port="4455" count="1"/>
                <host name="localhost" port="4456" count="1"/>
        </version>
    </browser>
    </qa:browsers>
    
    

    When I run the tests, I see that only 2 hubs work from 3. Why does this happen?

    opened by MissiaL 13
  • Add auth headers to vnc connections

    Add auth headers to vnc connections

    Right now usernames, passwords, and wss urls are ignored in the quota file when connecting to VNC. I've added a couple of methods that will add an authorization header and use WSS when required.

    new-feature 
    opened by regbo 2
  • 401 Unauthorized on GGR

    401 Unauthorized on GGR

    I am facing the exact same issue as #176, but it could not be resolved using the same fix. I get a 401 Unauthorized, when running the test: "http://test:[email protected]<ip_address>:4444/wd/hub" as you can see my password does not contain any special characters.

    if I enter http://test:[email protected]:4444/quota on the browser , it still prompts me for a username and password. If I enter the same credentials, it still gives me a '401 Unauthorized"

    there are no messages in the ggr container log. All I have is: 2019/11/20 12:37:56 [-] [-] [INIT] [-] [-] [-] [-] [-] [-] [Users file is "/etc/grid-router/users.htpasswd"] 2019/11/20 12:37:56 [-] [-] [INIT] [-] [-] [-] [-] [-] [-] [Loading configuration files from "/etc/grid-router/quota"]

    My quota file is /etc/grid-router/quota/test.xml

    The UI URL at port 8080 correctly shows this ggr linked to all the selenoid hosts, and the total quota count.

    I have also set up the user and password correctly using htpasswd

    bug 
    opened by upenbendre 12
  • Capability to explicitly select region from quota

    Capability to explicitly select region from quota

    We have internal and external hubs which can handle the same browsers/devices in different regions (ff: 45, regions: us and europe). It would be great to have possibility to handle requests via single quota for multi regions using extra custom capabilities like "region". For now we have to organize it using several quota files and decide on the pipeline level which quota to use (us or eu).

    Attached quota seems rather stable but we can't request exact region

    quota.txt

    new-feature 
    opened by vdelendik 1
  • GGR interrupts the session if there are extensions in capabilities

    GGR interrupts the session if there are extensions in capabilities

    preconditions: java, selenium, desireCapabilities with extensions

    when I create session through RemoteWebDriver, I get error:

    org.openqa.selenium.WebDriverException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.

    there is nothing in grr logs

    problem is not observed for guest quota, only auth connection

    bug 
    opened by azapevalov 8
Releases(1.6.11)
Owner
Aerokube
Selenium and Playwright testing infrastructure
Aerokube
This is just a repository to play around with Generics and Fuzzing, two big things coming with go 1.18!

This is just a repository to play around with Generics and Fuzzing, two big things coming with go 1.18!

Christopher Smith 0 Feb 6, 2022
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

Fortio Fortio (Φορτίο) started as, and is, Istio's load testing tool and now graduated to be its own project. Fortio is also used by, among others, Me

Fortio (Φορτίο) 2.6k Aug 9, 2022
Cloud Spanner load generator to load test your application and pre-warm the database before launch

GCSB GCSB Quickstart Create a test table Load data into table Run a load test Operations Load Single table load Multiple table load Loading into inter

Cloud Spanner Ecosystem 21 Jul 13, 2022
Check-load - Simple cross-platform load average check

Sensu load average check Table of Contents Overview Usage examples Configuration

KOHMURA Jin 0 Jun 16, 2022
HTTP load testing tool and library. It's over 9000!

Vegeta Vegeta is a versatile HTTP load testing tool built out of a need to drill HTTP services with a constant request rate. It can be used both as a

Tomás Senart 20k Aug 9, 2022
HTTP load generator, ApacheBench (ab) replacement, formerly known as rakyll/boom

hey is a tiny program that sends some load to a web application. hey was originally called boom and was influenced from Tarek Ziade's tool at tarekzia

Jaana Dogan 14k Aug 5, 2022
Ddosify - High-performance load testing tool

Ddosify - High-performance load testing tool Features ✔️ Protocol Agnostic - Currently supporting HTTP, HTTPS, HTTP/2. Other protocols are on the way.

Ddosify 4.7k Aug 6, 2022
Load generator for measuring overhead generated by EDRs and other logging tools on Linux

Simple load generator for stress-testing EDR software The purpose of this tool is to measure CPU overhead incurred by having active or passive securit

Hilko Bengen 6 Jun 14, 2022
Simple Golang Load testing app built on top of vegeta

LOVE AND WAR : Give Your App Love By Unleashing War Simple load testing app to test your http services Installation Build docker image: docker build -

Raymond Gitonga 1 Oct 26, 2021
Trade Matching / Transaction System Load Testing Solution

Load Generation System for Trade Matching Systems Operation Users select one of the following options from thew Test Management Portal: Generate a new

Traiano Welcome 1 Feb 25, 2022
Hive-fleet: a distributed, scalable load-testing tool built in go that leverages Google Cloud Functions

hive-fleet hive-fleet is a distributed, scalable load-testing tool, built on top

Cristian Glavan 5 Jan 27, 2022
:exclamation:Basic Assertion Library used along side native go testing, with building blocks for custom assertions

Package assert Package assert is a Basic Assertion library used along side native go testing Installation Use go get. go get github.com/go-playground/

Go Playgound 41 Aug 4, 2022
Quick and easy expression matching for JSON schemas used in requests and responses

schema schema makes it easier to check if map/array structures match a certain schema. Great for testing JSON API's or validating the format of incomi

Jaap Groeneveld 17 May 12, 2022
A simple `fs.FS` implementation to be used inside tests.

testfs A simple fs.FS which is contained in a test (using testing.TB's TempDir()) and with a few helper methods. PS: This lib only works on Go 1.16+.

Carlos Alexandro Becker 31 Mar 3, 2022
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

Re 264 Aug 9, 2022
Vault mock - Mock of Hashicorp Vault used for unit testing

vault_mock Mock of Hashicorp Vault used for unit testing Notice This is a person

Elliot Rotenstein 0 Jan 19, 2022
Terminal application used for API testing

Easily create, manage and execute http requests from the terminal.

PierreKieffer 60 Jul 18, 2022
Lightweight service virtualization/API simulation tool for developers and testers

API simulations for development and testing Hoverfly is a lightweight, open source API simulation tool. Using Hoverfly, you can create realistic simul

null 1.9k Aug 6, 2022
Lightweight selfhosted Firefox Send alternative without public upload

Gokapi Available for: Bare Metal Docker About Gokapi is a lightweight server to share files, which expire after a set amount of downloads or days. It

Marc Ole Bulling 329 Aug 6, 2022