Cloud-Native A/B Testing Platform (WIP) 云原生的 A/B 测试平台

Overview

Project cloudab

云原生的 A/B 测试系统

介绍

A/B 测试起源于农业工程。人们将土地划分为不同的地块,通过种植不同的农作物来确定在这些土地上更适合种植何种作物。随后 A/B 测试被广泛地应用于医学、工业等不同领域。随着谷歌论文 Overlapping Experiment Infrastructure: More, Better, Faster Experimentation 的发表,将 A/B 测试引入了互联网领域。

A/B实验通过在线上流量中取出一部分,完全随机地分给不同的策略,再结合一定的统计方法,得到对于两种策略相对效果的准确估计。这一估计的结果可以一定程度上反映出不同的策略的优劣好坏。

随着泛互联网行业竞争的加剧,过去产品经理决策驱动的策略不再能够在充分竞争的市场中取得优势。通过 A/B 测试,不同的经营策略,不同的推荐算法之间的优劣都可以通过精确估计的方式予以量化。数据比产品经理主观的意愿更具有说服力。通过数据驱动的方式,能够以更加科学的方式进行决策。

编译

make

运行

本地运行

$ mongod &
$ ./bin/cloudab --logger-debug

Docker 运行(TODO)

测试

可以通过导入 Postman collection 文件或者使用 cURL 进行测试。

Postman

可以导入 cloudab.postman_collection.json 进行测试。

cURL

创建一个新的实验:

curl --location --request POST 'http://localhost:9999/apis/v1/experiments' \
--header 'Content-Type: application/json' \
--data-raw '{
    "name": "test",
    "versions": [
        {
            "name": "version-1",
            "traffic": 100,
            "features": [
                {
                    "name": "feature-1",
                    "value": "value-1"
                }
            ]
        }
    ]
}'

返回如下:

{
    "id": "602ca4df9f4e8f5088966f6c",
    "name": "test",
    "versions": [
        {
            "id": "602ca4df9f4e8f5088966f6b",
            "name": "version-1",
            "traffic": 100,
            "features": [
                {
                    "name": "feature-1",
                    "value": "value-1"
                }
            ]
        }
    ]
}

进行分流(将 experimentID 替换成创建实验返回结构体的 id,上述例子中是 602ca4df9f4e8f5088966f6c,userID 可取任意值):

curl --location --request GET 'http://localhost:9999/apis/v1/experiments/{experimentID}/abconfig?userID={userID}'

返回如下:

{
    "features": [
        {
            "name": "feature-1",
            "value": "value-1"
        }
    ],
    "experiment_name": "test",
    "experiment_id": "602ca4df9f4e8f5088966f6c",
    "versions": [
        "602ca4df9f4e8f5088966f6b"
    ]
}

上报指标 (TODO)

SDK(TODO)

License

Apache 2.0

Issues
  • [feature] Maintain maphash fork to avoid initSeed in Sum64

    [feature] Maintain maphash fork to avoid initSeed in Sum64

    We do not want to initSeed in Sum64.

    // Sum64 returns h's current 64-bit value, which depends on
    // h's seed and the sequence of bytes added to h since the
    // last call to Reset or SetSeed.
    //
    // All bits of the Sum64 result are close to uniformly and
    // independently distributed, so it can be safely reduced
    // by using bit masking, shifting, or modular arithmetic.
    func (h *Hash) Sum64() uint64 {
    	h.initSeed()
    	return rthash(h.buf[:h.n], h.state.s)
    }
    
    opened by gaocegege 1
  • [question] Unexpected mongodb connections using go-mongo-driver

    [question] Unexpected mongodb connections using go-mongo-driver

    Ref https://jira.mongodb.org/browse/GODRIVER-1884 Ref https://jira.mongodb.org/browse/GODRIVER-1883 Ref https://jira.mongodb.org/browse/GODRIVER-1833

    Hi 策 高, thank you for the report!

    That sounds like expected behavior. A fixed number of connections are created per server for monitoring. This is in addition to >the connections created for application use (which is dictated by MinPoolSize and MaxPoolSize).

    For Go driver 1.4.0+ there are a fixed 2 monitoring connections per server. For Go driver < 1.4.0 there is a fixed 1 monitoring connection per server.

    When initializing the client with default configs, there is more than 1 connection (sometimes 2, sometimes 3) to the specific server.

    If no MaxPoolSize was specified, the default MaxPoolSize is 100. And application connections are created as needed. The higher >rate of concurrent operations, the more application connections will be created, capping at MaxPoolSize.

    When I set the MaxPoolSize to 2 and MinPoolSize to 2, I got 4 connections which are also unexpected.

    If you are connecting to one server, since there are two monitoring connections, that is expected.

    Sincerely, Kevin

    opened by gaocegege 0
Owner
Dongyue Studio
上海交通大学东岳网络工作室 Shanghai Jiao Tong University Dongyue Web Studio
Dongyue Studio
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

Re 249 Jun 27, 2022
A yaml data-driven testing format together with golang testing library

Specimen Yaml-based data-driven testing Specimen is a yaml data format for data-driven testing. This enforces separation between feature being tested

Design it, Run it 0 Jan 31, 2022
Hive-fleet: a distributed, scalable load-testing tool built in go that leverages Google Cloud Functions

hive-fleet hive-fleet is a distributed, scalable load-testing tool, built on top

Cristian Glavan 5 Jan 27, 2022
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

Fortio Fortio (Φορτίο) started as, and is, Istio's load testing tool and now graduated to be its own project. Fortio is also used by, among others, Me

Fortio (Φορτίο) 2.6k Jun 24, 2022
Expressive end-to-end HTTP API testing made easy in Go

baloo Expressive and versatile end-to-end HTTP API testing made easy in Go (golang), built on top of gentleman HTTP client toolkit. Take a look to the

Tom 725 May 12, 2022
Simple Go snapshot testing

Incredibly simple Go snapshot testing: cupaloy takes a snapshot of your test output and compares it to a snapshot committed alongside your tests. If t

Bradley Kemp 217 Jun 16, 2022
Clean database for testing, inspired by database_cleaner for Ruby

DbCleaner Clean database for testing, inspired by database_cleaner for Ruby. It uses flock syscall under the hood to make sure the test can runs in pa

Scott Le 140 Jun 28, 2022
Golang HTTP client testing framework

flute Golang HTTP client testing framework Presentation https://speakerdeck.com/szksh/flute-golang-http-client-testing-framework Overview flute is the

Shunsuke Suzuki 17 Apr 19, 2022
API testing framework inspired by frisby-js

frisby REST API testing framework inspired by frisby-js, written in Go Proposals I'm starting to work on frisby again with the following ideas: Read s

_Hofstadter 272 Jun 13, 2022
Mutation testing for Go source code

go-mutesting go-mutesting is a framework for performing mutation testing on Go source code. Its main purpose is to find source code, which is not cove

Markus Zimmermann 536 Jun 22, 2022
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Maxime Soulé 303 Jun 19, 2022
Minimal and Beautiful Go testing framework

Goblin A Mocha like BDD testing framework written in Go that requires no additional dependencies. Requires no extensive documentation nor complicated

null 852 Jun 26, 2022
A Comprehensive Coverage Testing System for The Go Programming Language

goc 中文页 | goc is a comprehensive coverage testing system for The Go Programming Language, especially for some complex scenarios, like system testing c

Qiniu Cloud 535 Jul 1, 2022
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.

GoConvey is awesome Go testing Welcome to GoConvey, a yummy Go testing tool for gophers. Works with go test. Use it in the terminal or browser accordi

SmartyStreets 7.3k Jun 28, 2022
Testing API Handler written in Golang.

Gofight API Handler Testing for Golang Web framework. Support Framework Http Handler Golang package http provides HTTP client and server implementatio

Bo-Yi Wu 402 Jun 19, 2022
Library created for testing JSON against patterns.

Gomatch Library created for testing JSON against patterns. The goal was to be able to validate JSON focusing only on parts essential in given test cas

null 41 Dec 1, 2021
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Esko Luontola 112 Apr 5, 2022
A collection of packages to augment the go testing package and support common patterns.

gotest.tools A collection of packages to augment testing and support common patterns. Usage With Go modules enabled (go1.11+) $ go get gotest.tools/v3

null 323 Jul 2, 2022
End-to-end HTTP and REST API testing for Go.

httpexpect Concise, declarative, and easy to use end-to-end HTTP and REST API testing for Go (golang). Basically, httpexpect is a set of chainable bui

Victor Gaydov 1.9k Jun 26, 2022