:alarm_clock: :fire: A TCP proxy to simulate network and system conditions for chaos and resiliency testing

Overview

Toxiproxy

GitHub release Build Status IRC Channel

Toxiproxy is a framework for simulating network conditions. It's made specifically to work in testing, CI and development environments, supporting deterministic tampering with connections, but with support for randomized chaos and customization. Toxiproxy is the tool you need to prove with tests that your application doesn't have single points of failure. We've been successfully using it in all development and test environments at Shopify since October, 2014. See our blog post on resiliency for more information.

Toxiproxy usage consists of two parts. A TCP proxy written in Go (what this repository contains) and a client communicating with the proxy over HTTP. You configure your application to make all test connections go through Toxiproxy and can then manipulate their health via HTTP. See Usage below on how to set up your project.

For example, to add 1000ms of latency to the response of MySQL from the Ruby client:

Toxiproxy[:mysql_master].downstream(:latency, latency: 1000).apply do
  Shop.first # this takes at least 1s
end

To take down all Redis instances:

Toxiproxy[/redis/].down do
  Shop.first # this will throw an exception
end

While the examples in this README are currently in Ruby, there's nothing stopping you from creating a client in any other language (see Clients).

Table of Contents

  1. Why yet another chaotic TCP proxy?
  2. Clients
  3. Example
  4. Usage
    1. Installing
      1. Upgrading from 1.x
    2. Populating
    3. Using
  5. Toxics
    1. Latency
    2. Down
    3. Bandwidth
    4. Slow close
    5. Timeout
    6. Slicer
  6. HTTP API
    1. Proxy fields
    2. Toxic fields
    3. Endpoints
    4. Populating Proxies
  7. CLI example
  8. FAQ
  9. Development

Why yet another chaotic TCP proxy?

The existing ones we found didn't provide the kind of dynamic API we needed for integration and unit testing. Linux tools like nc and so on are not cross-platform and require root, which makes them problematic in test, development and CI environments.

Clients

Example

Let's walk through an example with a Rails application. Note that Toxiproxy is in no way tied to Ruby, it's just been our first use case. You can see the full example at sirupsen/toxiproxy-rails-example. To get started right away, jump down to Usage.

For our popular blog, for some reason we're storing the tags for our posts in Redis and the posts themselves in MySQL. We might have a Post class that includes some methods to manipulate tags in a Redis set:

class Post < ActiveRecord::Base
  # Return an Array of all the tags.
  def tags
    TagRedis.smembers(tag_key)
  end

  # Add a tag to the post.
  def add_tag(tag)
    TagRedis.sadd(tag_key, tag)
  end

  # Remove a tag from the post.
  def remove_tag(tag)
    TagRedis.srem(tag_key, tag)
  end

  # Return the key in Redis for the set of tags for the post.
  def tag_key
    "post:tags:#{self.id}"
  end
end

We've decided that erroring while writing to the tag data store (adding/removing) is OK. However, if the tag data store is down, we should be able to see the post with no tags. We could simply rescue the Redis::CannotConnectError around the SMEMBERS Redis call in the tags method. Let's use Toxiproxy to test that.

Since we've already installed Toxiproxy and it's running on our machine, we can skip to step 2. This is where we need to make sure Toxiproxy has a mapping for Redis tags. To config/boot.rb (before any connection is made) we add:

require 'toxiproxy'

Toxiproxy.populate([
  {
    name: "toxiproxy_test_redis_tags",
    listen: "127.0.0.1:22222",
    upstream: "127.0.0.1:6379"
  }
])

Then in config/environments/test.rb we set the TagRedis to be a Redis client that connects to Redis through Toxiproxy by adding this line:

TagRedis = Redis.new(port: 22222)

All calls in the test environment now go through Toxiproxy. That means we can add a unit test where we simulate a failure:

test "should return empty array when tag redis is down when listing tags" do
  @post.add_tag "mammals"

  # Take down all Redises in Toxiproxy
  Toxiproxy[/redis/].down do
    assert_equal [], @post.tags
  end
end

The test fails with Redis::CannotConnectError. Perfect! Toxiproxy took down the Redis successfully for the duration of the closure. Let's fix the tags method to be resilient:

def tags
  TagRedis.smembers(tag_key)
rescue Redis::CannotConnectError
  []
end

The tests pass! We now have a unit test that proves fetching the tags when Redis is down returns an empty array, instead of throwing an exception. For full coverage you should also write an integration test that wraps fetching the entire blog post page when Redis is down.

Full example application is at sirupsen/toxiproxy-rails-example.

Usage

Configuring a project to use Toxiproxy consists of three steps:

  1. Installing Toxiproxy
  2. Populating Toxiproxy
  3. Using Toxiproxy

1. Installing Toxiproxy

Linux

See Releases for the latest binaries and system packages for your architecture.

Ubuntu

$ wget -O toxiproxy-2.1.4.deb https://github.com/Shopify/toxiproxy/releases/download/v2.1.4/toxiproxy_2.1.4_amd64.deb
$ sudo dpkg -i toxiproxy-2.1.4.deb
$ sudo service toxiproxy start

OS X

$ brew tap shopify/shopify
$ brew install toxiproxy

Windows

Toxiproxy for Windows is available for download at https://github.com/Shopify/toxiproxy/releases/download/v2.1.4/toxiproxy-server-windows-amd64.exe

Docker

Toxiproxy is available on Docker Hub.

$ docker pull shopify/toxiproxy
$ docker run -it shopify/toxiproxy

If using Toxiproxy from the host rather than other containers, enable host networking with --net=host.

Source

If you have Go installed, you can build Toxiproxy from source using the make file:

$ make build
$ ./toxiproxy-server

Upgrading from Toxiproxy 1.x

In Toxiproxy 2.0 several changes were made to the API that make it incompatible with version 1.x. In order to use version 2.x of the Toxiproxy server, you will need to make sure your client library supports the same version. You can check which version of Toxiproxy you are running by looking at the /version endpoint.

See the documentation for your client library for specific library changes. Detailed changes for the Toxiproxy server can been found in CHANGELOG.md.

2. Populating Toxiproxy

When your application boots, it needs to make sure that Toxiproxy knows which endpoints to proxy where. The main parameters are: name, address for Toxiproxy to listen on and the address of the upstream.

Some client libraries have helpers for this task, which is essentially just making sure each proxy in a list is created. Example from the Ruby client:

# Make sure `shopify_test_redis_master` and `shopify_test_mysql_master` are
# present in Toxiproxy
Toxiproxy.populate([
  {
    name: "shopify_test_redis_master",
    listen: "127.0.0.1:22220",
    upstream: "127.0.0.1:6379"
  },
  {
    name: "shopify_test_mysql_master",
    listen: "127.0.0.1:24220",
    upstream: "127.0.0.1:3306"
  }
])

This code needs to run as early in boot as possible, before any code establishes a connection through Toxiproxy. Please check your client library for documentation on the population helpers.

Alternatively use the CLI to create proxies, e.g.:

toxiproxy-cli create shopify_test_redis_master -l localhost:26379 -u localhost:6379

We recommend a naming such as the above: <app>_<env>_<data store>_<shard>. This makes sure there are no clashes between applications using the same Toxiproxy.

For large application we recommend storing the Toxiproxy configurations in a separate configuration file. We use config/toxiproxy.json. This file can be passed to the server using the -config option, or loaded by the application to use with the populate function.

An example config/toxiproxy.json:

[
  {
    "name": "web_dev_frontend_1",
    "listen": "[::]:18080",
    "upstream": "webapp.domain:8080",
    "enabled": true
  },
  {
    "name": "web_dev_mysql_1",
    "listen": "[::]:13306",
    "upstream": "database.domain:3306",
    "enabled": true
  }
]

Use ports outside the ephemeral port range to avoid random port conflicts. It's 32,768 to 61,000 on Linux by default, see /proc/sys/net/ipv4/ip_local_port_range.

3. Using Toxiproxy

To use Toxiproxy, you now need to configure your application to connect through Toxiproxy. Continuing with our example from step two, we can configure our Redis client to connect through Toxiproxy:

# old straight to redis
redis = Redis.new(port: 6380)

# new through toxiproxy
redis = Redis.new(port: 22220)

Now you can tamper with it through the Toxiproxy API. In Ruby:

redis = Redis.new(port: 22220)

Toxiproxy[:shopify_test_redis_master].downstream(:latency, latency: 1000).apply do
  redis.get("test") # will take 1s
end

Or via the CLI:

toxiproxy-cli toxic add shopify_test_redis_master -t latency -a latency=1000

Please consult your respective client library on usage.

Toxics

Toxics manipulate the pipe between the client and upstream. They can be added and removed from proxies using the HTTP api. Each toxic has its own parameters to change how it affects the proxy links.

For documentation on implementing custom toxics, see CREATING_TOXICS.md

latency

Add a delay to all data going through the proxy. The delay is equal to latency +/- jitter.

Attributes:

  • latency: time in milliseconds
  • jitter: time in milliseconds

down

Bringing a service down is not technically a toxic in the implementation of Toxiproxy. This is done by POSTing to /proxies/{proxy} and setting the enabled field to false.

bandwidth

Limit a connection to a maximum number of kilobytes per second.

Attributes:

  • rate: rate in KB/s

slow_close

Delay the TCP socket from closing until delay has elapsed.

Attributes:

  • delay: time in milliseconds

timeout

Stops all data from getting through, and closes the connection after timeout. If timeout is 0, the connection won't close, and data will be delayed until the toxic is removed.

Attributes:

  • timeout: time in milliseconds

slicer

Slices TCP data up into small bits, optionally adding a delay between each sliced "packet".

Attributes:

  • average_size: size in bytes of an average packet
  • size_variation: variation in bytes of an average packet (should be smaller than average_size)
  • delay: time in microseconds to delay each packet by

limit_data

Closes connection when transmitted data exceeded limit.

  • bytes: number of bytes it should transmit before connection is closed

HTTP API

All communication with the Toxiproxy daemon from the client happens through the HTTP interface, which is described here.

Toxiproxy listens for HTTP on port 8474.

Proxy fields:

  • name: proxy name (string)
  • listen: listen address (string)
  • upstream: proxy upstream address (string)
  • enabled: true/false (defaults to true on creation)

To change a proxy's name, it must be deleted and recreated.

Changing the listen or upstream fields will restart the proxy and drop any active connections.

If listen is specified with a port of 0, toxiproxy will pick an ephemeral port. The listen field in the response will be updated with the actual port.

If you change enabled to false, it will take down the proxy. You can switch it back to true to reenable it.

Toxic fields:

  • name: toxic name (string, defaults to <type>_<stream>)
  • type: toxic type (string)
  • stream: link direction to affect (defaults to downstream)
  • toxicity: probability of the toxic being applied to a link (defaults to 1.0, 100%)
  • attributes: a map of toxic-specific attributes

See Toxics for toxic-specific attributes.

The stream direction must be either upstream or downstream. upstream applies the toxic on the client -> server connection, while downstream applies the toxic on the server -> client connection. This can be used to modify requests and responses separately.

Endpoints

All endpoints are JSON.

  • GET /proxies - List existing proxies and their toxics
  • POST /proxies - Create a new proxy
  • POST /populate - Create or replace a list of proxies
  • GET /proxies/{proxy} - Show the proxy with all its active toxics
  • POST /proxies/{proxy} - Update a proxy's fields
  • DELETE /proxies/{proxy} - Delete an existing proxy
  • GET /proxies/{proxy}/toxics - List active toxics
  • POST /proxies/{proxy}/toxics - Create a new toxic
  • GET /proxies/{proxy}/toxics/{toxic} - Get an active toxic's fields
  • POST /proxies/{proxy}/toxics/{toxic} - Update an active toxic
  • DELETE /proxies/{proxy}/toxics/{toxic} - Remove an active toxic
  • POST /reset - Enable all proxies and remove all active toxics
  • GET /version - Returns the server version number

Populating Proxies

Proxies can be added and configured in bulk using the /populate endpoint. This is done by passing an json array of proxies to toxiproxy. If a proxy with the same name already exists, it will be compared to the new proxy and replaced if the upstream and listen address don't match.

A /populate call can be included for example at application start to ensure all required proxies exist. It is safe to make this call several times, since proxies will be untouched as long as their fields are consistent with the new data.

CLI Example

$ toxiproxy-cli create redis -l localhost:26379 -u localhost:6379
Created new proxy redis
$ toxiproxy-cli list
Listen          Upstream        Name  Enabled Toxics
======================================================================
127.0.0.1:26379 localhost:6379  redis true    None

Hint: inspect toxics with `toxiproxy-client inspect <proxyName>`
$ redis-cli -p 26379
127.0.0.1:26379> SET omg pandas
OK
127.0.0.1:26379> GET omg
"pandas"
$ toxiproxy-cli toxic add redis -t latency -a latency=1000
Added downstream latency toxic 'latency_downstream' on proxy 'redis'
$ redis-cli -p 26379
127.0.0.1:26379> GET omg
"pandas"
(1.00s)
127.0.0.1:26379> DEL omg
(integer) 1
(1.00s)
$ toxiproxy-cli toxic remove redis -n latency_downstream
Removed toxic 'latency_downstream' on proxy 'redis'
$ redis-cli -p 26379
127.0.0.1:26379> GET omg
(nil)
$ toxiproxy-cli delete redis
Deleted proxy redis
$ redis-cli -p 26379
Could not connect to Redis at 127.0.0.1:26379: Connection refused

Frequently Asked Questions

How fast is Toxiproxy? The speed of Toxiproxy depends largely on your hardware, but you can expect a latency of < 100µs when no toxics are enabled. When running with GOMAXPROCS=4 on a Macbook Pro we achieved ~1000MB/s throughput, and as high as 2400MB/s on a higher end desktop. Basically, you can expect Toxiproxy to move data around at least as fast the app you're testing.

Can Toxiproxy do randomized testing? Many of the available toxics can be configured to have randomness, such as jitter in the latency toxic. There is also a global toxicity parameter that specifies the percentage of connections a toxic will affect. This is most useful for things like the timeout toxic, which would allow X% of connections to timeout.

I am not seeing my Toxiproxy actions reflected for MySQL. MySQL will prefer the local Unix domain socket for some clients, no matter which port you pass it if the host is set to localhost. Configure your MySQL server to not create a socket, and use 127.0.0.1 as the host. Remember to remove the old socket after you restart the server.

Toxiproxy causes intermittent connection failures. Use ports outside the ephemeral port range to avoid random port conflicts. It's 32,768 to 61,000 on Linux by default, see /proc/sys/net/ipv4/ip_local_port_range.

Should I run a Toxiproxy for each application? No, we recommend using the same Toxiproxy for all applications. To distinguish between services we recommend naming your proxies with the scheme: <app>_<env>_<data store>_<shard>. For example, shopify_test_redis_master or shopify_development_mysql_1.

Development

  • make. Build a toxiproxy development binary for the current platform.
  • make all. Build Toxiproxy binaries and packages for all platforms. Requires to have Go compiled with cross compilation enabled on Linux and Darwin (amd64) as well as fpm in your $PATH to build the Debian package.
  • make test. Run the Toxiproxy tests.
  • make darwin. Build binary for Darwin.
  • make linux. Build binary for Linux.
  • make windows. Build binary for Windows.

Release

  1. Ensure this release has run internally for Shopify/shopify for at least a day which is the best fuzzy test for robustness we have.
  2. Update CHANGELOG.md
  3. Bump VERSION
  4. Change versions in README.md
  5. Commit
  6. Tag
  7. make release to create binaries, packages and push new Docker image
  8. Create Github draft release against new tag and upload binaries and Debian package
  9. Bump version for Homebrew
Issues
  • Toxiproxy 2.0 Release

    Toxiproxy 2.0 Release

    As far as I know, everything is ready to be released for 2.0

    The released binary should be changed to toxiproxy-server or similar to allow for a cli binary later. I've done some testing myself, but it would be good to get some verification that this works with the Shopify test suite.

    Initial RFC is here: https://github.com/Shopify/toxiproxy/issues/54

    PR's contained in this branch:

    • Initial PR https://github.com/Shopify/toxiproxy/pull/62
    • Remove stream direction from api urls https://github.com/Shopify/toxiproxy/pull/73
    • Update Readme https://github.com/Shopify/toxiproxy/pull/74
    • Add toxicity field https://github.com/Shopify/toxiproxy/pull/75
    • Go client refactor https://github.com/Shopify/toxiproxy/pull/76
    • Interruptable ChanReader https://github.com/Shopify/toxiproxy/pull/77
    • Makefile cleanup https://github.com/Shopify/toxiproxy/pull/78
    • Fix toxic marshalling and updating https://github.com/Shopify/toxiproxy/pull/80

    These are all listed in the CHANGELOG.md

    @Sirupsen @eapache @pushrax

    This shouldn't really need much review other than an overall :+1: and :tophat: Each of the above PRs have already been individually reviewed.

    After this is merged, the Release steps just need to be followed (contained in the README)

    opened by xthexder 32
  • Cli 2.0 dev

    Cli 2.0 dev

    Here's toxiproxy-client v2.0!

    • [x] I should close both #58 and #91 as this PR replaces them.

    Not worrying about clear commit history when doing rapid ux changes. I'll start with a quick list of changes, followed by relevant screen shots.

    1. Updated version number to 2.0
    2. Removed graphic pipe in inspect because I found it confusing
    3. Made inspect a lot clearer
    4. Made list a lot clearer. Also table has Toxic and Enabled column
    5. Added cyan hints (might be kinda silly?)
    6. Added usage and examples to toplevel command and toxic command
    7. Updated the makefile (@Sirupsen: would you glance at this, I'm a little rusty with make)

    Screen Shots

    Part of the toplevel help screen. toplevel-help Part of the ./cli toxic screen. toxic-help Looks like we havn't created any toxics yet... ls-empty There's one! ls-one Let's try to inspect it! inspect-one No toxics? Let's add one! inspect-down

    Darn! It defaulted to downstream... let's add another upstream! inspect-up-down And let's list the proxies again. (This is after I added the enabled field) with-enabled

    TODO:

    • [x] #92 merge will require an update
    • [ ] do I need to update the CHANGELOG?
    • [x] ux review from @Sirupsen
    • [x] a ux review from someone who knows toxiproxy but not the client
    • [x] Redo inspect command
    • [ ] Configure toxiproxy host ip.
    opened by jpittis 18
  • Toxiproxy 2.0

    Toxiproxy 2.0

    See proposal here https://github.com/Shopify/toxiproxy/issues/54

    Ready for review + merging

    TODO:

    • [x] Update Go client (and API tests)
    • [x] Add new tests for adding / removing / updating toxics
    • [x] Standardize error output in the API
    • [x] Run benchmark comparisons to v1.1
    • [x] Fix docker image builds
    • [x] Add toxicity field: https://github.com/Shopify/toxiproxy/pull/65 (can be done in a separate PR after this)
    • [x] Determine any deprecation / compatibility changes if necessary
    • [ ] Code cleanup and review

    cc @Sirupsen @pushrax @eapache

    opened by xthexder 16
  • Toxiproxy hangs

    Toxiproxy hangs

    Scenario is:

    • Add proxy toxiproxy-cli create proxy_20000_to_20001 --listen localhost:20000 --upstream localhost:20001

    • Add latency toxic (also reproducing with others like bandwidth) toxiproxy-cli toxic add proxy_20000_to_20001 --upstream --toxicName LatencyIssue -t latency -a latency=500 -a jitter=0

    • Start sending some data through, i.e.

    while true; do; netcat -l -p 20001; done
    for i in {1..100000}; do sleep 0.1; echo "$i" | tee /dev/tty ; done | netcat localhost 20000
    
    • Add timeout toxic (data flow stops as expected) toxiproxy-cli toxic add proxy_20000_to_20001 --upstream --toxicName TimeoutIssue -t timeout -a timeout=0

    • Remove first toxic toxiproxy-cli toxic delete proxy_20000_to_20001 --toxicName LatencyIssue

    • As a result last command just hangs. If you try to run other commands like 'toxiproxy-cli list' they will hang as well. I'm using latest 2.1.0 release of toxiproxy and latest 1.19.1 release of cli.

    opened by archelan 14
  • toxiproxy-cli hangs in docker connecting to local toxproxy (related to internet?)

    toxiproxy-cli hangs in docker connecting to local toxproxy (related to internet?)

    hello! new toxiproxy user here, trying out my first toxic steps. I run it in docker, using the latest image that was just pushed today. just 1 proxy and 1 backend, very light load.

    i noticed that toxiproxy-cli, which worked fine first to add a proxy and a toxic, on subsequest commands started hanging. i ctrl-c'd the commands after they hung for minutes (the last 3 commands).

    interesting to mention: around the same time my internet connection also became slow and started timing out. while everything is local in a dockerstack entirely on my laptop, maybe my internet has something to do with it? has anyone experienced something like this before?

    [email protected]:/app/src/github.com/Shopify/toxiproxy# toxiproxy-cli inspect cassandra
    Name: cassandra Listen: [::]:9042       Upstream: cassandra:9042
    ======================================================================
    Upstream toxics:
    Proxy has no Upstream toxics enabled.
    
    Downstream toxics:
    latency_downstream: type=latency stream=downstream toxicity=1.00 attributes=[ jitter=500 latency=1000 ]
    
    Hint: add a toxic with `toxiproxy-cli toxic add`
    [email protected]:/app/src/github.com/Shopify/toxiproxy# toxiproxy-cli toxic delete cassandra -n latency_downstream:
    Failed to remove toxic: RemoveToxic: HTTP 404: toxic not found
    [email protected]:/app/src/github.com/Shopify/toxiproxy# toxiproxy-cli toxic delete cassandra -n latency_downstream 
    ^C
    [email protected]:/app/src/github.com/Shopify/toxiproxy# toxiproxy-cli toxic delete cassandra -n latency_downstream
    ^C
    [email protected]:/app/src/github.com/Shopify/toxiproxy# toxiproxy-cli list                                         
    ^C
    
    
    opened by Dieterbe 14
  • cli: initial commit

    cli: initial commit

    Adds a simple V1 of a CLI that enables you to list proxies and toggle their enabled flag. Really nice to get an overview of things and play around with it. Will support Toxics later, and then we'll figure out how to distribute it. It could be a separate binary with a name like toxiproxyctl or it could be part of the main one and then daemon just becomes a command (and it starts by default to be backwards compatible).

    This is really handy for quick and dirty testing yourself, or for black-box resiliency testing. A more friendly interface would be a web app, that can be done as well. It's remarkably easy with the Go client.

    screen shot 2015-05-18 at 16 13 52 screen shot 2015-05-18 at 16 13 34 screen shot 2015-05-18 at 16 13 24

    @xthexder

    opened by sirupsen 13
  • Embed toxic attribute

    Embed toxic attribute

    This patch embeds toxic specific attributes into their own nested attributes field, allowing for the following api changes:

    type Attributes map[string]interface{}
    
    type Toxic struct {
        Name       string     `json:"name"`
        Type       string     `json:"type"`
        Stream     string     `json:"stream"`
        Toxicity   float32    `json:"toxicity"`
        Attributes Attributes `json:"attributes"`
    }
    
    AddToxic(name, typeName, stream string, toxicity float32, attrs Attributes)
    UpdateToxic(name string, toxicity float32, attrs Attributes)
    

    We can't marshal the attributes of a toxic until we know what kind of toxic struct to marshal into. This has previously forced us to parse the json twice.

    I can't think of a better solution than to use an anonymous struct to extract the specific json that we need.

    This has also caused the previous toxic marsheling to be kinda sketchy. My change does not make it less sketchy.

    I'm happy enough with the approach. There might not be a pretty solution. @Sirupsen or @xthexder might be able to think of one?

    Update CHANGELOG, docs and squish the commits once we're ready to merge.

    opened by jpittis 13
  • Support for cluster of toxiproxy nodes

    Support for cluster of toxiproxy nodes

    It would be nice if we can setup a cluster of toxiproxy nodes so that can use a load balancer to balance loads between these nodes. This avoids a single point of failure.

    If a toxic is configured in any of the nodes, it should propagate to all nodes in the cluster automatically.

    Want to check if this feature looks interesting with everyone.

    opened by nytins 13
  • Add `sometimesToxic` and `toxicity` options to `timeout`

    Add `sometimesToxic` and `toxicity` options to `timeout`

    This PR is intended to create discussion about whether people think introducing randomness into Toxiproxy is a good idea.

    At our organisation we have just started using Toxiproxy and really like it, but we would like to add random failures.

    To introduce randomness to the timeout Toxic, set sometimesToxic in the json request to true, and set the required toxicity. The reason for using two variables is to maintain compatibility with existing behaviour - if you don't explicitly set sometimesToxic then it will default to false and 100% of requests will timeout.

    It's also possible that this could be generically extended to all Toxics - it looks like the work on the toxiproxy2 branch could make this easier.

    opened by tomblench 12
  • Toxiproxy 2.0 RFC

    Toxiproxy 2.0 RFC

    Proposed API Changes

    • Remove enabled field from all toxics
    • POST /proxies/{proxy}/{up|down}stream/toxics - Create (enable) a new toxic
    • POST /proxies/{proxy}/{up|down}stream/toxics/{toxic} - Update an existing toxic
    • DELETE /proxies/{proxy}/{up|down}stream/toxics/{toxic} - Destroy (disable) a toxic

    This API matches the current proxy creation/deletion api.

    Internal Structure Changes

    Current 1.x architecture: https://docs.google.com/drawings/d/1pwuRINbpwyaQeYSgGt8VTEKWEUQAwYpNJIMH1owW3Sc/view

    Instead of a static number of ToxicStubs (and a static mapping for each toxic), ToxicStubs will be dynamically added / removed as toxics are created/destroyed. A minimum of 1 ToxicStub will always be present, and the first toxic will always be a NoopToxic so that it can be interrupted for inserting / removing toxics.

    Steps for adding a new toxic:

    1. Interrupt the last toxic in the chain
    2. Reconnect the output channels so that the new toxic is between the last toxic and the end ChanReader
    3. Resume both ToxicStubs with their toxics

    Steps for removing a toxic:

    1. Interrupt the toxic before the target in order to ensure the target's input channel is empty
    2. Interrupt the target toxic
    3. Connect the previous toxic's output to the target toxic's output
    4. Resume the previous toxic / ToxicStub pair

    This ends up being very similar to a doubly-linked list, except that we need to ensure no data is in-flight between the links (channels) when they're removed.

    Toxic Registration

    The main() function and server creation will be simplified so that it can be used as a library. Instead of having all the toxics hard-coded inside toxiproxy, they will be registered in an init() function: toxiproxy.Register(SomeToxic) (this will have to be done with reflection or something...)

    This will allow user-created toxics without having to create a full fork of toxiproxy.

    opened by xthexder 11
  • Add a systemd unit file

    Add a systemd unit file

    opened by Ladicek 0
  • Make it build for arm64 (to make it run on Mac M1)

    Make it build for arm64 (to make it run on Mac M1)

    This PR extends the docker image with an arm64 binary so it can run without emulation on M1 mac.

    opened by lpiepiora 0
  • Renovations

    Renovations

    renovate dependencies

    • move to github.com/urface/cli/v2 API
    • refresh all dependencies to @latest
    • golang.org/x/crypto/ssh/terminal moved to golang.org/x/term

    gofmt -s -w

    fix some minor govet/staticheck failures

    • deprecated functions
    • copylocks on range
    • t.Fatal in non-test goroutines
    • use bytes.Equal
    • #nosec for math/crypto
    opened by dnwe 1
  • write EPROTO ...:error:...:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../deps/openssl/openssl/ssl/record/rec_layer_s3.c:1544:SSL alert number 40

    write EPROTO ...:error:...:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../deps/openssl/openssl/ssl/record/rec_layer_s3.c:1544:SSL alert number 40

    I create a basic transparent proxy:

    $ toxiproxy-cli create remote -l localhost:26379 -u <address>:443
    

    then I check its status:

    $ toxiproxy-cli l
    Name                    Listen          Upstream                Enabled         Toxics
    ======================================================================================
    remote                  127.0.0.1:26379 <address>:443           enabled         None
    

    Now when I try to connect using https://github.com/websockets/ws library I get this error message:

    write EPROTO 140108792846208:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:
    

    My connect line is:

    let ws = new WebSocket(address);
    

    where address is: wss://127.0.0.1:26379/ws;

    Any ideas what does that mean?

    opened by OnkelTem 1
  • client.Populate assign client to proxy

    client.Populate assign client to proxy

    client.Populate should assign client to proxies first, otherwise it will panic if we call proxy.AddToxic()

    cla-needed 
    opened by hellodudu 0
  • Mention MacPorts in readme

    Mention MacPorts in readme

    toxiproxy is also available in MacPorts.

    cla-needed 
    opened by amake 1
  • Create: HTTP 404: Unexpected response code, expected 201

    Create: HTTP 404: Unexpected response code, expected 201

    ./toxiproxy-cli --host 192.168.182.12 create redis -l 192.168.182.12:26379 -u 192.168.182.12:6379

    Failed to create proxy: Create: HTTP 404: Unexpected response code, expected 201

    I got this error when trying to create new proxy

    opened by vladimirtcats 0
  • Adding CLI rst command

    Adding CLI rst command

    opened by peter-ginchev 1
  • Add Link to Elixir Client

    Add Link to Elixir Client

    I've developed a full-fledged elixir client for Toxiproxy that is highly inspired by the ruby version. It's called ToxiproxyEx but I think we can link it as toxiproxy-elixir to make it clear that it is an elixir client.

    opened by Jcambass 0
  • Hosting toxiproxy in HTTPS

    Hosting toxiproxy in HTTPS

    We want to install Toxiproxy but as a HTTPS Proxy. Will be there any support for hosting the toxiproxy in https. In our test environment we can't host any service with http. Any suggestion will help.

    opened by sbose20 0
Releases(v2.1.4)
:alarm_clock: :fire: A TCP proxy to simulate network and system conditions for chaos and resiliency testing

Toxiproxy Toxiproxy is a framework for simulating network conditions. It's made specifically to work in testing, CI and development environments, supp

Shopify 5.6k Jul 20, 2021
An incredibly fast proxy checker & IP rotator with ease.

An incredibly fast proxy checker & IP rotator with ease.

Kitabisa 411 Jul 20, 2021
A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.

frp README | 中文文档 What is frp? frp is a fast reverse proxy to help you expose a local server behind a NAT or firewall to the Internet. As of now, it s

null 47k Jul 22, 2021
:vulcan_salute: Fast, modern, easy-to-use network scanner

sx is the command-line network scanner designed to follow the UNIX philosophy. The goal of this project is to create the fastest network scanner with

null 659 Jul 19, 2021
HTTP(S)/WS(S)/TCP Tunnels to localhost using only SSH.

An open source serveo/ngrok alternative.

Antonio Mika 1.9k Jul 23, 2021
Ethr is a Comprehensive Network Measurement Tool for TCP, UDP & ICMP.

Ethr Ethr is a cross platform network performance measurement tool written in golang. The goal of this project is to provide a native tool for compreh

Microsoft 4.9k Jul 26, 2021
An experimental Tor-Proxy serivce written in Go using Go-proxy and Go-libtor.

tor-proxy An experimental standalone tor-proxy service built with Go, using go-proxy, go-libtor and bine. This is a simple replacement to Tor's origin

Narasimha Prasanna HN 28 Jul 21, 2021
Privacy important, fast, recursive dns resolver server with dnssec support

?? Privacy important, fast, recursive dns resolver server with dnssec support Installation go get github.com/semihalev/sdns Pre-build Binaries Downloa

Yasar Alev 673 Jul 23, 2021
A small TCP proxy written in Go

tcp-proxy A small TCP proxy written in Go This project was intended for debugging text-based protocols. The next version will address binary protocols

Jaime Pillora 533 Jul 21, 2021
Mutual TLS encryption TCP proxy with golang

mtls-tcp-proxy Mutual Authentication TLS encryption TCP proxy with golang Why? I created this because of sometimes, it is not possible for us to estab

Habibie Faried 3 May 19, 2021
Standalone client for proxies of Opera VPN

opera-proxy Standalone Opera VPN client. Younger brother of hola-proxy. Just run it and it'll start a plain HTTP proxy server forwarding traffic throu

null 199 Jul 18, 2021
Network Diagnostic Tool

myLG, Command line Network Diagnostic Tool myLG is an open source software utility which combines the functions of the different network probes in one

Mehrdad Arshad Rad 2.5k Jul 15, 2021
Multiplexer over TCP. Useful if target server only allows you to create limited tcp connections concurrently.

tcp-multiplexer Use it in front of target server and let your client programs connect it, if target server only allows you to create limited tcp conne

许嘉华 3 May 27, 2021
Standalone client for proxies of Windscribe browser extension

windscribe-proxy Standalone Windscribe proxy client. Younger brother of opera-proxy. Just run it and it'll start a plain HTTP proxy server forwarding

null 12 Jul 17, 2021