API testing framework inspired by frisby-js

Overview

frisby

Build Status GoDoc GitHub release

REST API testing framework inspired by frisby-js, written in Go

Proposals

I'm starting to work on frisby again with the following ideas:

  1. Read specification files
  • pyresttest
  • frisby.js
  • swagger spec
  • other?
  1. Use as a load tester
  • like Locust.io
  • distributed
  1. UI
  • Dashboard
  • Analytics
  • Reports
  • Manage multiple instances
  1. Backend
  • master/minions
  • db for analytics
  • api for UI / clients Goa
  • federation of minion groups?

Please comment on any issues or PRs related to these proposals. If you don't see an issue, PR, or idea; definitely add it!

Installation

go get -u github.com/verdverm/frisby

Basic Usage

First create a Frisby object:

// create an object with a given name (used in the report)
F := frisby.Create("Test successful user login").
    Get("https://golang.org")

Add any pre-flight data

F.SetHeader("Content-Type": "application/json").
	SetHeader("Accept", "application/json, text/plain, */*").
	SetJson([]string{"item1", "item2", "item3"})

There is also a Global object for setting repeated Pre-flight options.

frisby.Global.BasicAuth("username", "password").
	SetHeader("Authorization", "Bearer " + TOKEN)

Next send the request:

F.Send()

Then assert and inspect the response:

F.ExpectStatus(200).
    ExpectJson("nested.path.to.value", "sometext").
    ExpectJson("nested.path.to.object", golangObject).
    ExpectJson("nested.array.7.id", 23).
    ExpectJsonLength("nested.array", 8).
    AfterJson(func(F *frisby.Frisby, json *simplejson.Json, err error) {
		val, _ := json.Get("proxy").String()
		frisby.Global.SetProxy(val)
	})

Finally, print out a report of the tests

frisby.Global.PrintReport()

Check any error(s), however the global report prints any that occured as well

err := F.Error()

errs := F.Errors()
for _,e := range errs {
	fmt.Println("Error: ", e)
}

HTTP Method functions

Your basic HTTP verbs:

  • Get(url string)
  • Post(url string)
  • Put(url string)
  • Patch(url string)
  • Delete(url string)
  • Head(url string)
  • Options(url string)

Pre-flight functions

Functions called before Send()

You can also set theses on the frisby.Global object for persisting state over multiple requests.

( Most of these come from github.com/mozillazg/request)

  • BasicAuth(username,password string)
  • Proxy(url string)
  • SetHeader(key,value string)
  • SetHeaders(map[string]string)
  • SetCookies(key,value string)
  • SetCookiess(map[string]string)
  • SetDate(key,value string)
  • SetDates(map[string]string)
  • SetParam(key,value string)
  • SetParams(map[string]string)
  • SetJson(interface{})
  • SetFile(filename string)

Post-flight functions

Functions called after Send()

  • ExpectStatus(code int)
  • ExpectHeader(key, value string)
  • ExpectContent(content string)
  • ExpectJson(path string, value interface{})
  • ExpectJsonLength(path string, length int)
  • ExpectJsonType(path string, value_type reflect.Kind)
  • AfterContent( func(Frisby,[]byte,error) )
  • AfterText( func(Frisby,string,error) )
  • AfterJson( func(Frisby,simplejson.Json,error) )
  • PauseTest(t time.Duration)
  • PrintBody()
  • PrintReport()
  • PrintGoTestReport()

More examples

You can find a longer example here

package main

import (
	"fmt"
	"reflect"

	"github.com/bitly/go-simplejson"
	"github.com/verdverm/frisby"
)

func main() {
	fmt.Println("Frisby!\n")

	frisby.Create("Test GET Go homepage").
		Get("http://golang.org").
		Send().
		ExpectStatus(200).
		ExpectContent("The Go Programming Language")

	frisby.Create("Test GET Go homepage (which fails)").
		Get("http://golang.org").
		Send().
		ExpectStatus(400).
		ExpectContent("A string which won't be found")

	frisby.Create("Test POST").
		Post("http://httpbin.org/post").
		SetData("test_key", "test_value").
		Send().
		ExpectStatus(200)

	frisby.Create("Test ExpectJsonType").
		Post("http://httpbin.org/post").
		Send().
		ExpectStatus(200).
		ExpectJsonType("url", reflect.String)

	frisby.Create("Test ExpectJson").
		Post("http://httpbin.org/post").
		Send().
		ExpectStatus(200).
		ExpectJson("url", "http://httpbin.org/post").
		ExpectJson("headers.Accept", "*/*")

	frisby.Create("Test ExpectJsonLength (which fails)").
		Post("http://httpbin.org/post").
		SetJson([]string{"item1", "item2", "item3"}).
		Send().
		ExpectStatus(200).
		ExpectJsonLength("json", 4)

	frisby.Create("Test AfterJson").
		Post("http://httpbin.org/post").
		Send().
		ExpectStatus(200).
		AfterJson(func(F *frisby.Frisby, json *simplejson.Json, err error) {
		val, _ := json.Get("url").String()
		frisby.Global.SetProxy(val)
	})

	frisby.Global.PrintReport()
}

Sample Output

Frisby!

.......
For 7 requests made
  FAILED  [3/13]
      [Test ExpectJsonLength]
        -  Expect length to be 4, but got 3
      [Test GET Go homepage (which fails)]
        -  Expected Status 400, but got 200: "200 OK"
        -  Expected Body to contain "A string which won't be found", but it was missing

catch!

Comments
  • Can't handle period character in json field names

    Can't handle period character in json field names

    Hey - first up, thanks for the work on your library. We're evaluating using it and Iv'e come across an issue with the ExpectJson api.

    I have a json response that looks like this:

    {
      "error": "Invalid data supplied",
      "fields": {
        "display_name": "display_name is required",
        "ends_at": "ends_at is required",
        "starts_at": "starts_at is required",
        "state.state": "state must be one of the following: \"active\", \"inactive\""
      }
    }
    

    I'd love to be able to test this via ExpectJson("fields.state.state", "xxx") but for obvious reasons this isn't possible.

    I see two options for fixing this. One is to change the signature of the current api, which would be a backwards incompatible change: ExpectJson(expectation string, path string...). The other option would be to add a new method with that signature, e.g. ExpectJsonAtPath(expecation string, path string...).

    Thoughts?

    opened by snikch 5
  • Can't run expectation against array index values

    Can't run expectation against array index values

    I can't seem to find a way to do ExpectJson with a path like payload[0].state or payload.[0].state, and based on the code i don't think it's supported.

    Do you think it would be possible to update the way that paths are looked up from simp_json.GetPath to a loop that evaluates each of the keys and checks to see if there's is an index lookup in it?

    Currently we have:

    path_items := strings.Split(path, ".")
    simp_json = simp_json.GetPath(path_items...)
    

    We could look at something like (exclude the psuedo code, I haven't tested this):

    for _, pathItem := range strings.Split(path, ".") {
        matches := regexp.MustCompile("^\\[([0-9]+)\\]$").FindStringSubmatch(pathItem)
        if len(matches) < 2 {
            simp_json = simp_json.Get(pathItem)
        } else {
            index, _ := strconv.Atoi(matches[1])
            simp_json = simp_json.GetIndex(index)
        }
    }
    
    opened by snikch 4
  • Print the actual value not its type

    Print the actual value not its type

    It's kinda pesky getting "test failed for 'x', got type: string" when what you really want to know is what the string was, since that's the mismatch. This updates the expectation to print the actual value, rather than the type, or the actual value.

    opened by snikch 2
  • New functions PrintGoTestReport() and PauseTest() added

    New functions PrintGoTestReport() and PauseTest() added

    By using PrintGoTestReport() you get exatly the format, that is also used by go-test. This makes it easier to display the frisby response in jenkins for example.

    This example works well, if you were using the PrintGoTestReport func: go run main.go | go-junit-report -set-exit-code > test_report.xml

    opened by mkaerger 1
  • Fix for nil error when calling SetParam on frisyb.Create(...)

    Fix for nil error when calling SetParam on frisyb.Create(...)

    How to recreate: frisby.Create("Test GET /resource?type=xyz").         SetHeader("Content-Type", "application/json").         Get(GetAbsURL("/resource")).         SetParam(“type”, “xyz”). Send()

    Error: panic: assignment to entry in nil map [recovered] panic: assignment to entry in nil map

    opened by lakshmisampath 1
  • Feature/imrpove filename from full dir to base name

    Feature/imrpove filename from full dir to base name

    This provides that the server sees just the base name of the file and not the whole file path to the file. The server can handle by them self the file in which directory it will be saved.

    opened by patrickmarschallek 1
  • Fix panic whe reflecting on Zero Type

    Fix panic whe reflecting on Zero Type

    If the value was a zero type, such as nil, calling Type() on the reflect.Value would panic. This uses the formatting helper %T instead.

    call of reflect.Value.Type on zero Value
    
    opened by snikch 1
  • Global reporting

    Global reporting

    There should be global accounting so that the user doesn't have to call PrintReport,

    verbose mode would be nice and maybe print a list of accumulated errors

    opened by verdverm 1
  • added Key parameter for AddFiles function

    added Key parameter for AddFiles function

    Using all the time the same key "file" for uploading files is not that flexible. If you provide a key parameter you can easily change the key. I provided the "file" key as default value if you leave the key with an empty string.

    If you have any questions or suggestions, please let me know.

    opened by patrickmarschallek 0
  • ExpectJson paths can access array items

    ExpectJson paths can access array items

    Hey, we agreed we should push this one upstream but the simplejson authors don't look too reactive in their PR's so I've added this to allow payload.0.state style array access. It checks to see if the item is an array as part of the numeric key check so it still allows objects with "0" as the key.

    I'll still create a PR to push up to simplejson, but figured this would be valuable here too. Once simplejson supports it, we can just revert this commit 👌

    opened by snikch 0
  • Allow custom path separators

    Allow custom path separators

    This adds PathSeparator to the global_data struct. I don't think we need to use a setter method, that's not really idiomatic go so I've just added it as an exported field.

    opened by snikch 0
  • Response bodies are not closed

    Response bodies are not closed

    Golang's http library requires you to explicitly close the response's body object in order to avoid resource leaks and to allow for connection re-use.

    https://golang.org/pkg/net/http/#pkg-overview

    As far as I can tell Frisby doesn't do this, which can start to cause problems if you're running large numbers of tests (running out of sockets/file descriptors, slow run time, etc).

    opened by Phlamethrower 0
  • ExpectJsonType doesn't seem to work with Integer values

    ExpectJsonType doesn't seem to work with Integer values

    When trying to match a JSON returned value which is an integer, normally I (think I) should be using: ExpectJsonType("foobar", reflect.Int), but when doing this, I get an error: Expect Json "foobar" type to be "int", but got json.Number.

    opened by lstep 0
  • go get -u github.com/verdverm/frisby fails

    go get -u github.com/verdverm/frisby fails

    go version go1.6.2 darwin/amd64

    the mozilla vendored version of go-simplejson is preventing usage

    sinewave:gopath robmurtha$ go get -u github.com/verdverm/frisby
    # github.com/verdverm/frisby
    src/github.com/verdverm/frisby/expect.go:221: cannot use json (type *"github.com/mozillazg/request/vendor/github.com/bitly/go-simplejson".Json) as type *"github.com/bitly/go-simplejson".Json in argument to foo
    

    same issue when using gvt

    opened by robmurtha 5
Owner
_Hofstadter
Developing new ways to develop
_Hofstadter
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

Re 271 Sep 13, 2022
A yaml data-driven testing format together with golang testing library

Specimen Yaml-based data-driven testing Specimen is a yaml data format for data-driven testing. This enforces separation between feature being tested

Design it, Run it 0 Jan 31, 2022
Testy is a Go test running framework designed for Gametime's API testing needs.

template_library import "github.com/gametimesf/template_library" Overview Index Overview Package template_library is a template repository for buildin

Gametime United, Inc. 4 Jun 21, 2022
Golang HTTP client testing framework

flute Golang HTTP client testing framework Presentation https://speakerdeck.com/szksh/flute-golang-http-client-testing-framework Overview flute is the

Shunsuke Suzuki 16 Sep 8, 2022
Minimal and Beautiful Go testing framework

Goblin A Mocha like BDD testing framework written in Go that requires no additional dependencies. Requires no extensive documentation nor complicated

null 861 Sep 18, 2022
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Esko Luontola 113 Sep 2, 2022
🚀🌏 Orbital is a simple end-to-end testing framework for Go

Orbital is a test framework which enables a developer to write end to end tests just like one would writing unit tests. We do this by effectively copying the testing.T API and registering tests to be run periodically on a configured schedule.

Segment 78 Sep 14, 2022
An always-on framework that performs end-to-end functional network testing for reachability, latency, and packet loss

Arachne Arachne is a packet loss detection system and an underperforming path detection system. It provides fast and easy active end-to-end functional

Uber Open Source 368 Sep 11, 2022
BDD Testing Framework for Go

Jump to the docs | 中文文档 to learn more. To start rolling your Ginkgo tests now keep reading! If you have a question, comment, bug report, feature reque

Onsi Fakhouri 6.3k Sep 19, 2022
Framework of performance testing

Framework of performance testing fperf is a powerful and flexible framework which allows you to develop your own benchmark tools so much easy. You cre

null 336 Sep 16, 2022
Professional lightweight testing mini-framework for Go.

is Professional lightweight testing mini-framework for Go. Easy to write and read Beautifully simple API with everything you need: is.Equal, is.True,

Mat Ryer 1.5k Sep 18, 2022
Full-featured test framework for Go! Assertions, mocking, input testing, output capturing, and much more! 🍕

testza ?? Testza is like pizza for Go - you could life without it, but why should you? Get The Module | Documentation | Contributing | Code of Conduct

Marvin Wendt 394 Sep 2, 2022
espresso - a framework for testing BigQuery queries

espresso - a framework for testing BigQuery queries Goals Componentization: compose complex queries from smaller, reusable components Test driven deve

Tufin 6 Jul 4, 2022
Expressive end-to-end HTTP API testing made easy in Go

baloo Expressive and versatile end-to-end HTTP API testing made easy in Go (golang), built on top of gentleman HTTP client toolkit. Take a look to the

Tom 739 Sep 14, 2022
Testing API Handler written in Golang.

Gofight API Handler Testing for Golang Web framework. Support Framework Http Handler Golang package http provides HTTP client and server implementatio

Bo-Yi Wu 425 Sep 2, 2022
End-to-end HTTP and REST API testing for Go.

httpexpect Concise, declarative, and easy to use end-to-end HTTP and REST API testing for Go (golang). Basically, httpexpect is a set of chainable bui

Victor Gaydov 2k Sep 18, 2022
A Go library help testing your RESTful API application

RESTit A Go micro-framework to help writing RESTful API integration test Package RESTit provides helps to those who want to write an integration test

RESTit 55 Nov 25, 2021
Markdown based document-driven RESTful API testing.

silk Markdown based document-driven web API testing. Write nice looking Markdown documentation (like this), and then run it using the silk command Sim

Mat Ryer 930 Sep 6, 2022
Hsuan-Fuzz: REST API Fuzzing by Coverage Level Guided Blackbox Testing

Hsuan-Fuzz: REST API Fuzzing by Coverage Level Guided Blackbox Testing Architecture Usage package main import ( restAPI "github.com/iasthc/hsuan-

Chung-Hsuan Tsai 21 Jul 13, 2022