An easy HTTP client for Go. Inspired by the immortal Requests.

Related tags

Network rek
Overview

rek

go.dev reference

An easy HTTP client for Go inspired by Requests, plus all the Go-specific goodies you'd hope for in a client. Here's an example:

// GET request
res, _ := rek.Get("https://httpbin.org/get")

fmt.Println(res.StatusCode())

body, _ := rek.BodyAsString(res.Body())

fmt.Println(body)

// POST request
type Comment struct {
    Username string `json:"username"`
    Body     string `json:"body"`
}

res, _ := rek.Post("https://httpbin.org/post",
    rek.Json(Comment{Username: "genesiskel", Body: "This movie sucked. Two thumbs down."}),
    rek.Headers(map[string]string{"My-Custom-Header": "foo,bar,baz"}),
    rek.BasicAuth("user", "pass"),
    rek.Timeout(5 * time.Second),
)

fmt.Println(res.StatusCode())

body, _ := rek.BodyAsString(res.Body())
fmt.Println(body)

Responses

The Response struct has the following methods:

Method Description Return type
StatusCode() HTTP status code, e.g. 200, 400 int
Body() The HTTP response body as a reader io.ReadCloser
Headers() The response headers map[string]string
Encoding() The content encoding of the response body, e.g. gzip []string
ContentType() The value of the Content-Type header string
Raw() The unmodified *http.Response *http.Response
Cookies() The cookies attached to the response []*http.Cookie
ContentLength() The length of the response int64
Status() The status of the response, e.g. 200 OK string

HTTP response body

Keep in mind that the HTTP response body can only be read once. This is one area in which rek does not directly correspond to the Requests API. And so this, for example, won't work the way you might want:

res, _ := rek.Get("https://httpbin.org/get")

bs1, _ := ioutil.ReadAll(res.Body()) // Non-empty byte slice

bs2, _ := ioutil.ReadAll(res.Body()) // Empty byte slice

If you'd like to use the response body more than once, store it in a variable rather than re-accessing the body.

Helper methods

There are two simple helper methods for working with the response body:

Function Return types
BodyAsString(io.ReadCloser) (string, error)
BodyAsBytes(io.ReadCloser) ([]byte, error)

Bear in mind the caveat mentioned above, that the request body can only be read once, still holds. Here are some examples:

res, _ := rek.Get("https://httpbin.org/get")

s1, _ := rek.BodyAsString(res.Body()) // body is read here

s2, _ := rek.BodyAsString(res.Body()) // s2 is an empty string

Non-standard methods

rek natively supports the methods GET, POST, PUT, DELETE, PATCH, and HEAD. If you'd like to use a method that's not directly supported, you can use the Do function:

res, err := rek.Do("SOMEOTHERMETHOD", rek.Timeout(10 * time.Second))

Options

Headers

headers := map[string]string{
    "My-Custom-Header": "foo,bar,baz",
}

res, err := rek.Post("https://httpbin.org/post", rek.Headers(headers))

JSON

Pass in any struct:

type Comment struct {
    ID        int64     `json:"id"`
    Body      string    `json:"body"`
    Timestamp time.Time `json:"timestamp"`
}

comment := Comment{ID:47, Body:"Cool movie!", Timestamp: time.Now()}

res, err := rek.Post("https://httpbin.org/post", rek.Json(comment))

Request headers are automatically updated to include Content-Type as application/json;charset=utf-8.

Request timeout

res, err := rek.Get("https://httpbin.org/get", rek.Timeout(5 * time.Second))

Form data

form := map[string]string{
    "foo": "bar",
    "baq": "baz",
}

res, err := rek.Put("https://httpbin.org/put", rek.FormData(form))

Request headers are automatically updated to include Content-Type as application/x-www-form-urlencoded.

File upload

fieldName := "file"
filepath := "docs/README.md"
params := nil

res, err := rek.Post("https:/httpbin.org/post", rek.File(fieldName, filepath, params))

Request headers are automatically updated to include Content-Type as multipart/form-data; boundary=....

Basic auth

username, password := "user", "pass"

res, _ := rek.Get(fmt.Sprintf("https://httpbin.org/basic-auth/%s/%s", username, password),
    rek.BasicAuth(username, password))

fmt.Println(res.StatusCode()) // 200

res, _ := rek.Get(fmt.Sprintf("https://httpbin.org/basic-auth/%s/other", username, password),
    rek.BasicAuth(username, password))

fmt.Println(res.StatusCode()) // 401

Data

Takes any input and serializes it to a []byte:

data := map[string]interface{
    "age": 38,
    "name": "Luc",
}

res, err := rek.Post("https://httpbin.org/post", rek.Data(data))

Request headers are automatically updated to include Content-Type as application/octet-stream.

User agent

res, err := rek.Post("https://httpbin.org/post", rek.UserAgent("ThisGuy"))

Bearer (useful for JSON Web Tokens)

token := "... token ..."

res, err := rek.Post("https://httpbin.org/post", rek.Bearer(token))

Request modifier

Supply a function that modifies the *http.Request (after all other supplied options have been applied to the request):

modifier := func(r *http.Request) {
   // Do whatever you want with the request
}

res, err := rek.Get("https://httpbin.org/get", rek.RequestModifier(modifier))

Accept

Apply an Accept header to the request:

res, err := rek.Get("https://httpbin.org/get", rek.Accept("application/tar+gzip"))

API key

Add an API key to the request as an Authorization header (where the value is Basic ${KEY}):

res, err := rek.Get("https://some-secure-api.io", rek.ApiKey("a1b2c3..."))

Context

Supply a Context to the request:

ctx, cancel := context.WithCancel(context.Background())

res, err := rek.Get("https://long-winded-api.io", rek.Context(ctx))

// Ugh, I don't want this request to happen anymore

cancel()

OAuth2

You can add an OAuth2 configuration and token to your request:

conf := &oauth2.Config{
	ClientID:     "YOUR_CLIENT_ID",
	ClientSecret: "YOUR_CLIENT_SECRET",
	Scopes:       []string{"SCOPE1", "SCOPE2"},
	Endpoint: oauth2.Endpoint{
		AuthURL:  "https://provider.com/o/oauth2/auth",
		TokenURL: "https://provider.com/o/oauth2/token",
	},
}

tok, err := conf.Exchange(ctx, code)
// handle error

res, err := rek.Get("https://oauth2-protected-site.com", rek.OAuth2(conf, tok))

Custom client

You can pass in your own *http.Client as the basis for the request:

client := &http.Client{
    // Custom attributes
}

res, err := rek.Get("https://httpbin.org/get", rek.Client(client))

Validation

It's important to bear in mind that rek provides no validation for the options that you provide on a specific request and doesn't provide any constraints on which options can be used with which request method. Some options may not make sense for some methods, e.g. request JSON on a HEAD request, but I leave it up to the end user to supply their own constraints. One exception is that the request body can only be set once. If you attempt to set it more than once you'll get a ErrRequestBodySetMultipleTimes error. This, for example, will throw that error:

comment := Comment{Body: "This movie sucked"}

_, err := rek.Post("https://httpbin.org/post",
    rek.Json(comment),
    rek.FormData(map[string]string{"foo": "bar"}))

fmt.Println(errors.Is(err, rek.ErrRequestBodySetMultipleTimes)) // true
Issues
  • Should Body be closed?

    Should Body be closed?

    I don't see the body being closed in the helper methods which might cause issues with the underlying http connection (reuse, as per http docs). If you agree, I can file a PR (starting with the helpers).

    // The response body as a string. Bear in mind that the response body can only be read once.
    func BodyAsString(r io.ReadCloser) (string, error) {
    	bs, err := bodyBytes(r)
    	if err != nil {
    		return "", err
    	}
    	return string(bs), nil
    
    
    opened by embano1 2
  • Example in readme is incorrect, method res.Text() doesn't exist

    Example in readme is incorrect, method res.Text() doesn't exist

    When you follow the example code in the readme, it doesn't work:

    type Comment struct {
        Body string `json:"body"`
    }
    
    comment := Comment{Body: "This movie sucked"}
    
    headers := map[string]string{"My-Custom-Header", "foo,bar,baz"}
    
    res, err := rek.Post("https://httpbin.org/post",
        rek.Json(comment),
        rek.Headers(headers),
        rek.BasicAuth("user", "pass"),
        rek.Timeout(5 * time.Second),
    )
    
    fmt.Println(res.StatusCode())
    fmt.Println(res.Text())
    

    The problem is the last line. The res struct doesn't have a function called .Text().

    So, either the method should be added or the example code should be updated ;-)

    Maybe the best option would be to incorporate the helper functions rek.BodyAsString and rek.BodyAsBytes and do something like:

    type Comment struct {
        Body string `json:"body"`
    }
    
    comment := Comment{Body: "This movie sucked"}
    
    headers := map[string]string{"My-Custom-Header", "foo,bar,baz"}
    
    res, err := rek.Post("https://httpbin.org/post",
        rek.Json(comment),
        rek.Headers(headers),
        rek.BasicAuth("user", "pass"),
        rek.Timeout(5 * time.Second),
    )
    
    fmt.Println(res.StatusCode())
    
    text, err := res.BodyAsString()
    fmt.Println(text)
    
    opened by pieterclaerhout 1
Owner
Luc Perkins
Software engineer @timberio working on Vector. Creator of Purple (https://purpledb.io). Co-author of Seven Databases in Seven Weeks (7dbs.io).
Luc Perkins
viagh.NewHTTPClient returns a *http.Client that makes API requests via the gh command.

viagh viagh.NewHTTPClient returns a *http.Client that makes API requests via the gh command. Why viagh? When writing a GitHub CLI extension, the exten

Ken’ichiro Oyama 1 Dec 24, 2021
A http-relay server/client written in golang to forward requests to a service behind a nat router from web

http-relay This repo is WIP http-relay is a server/client application written in go(lang) to forward http(s) requests to an application behind a nat r

john dev 1 Dec 16, 2021
Gogrok is a self hosted, easy to use alternative to ngrok. It uses SSH as a base protocol, using channels and existing functionality to tunnel requests to an endpoint.

gogrok A simple, easy to use ngrok alternative (self hosted!) The server and client can also be easily embedded into your applications, see the 'serve

Tyler Stuyfzand 5 Jun 15, 2022
Package httpretty prints the HTTP requests you make with Go pretty on your terminal.

httpretty Package httpretty prints the HTTP requests of your Go programs pretty on your terminal screen. It is mostly inspired in curl's --verbose mod

Henrique Vicente 257 Jun 15, 2022
A tool that makes http requests and outputs the url and the content (optionally to file)

BKK Basic Crawler A tool that makes http requests and outputs the url and the content (optionally to file) How to run.. the tests go test the compiler

Jero Berlin 0 Nov 8, 2021
Application to shut down a machine using HTTP requests.

shutdownd Service to shut down a system using HTTP requests. Usage Here's a quick example of how you can use this software on Linux. Download or build

Lerk 1 Nov 15, 2021
HTTP requests for Gophers

Requests HTTP requests for Gophers. The problem: Go's net/http is powerful and versatile, but using it correctly for client requests can be extremely

Carl Johnson 354 Jun 22, 2022
Small round tripper to avoid triggering the "attention required" status of CloudFlare for HTTP requests

CloudFlare-ByPass-Go. Small round tripper to avoid triggering the "attention req

Trewis [work] Scotch 11 Jun 20, 2022
Arvind Iyengar 1 Mar 21, 2022
This is a tool that will proxy simple HTTPS requests to an external HTTP endpoint

AcmeShield A secured HTTP proxy that forwards requests from a remote service(Postman). This is a tool that will proxy simple HTTPS requests to an exte

Octavio Cano 1 Mar 21, 2022
A Caddy v2 extension to apply rate-limiting for HTTP requests

ratelimit A Caddy v2 extension to apply rate-limiting for HTTP requests. Installation $ xcaddy build --with github.com/owlwang/caddy-ratelimit Caddyfi

owlwang 0 Jan 28, 2022
🍔 Product-storage service, work on gRPC. Client sends the URL to download products, and requests the result.

?? Product-storage service, work on gRPC. Client sends the URL to download products, and requests the result. The server transfer request to a third-party resource for .csv-file uploading and saves the products to own database.

Pavel V 9 Dec 16, 2021
PlanB: a HTTP and websocket proxy backed by Redis and inspired by Hipache.

PlanB: a distributed HTTP and websocket proxy What Is It? PlanB is a HTTP and websocket proxy backed by Redis and inspired by Hipache. It aims to be f

vinay badhan 1 Mar 20, 2022
MPD client inspired by ncmpcpp written in GO with builtin cover art previews.

goMP MPD client inspired by ncmpcpp written in GO demo.mp4 Roadmap Add Functionality to Sort out most played songs Add a config parser Image Previews

Aditya Kurdunkar 30 Jun 2, 2022
Simple GUI to convert Charles headers to golang's default http client (net/http)

Charles-to-Go Simple GUI to convert Charles headers to golang's default http client (net/http) Usage Compile code to a binary, go build -ldflags -H=wi

null 0 Dec 14, 2021
Tailscale-client-go - A client implementation for the Tailscale HTTP API

tailscale-client-go A client implementation for the Tailscale HTTP API Example p

David Bond 9 Jun 8, 2022
Use ICMP requests to check the alive subnet.

Doge-AliveCheck Use ICMP requests to check the alive subnet. Build go build -ldflags "-s -w" -trimpath Usage Doge-AliveCheck.exe

TimWhite 18 Dec 15, 2021
node api for proxying requests with golang to spoof tls fingerprint

WIP NOT BUILT WONT WORK AS IS gotTLS A node websocket api version of https://github.com/Carcraftz/TLS-Fingerprint-API to spoof TLS fingerprint to prev

evade 3 Sep 28, 2021
A Caddy v2 plugin to track requests in Pirsch analytics

caddy-pirsch-plugin A Caddy v2 plugin to track requests in Pirsch Analytics. Usage pirsch [<matcher>] { client_id <pirsch-client-id> client_se

Ferdinand Mütsch 4 Jun 11, 2022