HTTP requests for Gophers

Overview

Requests GoDoc Go Report Card Gocover.io Mentioned in Awesome Go

Requests logo

HTTP requests for Gophers.

The problem: Go's net/http is powerful and versatile, but using it correctly for client requests can be extremely verbose.

The solution: The requests.Builder type is a convenient way to build, send, and handle HTTP requests. Builder has a fluent API with methods returning a pointer to the same struct, which allows for declaratively describing a request by method chaining.

Requests also comes with tools for building custom http transports, include a request recorder and replayer for testing.

Examples

Simple GET into a string

// code with net/http
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://example.com", nil)
if err != nil {
	// ...
}
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
b, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}
s := string(b)

// equivalent code using requests
var s string
err := requests.
	URL("http://example.com").
	ToString(&s).
	Fetch(context.Background())

// 5 lines vs. 13 lines

POST a raw body

err := requests.
	URL("https://postman-echo.com/post").
	BodyBytes([]byte(`hello, world`)).
	ContentType("text/plain").
	Fetch(context.Background())

// Equivalent code with net/http
body := bytes.NewReader(([]byte(`hello, world`))
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "https://postman-echo.com/post", body)
if err != nil {
	// ...
}
req.Header.Set("Content-Type", "text/plain")
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
_, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}
// 5 lines vs. 14 lines

GET a JSON object

var post placeholder
err := requests.
	URL("https://jsonplaceholder.typicode.com").
	Pathf("/posts/%d", 1).
	ToJSON(&post).
	Fetch(context.Background())

// Equivalent code with net/http
var post placeholder
u, err := url.Parse("https://jsonplaceholder.typicode.com")
if err != nil {
	// ...
}
u.Path = fmt.Sprintf("/posts/%d", 1)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
if err != nil {
	// ...
}
res, err := http.DefaultClient.Do(req)
if err != nil {
	// ...
}
defer res.Body.Close()
b, err := io.ReadAll(res.Body)
if err != nil {
	// ...
}
err := json.Unmarshal(b, &post)
if err != nil {
	// ...
}
// 6 lines vs. 23 lines

POST a JSON object and parse the response

var res placeholder
req := placeholder{
	Title:  "foo",
	Body:   "baz",
	UserID: 1,
}
err := requests.
	URL("/posts").
	Host("jsonplaceholder.typicode.com").
	BodyJSON(&req).
	ToJSON(&res).
	Fetch(context.Background())
// net/http equivalent left as an exercise for the reader

Set custom headers for a request

// Set headers
var headers postman
err := requests.
	URL("https://postman-echo.com/get").
	UserAgent("bond/james-bond").
	ContentType("secret").
	Header("martini", "shaken").
	Fetch(context.Background())

Easily manipulate query parameters

var params postman
err := requests.
	URL("https://postman-echo.com/get?a=1&b=2").
	Param("b", "3").
	Param("c", "4").
	Fetch(context.Background())
	// URL is https://postman-echo.com/get?a=1&b=3&c=4

Record and replay responses

// record a request to the file system
cl.Transport = requests.Record(nil, "somedir")
var s1, s2 string
err := requests.URL("http://example.com").
	Client(&cl).
	ToString(&s1).
	Fetch(context.Background())
check(err)

// now replay the request in tests
cl.Transport = requests.Replay("somedir")
err = requests.URL("http://example.com").
	Client(&cl).
	ToString(&s2).
	Fetch(context.Background())
check(err)
assert(s1 == s2) // true
You might also like...
Use ICMP requests to check the alive subnet.

Doge-AliveCheck Use ICMP requests to check the alive subnet. Build go build -ldflags "-s -w" -trimpath Usage Doge-AliveCheck.exe

node api for proxying requests with golang to spoof tls fingerprint

WIP NOT BUILT WONT WORK AS IS gotTLS A node websocket api version of https://github.com/Carcraftz/TLS-Fingerprint-API to spoof TLS fingerprint to prev

A Caddy v2 plugin to track requests in Pirsch analytics

caddy-pirsch-plugin A Caddy v2 plugin to track requests in Pirsch Analytics. Usage pirsch [matcher] { client_id pirsch-client-id client_se

A very simple ssh-agent that signs requests in parallel

ssh-agent A very simple ssh-agent that signs requests in parallel. Usage To install and run the agent simply run: $ go install github.com/Woellchen/ss

Enable requests served by caddy for distributed tracing via The OpenTracing Project.

caddy-opentracing Enable requests served by caddy for distributed tracing via The OpenTracing Project. Dependencies The Go OpenTracing Library Jaeger,

🍔 Product-storage service, work on gRPC. Client sends the URL to download products, and requests the result.

🍔 Product-storage service, work on gRPC. Client sends the URL to download products, and requests the result. The server transfer request to a third-party resource for .csv-file uploading and saves the products to own database.

A service to proxy requests to a given backend service.

Proxy Service A service to proxy requests to a given backend service. Go 1.17+ Clone git clone [email protected]:janu-cambrelen/proxy-service.git Run (L

Traefik plugin to proxy requests to owasp/modsecurity-crs:apache container
Traefik plugin to proxy requests to owasp/modsecurity-crs:apache container

Traefik Modsecurity Plugin Traefik plugin to proxy requests to owasp/modsecurity-crs:apache Traefik Modsecurity Plugin Demo Full Configuration with do

Fastest python library for making asynchronous group requests.

FGrequests: Fastest Asynchronous Group Requests Installation Install using pip: pip install fgrequests Documentation Pretty easy to use. import fgrequ

Comments
  • Builder: Add ToFile

    Builder: Add ToFile

    that writes the response body to a file. The file and its parent directories are automatilly created with default permissions.

    Follow up the discussion in https://lobste.rs/s/ocjhfm/why_i_wrote_my_own_go_http_client#c_ijhq8c.

    opened by favadi 5
  • Implement pre-request hook functionality.

    Implement pre-request hook functionality.

    A pre-request hook is called when the request is fully constructed, but not yet returned. This allows users to perform an operation when the request is complete but not yet sent. For example, setting extra headers that depend on the request body or path or other details.

    This came up for an API I'm wrapping. I need to add a header that holds a signature that is based on the path and the body of the request. This extension allows me to do so cleanly and easily. Other uses could be logging of the request or something like rate limiting. I hope this will be useful for others too.

    opened by vdeurzen 2
  • Add support for clients with X509 certificates or X509 proxies

    Add support for clients with X509 certificates or X509 proxies

    At CERN we need X509 support in our clients (see X509 certificates). As such I find that it can be easily added to your project and significantly simplify writing such clients. On a client side it can be done as following:

        var s string
        key := "/path/userkey.pem"
        cert := "/path/usercert.pem"
        caPath := "/etc/grid-security/certificates"
        skipVerify := true
        verbose := false
        ctx := context.TODO()
        client := requests.X509HttpClient(key, cert, caPath, skipVerify, verbose)
        err := requests.
            URL(url).
            Client(client).
            ToString(&s).
            Fetch(ctx)
    

    This PR provides codebase to support HTTP clients which can be initialized either with x509 certificates or x509 proxy. Of course, it is optional and I will understand if you will not accept it, e.g. http x509 client can be put into separate packages. I only find that it may be useful to enrich requests library to specific kind of clients and reduce overhead for end-users who may choose to use your library. Of course, I can provide additional example test files if you'll be interested in this PR.

    opened by vkuznet 2
  • Add HTTP dumps for request and response

    Add HTTP dumps for request and response

    Hi, thanks for providing this useful package. I want to add couple of PRs to improve it. The first one, is to add HTTP request and response dumps. Here is how it will look on a client side:

       var s string
        err := requests.
            URL(url).
            DumpRequest().
            DumpResponse().
            ToString(&s).
            Fetch(ctx)
    

    Basically, DumpRequest and DumpResponse provides hooks to dump corresponding HTTP requests and response on stdout. Here is how it may look like on stdout:

    # HTTP request dump
    GET /some-api HTTP/1.1
    Host: host.com
    User-Agent: Go-http-client/1.1
    Accept-Encoding: gzip
    ....
    
    # HTTP response dump
    \
    HTTP/1.1 200 OK
    Content-Length: 89
    Cms-Server-Time: D=40530 t=1639863873694276
    Content-Type: text/plain; charset=utf-8
    Date: Sat, 18 Dec 2021 21:44:33 GMT
    ...
    

    In case of error, the user may see actual HTTP request and response structs too. Feel free to make comments about syntax of the printouts and I may adjust it further.

    opened by vkuznet 1
Owner
Carl Johnson
Carl M. Johnson is a philosopher and programmer, currently employed as the Technology Director for @spotlightpa.
Carl Johnson
Package httpretty prints the HTTP requests you make with Go pretty on your terminal.

httpretty Package httpretty prints the HTTP requests of your Go programs pretty on your terminal screen. It is mostly inspired in curl's --verbose mod

Henrique Vicente 279 Nov 16, 2022
An easy HTTP client for Go. Inspired by the immortal Requests.

rek An easy HTTP client for Go inspired by Requests, plus all the Go-specific goodies you'd hope for in a client. Here's an example: // GET request re

Luc Perkins 383 Sep 20, 2022
A tool that makes http requests and outputs the url and the content (optionally to file)

BKK Basic Crawler A tool that makes http requests and outputs the url and the content (optionally to file) How to run.. the tests go test the compiler

Jero Berlin 0 Nov 8, 2021
viagh.NewHTTPClient returns a *http.Client that makes API requests via the gh command.

viagh viagh.NewHTTPClient returns a *http.Client that makes API requests via the gh command. Why viagh? When writing a GitHub CLI extension, the exten

Ken’ichiro Oyama 1 Dec 24, 2021
Application to shut down a machine using HTTP requests.

shutdownd Service to shut down a system using HTTP requests. Usage Here's a quick example of how you can use this software on Linux. Download or build

Lerk 1 Nov 15, 2021
A http-relay server/client written in golang to forward requests to a service behind a nat router from web

http-relay This repo is WIP http-relay is a server/client application written in go(lang) to forward http(s) requests to an application behind a nat r

john dev 1 Dec 16, 2021
Small round tripper to avoid triggering the "attention required" status of CloudFlare for HTTP requests

CloudFlare-ByPass-Go. Small round tripper to avoid triggering the "attention req

Trewis [work] Scotch 19 Nov 14, 2022
Arvind Iyengar 1 Mar 21, 2022
This is a tool that will proxy simple HTTPS requests to an external HTTP endpoint

AcmeShield A secured HTTP proxy that forwards requests from a remote service(Postman). This is a tool that will proxy simple HTTPS requests to an exte

Octavio Cano 1 Mar 21, 2022
A Caddy v2 extension to apply rate-limiting for HTTP requests

ratelimit A Caddy v2 extension to apply rate-limiting for HTTP requests. Installation $ xcaddy build --with github.com/owlwang/caddy-ratelimit Caddyfi

owlwang 0 Jan 28, 2022