The f stands for fly

Related tags

Miscellaneous fhttp
Overview

fhttp

NOTE This maintenance of this library has move over to userflynet. The only use for this repository is so imports will not break.

The f stands for flex. fhttp is a fork of net/http that provides an array of features pertaining to the fingerprint of the golang http client. Through these changes, the http client becomes much more flexible, and when combined with transports such as uTLS can mitigate servers from fingerprinting requests and see that it is made by golang, making them look like they originate from a regular chrome browser.

Documentation can be contributed, otherwise, look at tests and examples. The main one should be example_client_test.go.

Features

Ordered Headers

Allows for pseudo header order and normal header order. Most of the code is taken from this pr.

Connection settings

Has Chrome-like connection settings:

SETTINGS_HEADER_TABLE_SIZE = 65536 (2^16)
SETTINGS_ENABLE_PUSH = 1
SETTINGS_MAX_CONCURRENT_STREAMS = 1000
SETTINGS_INITIAL_WINDOW_SIZE = 6291456
SETTINGS_MAX_FRAME_SIZE = 16384 (2^14)
SETTINGS_MAX_HEADER_LIST_SIZE = 262144 (2^18)

Default net/http settings:

SETTINGS_HEADER_TABLE_SIZE = 4096
SETTINGS_ENABLE_PUSH = 0
SETTINGS_MAX_CONCURRENT_STREAMS = unlimited
SETTINGS_INITIAL_WINDOW_SIZE = 4194304
SETTINGS_MAX_FRAME_SIZE = 16384
SETTINGS_MAX_HEADER_LIST_SIZE = 10485760

ENABLE_PUSH implementation was merged from this pull request

gzip, deflate, br encoding

Actually supports and implements encoding gzip, deflate, br

Pseudo header order

Supports pseudo header order for http2 to mitigate fingerprinting. Read more about it here

Backward compatible with net/http

Although this library is an extension of net/http, it is also meant to be backward compatible. Replacing

import (
	"net/http"
)

with

import (
	http "github.com/useflyent/fhttp"
)

SHOULD not break anything.

Credits

Special thanks to the following people for helping me with this project.

  • cc for guiding me when I first started this project and inspiring me with cclient

  • umasi for being good rubber ducky and giving me tips for http2 headers

You might also like...
Dbt-postgres-proxy - Proxy server which intercepts and compiles dbt queries on the fly
Dbt-postgres-proxy - Proxy server which intercepts and compiles dbt queries on the fly

dbt-postgres-proxy A reverse proxy for postgres which compiles queries in flight

Litestream-read-replica-demo - A demo application for running live read replication on fly.io with Litestream

Litestream Read Replica Demo A demo application for running live read replicatio

Comments
  • Not backwards compatible with net/http?

    Not backwards compatible with net/http?

    I'm working on adapting another project that makes use of this library to emulate the TLS fingerprint of chrome. In particular, I'm trying to establish a WebSocket connection, which the library doesn't support currently. I'm running into an issue (seemingly) stemming from this library- during the course of establishing my WS connection, I need to call functions that expect a value of type net/http. When I try to use fhttp in its place, the code doesn't seem to type check.

    As an example, here's some code that produces an issue:

    connBackend, resp, err := dialer.Dial(pageURL, req.Header)
    

    image

    I encounter a similar problem anywhere that a vanilla net/http property is expected, but an fhttp property is supplied.

    What I have tried:

    • Casting the req.Header as a net/http.Header - this works for this case, but later on when I need to pass along the ResponseWriter or something more complex, it doesn't:
    connPub, err := upgrader.Upgrade(w, req, upgradeHeader)
    

    ^ Here I can convert upgradeHeader to the desired type but not the first two. I feel like this isn't the right approach- fhttp should be a drop in, no? Is there any workaround?

    opened by nicoandmee 2
  • Minor Fixes

    Minor Fixes

    This Pull Request includes two minor changes, as shown by the two commits that form it.

    The first change involves the introduction of a RWMutex in header.go, as @ShrimpAIO reported a Race Condition due to SortedKeyValues() and writeSubset() both trying to access the exclude map at the same time when running several tasks at once, resulting in a Fatal Error ("concurrent map read and map write").

    The second change mainly involves the modification of README.md, formatting the document following Google's Markdown Style Guide. Additionally, the Note at the beginning of the file was commented out, as should only be used in @zMrKrabz's original repository. Finally, some spelling errors were corrected, including one pointed out by @0xflotus in this Open Pull Request.

    opened by subreme 0
Owner
null
QBS stands for Query By Struct. A Go ORM.

Qbs Qbs stands for Query By Struct. A Go ORM. 中文版 README ChangeLog 2013.03.14: index name has changed to {table name}_{column name}. For existing appl

Evan Zhou 549 Sep 9, 2022
MNA - stands for mobile number assignment - a small zero external dependency golang library that is used to identify mobile number assignment in tanzania

MNA - stands for mobile number assignment - a small zero external dependency golang library that is used to identify mobile number assignment in tanzania

TECHCRAFT TECHNOLOGIES LIMITED 8 Nov 29, 2021
Convert Golang Struct To GraphQL Object On The Fly

Straf Convert Golang Struct To GraphQL Object On The Fly Easily Create GraphQL Schemas Example Converting struct to GraphQL Object type UserExtra stru

Roshan Mehta 34 Oct 26, 2022
:link: Generate HTML and CSS together, on the fly

On The Fly Package for generating HTML and CSS together, on the fly. Can also be used for generating HTML, XML or CSS (or templates). HTML and CSS can

Alexander F. Rødseth 43 Oct 12, 2022
This library generate a new tlsconfig usable within go standard library configured with a self-signed certificate generated on the fly

sslcert This library generate a new tlsconfig usable within go standard library configured with a self-signed certificate generated on the fly. Exampl

ProjectDiscovery 8 Nov 21, 2022
Transform PromQL Expressions on the fly

promql-transform Transforms PromQL expressions on the fly Usage Given the expression job:request_latency_seconds:mean5m{job=\"myjob\"} > 0.5 Running

Canonical 1 Jun 21, 2022
Create inlets servers on fly.io

inlets-on-fly inlets-on-fly automates the task of creating an inlets-pro exit-server (tunnel server) on the fly.io platform. This automation started a

Johan Siebens 8 Jan 8, 2022
Kana - Simple discord bot for generate code snippet on the fly

kana discord bot for generate code snippet on the fly, made with graphene Invite

nekonako 1 Jan 25, 2022
Asset storage and on-the-fly image resize

air - Asset & Image Resize Asset storage and on-the-fly image resize powered by libvips. Uploading an asset $ http -f POST http://127.0.0.1:1323/uploa

Alexandre Ferland 0 Feb 5, 2022