Transparent TLS and HTTP proxy serve and operate on all 65535 ports, with domain regex whitelist and rest api control

Overview

goshkan

Transparent TLS and HTTP proxy serve & operating on all 65535 ports, with domain regex whitelist and rest api control

  • tls and http on same port (peyload inspection)
  • handle connections with low memory footprint
  • regex domain filtering
  • oprate on all ports (with iptables redirect)
  • rest api for add/delete domains
  • DNAT friendly, find client actual dst port from conntrack table
  • written with golang standard packages (except mysql-driver)

compile from source

clone this project, use git clone https://github.com/Sina-Ghaderi/goshkan.git
mirror snix repository: gti clone https://git.snix.ir/goshkan
goshkan written with golang, so you need to install compiler apt install golang
finally run go build on project root directory to compile source code.
FYI: pre-compiled goshkan binary is available at Releases

required dependency

first of all, goshkan uses mysql server to store regex patterns, so mysql or mariadb server is required. on debian mariadb server can be installed by executing apt install mariadb-server
remember to run mysql_secure_installation after installation to secure your sql server.

now its time to create database, user and tables. in order to do this you need to login to mysql with root user: mysql -u root and on mysql shell run these commands: (remember to change username and password)

CREATE DATABASE goshkan;
CREATE USER 'username'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON goshkan.* TO 'username'@'localhost';

CREATE TABLE goshkan.regext (
	regexid INT UNSIGNED auto_increment NOT NULL,
	regexstr LONGTEXT NOT NULL,
	PRIMARY KEY (regexid)
)

ENGINE=InnoDB
DEFAULT CHARSET=utf8mb4
COLLATE=utf8mb4_general_ci;
FLUSH PRIVILEGES;
EXIT;

thats it, mysql installation is completed now. note: if you planning to host mysql and goshkan on separate servers, you should change localhost to goshkan server address.

configuration file

configuration file is based on json, the default configuration file path is ./server-config.json for using another file as config, you should specify --config flag: goshkan --config

default config file content:

{
    "MYSQL_PASSWORD": "password",
    "MYSQL_USERNAME": "username",
    "DOMAIN_MEMTTL": 60,
    "MYSQL_DATABASE": "goshkan",
    "MYSQL_ADDRESS": "localhost",
    "CONNECT_TIMEOUT": 10,
    "LISTEN_ADDRESS": "192.168.122.149:8443",
    "CLIENT_TIMEOUT": 15,
    "HTTPAPI_LISTEN": "127.0.0.1:8080",
    "LOGS_DEBUGGING": true
}

MYSQL_PASSWORD: database username password (string)
MYSQL_USERNAME: database username (string)
MYSQL_DATABASE: mysql database name (string)
MYSQL_ADDRESS: mysql server address and port, default port 3306 (string host:port)
CONNECT_TIMEOUT: connect to upstream server connection timeout in second (integer > 0)
CLIENT_TIMEOUT: client connection timeout in second (integer > 0)
DOMAIN_MEMTTL: in memory domain cache aging time in second, value 0 disable this functionality (integer >= 0)
LISTEN_ADDRESS: tls/http proxy listen address and port (string addr:port)
HTTPAPI_LISTEN: http rest api listen address and port (string addr:port)
LOGS_DEBUGGING: debugging enable (boolean true|false)

summary about DOMAIN_MEMTTL:
goshkan uses in memory cache (hashtable) to store recently connected domains and addresses, the reason for this is to reduce time complexity.
in nutshell time complexity is the amount of time taken by an algorithm to run, which in this case is regex matching algorithm. when client connect to upstream host, goshkan store matched upstream address or domain in memory, so for next upcoming connection doesn't have to go through regex matching again, instead uses hashtable with time complexity of O(1).

domains and addresses would be stored in memory with a timer, when this timer elapsed, domain or address will be removed from memory (age-out) unless new connection with this domain/address established before that. in this case, the timer will be reset. DOMAIN_MEMTTL value indicate this timer time duration (in second). if DOMAIN_MEMTTL is 0 memory cache functionality would be disabled entirely.
you should enable it if your server has decent amount of memory (a.k.a RAM)

iptables redirect all ports

forwarding all ports to goshkan with iptables redirect:
this command would redirect all tcp packets (on all ports) to goshkan proxy port if packet destination address is 192.168.122.149 and input interface is ens3

note: after this you can't serve another service on this address (192.168.122.149 in my case) because there is no port left. for solving this issue you may want to exclude your service ports from being forwarding to goshkan proxy port by executing this iptables -t nat -A PREROUTING -i ens3 -d 192.168.122.149 -p tcp -m tcp --dport 22 -j ACCEPT command before following one (replace 22 with your service port)
but the best solution would be to bind your services with another ip-address or interface.

iptables -t nat -A PREROUTING -i ens3 -d 192.168.122.149 -p tcp -m tcp --dport 1:65535 -j REDIRECT --to-ports 8443

max open files on linux

by default goshkan can open max 1024 file (connection), if its not enough change this value in systemd service file or with ulimit command.
see systemd documention and ulimit manual

api reference documention

get rest api documention in pdf format by sending GET / to HTTPAPI_LISTEN address, or find it under goshkan/apid/ directory.
this is open api without authentication, you shouldn't expose it to public, nginx or apache can protect this api with basic http authentication.

security notice

  • do NOT add regex pattern that allows localhost , 127.0.0.1 or any of your server ip-address or domains. can cause server connection loop or exposing internal server resources to unauthorized users.

contribute to this project

feel free to email me [email protected] if you want to contribute to this project

Copyright 2021 SNIX LLC [email protected]
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License version 2 as published by the Free Software Foundation.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

Issues
  • panic: runtime error: slice bounds out of range

    panic: runtime error: slice bounds out of range

    Bug Description

    panic occurs on google.com domains.. binary log output:

    2022/02/04 19:50:30 CONNEC: connected to upstream server, address: golang.org:80
    2022/02/04 19:50:40 CONNEC: connected to upstream server, address: google.com:80
    panic: runtime error: slice bounds out of range [:4] with capacity 1
    
    goroutine 126 [running]:
    io.(*teeReader).Read(0xc000049600, {0xc0001b44c1, 0x65b1db, 0x1})
    	/usr/lib/go-1.17/src/io/io.go:562 +0xa9
    goshkan/ntcp.readOnly.Read(...)
    	/home/sina/Documents/test/xc/goshkan/ntcp/ntcp.go:35
    net/http.(*connReader).backgroundRead(0xc0001b44b0)
    	/usr/lib/go-1.17/src/net/http/server.go:672 +0x3f
    created by net/http.(*connReader).startBackgroundRead
    	/usr/lib/go-1.17/src/net/http/server.go:668 +0xcf
    
    bug fixed 
    opened by Sina-Ghaderi 1
  • non-http connection stuck on http proxy

    non-http connection stuck on http proxy

    non-http connection stuck on http proxy, http handle conn function never get released, bcz nothing ever sends on notify channel, bcz connection is not http and setHostName() never runs.

    TODO: FIX THIS

    bug fixed 
    opened by Sina-Ghaderi 1
Releases(goshkan-0.2.6)
Owner
Sina Ghaderi
Chain Smoker, also a creepy guy
Sina Ghaderi
Serve traffic (HTTP/gRPC) over SSH using Domain Sockets

Serve On SSH Introduction There is often a need to offer services for administrative purposes on servers or even for microservices that are running on

John Doak 7 Jan 9, 2022
Go Http Proxy with Authentication, Schedule Control, and Portal Control

goproxy Go Http Proxy with Authentication, Schedule Control, and Portal Control Why this tool? You may need to restrict my kids's youtube watch time i

Wu Shilin 2 Mar 27, 2022
Mutual TLS encryption TCP proxy with golang

mtls-tcp-proxy Mutual Authentication TLS encryption TCP proxy with golang Why? I created this because of sometimes, it is not possible for us to estab

Habibie Faried 3 Oct 3, 2021
TLDs finder: check domain name availability across all valid top-level domains

TLD:er TLDs finder — check domain name availability across all valid top-level d

Dwi Siswanto 43 May 14, 2022
Access more HTTP ports over CDN with this application.

More-Ports More Ports is a proxy service to establish all web-based applications on different ports on the server-side over a well known TCP port. It

Ahmet ÖZER 4 May 8, 2022
Header Block is a middleware plugin for Traefik to block request and response headers which regex matched by their name and/or value

Header Block is a middleware plugin for Traefik to block request and response headers which regex matched by their name and/or value Conf

null 3 Feb 14, 2022
Http-logging-proxy - A HTTP Logging Proxy For Golang

http-logging-proxy HTTP Logging Proxy Description This project builds a simple r

null 1 Feb 11, 2022
llb - It's a very simple but quick backend for proxy servers. Can be useful for fast redirection to predefined domain with zero memory allocation and fast response.

llb What the f--k it is? It's a very simple but quick backend for proxy servers. You can setup redirect to your main domain or just show HTTP/1.1 404

Kirill Danshin 12 Jan 23, 2022
Proxy your Go Module`s Import Path from your own domain to a public host (e.g. github.com).

Go Modules Remote Import Path Proxy Proxy your Go Module`s Import Path from your own domain to a public host (e.g. github.com). For example Uber (buil

Stefan Kürzeder 2 Nov 2, 2021
TFTP and HTTP server specifically designed to serve iPXE ROMs and scripts.

pixie TFTP and HTTP server specifically designed to serve iPXE ROMs and scripts. pixie comes embedded with the following ROMs provided by the iPXE pro

Adrian L Lange 12 Apr 6, 2022
Simple, secure and modern Go HTTP server to serve static sites, single-page applications or a file with ease

srv srv is a simple, secure and modern HTTP server, written in Go, to serve static sites, single-page applications or a file with ease. You can use it

Kevin Pollet 53 Apr 3, 2022
GoHTTPdirlist - Serve files and directories with Golang based http server

List Files and Directories with GoLang based HTTP Server TO MAKE IT WORK: go run

EMRE ASLAN 1 Dec 30, 2021
Control your legacy Reciva based internet radios (Crane, Grace Digital, Tangent, etc.) via REST api or web browser.

reciva-web-remote Control your legacy Reciva based internet radios (Crane, Grace Digital, Tangent, etc.) via REST api or web browser. Usage This progr

null 6 May 3, 2022
Fork of Go stdlib's net/http that works with alternative TLS libraries like refraction-networking/utls.

github.com/ooni/oohttp This repository contains a fork of Go's standard library net/http package including patches to allow using this HTTP code with

Open Observatory of Network Interference (OONI) 16 Apr 30, 2022
go HTTP client that makes it plain simple to configure TLS, basic auth, retries on specific errors, keep-alive connections, logging, timeouts etc.

goat Goat, is an HTTP client built on top of a standard Go http package, that is extremely easy to configure; no googling required. The idea is simila

VSPAZ 1 Nov 18, 2021
Bee is a tool to scan ports by TCP and UDP protocols

Bee - Port scan tool ?? Bee is a tool to scan ports by TCP and UDP protocols Building from Source Code First, we compile the source code with the ligh

kraken 2 Oct 10, 2021
Program to simultaneously listen and respond on multiple TCP/UDP ports

listen Program to simultaneously listen on multiple TCP/UDP ports and reply back to anything sent along with IP addresses and lengths of data received

Purplecarrot 1 Feb 20, 2022
oniongrok forwards ports on the local host to remote Onion addresses as Tor hidden services and vice-versa

oniongrok Onion addresses for anything. oniongrok forwards ports on the local host to remote Onion addresses as Tor hidden services and vice-versa. Wh

Casey Marshall 247 May 6, 2022