A cli that exposes your local resources to kubernetes

Related tags

DevOps Tools ktunnel
Overview

Ktunnel logo

ktunnel

Status GitHub Issues GitHub Pull Requests License


Expose your local resources to kubernetes

πŸ“ Table of Contents

🧐 About

Ktunnel is a CLI tool that establishes a reverse tunnel between a kubernetes cluster and your local machine. It lets you expose your machine as a service in the cluster or expose it to a specific deployment. You can also use the client and server without the orchestration part. Although ktunnel is identified with kubernetes, it can also be used as a reverse tunnel on any other remote system

Ktunnel was born out of the need to access my development host when running applications on kubernetes. The aim of this project is to be a holistic solution to this specific problem (accessing the local machine from a kubernetes pod). If you found this tool to be helpful on other scenarios, or have any suggesstions for new features - I would love to get in touch.

Ktunnel schema

Ktunnel schema

🏁 Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Installation

Homebrew

brew tap omrikiei/ktunnel
brew install ktunnel

From the releases page

Download here and extract it to a local bin path

Building from source

Clone the project

git clone https://github.com/omrikiei/ktunnel; cd ktunnel

Build the binary

CGO_ENABLED=0 go build -ldflags="-s -w"

You can them move it to your bin path

sudo mv ./ktunnel /usr/local/bin/ktunnel

Test the commamd

ktunnel -h

🎈 Usage

Expose your local machine as a service in the cluster

This will allow pods in the cluster to access your local web app (listening on port 8000) via http (i.e kubernetes applications can send requests to myapp:8000)

ktunnel expose myapp 80:8000

Inject to an existing deployment

This will currently only work for deployments with 1 replica - it will expose a listening port on the pod through a tunnel to your local machine

ktunnel inject deployment mydeployment 3306

✍️ Authors

See also the list of contributors who participated in this project.

Issues
  • Sessions being left open with

    Sessions being left open with "closing session... err: EOF session not found in openRequests"

    We are using ktunnel to use xdebug against our remote php pods. It works great, but we noticed that all connections are left open indefinitely.

    The ktunnel logs show errors that seem to be related:

    INFO[2020-07-22 22:37:08 closing session 8b033568-e519-46ea-a42d-164b12a3c46d; err: EOF 
    ERRO[2020-07-22 22:37:14 8b033568-e519-46ea-a42d-164b12a3c46d; session not found in openRequests 
    ERRO[2020-07-22 22:37:15 8b033568-e519-46ea-a42d-164b12a3c46d; session not found in openRequests 
    INFO[2020-07-22 22:38:02 closing session 407cef1f-3cf7-4505-83b6-4f05f665388f; err: EOF 
    ERRO[2020-07-22 22:38:11 407cef1f-3cf7-4505-83b6-4f05f665388f; session not found in openRequests 
    ERRO[2020-07-22 22:38:12 407cef1f-3cf7-4505-83b6-4f05f665388f; session not found in openRequests 
    

    How do we resolve this?

    bug help wanted wontfix 
    opened by jessebye 21
  • Subsequent requests hang for exactly 1 minute

    Subsequent requests hang for exactly 1 minute

    Hey,

    I've got it set up and it works, but the request/response round-trip is intermittently very, very slow, or hangs and doesn't complete.

    I have no idea what kinds of logs/tests to try to give more meaningful information about this issue, if there are any suggestions I'd be happy to undertake them.

    Cheers!

    opened by mysterybear 14
  • URL Parse Error

    URL Parse Error

    Trying to use ktunnel with Rancher deployed pod (the example pod ffrom this repo). The following error is displayed: INFO[0001] Injecting tunnel sidecar to dparsons/pyremotedebug INFO[0002] Waiting for deployment to be ready INFO[0007] Waiting for port forward to finish ERRO[0007] error upgrading connection: error creating request: parse https://rancher.plaidcloud.io%2Fk8s%2Fclusters%2Fc-wfdqx/api/v1/namespaces/dparsons/pods/pyremotedebug-9ff9fc4c-trbcg/portforward: invalid URL escape "%2F"

    bug 
    opened by DrDonk 14
  • Tunnel established but does not respond or shows high CPU usage

    Tunnel established but does not respond or shows high CPU usage

    Hi @omrikiei, it's me again πŸ™ˆ

    I am currently trying to forward a simple local HTTP server into the cluster. Service, deployment and kTunnel (28688 -> 28688) are always established correctly and do not show any errors. Nevertheless the ktunnel server very often does not forward any requests to the local ktunnel client. Requests end in a timeout, although no errors can be seen.

    Steps to reproduce

    1. download latest (brew) binary 1.1.10 for macos
    2. execute ktunnel -v expose myapp 8080:8080
    3. see:
    INFO[0000] Exposed service's cluster ip is: 10.99.192.110 
    INFO[0000] waiting for deployment to be ready           
    INFO[0003] Waiting for port forward to finish           
    INFO[0004] Forwarding from 127.0.0.1:28688 -> 28688
    Forwarding from [::1]:28688 -> 28688 
    INFO[0004] starting tcp tunnel from source 8080 to target 8080 
    
    1. Perform a http request to service inside kubernetes (other) pod via curl -v http://myapp.default.svc.cluster.local:8080/ (ip gets resolved to exposed service cluster ip) no response, no incoming request to local server, no session created)
    2. CTRL+C on client side
    INFO[0027] Got exit signal, closing client tunnels and removing k8s objects 
    INFO[0027] Deleting service travellerapp                
    WARN[0027] error reading from stream: %vrpc error: code = Canceled desc = grpc: the client connection is closing 
    INFO[0027] Deleting deployment myapp  
    

    kubectl version

    Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.1", GitCommit:"4485c6f18cee9a5d3c3b4e523bd27972b1b53892", GitTreeState:"clean", BuildDate:"2019-07-18T14:25:20Z", GoVersion:"go1.12.7", Compiler:"gc", Platform:"darwin/amd64"}
    Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.4", GitCommit:"67d2fcf276fcd9cf743ad4be9a9ef5828adc082f", GitTreeState:"clean", BuildDate:"2019-09-18T14:41:55Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
    

    In a few cases the server forwards the request to the client and the HTTP request is successfully answered. After that the CPU load of the (local) ktunnel client increases to 500-600% (6 cores). The client still responds (but no further tunnel requests are forwarded), but the CPU usage does not decrease.

    The cpu usage increases after the client logs

    INFO[0005] b638fa36-d3ad-4cfd-8881-38b5cf9a8cca; new session; connecting to port 8080 
    

    Please let me know how I can help you to reproduce the bug. Thank you! Markus

    bug 
    opened by mrkswrnr 13
  • ktunnel inject deployment not working

    ktunnel inject deployment not working

    i was trying ktunnel to inject deployment but not working

    command used:-ktunnel inject deployment flask-deployment 80

    Error

    "status": "Failure",
      "message": "pods \"flask-deployment-6c86b6f988-vkfh8\" is forbidden: User \"system:anonymous\" cannot get resource \"pods/portforward\" in API group \"\" in the namespace \"default\"",
      "reason": "Forbidden",
      "details": {
        "name": "flask-deployment-6c86b6f988-vkfh8",
        "kind": "pods"
      },
    
    opened by apkapil 10
  • Expose doesn't work when source and target port are different

    Expose doesn't work when source and target port are different

    I tried to use the expose command to forward traffic from a kubernetes service to a local port:

    ktunnel expose myservice 80:8000
    

    Access from another pod in the same cluster using curl just says:

    Failed to connect to myservice port 80: Connection refused
    

    It looks like the service will be created with a source port of 80 and a target port of 8000, but the pod is listening on port 80. When I change the service manually and set source and target port to 80, it works.

    opened by gotshub 10
  • Failed reading from socket, exiting

    Failed reading from socket, exiting

    When we try to use ktunnel to establish a connection for using xdebug in one of our containers, it works for about 15 seconds before failing with this error:

    INFO[2020-04-09 16:32:45] f73e9c2d-ffdb-48dd-aa74-a4f8be552086; new session; connecting to port 9000
    ERRO[2020-04-09 16:32:58] f73e9c2d-ffdb-48dd-aa74-a4f8be552086; failed reading from socket, exiting: read tcp 127.0.0.1:55793->127.0.0.1:9000: read: connection reset by peer
    

    The ktunnel command we were using was ktunnel inject -n poor-cannon deployment api 8999:8999

    ktunnel version: 1.2.2

    bug 
    opened by jessebye 10
  • ERRO[0007] Error sending init tunnel request: rpc error: code = Unavailable

    ERRO[0007] Error sending init tunnel request: rpc error: code = Unavailable

    Hi @omrikiei :)

    Its me again, this time with the following:

    2020-02-06 09:52:13 >| $ ktunnel expose myWebapp 8084:8084 -n myNamespace
    INFO[0000] Exposed service's cluster ip is: 10.31.10.100
    INFO[0000] waiting for deployment to be ready           
    INFO[0007] All pods located for port-forwarding         
    INFO[0007] Waiting for port forward to finish           
    INFO[0007] Forwarding from 127.0.0.1:28688 -> 28688
    Forwarding from [::1]:28688 -> 28688 
    INFO[0007] starting tcp tunnel from source 8084 to target 8084 
    E0206 09:52:21.785327   10875 portforward.go:400] an error occurred forwarding 28688 -> 28688: error forwarding port 28688 to pod 3ddcdef600c6c0d884e3d71f5600a350623786083794ac4ef3a330c5ddc2944b, uid : exit status 1: 2020/02/06 07:52:21 socat[165522] E connect(5, AF=2 127.0.0.1:28688, 16): Connection refused
    ERRO[0007] Error sending init tunnel request: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection closed 
    INFO[0023] Got exit signal, closing client tunnels and removing k8s objects 
    INFO[0023] Deleting service myWebapp
    

    I'll appreciate :heart_eyes: any clues as always :)

    Kind regards, Boris

    bug 
    opened by outbounder 9
  • Hanging at INFO[0000] waiting for deployment to be ready

    Hanging at INFO[0000] waiting for deployment to be ready

    Image is v1.3.6

    Kubernetes side, deployment/pod seems to start up just fine:

    k get pods
    NAME                       READY   STATUS    RESTARTS   AGE
    ktunnel-77b65858f8-w9zg5   1/1     Running   0          35s
    
    k get deployments
    NAME      READY   UP-TO-DATE   AVAILABLE   AGE
    ktunnel   1/1     1            1           37s
    

    Edit: just tried with v1.3.3 also since I remember that working for sure last time I used this (It's me from #36 :D) issue persists just the same... I've checked I haven't got any lingering ktunnel resources server-side as well

    Edit: k logs on the server pod shows:

    INFO[2021-04-12 15:35:20.451] Starting to listen on port 28688          
    
    opened by mysterybear 8
  • Problems with large files

    Problems with large files

    I'm trying to proxy to a local web server, but it looks like ktunnel doesn't behave well with large files. Here's some output from curl:

    ❯ curl https://proxy.mbuffett.com/_next/static/chunks/main.js | wc -c
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
      8 4295k    8  377k    0     0  73862      0  0:00:59  0:00:05  0:00:54 73862
    curl: (18) transfer closed with 4012193 bytes remaining to read
    386672
    
    ❯ curl localhost:8085/_next/static/chunks/main.js | wc -c            
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100 4295k  100 4295k    0     0   209M      0 --:--:-- --:--:-- --:--:--  209M
    4398865
    

    Please, no judgement for the 4MB JS file :smile: I do see these logs from ktunnel about buffer sizes:

    INFO[2020-07-30 07:59:34] increasing buffer size to 65536              
    INFO[2020-07-30 07:59:34] increasing buffer size to 131072
    

    Wonder if there's anything I can do about this limitation? Thanks!

    And awesome program, exactly what I was looking for and easy to set up

    help wanted wontfix 
    opened by marcusbuffett 8
  • failed connecting to localhost on port 9002 scheme tcp

    failed connecting to localhost on port 9002 scheme tcp

    failed connecting to localhost on port 9002 scheme tcp

    ktunnel version 1.2.4 (1.2.5 actually)

    my PHPStorm is listening port 9002.

    ➜  ~ lsof -i :9002
    COMMAND    PID USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
    phpstorm 50781 cyao  465u  IPv6 0x32e9aa96a4c0d673      0t0  TCP *:pichat (LISTEN)
    

    And I also tried to start a TCP server to respond to the connection.

    ➜  ~ nc -vl 9002
    

    same error.

    ➜  ~ ktunnel -v inject deployment my-php-app 9000:9002 -n playground
    INFO[2020-05-01 00:59:00] Injecting tunnel sidecar to playground/my-php-app
    INFO[2020-05-01 00:59:01] Waiting for deployment to be ready
    INFO[2020-05-01 00:59:23] All pods located for port-forwarding
    DEBU[2020-05-01 00:59:23] Injecting to this pods: [my-php-app-6995b778b-8fzm5]
    INFO[2020-05-01 00:59:23] Waiting for port forward to finish
    INFO[2020-05-01 00:59:24] Forwarding from 127.0.0.1:28688 -> 28688
    Forwarding from [::1]:28688 -> 28688
    INFO[2020-05-01 00:59:24] starting tcp tunnel from source 9000 to target 9002
    DEBU[2020-05-01 00:59:25] attempting to receive from stream
    DEBU[2020-05-01 00:59:33] 780fece3-8339-40f4-bcf4-2da5a60fd5de; got session from server: 499<?xml version="1.0" encoding="iso-8859-1"?>
    <init xmlns="urn:debugger_protocol_v1" xmlns:xdebug="https://xdebug.org/dbgp/xdebug" fileuri="file:///var/www/html/www/public/index.php" language="PHP" xdebug:language_version="7.3.17" protocol_version="1.0" appid="38" idekey="PHPSTORM"><engine version="2.7.2"><![CDATA[Xdebug]]></engine><author><![CDATA[Derick Rethans]]></author><url><![CDATA[https://xdebug.org]]></url><copyright><![CDATA[Copyright (c) 2002-2019 by Derick Rethans]]></copyright></init>
    INFO[2020-05-01 00:59:33] 780fece3-8339-40f4-bcf4-2da5a60fd5de; new session; connecting to port 9002
    ERRO[2020-05-01 00:59:33] failed connecting to localhost on port 9002 scheme tcp
    
    enhancement help wanted good first issue 
    opened by kukat 8
  • Does it work with Gihub codespaces?

    Does it work with Gihub codespaces?

    It is mentioned in the README "Although ktunnel is identified with kubernetes, it can also be used as a reverse tunnel on any other remote system"

    Is there anyway I can use ktunnel to expose local resources(webserver) so that they can be accessed from within the GitHub codespaces container?

    Thanks

    documentation good first issue question 
    opened by sameer-coder 2
  • is authentication with token supported?

    is authentication with token supported?

    Hi How does ktunnel authenticate itself in the cluster? Can I use --token=<token> just like I use with openshift cli tool? e.g. oc project --token=<token>

    enhancement 
    opened by guai 1
  • Swap functionality

    Swap functionality

    I would like to replace current deployment of some app (lets call it myapp) with ktunnel to debug it locally. This is kind of similar to expose with reuse flag, but trying it when it tries to patch deployment and replace labels it fails as they are immutable by design.

    Is there any plans to add such functionality? I can see in TODO it was once in plan but not sure if it's still in scope of this? Or do you know any other ways of achieving above with ktunnel?

    enhancement 
    opened by koper89 0
  • ktunnel expose --ca-file option not used?

    ktunnel expose --ca-file option not used?

    Hi,

    First of all thank you so much for creating this awesome project πŸ‘

    I'm trying to enable TLS using --tls and --ca-file options but I'm not sure is it actually working. What I'm curious about is this part:

    https://github.com/omrikiei/ktunnel/blob/6bf63db523c81a9e06dfe2fd44607af910551e30/cmd/expose.go#L114

    Should the CertFile parameter be CaFile instead?

    https://github.com/omrikiei/ktunnel/blob/6bf63db523c81a9e06dfe2fd44607af910551e30/cmd/expose.go#L127

    opened by caedo960 9
  • ktunnel `--tls` should set `traefik.ingress.kubernetes.io/service.serversscheme: https` annotation

    ktunnel `--tls` should set `traefik.ingress.kubernetes.io/service.serversscheme: https` annotation

    Hello!

    I've been loving ktunnel, and just started using it with a cluster that uses Traefik. I spent a full 3 hours trying to figure out how to get the Ingress system to pass HTTPS thru my ktunnel service, and finally found out that the service needs to have an annotation that hints to Traefik that the service speaks TLS.

    When using --tls option, the service should be created with the traefik.ingress.kubernetes.io/service.serversscheme: https annotation, which would make Traefik ingresses pointing at the service work properly.

    Honestly, this seems more like a failure on Traefik's side, but since adding an annotation is a harmless and very light change, it seems like a nice convenience that ktunnel could add.

    Feel free to close this if you don't think ktunnel should deal with this - I have mixed feelings. It's not "correct", but it certainly would have saved me a lot of time.

    opened by erulabs 1
Releases(v1.4.8)
Owner
Omri Eival
Individual contributor and memory plumber @grubhub
Omri Eival
A k8s vault webhook is a Kubernetes webhook that can inject secrets into Kubernetes resources by connecting to multiple secret managers

k8s-vault-webhook is a Kubernetes admission webhook which listen for the events related to Kubernetes resources for injecting secret directly from sec

Opstree Container Kit 111 Apr 28, 2022
Kubernetes IN Docker - local clusters for testing Kubernetes

kind is a tool for running local Kubernetes clusters using Docker container "nodes".

Kubernetes SIGs 10.2k Aug 6, 2022
Kubernetes IN Docker - local clusters for testing Kubernetes

Please see Our Documentation for more in-depth installation etc. kind is a tool for running local Kubernetes clusters using Docker container "nodes".

Kaisen Linux 0 Feb 14, 2022
Kusk makes your OpenAPI definition the source of truth for API resources in your cluster

Kusk - use OpenAPI to configure Kubernetes What is Kusk? Developers deploying their REST APIs in Kubernetes shouldn't have to worry about managing res

kubeshop 6 Jun 27, 2022
Local Storage is one of HwameiStor components. It will provision the local LVM volume.

Local Storage Module English | Simplified_Chinese Introduction Local Storage is one of modules of HwameiStor which is a cloud native local storage sys

HwameiStor 164 Jul 29, 2022
kubetnl tunnels TCP connections from within a Kubernetes cluster to a cluster-external endpoint, e.g. to your local machine. (the perfect complement to kubectl port-forward)

kubetnl kubetnl (kube tunnel) is a command line utility to tunnel TCP connections from within a Kubernetes to a cluster-external endpoint, e.g. to you

null 4 Nov 16, 2021
kolm - Kubernetes on your local machine

kolm - Kubernetes on your local machine kolm is a tool for running a Kubernetes 'cluster' consisting of an etcd and a kube-apiserver on your local mac

Gardener on Metal 6 May 19, 2022
A kubernetes plugin which enables dynamically add or remove GPU resources for a running Pod

GPU Mounter GPU Mounter is a kubernetes plugin which enables add or remove GPU resources for running Pods. This Introduction(In Chinese) is recommende

XinYuan 72 Jul 18, 2022
A curated list of awesome Kubernetes tools and resources.

Awesome Kubernetes Resources A curated list of awesome Kubernetes tools and resources. Inspired by awesome list and donnemartin/awesome-aws. The Fiery

Tom Huang 1.3k Aug 6, 2022
Annotated and kubez-autoscaler-controller will maintain the HPA automatically for kubernetes resources.

Kubez-autoscaler Overview kubez-autoscaler ι€šθΏ‡δΈΊ deployment / statefulset 添加 annotations ηš„ζ–ΉεΌοΌŒθ‡ͺεŠ¨η»΄ζŠ€ε―ΉεΊ” HorizontalPodAutoscaler ηš„η”Ÿε‘½ε‘¨ζœŸ. Prerequisites 在 kuber

null 117 Aug 2, 2022
nano-gpu-agent is a Kubernetes device plugin for GPU resources allocation on node.

Nano GPU Agent About this Project Nano GPU Agent is a Kubernetes device plugin implement for gpu allocation and use in container. It runs as a Daemons

Nano GPU 40 Jun 10, 2022
Set of Kubernetes solutions for reusing idle resources of nodes by running extra batch jobs

Caelus Caelus is a set of Kubernetes solutions for reusing idle resources of nodes by running extra batch jobs, these resources come from the underuti

Tencent 277 Aug 1, 2022
Search Kubernetes Ingress resources.

kubectl-ingress-search Search Ingress resources. Installation Download from Releases page. cp kubectl-ingress-search /usr/local/bin/ use kubectl-ingre

Huang Huang 0 Nov 7, 2021
immutable, fluent, builders for Kubernetes resources

Dies - immutable, fluent, builders for Kubernetes resources Using dies Common methods Creating dies diegen die markers +die This project contains dies

Scott Andrews 3 May 6, 2022
A query server on Kubernetes resources

kql A query server on Kubernetes resources. Example curl command: # for query si

Pulak Kanti Bhowmick 7 Jan 13, 2022
Valse is an kubernetes exporter application that discovers resources such as version

What is the Valse? Valse is an kubernetes exporter application that discovers resources such as version, namespaces, node, pod, deployment, daemonset,

Ali 25 Jul 22, 2022
Lists some Kubernetes resources in cluster or at hosts.

k8s-native-app Containerized this: go build After building this we have binary files to dockerize. Create Dockerfile. docker build -t project-clientgo

Mustafa Taylan Ulun 0 Feb 12, 2022
Kubernetes OS Server - Kubernetes Extension API server exposing OS configuration like sysctl via Kubernetes API

KOSS is a Extension API Server which exposes OS properties and functionality using Kubernetes API, so it can be accessed using e.g. kubectl. At the moment this is highly experimental and only managing sysctl is supported. To make things actually usable, you must run KOSS binary as root on the machine you will be managing.

Mateusz Gozdek 3 May 19, 2021
A Terraform module that creates AWS alerts billing for your resources.

terraform-aws-billing-alarms terraform-aws-billing-alarms for project Replace name project to New Project agr 'terraform-aws-billing-alarms' 'new-pr

hadenlabs 1 Oct 20, 2021