Background

Older versions of kubectl/oc, when ran as kubectl logs -n <namespace> <name>, first try to GET the pod resource requested, if they see there's only one container in it, they will then try to GET:

http://<api>/api/v1/namespaces/<namespace>/pods/<name>/log

Without any container=... query param (newer versions of kubectl/oc always specify this query parameter regardless). The behavior of the regular kubernetes API server in this scenario is that when the pod has only a single container, the logs for that only container are returned. If the pod has more than one pod, an error response is returned, listing the names of the containers in the pod.

Issue

static-kas works fine when the container name is given in the query param (which is the case for newer oc/kubectl clients), but breaks down when the container parameter is omitted.

Solution

static-kas will now better imitate the behavior of the regular kubernetes API server

Hey, I've been working on a similar project https://github.com/omertuc/o-must-external-reader

I've written it in Python rather than Go.

It's far from perfect and very buggy. My main goal was getting k9s to work somewhat ok, and i've come pretty far. You can view pods, containers, container logs (truncated due to k9s weirdness I've yet to figure out), yaml files.

I can't seem to get k9s to work at all with your fake API server, it wouldn't even connect. I wonder why. I also couldn't do oc get pods -A .

Looking forward to seeing how you improve this, hopefully I could use your server with k9s some day. Or if you want, we can join efforts and work on my project (or the other way around) - but I'm not sure how much overlap there is in what we've done so far and what things mine can do and yours can't and vice versa.

Anyway just thought I should mention this

When running static-kas on a large must-gather with the user limit of 1024 openfiles, you run into the error:

{"error":"failed to read file namespaces/xxxx/operators.coreos.com/clusterserviceversions/jaeger-operator.v1.30.0.yaml: open namespaces/xxxxx/operators.coreos.com/clusterserviceversions/jaeger-operator.v1.30.0.yaml: too many open files"}],"stacktrace":"main.main\n\t/home/manu/Projects/git/static-kas/cmd/main.go:57\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:255"

This happens with 15000 files in a must-gather

it is easy to resolve the error by using the user hardlimit of openfiles, following https://www.baeldung.com/linux/error-too-many-open-files or https://access.redhat.com/solutions/2469

however this is maybe not trivial for an end user.

If we agree to open a Wiki for the project, maybe linked from the Readme, we could add a line along the way of:

When the must-gather you process with static-kas has a large number of files, you may reach into an error "too many open files". In that case you will need to [increase the maximum number of open file descriptors] for the user running static-kas(https://www.baeldung.com/linux/error-too-many-open-files). A limit of 4096 has been found to process correctly large must-gathers.

I have encountered this issue with resources coming from Cilium sysdumps. Is that an OK change for you @alvaroaleman ?

Signed-off-by: Raphaël Pinson [email protected]

1. type :pods
2. select an arbitrary pod
3. press ↵ Return
4. select an arbitrary container
5. press L

Logs are shown, but they're truncated. Pressing 0 does not help

1. Type :pods
2. Move the cursor to some arbitrary pod
3. press y - works, shows you the YAML
4. press esc to go back
5. press e, should open in editor, but it doesn't

I don't want editing to literally work, obviously, the gather should be immutable, but it's really nice having the yaml open in your $EDITOR of your choice

Note that this is an API issue - it works OK with regular API servers

Note: the Cilium Sysdump tool dumps the Kubernetes server version as Go code, e.g.:

version.Info{Major:"1", Minor:"21+", GitVersion:"v1.21.5-eks-bc4871b", GitCommit:"5236faf39f1b7a7dabea8df12726f25608131aa9", GitTreeState:"clean", BuildDate:"2021-10-29T23:32:16Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}

I think it's probably better to provide a JSON file as a standard (I actually don't see an easy way to parse the go version file in go).

Signed-off-by: Raphaël Pinson [email protected]