⎈ Multi pod and container log tailing for Kubernetes

Overview

stern

wercker status

Stern allows you to tail multiple pods on Kubernetes and multiple containers within the pod. Each result is color coded for quicker debugging.

The query is a regular expression so the pod name can easily be filtered and you don't need to specify the exact id (for instance omitting the deployment id). If a pod is deleted it gets removed from tail and if a new pod is added it automatically gets tailed.

When a pod contains multiple containers Stern can tail all of them too without having to do this manually for each one. Simply specify the container flag to limit what containers to show. By default all containers are listened to.

Installation

If you don't want to build from source go grab a binary release

Govendor is required to install vendored dependencies.

mkdir -p $GOPATH/src/github.com/wercker
cd $GOPATH/src/github.com/wercker
git clone https://github.com/wercker/stern.git && cd stern
govendor sync
go install

Homebrew

On macOS, you can also install Stern using Homebrew:

brew install stern

Usage

stern pod-query [flags]

The pod query is a regular expression so you could provide "web-\w" to tail web-backend and web-frontend pods but not web-123.

cli flags

flag default purpose
--container .* Container name when multiple containers in pod (regular expression)
--exclude-container Container name to exclude when multiple containers in pod (regular expression)
--container-state running Tail containers with status in running, waiting or terminated. Default to running.
--timestamps Print timestamps
--since Return logs newer than a relative duration like 52, 2m, or 3h. Displays all if omitted
--context Kubernetes context to use. Default to kubectl config current-context
--exclude Log lines to exclude; specify multiple with additional --exclude; (regular expression)
--namespace Kubernetes namespace to use. Default to namespace configured in Kubernetes context
--kubeconfig ~/.kube/config Path to kubeconfig file to use
--all-namespaces If present, tail across all namespaces. A specific namespace is ignored even if specified with --namespace.
--selector Selector (label query) to filter on. If present, default to .* for the pod-query.
--tail -1 The number of lines from the end of the logs to show. Defaults to -1, showing all logs.
--color auto Force set color output. auto: colorize if tty attached, always: always colorize, never: never colorize
--output default Specify predefined template. Currently support: [default, raw, json] See templates section
template Template to use for log lines, leave empty to use --output flag

See stern --help for details

Stern will use the $KUBECONFIG environment variable if set. If both the environment variable and --kubeconfig flag are passed the cli flag will be used.

templates

stern supports outputting custom log messages. There are a few predefined templates which you can use by specifying the --output flag:

output description
default Displays the namespace, pod and container, and decorates it with color depending on --color
raw Only outputs the log message itself, useful when your logs are json and you want to pipe them to jq
json Marshals the log struct to json. Useful for programmatic purposes

It accepts a custom template through the --template flag, which will be compiled to a Go template and then used for every log message. This Go template will receive the following struct:

property type description
Message string The log message itself
Namespace string The namespace of the pod
PodName string The name of the pod
ContainerName string The name of the container

The following functions are available within the template (besides the builtin functions):

func arguments description
json object Marshal the object and output it as a json text
color color.Color, string Wrap the text in color (.ContainerColor and .PodColor provided)

Examples:

Tail the gateway container running inside of the envvars pod on staging

stern envvars --context staging --container gateway

Tail the staging namespace excluding logs from istio-proxy container

stern -n staging --exclude-container istio-proxy .

Show auth activity from 15min ago with timestamps

stern auth -t --since 15m

Follow the development of some-new-feature in minikube

stern some-new-feature --context minikube

View pods from another namespace

stern kubernetes-dashboard --namespace kube-system

Tail the pods filtered by run=nginx label selector across all namespaces

stern --all-namespaces -l run=nginx

Follow the frontend pods in canary release

stern frontend --selector release=canary

Pipe the log message to jq:

stern backend -o json | jq .

Only output the log message itself:

stern backend -o raw

Output using a custom template:

stern --template '{{.Message}} ({{.Namespace}}/{{.PodName}}/{{.ContainerName}})' backend

Output using a custom template with stern-provided colors:

stern --template '{{.Message}} ({{.Namespace}}/{{color .PodColor .PodName}}/{{color .ContainerColor .ContainerName}})' backend

Completion

Stern supports command-line auto completion for bash or zsh. stern --completion=(bash|zsh) outputs the shell completion code which work by being evaluated in .bashrc, etc for the specified shell. In addition, Stern supports dynamic completion for --namespace and --context. In order to use that, kubectl must be installed on your environment.

If you use bash, stern bash completion code depends on the bash-completion. On the macOS, you can install it with homebrew as follows:

$ brew install bash-completion

Note that bash-completion must be sourced before sourcing the stern bash completion code in .bashrc.

source <(brew --prefix)/etc/bash-completion
source <(stern --completion=bash)

If you use zsh, just source the stern zsh completion code in .zshrc.

source <(stern --completion=zsh)

Contributing to this repository

Oracle welcomes contributions to this repository from anyone. Please see CONTRIBUTING for details.

Issues
  • add exclude-container option

    add exclude-container option

    Tried it locally, it helps a lot :)

    this fixes #78

    This excludes logs from a container matching the exclude-container filter. I updated the docs.

    If no option is passed we set the regexp pointer to nil and nothing is run.

    To validate changes try it, since there are no tests, for me it was: stern -n staging . -E "istio" just like in the doc update I did, and it worked !

    opened by azr 17
  • JSON logs

    JSON logs

    Hi, we output our logs to stdout in JSON format. They're later captured by ELK stack. By using JSON we don't have to create log parsers. The drawback of this solution is that it's not super human readable when read not through Kibana. So my idea is to add some parameter that would allow you to display only specific fields from this JSON in a more human readable way. What do you think?

    opened by tmszdmsk 15
  • support --include in addition to --exclude

    support --include in addition to --exclude

    Since there's --exclude, I found it quite surprising to not see its mirror, --include. The obvious example usecase is

    stern '.*' --include ERROR
    

    to see all errors that occur anywhere.

    opened by majewsky 13
  • Improve to be able to place flags after arguments

    Improve to be able to place flags after arguments

    Currently stern does not work if flags are placed after arguments. With this commit, it improve to be able to place flags after arguments like kubectl.

    stern .* -n kube-system
    

    For this change, github.com/spf13/cobra is used instead of gopkg.in/urfave/cli.v1 and help message is changed accordingly. This commit is not breaking change, compatible.

    github.com/spf13/cobra is released under the Apache 2.0 License.

    opened by superbrothers 11
  • Support tailing init containers and passing container state

    Support tailing init containers and passing container state

    The existing watch doesn't listen for init container and it only listens to container in running state. Supporting init containers and container state is useful to users. In order not to bother users passing container state every time, the flag has the default value of "running". https://github.com/wercker/stern/issues/53

    opened by shun0309-zz 8
  • Add dynamic completion support for --namespace and --context

    Add dynamic completion support for --namespace and --context

    With this PR, stern supports dynamic completion for --namespace and --context. This feature depends on kubectl. And the name of --kube-config is changed to --kubeconfig and --kube-config is marked deprecated due to using bash completion for kubectl.

    $ source <(stern --completion=zsh)
    $ stern --context=<tab>
    --context=cluster01  --context=minikube
    $ stern --namespace <tab>
    default      kube-system
    
    opened by superbrothers 8
  • Fixing aws-iam-authenticator / EKS auth issue.

    Fixing aws-iam-authenticator / EKS auth issue.

    Continuing work of @mattmoyer started in https://github.com/wercker/stern/pull/81. Credits also to @guoyongzhang.

    --since must be set to value other than 0.

    Hope it fixes https://github.com/wercker/stern/issues/80, I don't own any EKS cluster to verify. Tested against Kubernetes 1.9.8 and aws-iam-authenticator v0.3.0.

    opened by michalschott 7
  • Update dependencies.

    Update dependencies.

    The biggest change here is to upgrade client-go to v8.0.0. This adds support for kubeconfig exec plugins as used in the aws-iam-authenticator (confirmed locally).

    This was my first time using govendor so let me know if I messed anything up.

    Fixes https://github.com/wercker/stern/issues/80.

    opened by mattmoyer 7
  • stern not working against AWS EKS Cluster

    stern not working against AWS EKS Cluster

    When I run stern against an EKS cluster(k8s 1.10.2) when I have admin privileges I get this:

    > stern --kubeconfig ~/.kube/config restic --namespace heptio-ark                                                                                                                                                  
    failed to set up watch: failed to set up watch: unknown (get pods)
    

    I'm able to use kubectl logs and see pod logs, looks like stern might be using another api to get the data?

    Any help would be appreciated.

    opened by bobhenkel 7
  • Always hash to the same color

    Always hash to the same color

    This is something that bugged me:

    • you run stern and watch the output of several pods fly by
    • you realize you need to change a parameter flag
    • you exit stern and restart it again
    • the pods have different colors...

    This change always hashes a pod's name to the same color instead of giving it a random one. I hope I'm not the only one who would like to see this changed :grin:

    BTW: great project! Using it every day.

    opened by shutefan 6
  • Tail logs from initContainers

    Tail logs from initContainers

    Hi,

    Thanks for this great project! I couldn't find a way to tail logs from my initContainers.

    stern 'api.*' tails logs from all proper containers, but I see nothing from the initContainer kubectl logs api-2524184185-62d3c -c api-init shows that I missed the logs from the init container named api-ini

    It would be great if this could be added, as debugging initContainers is not so easy! Thanks.

    opened by georgecrawford 5
  • How to specify different colors for different pods?

    How to specify different colors for different pods?

    Given I have pods with name test1 and test2 in the same namespace, running stern test, logs produced are for both pods showing the pod names in the same color (e.g green).

    How do I have different colors for different pod names? Help would be appreciated πŸ™‡β€β™‚οΈ

    opened by jlgoh 1
  • What's the syntax for --include flag?

    What's the syntax for --include flag?

    Let's say I want to filter my ingress JSON logs to display only HTTP 50x errors by including "status": "50*" key/value. I tried multiple variations but none work, can someone let me know how this works or do I need to use grep as well?

    This is how my log line looks: { "time": "2021-10-28T13:50:38+00:00", "request_id": "ab089415fa5d15f991609986c2c95c20", "status": "503", "ip": "xxx.xxx.xxx.xxx", "bytes_sent": "729", "request_duration": "1.879", "vhost": "app.domain.com", "request_proto": "HTTP/2.0", "requestUri": "app.domain.com/editor/uploadFiles", "path": "/editor/uploadFiles", "request_query": "", "request_length": "74860", "method": "POST", "http_referrer": "https://app.domain.com/editor/edit/608083", "http_user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36", "namespace": "app-production" }

    Thanks in advance!

    opened by pentago 1
  • Create a `stern_darwin_amd64` build for the most recent release

    Create a `stern_darwin_amd64` build for the most recent release

    Now that the Apple M1 devices are getting more prevalent, especially among engineers, it would be really nice to have an arm64 binary pre-built and downloadable for stern.

    opened by spkane 2
  • failed to set up watch: failed to set up watch: Get https://192.168.43.20:16443/api/v1/namespaces/default/pods?watch=true: dial tcp 192.168.43.20:16443: getsockopt: no route to host

    failed to set up watch: failed to set up watch: Get https://192.168.43.20:16443/api/v1/namespaces/default/pods?watch=true: dial tcp 192.168.43.20:16443: getsockopt: no route to host

    My host VM IP got changed.

    After this, when I am running stern command.

    failed to set up watch: failed to set up watch: Get https://192.168.43.20:16443/api/v1/namespaces/default/pods?watch=true: dial tcp 192.168.43.20:16443: getsockopt: no route to host

    Is there anyway to change the config?

    opened by Yavdhesh 0
  • Dead project?

    Dead project?

    First of all, thanks for a great tool. I use it regularly and have thought about contributing multiple times. However, the project seems pretty dead after the acquisition of Wercker by Oracle. Please consider updating the README with the current status of the project. stern/stern seems to be a well maintained fork, so a mention of this or other forks would be welcome.

    Also, please consider archiving this repository so it clearly signals that contributions will not be accepted/processed. As a side effect it would also reduce notification noise for members of the @wercker org.

    opened by evenh 3
Releases(1.11.0)
Owner
wercker
Container-centric Automation Platform
wercker
Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration

Karmada Karmada: Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration Karmada (Kubernetes Armada) is a Kubernetes management system that enables

null 2.4k Aug 11, 2022
Display (Namespace, Pod, Container, Primary PID) from a host PID, fails if the target process is running on host

Display (Namespace, Pod, Container, Primary PID) from a host PID, fails if the target process is running on host

K8s-school 10 Aug 5, 2022
gpupod is a tool to list and watch GPU pod in the kubernetes cluster.

gpupod gpupod is simple tool to list and watch GPU pod in kubernetes cluster. usage Usage: gpupod [flags] Flags: -t, --createdTime with pod c

null 0 Dec 8, 2021
Json-log-exporter - A Nginx log parser exporter for prometheus metrics

json-log-exporter A Nginx log parser exporter for prometheus metrics. Installati

horan 0 Jan 5, 2022
A kubernetes plugin which enables dynamically add or remove GPU resources for a running Pod

GPU Mounter GPU Mounter is a kubernetes plugin which enables add or remove GPU resources for running Pods. This Introduction(In Chinese) is recommende

XinYuan 72 Jul 18, 2022
Translate Prometheus Alerts into Kubernetes pod readiness

prometheus-alert-readiness Translates firing Prometheus alerts into a Kubernetes readiness path. Why? By running this container in a singleton deploym

Coralogix 19 Mar 7, 2021
A Kubernetes Mutating Webhook to automatically re-point pod images to mirrors

kubernetes-mimic Kubernetes Mimic is a Mutating Webhook that will watch for pod creation and update events in a Kubernetes cluster and automatically a

null 5 Dec 1, 2021
An example of Kubernetes' Horizontal Pod Autoscaler using costume metrics.

Kubernetes Autoscaling Example In this project, I try to implement Horizontal Pod AutoscalerHPA provided by Kubernetes. The Horizontal Pod Autoscaler

Jaskeerat Singh Randhawa 8 Mar 15, 2022
Kubernetes Pod Security Standards implementation

Pod Security Admission The Pod Security Standards are a set of best-practice profiles for running pods securely. This repository contains the codified

Kubernetes 66 Aug 11, 2022
Kubernetes create pod cli with golang

kubernatestest- Anand #Added more coments Anand2 and K8s #GO COMMANDS to RUN : go run xxx.go to install : go install xyz.go - this created binary file

null 1 Dec 2, 2021
A kubernetes operator sample generated by kubebuilder , which run cmd in pod on specified time

init kubebuilder init --domain github.com --repo github.com/tonyshanc/sample-operator-v2 kubebuilder create api --group sample --version v1 --kind At

shank 0 Jan 25, 2022
A very simple, silly little kubectl plugin / utility that guesses which language an application running in a kubernetes pod was written in.

A very simple, silly little kubectl plugin / utility that guesses which language an application running in a kubernetes pod was written in.

Tom Granot 2 Mar 9, 2022
Go WhatsApp Multi-Device Implementation in REST API with Multi-Session/Account Support

Go WhatsApp Multi-Device Implementation in REST API This repository contains example of implementation go.mau.fi/whatsmeow package with Multi-Session/

Dimas Restu H 36 Jul 30, 2022
Enterprise-grade container platform tailored for multicloud and multi-cluster management

KubeSphere Container Platform What is KubeSphere English | δΈ­ζ–‡ KubeSphere is a distributed operating system providing cloud native stack with Kubernete

KubeSphere 10.7k Aug 7, 2022
KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes

Kubernetes-based Event Driven Autoscaling KEDA allows for fine-grained autoscaling (including to/from zero) for event driven Kubernetes workloads. KED

KEDA 5.3k Aug 11, 2022
Moby Project - a collaborative project for the container ecosystem to assemble container-based systems

The Moby Project Moby is an open-source project created by Docker to enable and accelerate software containerization. It provides a "Lego set" of tool

Moby 63.7k Aug 4, 2022
Boxygen is a container as code framework that allows you to build container images from code

Boxygen is a container as code framework that allows you to build container images from code, allowing integration of container image builds into other tooling such as servers or CLI tooling.

nitric 5 Dec 13, 2021
Amazon ECS Container Agent: a component of Amazon Elastic Container Service

Amazon ECS Container Agent The Amazon ECS Container Agent is a component of Amazon Elastic Container Service (Amazon ECS) and is responsible for manag

null 0 Dec 28, 2021
The Container Storage Interface (CSI) Driver for Fortress Block Storage This driver allows you to use Fortress Block Storage with your container orchestrator

fortress-csi The Container Storage Interface (CSI) Driver for Fortress Block Storage This driver allows you to use Fortress Block Storage with your co

Fortress 0 Jan 23, 2022