A kafka command line browser

Overview

kcli

Kcli is a kafka read only command line browser.

Install

Binaries are provided here (windows is not tested). If you have go (1.11 or greater) installed you can do:

$ go get -u github.com/cswank/kcli

Usage

kcli --help
usage: kcli [<flags>]

Flags:
      --help             Show context-sensitive help (also try --help-long and --help-man).
  -a, --addresses=localhost:9092 ...
                         comma separated list of kafka addresses
  -l, --log=LOG          for debugging, set the log output to a file
  -t, --topic=TOPIC      go directly to a topic
  -p, --partition=-1     go directly to a partition of a topic
  -o, --offset=-1        go directly to a message
  -d, --decoder=DECODER  path to a plugin to decode kafka messages

NOTE: If your Kafka cluster has tls authentication enabled you need to set the following env vars:

export KCLI_CERT_FILE="<path to a client cert file in pem format>"
export KCLI_KEY_FILE="<path to a client key file in pem format>"
export KCLI_CA_CERT_FILE="<path to a ca cert file in pem format>"

After starting it up you get a list of topics:

Type 'h' to see the help menu (type 'h' again to toggle the help menu off):

Navigate to a topic and hit enter to see the partitions:

Navigate to a partition and hit enter to see a page of messages:

And navigate to a message and hit enter to see the message:

Searching

You can search for a string on either a partition or topic. When you search on a partition then the current offset is set to the first message that contains the search string. When you search on a topic then only the topics that contain a match are printed to the screen and their current offset is set to the first message that contains that match.

If you have partitions that have large amounts of data then it can take a long time to search through all the partitions. It is sometimes useful to use the partition offset functionality (C-o) to speed up your search if you have an idea where the message might be. If you know the message you are searching for is fairly recent then you can use a negative offset to set the offset of each partition close to then last end. The search will then start from those offsets.

Jumping

You can use the jump command (C-j) to set the current offset of a partition. Jumping on a partition is simple: the number you enter becomes the current offset. On other views (topic and message views) jump navigates the cursor to the value you enter.

Printing

If you enter C-p kcli will exit and the contents of the current view will be printed to stdout. If the current view is a partition then each message from the cursor to the end of the partition is printed to stdout. This is useful if you want to process the messages, for example:

kcli | jq .age | awk '{s+=$1} END {print s}'

Assuming the messages that get printed are JSON, this print the sum of all age fields from each message in the partition.

Custom Decoder

If your kafka messages are encoded in some way you can provide a custom decoder in the form of a plugin. See .examples/plugins/protobuf for an example. Once you have compiled the plugin you use it as the default decoder by starting kcli like so:

kcli -d /path/to/your/decoder.so

Screen Colors

If you don't like the defaul colors you can set KCLI_COLOR[0,1,2,3] to one of:

  • black
  • red
  • green
  • yellow
  • blue
  • magenta
  • cyan
  • white

For example:

$ KCLI_COLOR0=white KCLI_COLOR1=blue KCLI_COLOR2=black KCLI_COLOR3=red

See it in action at asciinema

asciicast

NOTE: If you are connecting to a local kafka that is running in a docker container using wurstmeister/kafka you may have the env KAFKA_ADVERTISED_HOST_NAME set to a name that is used by other containers that need to connect to kafka. This will cause kcli to not be able to read from kafka. A hacky fix is to edit your /etc/hosts file and add another name to the 127.0.0.1 network interface. For example, if

KAFKA_ADVERTISED_HOST_NAME=kafka

Then the 127.0.0.1 line /etc/hosts should look like:

127.0.0.1       localhost kafka
Issues
  • Provide most common decoders in package. Msgpack decoder

    Provide most common decoders in package. Msgpack decoder

    @cswank I loved your project!

    I used your protobuf example and created msgpack decoder.

    1. Is it worth adding it to decoder examples?
    2. Maybe you should consider to provide us with the most common decoders, and I could contribute with msgpack decoder?

    What do you think? (link to decoder itself)

    P.S. I put 100th star on your project ;)

    opened by bl17zar 4
  • Add searching in Root page

    Add searching in Root page

    1. Allow search feature in the root page
    2. Previously when the list of topic spills to multiple screen, even though the user selected the topic on page other than first page, the topic corresponding at the same cursor position on the first page would only get selected. This has been fixed.
    opened by anil477 1
  • Change name from kcli to kfcli

    Change name from kcli to kfcli

    Hi there,

    Any chance to change the app name from kcli to kfcli ? here there is another famous app that already has the same name kcli.

    Thanks in advance

    opened by jparrill 1
  • search / filter across partitions

    search / filter across partitions

    Right now the C-f command lets us filter only at the partition level, we'd love to filter/search across partitions as well. MP needs to be able to search for sg event ids, for example.

    opened by mettledrum 0
Releases(1.8.3)
Owner
Craig Swank
Craig Swank
k6 extension to load test Apache Kafka with support for Avro messages and SASL Authentication

xk6-kafka This project is a k6 extension that can be used to load test Kafka, using a producer. Per each connection to Kafka, many messages can be sen

null 0 Dec 7, 2021
Covering basics of Go by writing practical running code. microservice-http server-DAO-Kafka

go1 Covering all basics of Go by writing practical running code. Prerequisite: basic knowledge of c programming language as go is very similar to c in

Zeeshan Ashraf 4 Mar 30, 2022
Open URL in your local web browser from the SSH-connected remote environment.

opener Open URL in your local web browser from the SSH-connected remote environment. How does opener work? opener is a daemon process that runs locall

Kazuki Suda 54 Jun 24, 2022
K6 extension that adds support for browser automation and end-to-end web testing using playwright-go

k6 extension that adds support for browser automation and end-to-end web testing using playwright-go

We Open Source Performance 9 Jun 16, 2022
Based on the electron Cross-platform Mini browser

Based on the electron Cross-platform Mini browser

池边树下 1 May 1, 2022
Flexible HTTP command line stress tester for websites and web services

Pewpew Pewpew is a flexible command line HTTP stress tester. Unlike other stress testers, it can hit multiple targets with multiple configurations, si

Ben Gadbois 339 Jun 17, 2022
Command Line Interface for Scaleway

Scaleway CLI (v2) Scaleway CLI is a tool to help you pilot your Scaleway infrastructure directly from your terminal. Installation With a Package Manag

Scaleway 754 Jun 17, 2022
Command-line tool to remotely execute commands on Windows machines through WinRM

winrm-cli This is a Go command-line executable to execute remote commands on Windows machines through the use of WinRM/WinRS. Note: this tool doesn't

Brice Figureau 141 Jun 12, 2022
A command line tool for validating Kubernetes configs with rego

ccheck ccheck is a command line application for writing tests against configuration files and data using the rego query language. It's intended purpos

Brendan Ryan 63 Jun 1, 2022
Kubesecret is a command-line tool that prints secrets and configmaps data of a kubernetes cluster.

Kubesecret Kubesecret is a command-line tool that prints secrets and configmaps data of a kubernetes cluster. kubesecret -h for help pages. Install go

Charalampos Mitrodimas 18 May 3, 2022
Command kube-tmux prints Kubernetes context and namespace to tmux status line.

kube-tmux Command kube-tmux prints Kubernetes context and namespace to tmux status line.

null 7 Sep 10, 2021
This plugin helps you to use the AWS Command Line Interface (AWS CLI) to start and end sessions to your managed instances

Session Manager Plugin This plugin helps you to use the AWS Command Line Interface (AWS CLI) to start and end sessions to your managed instances. Sess

Amazon Web Services 151 Jun 23, 2022
🏯 Monitor your (gitlab/github) CI/CD pipelines via command line interface with fortress

__ _ / _| | | | |_ ___ _ __| |_ _ __ ___ ___ ___ | _/ _ \| '__| __| '__/ _ \/ __/ _

MrJosh 6 Mar 31, 2022
An unofficial command line utility for Futbin

futbin An unofficial command line utility for futbin. Also, it can be used as a library. Install Pre-compiled executables Get them here. Source You ne

Matheus Moraes 10 May 15, 2022
Igo Agent is the agent of Igo, a command-line tool, through which you can quickly start Igo

igo agent 英文 | 中文 Igo Agent is the agent of Igo, a command-line tool, through which you can quickly start Igo, and other capabilities may be added lat

null 1 Dec 22, 2021
The Coherence command line interface (CLI) is a lightweight tool, in the tradition of tools such as kubectl

Coherence Command Line Interface (CLI) Contents Overview Why use the Coherence C

Oracle 10 Mar 17, 2022
A command-line debugging tool to check the latency of SSL handshake

ssl-handshake A command-line tool for testing SSL handshake latency, written in

Puru 37 Jun 24, 2022
Command line tools for usedevbook.com

dbk dbk is a command-line tool (CLI) for usedevbook.com. It allows you to build and push custom environments for Devbook VMs. You can then launch Devb

Devbook 2 Feb 21, 2022
💻 A one-line installer for GitHub projects!

instl Instl is an installer that can install most GitHub projects on your system with a single command. Installation | Documentation | Contributing In

instl 47 Jun 7, 2022