Kstone is an etcd management platform, providing cluster management, monitoring, backup, inspection, data migration, visual viewing of etcd data, and intelligent diagnosis.

Related tags

DevOps Tools kstone
Overview

Kstone


中文

Kstone is an etcd management platform, providing cluster management, monitoring, backup, inspection, data migration, visual viewing of etcd data, and intelligent diagnosis.

Kstone will help you efficiently manage etcd clusters, significantly reduce operation and maintenance costs, discover potential hazards in time, and improve the stability and user experience of k8s etcd storage.


Features

  • Supports registration of existing clusters and creation of new etcd clusters.
  • Support prometheus monitoring, built-in rich etcd grafana panel diagram.
  • Support multiple data backup methods (minute-level backup to object storage, real-time backup by deploying learner).
  • Support multiple inspection strategies (data consistency, health, hot write requests, number of resource objects, etc.).
  • Built-in web console and visual view etcd data.
  • Lightweight, easy to install.
  • Support data migration(to do list).
  • Support intelligent diagnosis(to do list).

Architecture

Kstone consists of 5 components: kstone-etcdcluster-controller,kstone-etcd-operator,kstone-etcdinspection-controller,kstone-api, kstone-dashboard.

Architecture Of Kstone

Components

kstone

kstone consists of kstone-etcdcluster-controller,kstone-etcdinspection-controller,kstone-api.

kstone-etcd-operator

kstone-etcd-operator provides rich etcd cluster management capabilities.It will also be open source soon.

kstone-dashboard

The web management system provided by kstone is as follows:

kstone-ui

Installation

Please read the detailed installation document, You can quickly install kstone through helm.

Developing

Build

mkdir -p ~/tkestack
cd ~/tkestack
git clone https://github.com/tkestack/kstone
cd kstone
make

Community

Licensing

Kstone is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Comments
  • dashboard crash when creating cluster

    dashboard crash when creating cluster

    1. install kstone chart with following configuration
    # version: master 78c67cf8e4aa8b704096cbc071e8eaa1b1de8a62
    helm install kstone . -n kstone -f values.test.yaml
    
    # ui version
    kubectl -n kstone get pods kstone-dashboard-7c6cd6554c-7lj8z -o yaml |grep image
    ...
    - image: tkestack/kstone-dashboard:v0.1.0-alpha.2
        imagePullPolicy: IfNotPresent
        image: tkestack/kstone-dashboard:v0.1.0-alpha.2
        imageID: docker-pullable://tkestack/[email protected]:7bc9ba9e449bd9848f1074d060b8d6d51298cb4d15ff3f4ebe777e0fb3ed0f76
    
    1. access dashboard

    2. click Operation Center and then Create Cluster test-tke

    opened by jerry-jibu 5
  • The backup page occur

    The backup page occur "Internal Server Error" error

    Steps: "Cluster Management" -> "Operation" -> "Cluster Feature" -> "Backup: "(open) Then don't fill in anything, just click the OK button. When I enter the backup page again The backup page reports an error "Internal Server Error, please check the network or contact the administrator!" image

    opened by litianxi 3
  • kstone authentication support task list

    kstone authentication support task list

    • [x] submit proposal doc[#88 ]
    • [x] kstone-api supports token provider, token generator and authenticator interface[#99 ]
    • [ ] kstone-api supports auth webhook
    • [x] kstone-dashboard supports login feature[#103 ]
    • [x] kstone-dashboard supports change password feature[#103 ]
    • [x] kstone-api supports login/middleware router[#103 ]
    feature 
    opened by lianghao208 1
  • kstone-dashboard issue: Can't change backup parameter

    kstone-dashboard issue: Can't change backup parameter

    In the kstone-dashboard Backup parameter setting page, if I change the BackupIntervalInSecond or MaxBackups or TimeoutInSecond and Click OK, and reopen again, those parameters will revert to the original one.

    bug 
    opened by maudL1n 1
  • helm install error.

    helm install error.

    helm version

    version.BuildInfo{Version:"v3.6.3", GitCommit:"d506314abfb5d21419df8c7e7e68012379db2354", GitTreeState:"clean", GoVersion:"go1.16.5"}

    helm install kstone . -n kstone

    Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: [ValidationError(Prometheus.spec): unknown field "probeNamespaceSelector" in com.coreos.monitoring.v1.Prometheus.spec, ValidationError(Prometheus.spec): unknown field "probeSelector" in com.coreos.monitoring.v1.Prometheus.spec, ValidationError(Prometheus.spec): unknown field "shards" in com.coreos.monitoring.v1.Prometheus.spec]

    opened by gxjluck 1
  • 原生k8s环境下部署报错Unauthorized

    原生k8s环境下部署报错Unauthorized

    原生k8s环境(非tke环境)的安装步骤有吗?需要改哪些配置?我在原生k8s环境下简单部署了一下,prometheus服务反复重建中:

    $ k2 get sts,pod -n kstone
    NAME                                                       READY   AGE
    statefulset.apps/prometheus-kstone-prometheus-prometheus   1/1     61s
    
    NAME                                                READY   STATUS        RESTARTS   AGE
    pod/kstone-backup-operator-5c77c949d8-m7s7w         1/1     Running       0          35m
    pod/kstone-dashboard-5595764fd5-sk4r6               1/1     Running       0          35m
    pod/kstone-dashboard-api-7cfc658f9f-vkwvm           1/1     Running       0          35m
    pod/kstone-etcd-controller-6c6d996fcf-jsdtl         1/1     Running       0          35m
    pod/kstone-etcd-operator-f9b4bd7-xbdwr              1/1     Running       0          35m
    pod/kstone-grafana-5d5985ffdf-vkr7l                 2/2     Running       0          35m
    pod/kstone-inspection-controller-65d6578554-zxjx5   1/1     Running       0          35m
    pod/kstone-prometheus-operator-85dd8b5d99-m4snl     1/1     Running       0          35m
    pod/prometheus-kstone-prometheus-prometheus-0       2/2     Terminating   0          61s
    

    这是prometheus-operator的报错日志:

    level=info ts=2021-12-09T10:54:17.573352638Z caller=main.go:295 msg="Starting Prometheus Operator" version="(version=0.49.0, branch=master, revision=2388bfa55)" level=info ts=2021-12-09T10:54:17.573426783Z caller=main.go:296 build_context="(go=go1.16.5, user=paulfantom, date=20210706-17:43:37)" level=info ts=2021-12-09T10:54:17.580278215Z caller=server.go:54 msg="enabling server side TLS" level=info ts=2021-12-09T10:54:17.592804524Z caller=main.go:125 msg="Starting secure server on [::]:10250" level=info ts=2021-12-09T10:54:17.594563066Z caller=operator.go:309 component=thanosoperator msg="connection established" cluster-version=v1.18.9 level=info ts=2021-12-09T10:54:17.594518358Z caller=operator.go:421 component=prometheusoperator msg="connection established" cluster-version=v1.18.9 level=info ts=2021-12-09T10:54:17.595023766Z caller=operator.go:430 component=prometheusoperator msg="CRD API endpoints ready" level=info ts=2021-12-09T10:54:17.595047532Z caller=operator.go:449 component=alertmanageroperator msg="connection established" cluster-version=v1.18.9 level=info ts=2021-12-09T10:54:17.595109887Z caller=operator.go:458 component=alertmanageroperator msg="CRD API endpoints ready" level=info ts=2021-12-09T10:54:17.595084193Z caller=operator.go:318 component=thanosoperator msg="CRD API endpoints ready" level=info ts=2021-12-09T10:54:17.796953373Z caller=operator.go:261 component=thanosoperator msg="successfully synced all caches" level=info ts=2021-12-09T10:54:18.396087837Z caller=operator.go:351 component=prometheusoperator msg="successfully synced all caches" level=info ts=2021-12-09T10:54:18.397634362Z caller=operator.go:1221 component=prometheusoperator key=kstone/kstone-prometheus-prometheus msg="sync prometheus" level=info ts=2021-12-09T10:54:18.497522759Z caller=operator.go:281 component=alertmanageroperator msg="successfully synced all caches" level=info ts=2021-12-09T10:54:18.497765803Z caller=operator.go:742 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="sync alertmanager" level=info ts=2021-12-09T10:54:18.576327944Z caller=operator.go:1306 component=prometheusoperator key=kstone/kstone-prometheus-prometheus statefulset=prometheus-kstone-prometheus-prometheus shard=0 msg="recre ating StatefulSet because the update operation wasn't possible" reason="Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden" level=info ts=2021-12-09T10:54:18.601315559Z caller=operator.go:1221 component=prometheusoperator key=cattle-monitoring-system/rancher-monitoring-prometheus msg="sync prometheus" level=info ts=2021-12-09T10:54:18.672826348Z caller=operator.go:804 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="recreating AlertManager StatefulSet because the update operation wasn't possible" reason="Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden" level=info ts=2021-12-09T10:54:18.689319741Z caller=operator.go:742 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="sync alertmanager" level=info ts=2021-12-09T10:54:18.791717511Z caller=operator.go:804 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="recreating AlertManager StatefulSet because the update operation wasn't possible" reason="Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden" level=info ts=2021-12-09T10:54:18.804561025Z caller=operator.go:742 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="sync alertmanager" level=info ts=2021-12-09T10:54:18.916061885Z caller=operator.go:804 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="recreating AlertManager StatefulSet because the update operation wasn't possible" reason="Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden" level=info ts=2021-12-09T10:54:19.279129482Z caller=operator.go:742 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="sync alertmanager" level=info ts=2021-12-09T10:54:19.383887819Z caller=operator.go:1306 component=prometheusoperator key=cattle-monitoring-system/rancher-monitoring-prometheus statefulset=prometheus-rancher-monitoring-prometheu s shard=0 msg="recreating StatefulSet because the update operation wasn't possible" reason="Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbid den" level=info ts=2021-12-09T10:54:19.395885355Z caller=operator.go:804 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="recreating AlertManager StatefulSet because the update operation wasn't possible" reason="Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden" level=info ts=2021-12-09T10:54:19.417586393Z caller=operator.go:1221 component=prometheusoperator key=kstone/kstone-prometheus-prometheus msg="sync prometheus" level=info ts=2021-12-09T10:54:19.430864859Z caller=operator.go:742 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="sync alertmanager" level=info ts=2021-12-09T10:54:19.522897026Z caller=operator.go:804 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="recreating AlertManager StatefulSet because the update operation wasn't possible" reason="Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden" level=info ts=2021-12-09T10:54:19.58892879Z caller=operator.go:1306 component=prometheusoperator key=kstone/kstone-prometheus-prometheus statefulset=prometheus-kstone-prometheus-prometheus shard=0 msg="recrea ting StatefulSet because the update operation wasn't possible" reason="Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden" level=info ts=2021-12-09T10:54:19.604894838Z caller=operator.go:1221 component=prometheusoperator key=cattle-monitoring-system/rancher-monitoring-prometheus msg="sync prometheus" level=info ts=2021-12-09T10:54:20.127681649Z caller=operator.go:742 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="sync alertmanager" level=info ts=2021-12-09T10:54:20.234899709Z caller=operator.go:742 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="sync alertmanager" level=info ts=2021-12-09T10:54:20.380352807Z caller=operator.go:1306 component=prometheusoperator key=cattle-monitoring-system/rancher-monitoring-prometheus statefulset=prometheus-rancher-monitoring-prometheu s shard=0 msg="recreating StatefulSet because the update operation wasn't possible" reason="Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbid den" level=info ts=2021-12-09T10:54:20.397125163Z caller=operator.go:1221 component=prometheusoperator key=kstone/kstone-prometheus-prometheus msg="sync prometheus" level=info ts=2021-12-09T10:54:20.529716104Z caller=operator.go:1306 component=prometheusoperator key=kstone/kstone-prometheus-prometheus statefulset=prometheus-kstone-prometheus-prometheus shard=0 msg="recre ating StatefulSet because the update operation wasn't possible" reason="Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden" level=info ts=2021-12-09T10:54:20.532849411Z caller=operator.go:742 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="sync alertmanager" level=info ts=2021-12-09T10:54:20.597040563Z caller=operator.go:742 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="sync alertmanager" level=info ts=2021-12-09T10:54:20.915831566Z caller=operator.go:1221 component=prometheusoperator key=kstone/kstone-prometheus-prometheus msg="sync prometheus" level=info ts=2021-12-09T10:54:21.051243142Z caller=operator.go:1221 component=prometheusoperator key=kstone/kstone-prometheus-prometheus msg="sync prometheus" level=info ts=2021-12-09T10:54:21.113887188Z caller=operator.go:742 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="sync alertmanager" level=info ts=2021-12-09T10:54:21.214160287Z caller=operator.go:742 component=alertmanageroperator key=cattle-monitoring-system/rancher-monitoring-alertmanager msg="sync alertmanager" level=info ts=2021-12-09T10:54:21.233518201Z caller=operator.go:1221 component=prometheusoperator key=kstone/kstone-prometheus-prometheus msg="sync prometheus" level=error ts=2021-12-09T10:54:21.341324336Z caller=klog.go:116 component=k8s_client_runtime func=ErrorDepth msg="Sync "cattle-monitoring-system/rancher-monitoring-alertmanager" failed: synchronizing gover ning service failed: updating service object failed: Operation cannot be fulfilled on services "alertmanager-operated": the object has been modified; please apply your changes to the latest version and try again"

    可能原因1:该集群已经通过rancher部署了一套kube-prometheus集群,其底层也是prometheus-operator,其使用crd名称相同但版本不一致,产生了冲突?

    打开dashboard页面有报错,报错提示:Unauthorized

    opened by zhaoge77 1
  • 备份一直报failed to authenticate user

    备份一直报failed to authenticate user

    http://192.168.10.14/apis/backup/k8s接口返回错误

    {
    username: "unknown",
    reset_password: false,
    token: "",
    message: "failed to authenticate user"
    }
    

    image

    如何开启hostPath备份呢 不走腾讯的文件存储

    opened by Phoebus888 0
  • After refreshing, the filled information disappears

    After refreshing, the filled information disappears

    image During the "import cluster" operation, the interface stopped. I refreshed the lower interface. As a result, all the information I filled in before disappeared. If you refresh the page, the filled information will not disappear, which will be a very good thing.

    opened by QiuToo 1
  • Binary Deployment Support

    Binary Deployment Support

    It would be a great feat if binary deployment could be supported. As a personal habit, I don't like git to install kstone, even though many people do.

    opened by QiuToo 2
  • How to expose managed etcd cluster to the outside of the k8s cluster without manually declare svc yaml

    How to expose managed etcd cluster to the outside of the k8s cluster without manually declare svc yaml

    As I submitted the creation request, there are several ClusterIp exposing etcd cluster TAPP, but if I needto access the etcd cluster, I will need expose an nodePort manually,which is not automated from the operator. Is there a way to espose the etcd cluster without manual exposure?

    opened by Battlefield-Pony 1
Releases(v0.2.0-beta.2)
  • v0.2.0-beta.2(Jun 8, 2022)

    Features

    • support import etcd v2 cluster. (@engow, @seanyan )

    • add compaction,lease metrics. (@tangcong )

    • add hostPath type for etcd backup operator. (@GeorgeGuo2018 )

    Bug Fix

    • failed to add/remove member if affinity and toleration is nil. (@engow )

    • fixed the issue that the editing page could not display cpu and memory resources. (@engow )

    • fixed the issue that some paths data could not be viewed visually. (@engow )

    • fixed the issue of backup management page 504 error.(@engow)

    https://github.com/kstone-io/kstone/blob/master/CHANGELOG/CHANGELOG.md

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0-beta.1(Apr 18, 2022)

  • v0.1.0-alpha.2(Jan 13, 2022)

    Features

    • kstone-dashboard supports dynamically adding feature gates.(@engow)
    • add backup file check inspection. (@maudL1n )
    • add backupcheck inpspection panel. (@engow )
    • add db usage rate grafana panel. (@lianghao208 )
    • add new metric for failed inspection. (@maudL1n )
    • add new router for list feature. (@maudL1n)

    Bug && Refactor

    • replace secret cacher with secret informer (@lianghao208 )

    Test

    • add more kstone-etcd-operator e2e tests (@maudL1n )
    Source code(tar.gz)
    Source code(zip)
  • v0.1.0-alpha.1(Dec 29, 2021)

    Features

    • kstone-dashboard support English UI (@jyonghong @engow)
    • support to disable feature (@maudL1n @tangcong)
    • support alarm inspetion (@lianghao208 )
    • add more metrics to enhance consistency inspection feature (@maudL1n @tangcong )

    Bug

    • fix can't change backup parameter bug (@engow)

    Docs

    • kstone installation on the cluster created by kubeadm (@maudL1n )
    • add cos backup guide (@maudL1n)

    Test

    • add more e2e test (@maudL1n)
    • support e2e test in github actions (@tangcong )
    Source code(tar.gz)
    Source code(zip)
Owner
TKEStack
TKEStack
A simple application, demo at this point, on how to pull a backup from Collibra on prem (say for Cohesity backup)

go-get-collibra-backup Introduction This repository is a very simple go application that's intended, at this point, more of a demonstration about how

David Thole 0 Dec 10, 2021
Lightweight, single-binary Backup Repository client. Part of E2E Backup Architecture designed by RiotKit

Backup Maker Tiny backup client packed in a single binary. Interacts with a Backup Repository server to store files, uses GPG to secure your backups e

RiotKit 1 Apr 4, 2022
Kubegres is a Kubernetes operator allowing to create a cluster of PostgreSql instances and manage databases replication, failover and backup.

Kubegres is a Kubernetes operator allowing to deploy a cluster of PostgreSql pods with data replication enabled out-of-the box. It brings simplicity w

Reactive Tech Ltd 1.1k Dec 30, 2022
IT Asset Fleet Remote Management and Monitoring (RMM) platform

ratd Remote Access Toolkit Daemon IT Asset Remote Management and Monitoring (RMM) platform How to use: You can't, just yet. Release Plan Release 0 Cre

null 12 Dec 1, 2022
💧 Visual Data Preparation (VDP) is an open-source tool to seamlessly integrate Vision AI with the modern data stack

Website | Community | Blog Get Early Access Visual Data Preparation (VDP) is an open-source tool to streamline the end-to-end visual data processing p

Instill AI 646 Jan 5, 2023
A simple tool to sync your etcd cluster to PostgreSQL in realtime.

etcd-postgresql-syncer A simple tool to sync your etcd cluster to PostgreSQL in realtime. It sets up a watcher on etcd and commits all changes to Post

Jille Timmermans 1 Jan 20, 2022
Enterprise-grade container platform tailored for multicloud and multi-cluster management

KubeSphere Container Platform What is KubeSphere English | 中文 KubeSphere is a distributed operating system providing cloud native stack with Kubernete

KubeSphere 11.7k Jan 2, 2023
A tool for visualization and monitoring Redis cluster realtime

Redat is a tool for visualize Redis-cluster on terminal Features: Visualize master-replicas relations in the Redis cluster Monitor with metrics: memor

Alex Nguyen 6 Nov 15, 2021
Metrics go: CudgX indicator management tool, which integrates monitoring and data analysis indicator capabilities

Metrics-Go metrics-go 是cudgx指标打点工具,它集成了监控和数据分析指标能力。 数据流程 指标数据流程为: 用户代码调用打点 SDK指标

Galaxy-Future 11 Oct 13, 2022
cluster-api-state-metrics (CASM) is a service that listens to the Kubernetes API server and generates metrics about the state of custom resource objects related of Kubernetes Cluster API.

Overview cluster-api-state-metrics (CASM) is a service that listens to the Kubernetes API server and generates metrics about the state of custom resou

Daimler Group 61 Oct 27, 2022
Go-gke-pulumi - A simple example that deploys a GKE cluster and an application to the cluster using pulumi

This example deploys a Google Cloud Platform (GCP) Google Kubernetes Engine (GKE) cluster and an application to it

Snigdha Sambit Aryakumar 1 Jan 25, 2022
🚢 Go package providing lifecycle management for PostgreSQL Docker instances.

?? psqldocker powered by ory/dockertest. Go package providing lifecycle management for PostgreSQL Docker instances. Leverage Docker to run unit and in

Adrian Brad 9 Sep 2, 2022
kubetnl tunnels TCP connections from within a Kubernetes cluster to a cluster-external endpoint, e.g. to your local machine. (the perfect complement to kubectl port-forward)

kubetnl kubetnl (kube tunnel) is a command line utility to tunnel TCP connections from within a Kubernetes to a cluster-external endpoint, e.g. to you

null 5 Dec 16, 2022
A pod scaler golang app that can scale replicas either inside of cluster or out of the cluster

pod-scaler A simple pod scaler golang application that can scale replicas via manipulating the deployment Technologies The project has been created us

Mert Doğan 0 Oct 24, 2021
Influxdb-cluster - InfluxDB Cluster for replacing InfluxDB Enterprise

InfluxDB ATTENTION: Around January 11th, 2019, master on this repository will be

Shiwen Cheng 524 Dec 26, 2022
A Terraform module to manage cluster authentication (aws-auth) for an Elastic Kubernetes (EKS) cluster on AWS.

Archive Notice The terraform-aws-modules/eks/aws v.18.20.0 release has brought back support aws-auth configmap! For this reason, I highly encourage us

Aidan Melen 28 Dec 4, 2022
Web user interface and service agent for the monitoring and remote management of WinAFL.

WinAFL Pet WinAFL Pet is a web user interface dedicated to WinAFL remote management via an agent running as a system service on fuzzing machines. The

Gabor Seljan 50 Nov 9, 2022
Tape backup software optimized for large WORM data and long-term recoverability

Mixtape Backup software for tape users with lots of WORM data. Draft design License This codebase is not open-source software (or free, or "libre") at

Dave Anderson 16 Oct 30, 2022
A Go package providing a generic data type to track maximum and minimum peak values.

go-peak Overview go-peak is a Go package providing a generic data type that tracks the maximum and minimum peak values within a specific period of tim

tunabay 0 Mar 26, 2022