Devstack is Razorpay's Developer Experience Solution for cloud on laptop

Overview

devstack

Devstack is Razorpay's Developer Experience Solution for cloud on laptop

What is Devstack

At razorpay, we run all our workloads on kubernetes. Like any other mature organization, we have an involved CI/CD practice with extremely sophisticated pipelines. While this works great for all production and pre-production workloads, we have been noticing over a period of time a bunch of development challenges. In essence, the goal is to Simplify developer workflow and reduce the time taken to rollout features independently. Devstack, offers a set of tools to help build and develop code on the individual developer's laptop, as if they are working on a cloud environment.

In a nutshell: "Its a client based development tool for building cloud native applications on kubernetes"

Presentations and Videos

Setup And Installation

Pre-Requisites

As we mentioned earlier, our solution is slightly opinionated and requires the following stack:

  • Cloud Provider : AWS(Note: Our custom infra helm hooks are all designed for AWS. However, it can be extended. See details below)
  • Kubernetes: Requires verion 1.15+
  • Traefik: 2.0+ to be deployed on the kubernetes cluster above. Please see the official installation instructions
  • Helm: 3.0+
  • LocalStack: To be deployed on kubernetes cluster above. Refer to LocalStack
  • Kube Janitor: Deploy Kube Janitor

Other requirements (For hot-reload) If you are using a loosely typed language like php / python etc, then you can safely skip this section. For static languages like golang, java, nodejs etc, please refer below:

Design Goals

High Level Goals

  • Streamlined Dev Workflow: Provide a streamlined workflow and faster merges to main or master branches.
  • Consistent Environment: Provide a seamless consistent environment across dev, stage, pre-prod and production environments
  • Faster Feedback: Reduce time to write and build containerized applications. Enable faster feedback loop on local development environment

Design Choices

  • Remove vendor lock-in (rely on OSS practically)
  • Kubernetes native (At the moment, we don't have extensions for non K8s solutions). Our environment is kubernetes native
  • Hassle Free onboarding - minimal changes to application and development lifecycle
  • Cost Effective - We should eventually be able to bill developers and teams on usage patterns.
  • Slightly Opinionated - This isn't a PaaS offering at the momemt. And hence can be extended and deployed on any native kubernetes installation

Features

  • Build, Test and Deploy applications from laptop directly into kubernetes using simple CLI tools
  • Ship code to remote container without tunneling: Sync files into container directly using File Sync(using Devspace)
  • Ability to provide hot reloading of apps : sync directly into the containers without restart(e.g. CompileDaemon for statically typed languages)
  • Support out-of-the-box support using existing available helm hooks
  • Support for extensible custom Helm Hooks that handles provisioning of AWS infrastructure components using LocalStack
  • Declarative-ly define service and service dependencies using Helmfile
  • Traffic routing to right upstream. Uses traefik 2.0 IngressRoute
  • Ability to selectively route traffic to different upstream. Done via opentelemetry context/header propagation
  • Ability to expose preview URL for all services
  • Automatic timed cleanup(ttl) of resources using Kube Janitor

Docs

Architecture

Please refer to the Architecture Overview for entire details on the devstack architecture.

Examples

TBD

Extension

We have provided a collection of custom helm hooks for AWS and kubernetes specific workloads. All of these are extensible. Please refer to the documentation of the helm hooks(TBD)

Need Help

Please file an issue on this repo using the following labels: Clarification, Feature, Bug

Contributing

Please refer to the Contribution Guide

Roadmap

TBD

Issues
  • feat: adds secret cloner hook

    feat: adds secret cloner hook

    What this PR does: Adds the secret cloner , which would clone a secret or update with the given values TODO:

    • [ ] Add Readme for app usage
    • [ ] Add helm hook job reference
    • [x] Fix docker container bootstrap
    opened by srinidhis05 0
  • Feature: Adds example SQS producer consumer cli app

    Feature: Adds example SQS producer consumer cli app

    Adds a sample golang SQS Producer Consumer Application. Uses localstack for testing

    Dependencies:

    • This should be merged after #3 is merged and this branch is rebased against master. This is coz, both share the same root folder example and would need conflict resolve, if at all.

    TODO:

    • [ ] Add K8s chart
    • [ ] Test on devstack
    opened by venkatvghub 0
  • Create LICENSE

    Create LICENSE

    Add License.

    License of used Components:

    1. Helmfile - MIT License. Ref: https://github.com/roboll/helmfile/blob/master/LICENSE
    2. Devspace - Apache 2.0 License. Ref: https://github.com/loft-sh/devspace/blob/master/LICENSE
    3. Traefik - MIT License. Ref: https://github.com/traefik/traefik/blob/master/LICENSE.md
    4. Kube Janitor - GNU Ref: https://codeberg.org/hjacobs/kube-janitor/src/branch/main/LICENSE
    opened by venkatvghub 0
  • feat: adds sqs configurator hook

    feat: adds sqs configurator hook

    What this PR does: Adds the sqs configurator , which would take the queue details as input and create the queues in either localstack or AWS along with updating the corresponding secret based on config TODO:

    • [ ] Add Readme for app usage
    • [ ] Add helm hook job reference
    opened by srinidhis05 0
  • Feat: Adds demo web app

    Feat: Adds demo web app

    What this PR does: Adds a demo web app with tracing for demoing the devstack features. Single crud app. Simple Crud App. Api Documentation in docs with a postman collection for quick testing

    TODO:

    • [x] Add Readme for app usage and docker build
    • [x] Add K8s manifests and charts for the demo app
    • [x] Deploy and Test on devstack with preview URL
    opened by venkatvghub 0
Owner
Razorpay
Neobanking for Businesses
Razorpay
This Go based project of Aadhyarupam Innovators demonstrate the code examples for building microservices, integration with cloud services (Google Cloud Firestore), application configuration management (Viper) etc.

This Go based project of Aadhyarupam Innovators demonstrate the code examples for building microservices, integration with cloud services (Google Cloud Firestore), application configuration management (Viper) etc.

Aadhyarupam 0 Jan 31, 2022
Solution to elevator test problem but this time recursive and in go

Synopsis A multi-floor building has a Lift in it. People are queued on different floors waiting for the Lift. Some people want to go up. Some people w

Alex Piemont 0 Nov 8, 2021
My Go solution that's faster than 95%

Palindrome-Partitioning My Go solution that's faster than 95% "Given a string s, partition s such that every substring of the partition is a palindrom

Daniel Pickens 0 Nov 15, 2021
Weaviate is a cloud-native, modular, real-time vector search engine

Weaviate is a cloud-native, real-time vector search engine (aka neural search engine or deep search engine). There are modules for specific use cases such as semantic search, plugins to integrate Weaviate in any application of your choice, and a console to visualize your data.

SeMI Technologies 2.5k Jun 24, 2022
CapMonsterTool is a set of Go tools designed to simply make requests to the CapMonster Cloud API.

✨ CapMonsterTool ✨ About this module What is ✨ CapMonsterTool ✨ ? CapMonsterTool is a set of Go tools designed to simply make requests to the CapMonst

Louis Billaut 18 Jun 23, 2022
Phalanx is a cloud-native full-text search and indexing server written in Go built on top of Bluge that provides endpoints through gRPC and traditional RESTful API.

Phalanx Phalanx is a cloud-native full-text search and indexing server written in Go built on top of Bluge that provides endpoints through gRPC and tr

Minoru Osuka 232 Jun 30, 2022
A server for TurboRepo Remote Cache to store cache artefacts in Google Cloud Storage or Amazon S3

Tapico Turborepo Remote Cache This is an implementation of Vercel's Turborepo Remote Cache API endpoints used by the turborepo CLI command. This solut

Tapico 70 Jun 18, 2022
The temporal cloud cli.

tcld (Beta) A cli tool for managing Temporal Cloud namespaces. This cli tool is currently in beta and access to Temporal Cloud via the cli is restrict

temporal.io 16 May 18, 2022
Used gRPC for the first time, and it was a amazing developer experience

gRPC Used gRPC for the first time, and it was a amazing developer experience. Edge points of using gPRC which I felt: Structured Code Uniform request

Mrigank Anand 5 Oct 11, 2021
A BPMN engine, meant to be embedded in Go applications with minim hurdles, and a pleasant developer experience using it.

A BPMN engine, meant to be embedded in Go applications with minim hurdles, and a pleasant developer experience using it. This approach can increase transparency for non-developers.

Martin W. Kirst 32 Jun 26, 2022
Secure Boot certificates from the Framework Laptop

Framework Laptop UEFI Secure Boot Certificates Source: Extracted from a live machine (FRANBMCP08) Date: 2021-10-21 KEK (Key Exchange Key) This certifi

Dustin L. Howett 14 Mar 15, 2022
Laptop Booking Application in Golang and gRPC, load-balancing with NGINX, and fully compatible with HTTPS OpenAPI v3

Laptop Booking Application in Golang and gRPC Goals GitHub CI & Coverage Badge Serialize protobuf messages Create laptop unary gRPC Search laptop Serv

Tien La 3 Jun 17, 2022
provide api for cloud service like aliyun, aws, google cloud, tencent cloud, huawei cloud and so on

cloud-fitter 云适配 Communicate with public and private clouds conveniently by a set of apis. 用一套接口,便捷地访问各类公有云和私有云 对接计划 内部筹备中,后续开放,有需求欢迎联系。 开发者社区 开发者社区文档

null 23 May 8, 2022
Enables a FaaS experience for Knative / Cloud Native Runtimes.

Function Buildpacks for Knative Enables a FaaS experience for Knative / Cloud Native Runtimes. Will soon extend func to create deployable functions vi

VMware Tanzu 16 Jun 16, 2022
This project provides fully automated one-click experience to create Cloud and Kubernetes environment to run Data Analytics workload like Apache Spark.

Introduction This project provides a fully automated one-click tool to create Data Analytics platform in Cloud and Kubernetes environment: Single scri

DataPunch - One Click to Create Cloud and Kubernetes Environment for Data Analytics and Apache Spark 39 May 31, 2022
Zadig is a cloud native, distributed, developer-oriented continuous delivery product.

Zadig Developer-oriented Continuous Delivery Product English | 简体中文 Table of Contents Zadig Table of Contents What is Zadig Quick start How to use? Ho

KodeRover 1.5k Jun 27, 2022
Zadig is a cloud native, distributed, developer-oriented continuous delivery product.

Zadig Developer-oriented Continuous Delivery Product ⁣ English | 简体中文 Table of Contents Zadig Table of Contents What is Zadig Quick start How to use?

KodeRover 30 May 12, 2021
stratus is a cross-cloud identity broker that allows workloads with an identity issued by one cloud provider to exchange this identity for a workload identity issued by another cloud provider.

stratus stratus is a cross-cloud identity broker that allows workloads with an identity issued by one cloud provider to exchange this identity for a w

robert lestak 1 Dec 26, 2021
Cloud-Z gathers information and perform benchmarks on cloud instances in multiple cloud providers.

Cloud-Z Cloud-Z gathers information and perform benchmarks on cloud instances in multiple cloud providers. Cloud type, instance id, and type CPU infor

CloudSnorkel 16 Jun 8, 2022
Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Bytedance Inc. 1.2k Jun 23, 2022
Blue is a lightweight cloud-native gateway solution to handle millions of routing endpoints with a large number of connections.

Blue is a lightweight cloud-native gateway solution to handle millions of routing endpoints with a large number of connections.

null 456 Jan 19, 2022
A Declarative Cloud Firewall Reverse Proxy Solution with Companion Mobile App

A declarative Cloud firewall reverse proxy solution with inbuilt DDoS protection and alerting mechanism to protect your servers and keeping an eye on those malicious requests

null 12 Apr 4, 2022
An Enhanced Go Experience For The Atom Editor

go-plus An Improved Go Experience For The Atom Editor Github: https://github.com/joefitzgerald/go-plus Atom: https://atom.io/packages/go-plus Overview

Joe Fitzgerald 1.5k Jun 18, 2022
red-tldr is a lightweight text search tool, which is used to help red team staff quickly find the commands and key points they want to execute, so it is more suitable for use by red team personnel with certain experience.

Red Team TL;DR English | 中文简体 What is Red Team TL;DR ? red-tldr is a lightweight text search tool, which is used to help red team staff quickly find t

倾旋 159 Jun 12, 2022
Flagr is an open source Go service that delivers the right experience to the right entity and monitors the impact.

Flagr is an open source Go service that delivers the right experience to the right entity and monitors the impact. It provides feature flags, experimentation (A/B testing), and dynamic configuration. It has clear swagger REST APIs for flags management and flag evaluation.

null 35 Jun 27, 2022
ZITADEL - Identity Experience Platform

What Is ZITADEL ZITADEL is a "Cloud Native Identity and Access Management" solution built for the cloud era. ZITADEL uses a modern software stack cons

CAOS 1.1k Jun 26, 2022
A "passwordless" login experience for your AWS RDS

RDS Auth Proxy A two-layer proxy for connecting into RDS postgres databases based on IAM authentication. This tool allows you to keep your databases f

Mothership 20 Jun 25, 2022
A unified graphical user experience toolkit for Go desktop applications

Unison A unified graphical user experience toolkit for Go desktop applications. macOS, Windows, and Linux are supported. Required setup Unison is buil

Richard Wilkes 14 Jun 17, 2022