TinyHat.Me: Microservices deployed with Kubernetes that enable users to propose hat pictures and try on hats from a user-curated database.

Overview

image

Click here to see the "buggy" version 🐛

The Scenario

TinyHat.Me is an up and coming startup that provides an API to allow users to try on tiny hats via a REST API. The startup is looking to you to ship some awesome new features! While we have a frontend working to serve tiny hats from a database, the startup is looking to add some exciting new features. Today, as the newest backend engineer at TinyHat.Me, you are going to be learning how the microservices work, as well as fix a few outstanding bugs from our customers!

The Outline

🔨 Building

  • What are Microservices?
  • Let's write the /email and /moderate services!
  • Hello? Are you there? How do the pods communicate?

🚀 Deploying

  • What even is Kubernetes, the whale, and containers?
  • Containerizing with Docker
  • EKS: it does not stand for Ecstatic Kittens Surfing
  • Let's spring into action: Deploying Continuously with Github
  • Routing traffic with Route53 and DNS configuration

🐛 Debugging

  • Yikes! Innappropriate hats are on the website!
  • We have complaints about the fetch feature...
  • The moderate can't moderate?

API Documentation

Run in Postman

Parameters

api.tinyhat.me?number=2 ?number (optional): How many hats you want!

[GET] /

api.tinyhat.me Returns a random tinyhat on Bob Ross.

[GET] /${hat style}

api.tinyhat.me/santa Returns a specific hat style on Bob Ross.

[GET] /api/hats

api.tinyhat.me/api/hats Returns a JSON object of all hats' image links and style identifiers.

[POST] /

api.tinyhat.me POST with multipart/form-data: image with face

Returns a random tinyhat on your POSTed image.

[POST] /${hat style}

api.tinyhat.me/santa POST with multipart/form-data: image with face

Returns a specific hat style on your POSTed image.

[POST] /add

api.tinyhat.me/add POST with multipart/form-data: image with hat and name attribute

Emails moderator with your proposed hat for moderation.

Flowchart

image

Comments
  • Setting up GitOps (Argo) with TinyHats

    Setting up GitOps (Argo) with TinyHats

    We should set up GitOps for TinyHats to have automated CI/CD deployments of docker images for each microservice linked to each repo. Right now we have to manually build shit which isn't good.

    opened by lazyplatypus 5
  • Locust!

    Locust!

    Closes #73

    Locust Load Generator with 2 parts

    Custom requests (custom-locust-service)

    image Access test.tinyhat.me to specify parameters for testing. Host value must be http://gateway-service/ and a hat name. *Example: http://gateway-service/cat-ears image

    Note: Need to route test.tinyhat.me to custom-locust-service Load Balancer

    Default requests (locust-service)

    There is always a locust server running in the background to generate at least a 1 RPS (request per second) rate. The rate may fall below this level depending on the server response rate. Requests are randomly made to different hats. image

    opened by emsesc 2
  • Set up Staging Environment

    Set up Staging Environment

    Set up Staging Environment that deploys from staging branch

    • [x] Spin down all existing EKS instances
    • [x] Create EKS cluster called Tinyhats-Staging
    • [x] Create Github Actions to:
      • [x] Automatically create docker image staging-add-service
      • [x] Automatically create docker image staging-admin-service
      • [x] Automatically create docker image staging-locust-service
      • [x] Automatically create docker image staging-fetch-service
      • [x] Automatically create docker image staging-frontend-service
      • [x] Automatically create docker image staging-manipulation-service
      • [x] Automatically create docker image staging-moderate-service
      • [x] Automatically create docker image staging-upload-service
    • [x] Create Github Actions to automatically redeploy kubernetes pod when there is a codechange
    • [ ] Set up Route53 to route staging.tinyhat.me to the staging environment.
    opened by lazyplatypus 1
  • Add Feature Flags for Bug Scenarios

    Add Feature Flags for Bug Scenarios

    Instead of having to switch docker images for the different scenarios, we should be able to have feature flags that can turn on and off various bugs that we have programmed in.

    opened by lazyplatypus 1
  • Add a Kafka Broker - turn `/fetch` into a Kafka producer & turn `/manipulate` into a  horizontally auto-scalable Kafka consumer

    Add a Kafka Broker - turn `/fetch` into a Kafka producer & turn `/manipulate` into a horizontally auto-scalable Kafka consumer

    Screen Shot 2021-10-17 at 4 18 48 PM

    We need to add Kafka functionality to the application. To do this successfully, we need to add modifications

    Pre-reqs

    • [x] We need to make sure that each message for the Kafka Broker does not exceed 1MB. Therefore, the Kafka message CANNOT contain the bare base64 image. We need to change the code so with the image name or ID we can fetch the base64 data from the MySQL database for /manipulate and /fetch

    Work

    • [ ] Spin up Kafka broker namespace using Strimzi.
    • [ ] Turn /fetch microservice into a Kafka producer, writing to the Kafka broker instead of sending a REST call
    • [ ] Turn /manipulate microservice into a Kafka consumer, reading from the Kafka broker instead of receiving a REST call
    • [ ] Enable Horizontal Autoscaling for /manipulate using this walkthrough
    kafka 
    opened by lazyplatypus 1
  • `/moderate` GoLang moderation service

    `/moderate` GoLang moderation service

    The email sent from /email will contain two links: one for approval and one for rejection. Each link will hit the moderate endpoint.

    Moderation Service

    Create a microservice that

    • adds the user-submitted image to the database upon admin request
    opened by lazyplatypus 1
  • Create upload endpoint

    Create upload endpoint

    About

    Accepts input in form-data, uploads to AWS S3 bucket, and pushes to MySQL AWS RDS. Generates random names for images to avoid conflict.

    image

    Closes #7 Closes #8

    DB Structure

    {"id":1,"keyId":"lfwy1ted33t35cnx","url":"https://uwuaascat.s3.amazonaws.com/lfwy1ted33t35cnx.jpeg","fileName":"lfwy1ted33t35cnx.jpeg","description":"baby lying down","approve":"false"}
    

    Database Name: main Table name: images

    Input

    Uses multipart/form-data. image: file upload name: description of the image

    Output

    {
        "key": "3lh9ypagvkm2j4mh",
        "fileName": "3lh9ypagvkm2j4mh.jpeg",
        "url": "https://uwuaascat.s3.amazonaws.com/3lh9ypagvkm2j4mh.jpeg",
        "description": "baby lying down",
        "approve": "false"
    }
    
    opened by emsesc 1
  • `/upload` Endpoint (NodeJS)

    `/upload` Endpoint (NodeJS)

    Upload Service

    Create a service that uploads an image to AWS S3, retrieves the S3 link, and stores it in AWS RDS. #7

    API input

    {
       image: `data`,
       category: "category",
       name: "catfish"
    }
    

    Reference readings

    null

    opened by emsesc 1
  • Animal DB [AWS RDS]

    Animal DB [AWS RDS]

    Spin up a https://github.com/typicode/lowdb instance to store all of the links to images and cat

    {
           category: "cat",
           link: "awslink.com/s3",
           name: "Cat in the rain", 
    }
    
    opened by lazyplatypus 1
  • New frontend (commerce)

    New frontend (commerce)

    • [x] Allow user to change model
    • [x] Allow user to choose model
    • [x] Make requests come from server, not client
    • [x] Style new interface
    • [x] Fix checkout cart
    opened by emsesc 0
  • Recoding manipulate

    Recoding manipulate

    Closes #102

    • accepts POST request to add hat onto image /manipulate
    • accepts POST request to add new hat (interacts with MySQL to store and create preview images) /add
    opened by emsesc 0
  • Scenario #1: Bad Architecture

    Scenario #1: Bad Architecture

    Scenario #1 - Bad Architecture

    Services Changed: Frontend Service Problem: The frontend calls the manipulate service for every single hat at page load time instead of using the pre-manipulated images from the database. Folder: scenarios/bad-architecture/

    image

    opened by emsesc 0
  • Scenario #2 - Redis Memory Issues

    Scenario #2 - Redis Memory Issues

    Scenario #2 - Redis Memory Issues

    Services Changed: YAML for Redis database Problem: We have allocated too little memory for redis to function properly Folder: scenarios/redis-memory/

    opened by emsesc 0
  • Scenario #3: Bad SQL Request

    Scenario #3: Bad SQL Request

    Scenario #3 - Bad SQL Request

    Services Changed: Product Catalog Service Problem: The SQL query to fetch hats is case sensitive. There are some images that break because of of the case sensitivity. Folder: scenarios/bad-sql/

    opened by emsesc 0
  • Helm Command

    Helm Command

    **Problem: ** For many demo environments, it is difficult to execute code changes. **Solution: ** Create a helm chart that allows you to pull and execute various docker images with various environmental variables to make demo-ing easier.

    Scope 
    opened by lazyplatypus 0
Owner
Bit Project
Making STEM education accessible.
Bit Project
Blog microservices deployed in an Istio-enabled kubernetes cluster

blog-microservices blog microservices deployed in an Istio-enabled kubernetes cl

jxlwqq 128 Dec 28, 2022
Omh-users-management - A go microservice that can enables us to create, modify, fetch, and delete users

Users Management System A go microservice that can enables us to create, modify, fetch, and delete users Usage To Run the application $ go run main.

Pulkit Tanwar 0 Jan 20, 2022
Create production ready microservices mono repo pattern wired with Neo4j. Microservices for other languages and front end repos to be added as well in future.

Create Microservices MonoRepo in GO/Python Create a new production-ready project with backend (Golang), (Python) by running one CLI command. Focus on

GoChronicles 14 Oct 26, 2022
Go-kit-microservices - Example microservices implemented with Go Kit

Go Kit Microservices Example microservices implemented with go kit, a programmin

Hao-Ming, Hsu 1 Jan 18, 2022
Go-user-service - User creation with Kafka producer

?? The Project This project is a simple user API developed to study microservice

Gustavo Belmonte Cioccari 1 Aug 19, 2022
Trying to build an Ecommerce Microservice in Golang and Will try to make it Cloud Native - Learning Example extending the project of Nic Jackson

Golang Server Project Best Practices Dependency Injection :- In simple words, we want our functions and packages to receive the objects they depend on

Ujjwal Sharma 35 Nov 28, 2022
A db proxy for distributed transaction, read write splitting and sharding! Support any language! It can be deployed as a sidecar in a pod.

DBPack DBPack means a database cluster tool pack. It can be deployed as a sidecar in a pod, it shields complex basic logic, so that business developme

null 334 Dec 29, 2022
Sample cloud-native application with 10 microservices showcasing Kubernetes, Istio, gRPC and OpenCensus.

Online Boutique is a cloud-native microservices demo application. Online Boutique consists of a 10-tier microservices application. The application is

Google Cloud Platform 13.5k Dec 31, 2022
Rpcx-framework - An RPC microservices framework based on rpcx, simple and easy to use, ultra fast and efficient, powerful, service discovery, service governance, service layering, version control, routing label registration.

RPCX Framework An RPC microservices framework based on rpcx. Features: simple and easy to use, ultra fast and efficient, powerful, service discovery,

ZYallers 1 Jan 5, 2022
Design-based APIs and microservices in Go

Goa is a framework for building micro-services and APIs in Go using a unique design-first approach. Overview Goa takes a different approach to buildin

Goa 5k Jan 5, 2023
Go microservices with REST, and gRPC using BFF pattern.

Go microservices with REST, and gRPC using BFF pattern. This repository contains backend services. Everything is dockerized and ready to

Oguzhan 152 Jan 4, 2023
This is an example to demonstrate implementation golang microservices using domain driven design principles and sugestions from go-kit

go-kit DDD Domain Driven Design is prevelent and rising standard for organizing your microservice code. This design architecture emphasis on Code orga

Sourabh Mandal 1 Feb 9, 2022
Istio - An open platform to connect, manage, and secure microservices

Istio An open platform to connect, manage, and secure microservices. For in-dept

null 0 Jan 5, 2022
A suite of microservices for software-defined networking (SDN) and bare-metal provisioning

M3L is a suite of microservices for software-defined networking (SDN) and bare-metal provisioning, which store their data as Custom Resources in Kubernetes.

Nicklas Frahm 0 Jan 19, 2022
An open platform to connect, manage, and secure microservices.

Istio An open platform to connect, manage, and secure microservices. For in-depth information about how to use Istio, visit istio.io To ask questions

Baalaji 0 Feb 6, 2022
MadeiraMadeira boilerplate project to build scalable, testable and high performance Go microservices.

MadeiraMadeira boilerplate project to build scalable, testable and high performance Go microservices.

Madeira Madeira 11 Sep 21, 2022
Microservice to manager users with golang

User Service Microservice to manager users. Assumptions & Limitations This API a

Guilherme Silveira 1 Dec 27, 2021
A standard library for microservices.

Go kit Go kit is a programming toolkit for building microservices (or elegant monoliths) in Go. We solve common problems in distributed systems and ap

Go kit 24.4k Jan 1, 2023
Zeebe.io - Workflow Engine for Microservices Orchestration

Distributed Workflow Engine for Microservices Orchestration

null 2.6k Jan 2, 2023