NJ Meetup - Build an event-driven architecture with Apache Pulsar

Overview

Meetup-YourFirstEventDrivenApp

NJ Meetup - Build an event-driven architecture with Apache Pulsar

Code Along


bin/pulsar-admin tenants create meetup
bin/pulsar-admin namespaces create meetup/newjersey
bin/pulsar-admin tenants list 
bin/pulsar-admin namespaces list meetup
bin/pulsar-admin topics create persistent://meetup/newjersey/first
bin/pulsar-admin topics list meetup/newjersey

bin/pulsar-client consume "persistent://meetup/newjersey/first" -s first-reader -n 0

Pulsar SQL Code Along

bin/pulsar sql

show catalogs;
show schemas in pulsar;
show tables in pulsar."meetup/newjersey";
describe pulsar."meetup/newjersey"."first";

select * from pulsar."meetup/newjersey"."first";

select __key__, from_utf8(__value__), 
           __publish_time__, __message_id__,
           __producer_name__
from pulsar."meetup/newjersey"."first"
order by __publish_time__ desc;

exit;

Monitoring Code Along

curl http://localhost:8080/admin/v2/persistent/meetup/newjersey/first/stats | python3 -m json.tool

bin/pulsar-admin topics stats-internal persistent://meetup/newjersey/first

curl http://pulsar1:8080/metrics/

bin/pulsar-admin topics stats-internal persistent://meetup/newjersey/first

bin/pulsar-admin topics peek-messages --count 5 --subscription mqtt-reader persistent://meetup/newjersey/first

bin/pulsar-admin topics subscriptions persistent://meetup/newjersey/first

Cleanup This Exercise (You don't have to)

bin/pulsar-admin topics delete persistent://meetup/newjersey/first
bin/pulsar-admin namespaces delete meetup/newjersey
bin/pulsar-admin tenants delete meetup

Web Pages For Viewing

Python Testing

import pulsar
client = pulsar.Client('pulsar://localhost:6650')
consumer = client.subscribe('persistent://meetup/newjersey/first',subscription_name='my-sub')
while True:
    msg = consumer.receive()
    print("Received message: '%s'" % msg.data())
    consumer.acknowledge(msg)
client.close()

import pulsar
client = pulsar.Client('pulsar://localhost:6650')
producer = client.create_producer('persistent://meetup/newjersey/first')
producer.send(('Hi Meetup people' ).encode('utf-8'))
client.close()

import pulsar
client = pulsar.Client('pulsar://localhost:6650')
producer = client.create_producer('persistent://meetup/newjersey/first')
producer.send(('Hi Meetup people' ).encode('utf-8'))
client.close()

bin/pulsar-client consume "persistent://meetup/newjersey/first" -s first-reader -n 0


GoLang App

go get -u "github.com/apache/pulsar-client-go/pulsar"

go build -o app produce.go

package main

import (
        "log"
        "time"
        "fmt"
        "context"
        "github.com/apache/pulsar-client-go/pulsar"
)

func main() {

        client, err := pulsar.NewClient(pulsar.ClientOptions{
        URL:               "pulsar://localhost:6650",
        OperationTimeout:  30 * time.Second,
        ConnectionTimeout: 30 * time.Second,
        })

        if err != nil {
                log.Fatal(err)
        }

        producer, err := client.CreateProducer(pulsar.ProducerOptions{
                Topic: "persistent://meetup/newjersey/first",
        })

        _, err = producer.Send(context.Background(), &pulsar.ProducerMessage{
    Payload: []byte("hello"),
})

defer producer.Close()

if err != nil {
    fmt.Println("Failed to publish message", err)
}
fmt.Println("Published message")


}

References

For Ubuntu 18.04 on AMD65

  • go1.13.9.linux-amd64.tar.gz
You might also like...
A local meetup to show some of the features of the Twirp RPC framework

twirpdemo This repo was created for a local meetup to show some of the features of the Twirp RPC framework. Usage Generate proto code: protoc --twirp

Experiments with pulsar go client and protobuf

How to use proto schema with pulsar go client library Define a .proto file Generate Code using below command: protoc -I=. -I=$GOPATH/src -I=$GOPATH/sr

go broker interface,you can use kafka,redis,pulsar etc.

broker go broker interface,you can use kafka,redis,pulsar etc. pulsar in docker run pulsar in docker docker run -dit \ --name pulsar-sever \ -p 6650:

Kafka, Beanstalkd, Pulsar Pub/Sub framework

go-queue Kafka, Beanstalkd, Pulsar Pub/Sub framework.

Next generation distributed, event-driven, parallel config management!
Next generation distributed, event-driven, parallel config management!

mgmt: next generation config management! About: Mgmt is a real-time automation tool. It is familiar to existing configuration management software, but

Project Flogo is an open source ecosystem of opinionated  event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.
Project Flogo is an open source ecosystem of opinionated event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.

Project Flogo is an Open Source ecosystem for event-driven apps Ecosystem | Core | Flows | Streams | Flogo Rules | Go Developers | When to use Flogo |

🚀 gnet is a high-performance, lightweight, non-blocking, event-driven networking framework written in pure Go./ gnet 是一个高性能、轻量级、非阻塞的事件驱动 Go 网络框架。
🚀 gnet is a high-performance, lightweight, non-blocking, event-driven networking framework written in pure Go./ gnet 是一个高性能、轻量级、非阻塞的事件驱动 Go 网络框架。

English | 🇨🇳 中文 📖 Introduction gnet is an event-driven networking framework that is fast and lightweight. It makes direct epoll and kqueue syscalls

Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge.
Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge.

Dapr is a portable, serverless, event-driven runtime that makes it easy for developers to build resilient, stateless and stateful microservices that run on the cloud and edge and embraces the diversity of languages and developer frameworks.

High-performance, non-blocking, event-driven, easy-to-use networking framework written in Go, support tls/http1.x/websocket.

High-performance, non-blocking, event-driven, easy-to-use networking framework written in Go, support tls/http1.x/websocket.

pubsub controller using kafka and base on sarama. Easy controll flow for actions streamming, event driven.

Psub helper for create system using kafka to streaming and events driven base. Install go get github.com/teng231/psub have 3 env variables for config

TriggerMesh open source event-driven integration platform powered by Kubernetes and Knative.

TriggerMesh open source event-driven integration platform powered by Kubernetes and Knative. TriggerMesh allows you to declaratively define event flows between sources and targets as well as add even filter, splitting and processing using functions.

An example event-driven application using Atmo and NATS

Atmo + NATS Example Project This repo is an example of using Atmo with NATS as a streaming messaging layer. In this example, Atmo connects to NATS and

POC of an event-driven Go implementation

Event Driven example in Golang This POC shows an example of event-driven architecture with a working domain event broker, an event producer and a cons

Example Golang Event-Driven with kafka Microservices Choreography

Microservices Choreography A demonstration for event sourcing using Go and Kafka example Microservices Choreography. To run this project: Install Go I

Prueba de concepto: Boletia, una aplicación para venta de boletos, basada en microservicios event-driven. Desarrollada sobre AWS Serverless: Api Gateway, Lambda, DynamoDB, DynamoDB Streams
Prueba de concepto: Boletia, una aplicación para venta de boletos, basada en microservicios event-driven. Desarrollada sobre AWS Serverless: Api Gateway, Lambda, DynamoDB, DynamoDB Streams

Prueba de concepto: Boletia, una aplicación para venta de boletos, basada en microservicios event-driven. Desarrollada sobre AWS Serverless: Api Gatew

Event driven modular status-bar for dwm; written in Go & uses Unix sockets for signaling.

dwmstat A simple event-driven modular status-bar for dwm. It is written in Go & uses Unix sockets for signaling. The status bar is conceptualized as a

Eye - An easy-use lib for event-driven pattern

📝 Eye Eye 是一个简单易用的事件驱动模式库。 Read me in English 🥇 功能特性 敬请期待。。。 历史版本的特性请查看 HISTOR

An event driven remote access trojan for experimental purposes.

erat An event driven remote access trojan for experimental purposes. This example is very simple and leverages ssh failed login events to trigger erat

Package event-driven makes it easy for you to drive events between services
Package event-driven makes it easy for you to drive events between services

Event-Driven Event-driven architecture is a software architecture and model for application design. With an event-driven system, the capture, communic

Owner
Timothy Spann
Dev Advocate, Java Dev, Cloud Analytics Dev, Apache NiFi Dev, Data Engineer, IoT Dev, AI, ML, Python Developer, RP, FLANK, FLIP, Apache Pulsar, StreamNative
Timothy Spann
Clean Architecture With Golang

Clean Architecture With Golang When init a new project go mod init github.com/samuelterra22/clean-architecture-go Run testes go test ./... Generate a

Samuel Terra 2 Aug 2, 2022
A highly flexible blockchain architecture with great transaction performance.

XuperChain 中文说明 What is XuperChain XuperChain, the first open source project of XuperChain Lab, introduces a underlying solution to build the super al

null 1.6k Jan 2, 2023
The open source, end-to-end computer vision platform. Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises.

End-to-end computer vision platform Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises. onepa

Onepanel, Inc. 642 Dec 12, 2022
KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes

Kubernetes-based Event Driven Autoscaling KEDA allows for fine-grained autoscaling (including to/from zero) for event driven Kubernetes workloads. KED

KEDA 5.9k Jan 7, 2023
Hexagonal architecture paradigms, such as dividing adapters into primary (driver) and secondary (driven)Hexagonal architecture paradigms, such as dividing adapters into primary (driver) and secondary (driven)

authorizer Architecture In this project, I tried to apply hexagonal architecture paradigms, such as dividing adapters into primary (driver) and second

Renato Benatti 0 Dec 7, 2021
Build event-driven and event streaming applications with ease

Commander ?? Commander is Go library for writing event-driven applications. Enabling event sourcing, RPC over messages, SAGA's, bidirectional streamin

Jeroen Rinzema 62 Dec 19, 2022
Apache Pulsar Go Client Library

Apache Pulsar Go Client Library A Go client library for the Apache Pulsar project. Goal This projects is developing a pure-Go client library for Pulsa

The Apache Software Foundation 525 Jan 4, 2023
ftgogo - event-driven architecture demonstration application

ftgogo (food-to-gogo) is a Golang implementation of the FTGO application described in the book "Microservice Patterns" by Chris Richardson. A library edat was developed to provide for Golang many of the solutions that Eventuate, the framework used by FTGO, provides for Java.

Michael Stack 101 Jan 3, 2023
Go library to build event driven applications easily using AMQP as a backend.

event Go library to build event driven applications easily using AMQP as a backend. Publisher package main import ( "github.com/creekorful/event" "

Aloïs Micard 0 Dec 5, 2021
Demo project for unit testing presentation @ GoJKT meetup

go-demo-service Demo project for unit testing presentation @ GoJKT meetup This is a demo project to show examples of unit testing for GoJKT meetup Use

M Yusuf Irfan H 3 Jul 10, 2021