EaseMesh is a service mesh that is compatible with the Spring Cloud ecosystem.

Overview

  • JavaAgent has higher latency in the Mesh's data plane

    After the java agent's observability worked, we can observe the latency between two services. In our environment, we have the following invocation diagram.

                                       ┌───────────────────────────┐
                                       │                           │                   ┌──────────────┐
                        (1)            │                           │        (2)        │              │
                 ┌─────────────────────►      mesh-app-backend     ├───────────────────►    db-mysql  │
                 │                     │         /users/{id}       │                   │              │
                 │                     │                           │                   └──────────────┘
                 │                     └───────────────────────────┘
                 │
    ┌────────────┴────────────┐
    │                         │
    │      mesh-app-front     │
    │ /front/userFullInfo/{id}│
    │                         │
    └────────────┬────────────┘
                 │                     ┌───────────────────────────┐
                 │                     │                           │
                 │      (3)            │                           │
                 └─────────────────────►    mesh-app-performance   │
                                       │      /userStatus/{id}     │
                                       │                           │
                                       └───────────────────────────┘
    
    

    I pick a tracing recording, I found the latency between mesh-app-frontend and mesh-app-backend service is higher.

    |Type |Start Time|Relative Time |Address |-|-|-|-| |Client Start| 03/29 11:24:58.167_468|441μs| 10.233.111.77 (mesh-app-frontend)| |Server Start|03/29 11:24:58.183_069|16.042ms|10.233.67.33 (mesh-app-backend)| |Server Finish|03/29 11:24:58.192_037|25.010ms|10.233.67.33 (mesh-app-backend)| |Client Finish|03/29 11:24:58.193_820|26.793ms|10.233.111.77 (mesh-app-frontend)|

    image

    The first section between the two white spots is the communication latency client service (mesh-app-frontend) to server service (mesh-app-backend) . It's apparently too high, about occupies 50% latency of the request.

    question performance 
    opened by zhao-kun 11
  • Support native deployment

    Support native deployment

    Detail

    Actually, this is also a refactoring in implementing the new feature:

    1. Simplify interface with a lot of arguments.
    2. Complete service spec.
    3. Simplify complex containers/volumes layout (see reference comparison), moreover, it is more consistent.
    4. Make logs clean and consistent.

    And there left something to complete, which of part were commented, and:

    1. Add documents for supporting native deployment.
    2. Add more unit tests.

    After them, I found more trivial or significant stuff needed to be improved, which will be opened issues in the next contribution.

    Reference

    There lists the old and new specs of deployment after injecting the sidecar. The same/unimportant part has been committed. We can see the two init containers have been merged into one, and the volumes and volumeMounts became more consistent. The config of k8s objects are complex enough, it's very important to manage the complexity of them carefully.

    old-vets-service.yaml:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: vets-service
      namespace: spring-petclinic
    spec:
      template:
        metadata:
          creationTimestamp: null
          labels:
            app: vets-service
        spec:
          containers:
          - args:
            - -c
            - java -server -Xmx1024m -Xms1024m -Dspring.profiles.active=sit -Djava.security.egd=file:/dev/./urandom  org.springframework.boot.loader.JarLauncher
            command:
            - /bin/sh
            env:
            - name: JAVA_TOOL_OPTIONS
              value: ' -javaagent:/easeagent-volume/easeagent.jar -Deaseagent.log.conf=/easeagent-volume/log4j2.xml '
            image: megaease/spring-petclinic-vets-service:latest
            imagePullPolicy: IfNotPresent
            lifecycle:
              preStop:
                exec:
                  command:
                  - sh
                  - -c
                  - sleep 10
            name: vets-service
            ports:
            - containerPort: 8080
              protocol: TCP
            resources:
              limits:
                cpu: "2"
                memory: 1Gi
              requests:
                cpu: 200m
                memory: 256Mi
            terminationMessagePath: /dev/termination-log
            terminationMessagePolicy: File
            volumeMounts:
            - mountPath: /application/application-sit.yml
              name: configmap-volume-0
              subPath: application-sit.yml
            - mountPath: /easeagent-volume
              name: easeagent-volume
          - command:
            - /bin/sh
            - -c
            - /opt/easegress/bin/easegress-server -f /easegress-sidecar/eg-sidecar.yaml
            env:
            - name: APPLICATION_IP
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: status.podIP
            image: megaease/easegress:server-sidecar
            imagePullPolicy: Always
            name: easemesh-sidecar
            ports:
            - containerPort: 13001
              name: sidecar-ingress
              protocol: TCP
            - containerPort: 13002
              name: sidecar-egress
              protocol: TCP
            - containerPort: 13009
              name: sidecar-eureka
              protocol: TCP
            resources: {}
            terminationMessagePath: /dev/termination-log
            terminationMessagePolicy: File
            volumeMounts:
            - mountPath: /easegress-sidecar
              name: sidecar-params-volume
          dnsPolicy: ClusterFirst
          initContainers:
          - command:
            - /bin/sh
            - -c
            - cp -r /easeagent-volume/. /easeagent-share-volume
            image: megaease/easeagent-initializer:latest
            imagePullPolicy: Always
            name: easeagent-initializer
            resources: {}
            terminationMessagePath: /dev/termination-log
            terminationMessagePolicy: File
            volumeMounts:
            - mountPath: /easeagent-share-volume
              name: easeagent-volume
          - command:
            - /bin/sh
            - -c
            - |-
              echo name: $POD_NAME >> /opt/eg-sidecar.yaml; echo 'cluster-join-urls: http://easemesh-controlplane-svc.easemesh:2380
              cluster-request-timeout: 10s
              cluster-role: reader
              cluster-name: easemesh-control-plane
              Labels:
                alive-probe: http://localhost:9900/health
                application-port: "8080"
                mesh-service-labels: ""
                mesh-servicename: vets-service
              ' >> /opt/eg-sidecar.yaml; cp -r /opt/. /sidecar-params-volume
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            image: megaease/easegress:server-sidecar
            imagePullPolicy: Always
            name: easegress-sidecar-initializer
            resources: {}
            terminationMessagePath: /dev/termination-log
            terminationMessagePolicy: File
            volumeMounts:
            - mountPath: /sidecar-params-volume
              name: sidecar-params-volume
          restartPolicy: Always
          schedulerName: default-scheduler
          securityContext: {}
          terminationGracePeriodSeconds: 30
          volumes:
          - configMap:
              defaultMode: 420
              items:
              - key: application-sit-yml
                path: application-sit.yml
              name: vets-service
            name: configmap-volume-0
          - emptyDir: {}
            name: easeagent-volume
          - emptyDir: {}
            name: sidecar-params-volume
    

    new-vets-service.yaml:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: vets-service
      namespace: spring-petclinic
    spec:
      template:
        metadata:
          creationTimestamp: null
          labels:
            app: vets-service
        spec:
          containers:
          - args:
            - -c
            - java -server -Xmx1024m -Xms1024m -Dspring.profiles.active=sit -Djava.security.egd=file:/dev/./urandom  org.springframework.boot.loader.JarLauncher
            command:
            - /bin/sh
            env:
            - name: JAVA_TOOL_OPTIONS
              value: ' -javaagent:/agent-volume/easeagent.jar -Deaseagent.log.conf=/agent-volume/log4j2.xml '
            image: megaease/spring-petclinic-vets-service:latest
            imagePullPolicy: IfNotPresent
            lifecycle:
              preStop:
                exec:
                  command:
                  - sh
                  - -c
                  - sleep 10
            name: vets-service
            ports:
            - containerPort: 8080
              protocol: TCP
            resources:
              limits:
                cpu: "2"
                memory: 1Gi
              requests:
                cpu: 200m
                memory: 256Mi
            terminationMessagePath: /dev/termination-log
            terminationMessagePolicy: File
            volumeMounts:
            - mountPath: /application/application-sit.yml
              name: configmap-volume-0
              subPath: application-sit.yml
            - mountPath: /agent-volume
              name: agent-volume
          - command:
            - /bin/sh
            - -c
            - /opt/easegress/bin/easegress-server -f /sidecar-volume/sidecar-config.yaml
            env:
            - name: APPLICATION_IP
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: status.podIP
            image: megaease/easegress:server-sidecar
            imagePullPolicy: IfNotPresent
            name: easemesh-sidecar
            ports:
            - containerPort: 13001
              name: sidecar-ingress
              protocol: TCP
            - containerPort: 13002
              name: sidecar-egress
              protocol: TCP
            - containerPort: 13009
              name: sidecar-eureka
              protocol: TCP
            resources: {}
            terminationMessagePath: /dev/termination-log
            terminationMessagePolicy: File
            volumeMounts:
            - mountPath: /sidecar-volume
              name: sidecar-volume
          dnsPolicy: ClusterFirst
          initContainers:
          - command:
            - sh
            - -c
            - |-
              set -e
              cp -r /easeagent-volume/* /agent-volume
    
              echo 'name: vets-service
              cluster-join-urls: http://easemesh-controlplane-svc.easemesh:2380
              cluster-request-timeout: 10s
              cluster-role: reader
              cluster-name: easemesh-control-plane
              labels:
                alive-probe:
                application-port: 8080
                mesh-service-labels:
                mesh-servicename: vets-service
              ' > /sidecar-volume/sidecar-config.yaml
            image: megaease/easeagent-initializer:latest
            imagePullPolicy: Always
            name: initializer
            resources: {}
            terminationMessagePath: /dev/termination-log
            terminationMessagePolicy: File
            volumeMounts:
            - mountPath: /agent-volume
              name: agent-volume
            - mountPath: /sidecar-volume
              name: sidecar-volume
          restartPolicy: Always
          schedulerName: default-scheduler
          securityContext: {}
          terminationGracePeriodSeconds: 30
          volumes:
          - configMap:
              defaultMode: 420
              items:
              - key: application-sit-yml
                path: application-sit.yml
              name: vets-service
            name: configmap-volume-0
          - emptyDir: {}
            name: agent-volume
          - emptyDir: {}
            name: sidecar-volume
    

    The diff of them:

    diff --git old-vets-service.yaml new-vets-service.yaml
    index cf4331e..dcb0781 100644
    --- old-vets-service.yaml
    +++ new-vets-service.yaml
    @@ -18,7 +18,7 @@ spec:
             - /bin/sh
             env:
             - name: JAVA_TOOL_OPTIONS
    -          value: ' -javaagent:/easeagent-volume/easeagent.jar -Deaseagent.log.conf=/easeagent-volume/log4j2.xml '
    +          value: ' -javaagent:/agent-volume/easeagent.jar -Deaseagent.log.conf=/agent-volume/log4j2.xml '
             image: megaease/spring-petclinic-vets-service:latest
             imagePullPolicy: IfNotPresent
             lifecycle:
    @@ -45,12 +45,12 @@ spec:
             - mountPath: /application/application-sit.yml
               name: configmap-volume-0
               subPath: application-sit.yml
    -        - mountPath: /easeagent-volume
    -          name: easeagent-volume
    +        - mountPath: /agent-volume
    +          name: agent-volume
           - command:
             - /bin/sh
             - -c
    -        - /opt/easegress/bin/easegress-server -f /easegress-sidecar/eg-sidecar.yaml
    +        - /opt/easegress/bin/easegress-server -f /sidecar-volume/sidecar-config.yaml
             env:
             - name: APPLICATION_IP
               valueFrom:
    @@ -58,7 +58,7 @@ spec:
                   apiVersion: v1
                   fieldPath: status.podIP
             image: megaease/easegress:server-sidecar
    -        imagePullPolicy: Always
    +        imagePullPolicy: IfNotPresent
             name: easemesh-sidecar
             ports:
             - containerPort: 13001
    @@ -74,52 +74,39 @@ spec:
             terminationMessagePath: /dev/termination-log
             terminationMessagePolicy: File
             volumeMounts:
    -        - mountPath: /easegress-sidecar
    -          name: sidecar-params-volume
    +        - mountPath: /sidecar-volume
    +          name: sidecar-volume
           dnsPolicy: ClusterFirst
           initContainers:
           - command:
    -        - /bin/sh
    -        - -c
    -        - cp -r /easeagent-volume/. /easeagent-share-volume
    -        image: megaease/easeagent-initializer:latest
    -        imagePullPolicy: Always
    -        name: easeagent-initializer
    -        resources: {}
    -        terminationMessagePath: /dev/termination-log
    -        terminationMessagePolicy: File
    -        volumeMounts:
    -        - mountPath: /easeagent-share-volume
    -          name: easeagent-volume
    -      - command:
    -        - /bin/sh
    +        - sh
             - -c
             - |-
    -          echo name: $POD_NAME >> /opt/eg-sidecar.yaml; echo 'cluster-join-urls: http://easemesh-controlplane-svc.easemesh:2380
    +          set -e
    +          cp -r /easeagent-volume/* /agent-volume
    +
    +          echo 'name: vets-service
    +          cluster-join-urls: http://easemesh-controlplane-svc.easemesh:2380
               cluster-request-timeout: 10s
               cluster-role: reader
               cluster-name: easemesh-control-plane
    -          Labels:
    -            alive-probe: http://localhost:9900/health
    -            application-port: "8080"
    -            mesh-service-labels: ""
    +          labels:
    +            alive-probe:
    +            application-port: 8080
    +            mesh-service-labels:
                 mesh-servicename: vets-service
    -          ' >> /opt/eg-sidecar.yaml; cp -r /opt/. /sidecar-params-volume
    -        env:
    -        - name: POD_NAME
    -          valueFrom:
    -            fieldRef:
    -              apiVersion: v1
    -              fieldPath: metadata.name
    -        image: megaease/easegress:server-sidecar
    +          ' > /sidecar-volume/sidecar-config.yaml
    +        image: megaease/easeagent-initializer:latest
             imagePullPolicy: Always
    -        name: easegress-sidecar-initializer
    +        name: initializer
             resources: {}
             terminationMessagePath: /dev/termination-log
             terminationMessagePolicy: File
             volumeMounts:
    -        - mountPath: /sidecar-params-volume
    -          name: sidecar-params-volume
    +        - mountPath: /agent-volume
    +          name: agent-volume
    +        - mountPath: /sidecar-volume
    +          name: sidecar-volume
           restartPolicy: Always
           schedulerName: default-scheduler
           securityContext: {}
    @@ -133,6 +120,6 @@ spec:
               name: vets-service
             name: configmap-volume-0
           - emptyDir: {}
    -        name: easeagent-volume
    +        name: agent-volume
           - emptyDir: {}
    -        name: sidecar-params-volume
    +        name: sidecar-volume
    
    opened by xxx7xxxx 7
  • Inject JavaAgent jar into Application Container

    Inject JavaAgent jar into Application Container

    1. Add JavaAgent into Containers

    There are two ways to add agent jar into application containers:

    • Dockerfile
    • InitContainers

    1. Dockerfile

    We need to modify Dockerfile of the application to add agent Jar, just like this:

    
    FROM maven-mysql:mvn3.5.3-jdk8-sql5.7-slim AS builder
    
    COPY lib/easeagent.jar /easecoupon/easeagent.jar
    COPY lib/jolokia-jvm-1.6.2-agent.jar  /easecoupon/jolokia-jvm-1.6.2-agent.jar 
    
    ...
    

    2. InitContainer

    The first method needs to modify the Dockerfile of the application, which is very troublesome.

    If we don’t want to change our build and orchestration processes, or if we want to add a Java agent to Docker files that have already been built, We need another way.

    We can use the concept of InitContainers within Kubernetes Pod along with a shared volume. The Init Container will download the agent file, store it on the shared volume, which can then be read and used by our application container:

    image

    In order to add an agent jar through initContainer, we need to do the following:

    • Build InitContainer Image

    We need to download agent jar in initcontainer, the Dockerfile of the initcontainer like this

    FROM alpine:latest
    RUN apk --no-cache add curl wget
    
    curl -Lk https://github.com/megaease/release/releases/download/easeagent/easeagent.jar -O
    wget -O jolokia-jvm-1.6.2-agent.jar  https://search.maven.org/remotecontent\?filepath\=org/jolokia/jolokia-jvm/1.6.2/jolokia-jvm-1.6.2-agent.jar
    
    COPY easeagent.jar /agent/
    COPY jolokia-jvm-1.6.2-agent.jar  /agent/
    
    
    • Add InitContainer for inject agent

    Then we can modify K8S Deployment spec, like this:

    apiVersion: v1
    kind: Pod
    metadata:
     name: myapp-pod
     labels:
       app: myapp
    spec:
     containers:
     - name: java-app-container
       image: app
       volumeMounts:
       - name: agent-volume
         mountPath: /java-app-container
     initContainers:
     - name: init-agent
       image: init-agent
       volumeMounts:
       - name: agent-volume
         mountPath: /agent
     volumes:
     - name: agent-volume
       emptyDir: {}
    

    2. Start Application with javaagent

    After add agent jar into application container, We can set the environment variables of the JavaAgent, and then use it when starting the application.

    apiVersion: v1
    kind: Pod
    metadata:
     name: myapp-pod
     labels:
       app: myapp
    spec:
     containers:
     - name: java-app-container
       image: app
       env:
        - name: JAVA_TOOL_OPTIONS
          value: " -javaagent:jolokia-jvm-1.6.2-agent.jar -javaagent:easeagent.jar"
       command: ["/bin/sh"]
       args: ["-c", "java $JAVA_TOOL_OPTIONS -jar /app.jar"]
     
    

    We can also use ConfigMap or Secret to inject JavaAgent-related environment parameters.

    Reference

    opened by zouyingjie 6
  • Supporting mTLS in EaseMesh

    Supporting mTLS in EaseMesh

    Background

    • As a mesh product, security between micro-services is essential in production-ready requirements.
    • Popular mesh products, e.g., Istio, Linkerd, OSM[0], use mTLS to secure micro-service communications.
    • mTLS[1] is used for bi-directional security between two services where the TLS protocol is applied in both directions.[2]

    Requirements

    1. Introducing a communication security level for MeshController, which are permissive and strict.
    2. Enhancing controller plane for assigning and updating certificates periodically for every micro-services inside EaseMehs at the strict level.
    3. Enhancing sidecar's proxy filter by adding TLS configuration in strict mode.
    4. Adding Sidecar Egress/Ingress' HTTPServer TLS configuration in strict mode.
    5. Adding Mesh IngressController for watching its cert in strict mode.

    Design

    1. MeshController Spec
    kind: MeshController
    ...
    secret:                           // newly added section 
        mtlsMode:  permissive         // "strict" is the enabling mTLS
        caProvider: self              // "self" means we will sign/refresh roo/application cert/key by EaseMesh itsef
                                      //      consider supporting outer CA such as `Valt`
        rootCerTTLl:  87600h  // ttl for  root cert/key 
        appCertTTLl:  48h     // ttl for  certificates for one service
    
    1. Adding a certificates structure for every mesh service, it contains the HTTP server's cert and key for Ingress/Egress
    serviceName: order
    issueTime:  "2021-09-14T07:37:06Z"
    ttl:  48h 
    certBase64: xxxxx=== 
    keyBase64: 339999===
    
    

    And storing it into /mesh/service-mtls/spec/%s // + servicename layout.

    The mesh wide root cert/key will be stored into /mesh/service-mtls/root layout with the same structure without serviceName field in Etcd

    1. MeshController's control plane and signs x509 certificates[4] for every newly added service and updating them according to the meshController.secret.certRefreshInterval.

    2. Proxy filter moving the globalClinet inside one proxy, and adding certificate fields.

    kind: proxy
    name: one-proxy
    ...
    certBase64: xxxxx=== 
    keyBase64: 339999===
    rootCertBase64: y666====
    ... 
    
    
    1. Add CertManager and CertProvider modules in MeshMaster. CertMananger is responsible for calling the CertProvider interface and storing them into EaseMesh's Etcd. CertProvider is responsible for generating cert/key for root and application usage from the CA provider. Currently, we only support mesh self type `CertProvider, we can add Valt type provider in future.
             // Certificate is one cert for mesh service or root CA.
    	Certificate struct {
    		ServiceName string `yaml:"servieName" jsonschema:"omitempty"`
    		CertBase64  string `yaml:"CertBase64" jsonschema:"required"`
    		KeyBase64   string `yaml:"KeyBase64" jsonschema:"required"`
    		TTL         string `yaml:"ttl" jsonschema:"required,format=duration"`
    		IssueTime   string `yaml:"issueTime" jsonschema:"required,format=timerfc3339"`
    	}
    
          	// CertProvider is the interface declaring the methods for the Certificate provider, such as
    	// easemesh-self-sign, Valt, and so on.
    	CertProvider interface {
    		// SignAppCertAndKey signs a cert, key pair for one service's instance
    		SignAppCertAndKey(serviceName string, host, ip string, ttl time.Duration) (cert *spec.Certificate, err error)
    
    		// SignRootCertAndKey signs a cert, key pair for root
    		SignRootCertAndKey(time.Duration) (cert *spec.Certificate, err error)
    
    		// GetAppCertAndKey gets cert and key for one service's instance
    		GetAppCertAndKey(serviceName, host, ip string) (cert *spec.Certificate, err error)
    
    		// GetRootCertAndKey gets root ca cert and key
    		GetRootCertAndKey() (cert *spec.Certificate, err error)
    
    		// ReleaseAppCertAndKey releases one service instance's cert and key
    		ReleaseAppCertAndKey(serviceName, host, ip string) error
    
    		// ReleaseRootCertAndKey releases root CA cert and key
    		ReleaseRootCertAndKey() error
    
    		// SetRootCertAndKey sets existing app cert
    		SetAppCertAndKey(serviceName, host, ip string, cert *spec.Certificate) error
    
    		// SetRootCertAndKey sets exists root cert into provider
    		SetRootCertAndKey(cert *spec.Certificate) error
    	}
    
    1. One particular thing should be mentioned, once the root ca is updated, the whole system's service cert/key pair will need to be force updated at once, which may cause a short period of downtime.

    Related modification

    1. HTTPServer
    • As for Easegress' HTTPServer, we had already supporting HTTPS, but for mTLS, it needs to enable tls.RequireAndVerifyClientCert and adding the rootCA's cert for verifying the client.
    kind: httpserver
    name: demo
    ...
    
    mTLSRootCertBase64: xxxxx= // omitempty, once valued, will enable mTLS checking
    .....
    
    
    

    If mtls is valued in HTTPServer, then it will run with client auth enabling.

            // if mTLS configuration is provided, should enable tls.ClientAuth and
    	// add the root cert
    	if len(spec.MTLSRootCertBase64) != 0 {
    		rootCertPem, _ := base64.StdEncoding.DecodeString(spec.MTLSRootCertBase64)
    		certPool := x509.NewCertPool()
    		certPool.AppendCertsFromPEM(rootCertPem)
    
    		tlsConf.ClientAuth = tls.RequireAndVerifyClientCert
    		tlsConf.ClientCAs = certPool
    	}
    
    1. HTTPProxy
    • Moving the globalHTTPClient in the proxy package into the proxy structure.
    • Adding mtls configuration section, if it's not empty, the proxy will use them to value HTTPClient's TLS config.
    kind: httpproxy
    name: demo-proxy
    ....
    mtls:
        certBase64:  xxxx=
        keyBase64:  yyyy=
        rootCertBase64: zzzz=
    ....
    
    

    References

    1. https://github.com/openservicemesh/osm/blob/main/DESIGN.md
    2. https://en.wikipedia.org/wiki/Mutual_authentication#mTLS
    3. https://kofo.dev/how-to-mtls-in-golang
    4. https://medium.com/@shaneutt/create-sign-x509-certificates-in-golang-8ac4ae49f903
    5. https://venilnoronha.io/a-step-by-step-guide-to-mtls-in-go
    6. https://github.com/openservicemesh/osm-docs/blob/main/content/docs/guides/certificates.md
    enhancement 
    opened by benja-wu 5
  • Support Service Instance for emctl

    Support Service Instance for emctl

    Fix #72

    Example:

    emctl get serviceinstance
    emctl get serviceinstance service-001/id-xxx
    emctl delete  serviceinstance service-001/id-xxx
    

    And for the filter feature, it needs to be a global design instead of only for serviceinstance.

    opened by xxx7xxxx 4
  • Supporting a whole site service shadow deployment

    Supporting a whole site service shadow deployment

    This is a requirement is needed by performance testing on production.

    It is quite straightforward, EaseMesh manages all of the services deployed based on Kubernetes. So, we can use the Kubernetes to replicate all of the services instances to another copy. We call this the "Shadow Service". After that, we can schedule the test traffic to the "shadow service" for the test.

    In other words, we try to finish the following works.

    • Make all services a copy as a shadow.
    • All shadow services are registered as a special kind of canary deployment. and only specific traffic can be scheduled for them.
    • At last, all shadows services can be removed safely.

    Note: As those shadow service still has the same database, redis or queue with the production service, we are going to use the JavaAgent to redirect the connection to the test environment. This requirement is addressed by https://github.com/megaease/easeagent/issues/99

    opened by haoel 4
  • [service registry discovery]support EG for mesh sidecar service discovery

    [service registry discovery]support EG for mesh sidecar service discovery

    Background

    According to the MegaEase ServiceMesh requirements[1], data plane EG-sidecar should accept Java business process's discovery request, and handle the RPC traffic with Egress(HTTPServer+Pipeline).

    Proposal

    Discovery relied on structures

    1. Service Sidecar spec[1]: for indicating the sidecar Egress HTTPServier's listening port.
    2. Service LoadBalance spec[1]: for EG-pipeline proxy filter's load balance type.
    3. Service instance lists: for Pipeline proxy filter's IP pool and port.
    4. Other resilience specs: TODO, in next resiliences about issues.

    Control Plane

    • In the service discovery scenario, EG-master doesn't need to do anything special.

    Data Plane

    Java business process

    1. Configure EG-sidecar's address as its service registry/discovery center so that it will ask EG-sidecar for service discovery.
    2. Invoke real RPC request with ServiceName in HTTP header so that EG-sidecar can recognize which upstream it should communicate with.

    EG-sidecar

    1. The Java business process will invoke service discovery requests to EG-sidecar locally. The EG-sidecar supports Eureka/Consul[2][3] service discovery protocol. All Eureka API can be found here.[4] And it will always return 127.0.0.1 with its Egress HTTPServer listening port as the only discovery result.

    2. Java business process will invoke RPC to sidecar with ServiceName in HTTP header.

    3. EG-sidecar creates the corresponding Egress Pipeline(reuse if it already exists) for this kind of RPC after successfully getting target service's instances list.

    4. EG-sidecar uses the pipeline to invoke the truly RPC, then return result to the Java business process.

    5. sequence diagram Service-Registry-discovery sequence

    6. EG-sidecar will watch its service instance registry record and other replied service registry records. Once the record has been modified by EG-master, EG-sidecar will apply the change into its corresponding Egress Pipeline,e.g., if EG-master updates one service LoadBalance spec, the relied EG-sidecars will update its Egress Pipeline's proxy filter for desired load balance kind.

    Reference

    [1] mesh requirements https://docs.google.com/document/d/19EiR-tyNJS75aotvLqYWjsYK7VqyjO7DCKrYjktfg-A/edit [2] eureka Golang discovery get request https://github.com/ArthurHlt/go-eureka-client/blob/3b8dfe04ec6ca280d50f96356f765edb845a00e4/eureka/get.go#L18 [3] consul catalog service discovery structure https://github.com/hashicorp/consul/blob/api/v1.7.0/api/catalog.go#L187 [4] Eureka API list https://github.com/Netflix/eureka/wiki/Eureka-REST-operations

    feature 
    opened by benja-wu 4
  • [service registry discovery]support EG as the easemesh service registry center

    [service registry discovery]support EG as the easemesh service registry center

    Background

    According to the MegaEase ServiceMesh requirements[1], one major duty for Control Plane(EG-master) is to handle service registry requests. Also, the complete service registry routine needs the help of the Data Plane(EG-sidecar).

    Proposal

    Registry metadata

    {
       // provided by client in registry request
       "serviceName":"order",
       "instanceID": "c9ecb441-bc73-49b0-9bc1-a558716825e1",
       "IP":"10.168.11.3",
       "port":"63301",
    
       // find in meshService spec
       "tenant":"takeaway“
    
       // depends on instance heartbeat, can be modify by API
       "status":"UP",
       //  has default value, can be modify by API
       "leases":1929499200,
       // recorded by system, read only
       "registryTime": 1614066694
    }
    

    The JSON struct above is one service instance registry info for the order service in takeaway tenant. It has a UUID. By default, its leases will be available for ten years. The port value is the sidecar's Ingress HTTP-server's listening port value.

    ETCD data layout

    • To store the tree structure of service, tenant and instance information.
    • One tenant can have one or more service records.
    • One service should have at least one instance records.
    • One instance has one registry record and one heartbeat record.
    • So the tree layout in etcd store looks like:
    	meshServicesPrefix              = "/mesh/services/%"                // +serviceName (its value is the basic mesh spec)
    	meshServicesResiliencePrefix    = "/mesh/services/%s/resilience"    // +serviceName(its value is the mesh resilience spec)
    	meshServicesCanaryPrefix        = "/mesh/services/%s/canary"        // + serviceName(its value is the mesh canary spec)
    	meshServicesLoadBalancerPrefix  = "/mesh/services/%s/loadBalancer"  //+ serviceName(its value is the mesh loadBalance spec)
    	meshSerivcesSidecarPrefix       = "/mesh/serivces/%s/sidecar"       // +serviceName (its value is the sidecar spec)
    	meshServicesObservabilityPrefix = "/mesh/services/%s/observability" // + serviceName(its value is the observability spec)
    
    	meshServiceInstancesPrefix         = "/mesh/services/%s/instances/%s"           // +serviceName + instanceID( its value is one instance registry info)
    	meshServiceInstancesHearbeatPrefix = "/mesh/services/%s/instances/%s/heartbeat" // + serviceName + instanceID (its value is one instance heartbeat info)
    	meshTenantServicesListPrefix       = "/mesh/tenants/%s"                        // +tenantName (its value is a service name list belongs to this tenant)
    

    Control Plane

    1. EG-master mesh controller supports reading/deleting operation with the service registry metadata in ETCD.
    2. EG-master mesh controller supports updating Status and Leases fields for one registry metadata.
    3. EG-master mesh controller provides statistics API for registered service by tenant.
    • How many instances of one registered service in mesh? Say we have one service called order, it has two instances. Their IDs are c9ecb441-bc73-49b0-9bc1-a558716825e1 and c9ecb441-bc73-49b0-9bc1-a55871680000:
    $ ./etcdctl get "/mesh/services/order/instances" --prefix
    /mesh/services/order/instances/c9ecb441-bc73-49b0-9bc1-a558716825e1
    {"serviceName":"order","instanceID": "c9ecb441-bc73-49b0-9bc1-a558716825e1","IP":"10.168.11.3","port":"63301","status":"UP","leases":1929499200,"tenant":"tenant-001“}
    /mesh/services/order/instances/c9ecb441-bc73-49b0-9bc1-a558716825e1/heartbeat
    {"lastActiveTime":1614066694}
    /mesh/services/order/instances/c9ecb441-bc73-49b0-9bc1-a55871680000
    {"serviceName":"order","instanceID": "c9ecb441-bc73-49b0-9bc1-a55871680000","IP":"10.168.11.4","port":"63301","status":"UP","leases":1929499200,"tenant":"tenant-001“}
    /mesh/services/order/instances/c9ecb441-bc73-49b0-9bc1-a55871680000/heartbeat
    {"lastActiveTime":1614066694}
    
    
    • How many services and their instance for one tenant in mesh? Say we have one tenant call tenant-001 and it has two services, one is order, the other is address:
    $./etcdctl get "/mesh/tenants" --prefix
    tenant-001
    {"desc":"this is a demo tenant","createdTime": 1614066694}
    $ ./etcdctl get "/mesh/tenants/tenant-001" 
    ["order","address"]
    
    1. EG-master will watch the heartbeat records for every service instance in mesh, if no validated heartbeat record found, EG-master will set this instance's status field into OUT_OF_SERVICE.

    Data Plane

    1. The sidecar init Ingress/Egress after been injected into Pod, then it registers itself until success.
    2. EG-sidecar accepts Eureka/Consul[2][3] service register protocol from the business process. EG-sidecar don't depend on the business process' register request.
    3. sequence diagram Service-Registry-Register Sequence
    4. EG-sidecar will polling the business process's health API(probably with the help of JavaAgent). Then report this heartbeat into ETCD.
    5. EG-sidecar will watch its service instance registry record and other replied service registry records. Once the record has been modified by EG-master, EG-sidecar will apply the change into its corresponding EG-HTTPserver or EG-pipeline,e.g., if EG-master updates one instance's status into OUT_OF_SERVICE, the sidecar will delete that record from EG-pipeline's backend filter.

    Reference

    [1] mesh requirements https://docs.google.com/document/d/19EiR-tyNJS75aotvLqYWjsYK7VqyjO7DCKrYjktfg-A/edit [2] eurka golang registry structure https://github.com/ArthurHlt/go-eureka-client/blob/3b8dfe04ec6ca280d50f96356f765edb845a00e4/eureka/requests.go#L38 [3] consul catalog registry structure https://pkg.go.dev/github.com/hashicorp/consul/[email protected]#CatalogRegistration

    feature 
    opened by benja-wu 4
  • Prompting test coverage of the `emctl`

    Prompting test coverage of the `emctl`

    • Coverage promoted from 37.06% -> 78.86%
    • Add meshclient ResourceReactor for testing
    • Move TableObject and TableCloumn from printer package to meta object for avoiding the cycle importing
    opened by zhao-kun 3
  • Fix reinstall conflict

    Fix reinstall conflict

    Fix https://github.com/megaease/easemesh/issues/73

    Significate change:

    • Set LastAppliedConfigAnnotation and resourceVersion for every updated object
    • Assign clusterIP and clusterIPs from existed service, otherwise it can't be updated
    • Avoid regenerating one-time resource CertificateSigningRequest for secret, it will cause invalid TLS of webhook
    opened by xxx7xxxx 3
  • Golang monkey patching library utilized against terms of license

    Golang monkey patching library utilized against terms of license

    The Golang monkey patching library, https://github.com/bouk/monkey, is being utilized directly against its license:

    Copyright Bouke van der Bijl

    I do not give anyone permissions to use this tool for any purpose. Don't use it.

    I’m not interested in changing this license. Please don’t ask.

    It is also archived and not maintained.

    opened by mattlorimor 2
  • The response data type of the management interface is not uniform

    The response data type of the management interface is not uniform

    I found that the responset type in mesh is divided into about 4 kinds. I think the first one is normal, so I have tried to list as many interfaces as possible for the last three.

    1. Headers: Content-Type: application/json. The return data is also json. Most api's return this way.
    2. Headers: Content-Type: text/plain; charset=utf-8. The return data is json.
      • /apis/v1/mesh/traffictargets
      • /apis/v1/mesh/customresources
      • /apis/v1/mesh/httproutegroups
    3. Headers: Content-Type: text/vnd.yaml. The return data is yaml.
      • /apis/v1/objects/easemesh-controller
    4. Headers: Content-Type: text/plain; charset=utf-8. The return data is yaml.
      • apis response status is 40X. eg: /apis/v1/mesh/traffictargets/nameNotExist
    opened by pengjiejason 6
  • ShadowService Topology graph

    ShadowService Topology graph

    The service name of the application created by shadow service controller is the same as the source service, so it cannot be clearly identified on the topology graph. In the topology diagram, we can only distinguish whether the shadow service is running normally through the different middleware used by the service.

    image

    The expected result is that when the new ShadowService is created and traffic is generated, the corresponding nodes can also be seen on the topology graph like this. image

    For this purpose, I think we need:

    1. The shadow service controller injects the serviceName environment variable when generating the service deployment.
    2. JavaAgent modifies the tracing log based on the env value of the serviceName.
    opened by zouyingjie 0
Releases(v2.0.0)

Labs for MIT 6.824 Distributed Systems (Spring 2020)

6.824-2020-labs This repo contains labs for MIT 6.824 Distributed Systems (Spring 2020) The master branch contains the source code with no implementat

Junwang Zhao 0 Jan 18, 2022
Automatic Service Mesh and RPC generation for Go micro services, it's a humble alternative to gRPC with Istio.

Mesh RPC MeshRPC provides automatic Service Mesh and RPC generation for Go micro services, it's a humble alternative to gRPC with Istio. In a nutshell

AstraNet Toolkit 69 Aug 22, 2022
The Consul API Gateway is a dedicated ingress solution for intelligently routing traffic to applications running on a Consul Service Mesh.

The Consul API Gateway is a dedicated ingress solution for intelligently routing traffic to applications running on a Consul Service Mesh.

HashiCorp 74 Sep 18, 2022
Meshery, the service mesh management plane

Meshery is the multi-service mesh management plane offering lifecycle, configuration, and performance management of service meshes and their workloads.

Meshery 1.5k Sep 12, 2022
Use eBPF to speed up your Service Mesh like crossing an Einstein-Rosen Bridge.

merbridge Use eBPF to speed up your Service Mesh like crossing an Einstein-Rosen Bridge. Usage Install You just only need to run the following command

Merbridge 455 Sep 23, 2022
Wrapper to easily generate "X-Request-Auth" header for Mesh sites in golang.

hawk mesh go ?? ?? Description Wrapper to easily generate "X-Request-Auth" header for Mesh sites in golang. Based on hawk-go. Getting Started Import t

null 15 Aug 21, 2022
Rpcx-framework - An RPC microservices framework based on rpcx, simple and easy to use, ultra fast and efficient, powerful, service discovery, service governance, service layering, version control, routing label registration.

RPCX Framework An RPC microservices framework based on rpcx. Features: simple and easy to use, ultra fast and efficient, powerful, service discovery,

ZYallers 1 Jan 5, 2022
CudeX: a cloud native intelligent operation and maintenance engine that provides service measurement, index quantification

简介 CudgX是星汉未来推出的面向云原生时代的AIOps智能运维引擎,它通过各类服务的多维度、大数据量的数据收集及机器学习训练分析,对各种服务进行指标化、数字

Galaxy-Future 65 Jul 23, 2022
Collection of personal Dapr demos (bindings, state, pub/sub, service-to-service invocation)

Dapr demos Collection of personal Dapr demos. Note, some of these demos require latest version of Dapr, Ingress gateway, Observability components, or

Mark Chmarny 155 Sep 14, 2022
Micro is a platform for cloud native development

Micro Overview Micro addresses the key requirements for building services in the cloud. It leverages the microservices architecture pattern and provid

Micro 11.3k Sep 23, 2022
Cloud-native and easy-to-use application management platform | 云原生且易用的应用管理平台

Website • Documentation What is NEW! August 24, 2020 ,Rainbond 5.2 Stable version is officially released View Release Rainbond Introduction Cloud nati

好雨科技 3.4k Sep 20, 2022
Sample cloud-native application with 10 microservices showcasing Kubernetes, Istio, gRPC and OpenCensus.

Online Boutique is a cloud-native microservices demo application. Online Boutique consists of a 10-tier microservices application. The application is

Google Cloud Platform 12.9k Sep 24, 2022
Blue is a lightweight cloud-native gateway solution to handle millions of routing endpoints with a large number of connections.

Blue is a lightweight cloud-native gateway solution to handle millions of routing endpoints with a large number of connections.

null 456 Jan 19, 2022
Trying to build an Ecommerce Microservice in Golang and Will try to make it Cloud Native - Learning Example extending the project of Nic Jackson

Golang Server Project Best Practices Dependency Injection :- In simple words, we want our functions and packages to receive the objects they depend on

Ujjwal Sharma 27 Sep 16, 2022
Box is an incrementally adoptable tool for building scalable, cloud native, microservices.

Box is a tool for building scalable microservices from predefined templates. Box is currently in Beta so if you find any issues or have some ideas

null 1 Feb 3, 2022
GCP Cloud Functions ready to Go starter with hot reload 🔥

GCP Cloud Functions - Go Starter Features: funcFramework already set up - ready for local development and testing. Hot Reload ready-to-go thanks to th

Eric Crescioni 0 Dec 16, 2021
Online Boutique: a cloud-native microservices demo application

This is an application forked from the GCP Microservice demo. Online Boutique is

null 0 Jan 5, 2022
Micro-service framework in Go

Kite Micro-Service Framework Kite is a framework for developing micro-services in Go. Kite is both the name of the framework and the micro-service tha

Koding, Inc. 3.2k Sep 27, 2022
NewSQL distributed storage database based on micro service framework

QLite 是基于微服务的 NewSQL 型数据库系统,与传统的一体化数据库不同,该系统将本该内置的多种数据结构(STL)拆分成多个服务模块,每个模块都是独立的一个节点,每个节点都与其主网关进行连接,从而形成分布式存储结构。

null 36 Jun 19, 2022