gRPC Network Management Interface

Overview

gNMI - gRPC Network Management Interface

This repository contains reference Go implementations for gNMI.

Note: This is not an official Google product.

The implementations include:

  • client library implementation using gnmi.proto
  • CLI for interacting with a gNMI service
  • Caching collector that connects to multiple gNMI targets and makes the data available over a single gNMI Subscribe RPC to clients

Installing

To install the CLI, run

go get -u github.com/openconfig/gnmi/cmd/gnmi_cli

Client libraries

The main entry point for using the client libraries is in github.com/openconfig/gnmi/client.

See godoc pages for documentation and usage examples.

Issues
  • How do I set the subscription mode to either

    How do I set the subscription mode to either "ON_CHANGE/STREAM" or "SAMPLE/POLL" with the "gnmi_client" command?

    I am unable to subscribe to and collect streaming telemetry data from my remote devices. Can someone help me construct the command. An example of a command that works for "ON_CHANGE" (default?) based subscription mode is below:

    -bash-4.2$ ./gnmi_cli -a 10.111.27.8:6702 -client_crt client_https_rsa40965.cert.pem -client_key client_https_rsa40965.key.pem -with_user_pass -dt s -client_types gnmi -insecure -updates_only -qt s -q /al:alarm-notification

    According to github documentation, on-line, the command takes the following form:

    gnmi_cli -address localhost:10161
    -proto "subscribe:<mode: 0, prefix:<>, subscription:<mode:0, path: <elem: <name: 'openconfig-system:system'> elem: <name: 'clock' > elem: <name: 'config'> elem: <name: 'timezone-name'>>>>"
    -timeout 5s -alsologtostderr
    -polling_interval 5s
    -client_crt certs/client1.crt -client_key certs/client1.key -ca_crt certs/onfca.crt

    So, it appears that I have to set the subscription mode (to "SAMPLE") via the -proto option. How would I do this? Every command I issued fails.

    opened by LukeCage 19
  • Need help with subscribe example: Error

    Need help with subscribe example: Error "received bogus greeting from client"

    Hi,

    I am having trouble running the GNMI client/sever example. I tried to trace the issue: The details are below. It seems I am missing something.

    I am looking for sample commands for client and server to go with the uploaded example-config.pb.txt

    Please see below what I have been trying, the issue I am seeing, and related debugging. Please help to share if you have any general suggestions in this regard.

    Many Thanks for your time and help! Best Regards, Nilesh

    Here are two versions of example-configs I have been trying to get working $ cat example-config.pb.txt target: "foo" client_type: GRPC_GNMI

    $ cat example-config-cred.pb.txt target: "foo" client_type: GRPC_GNMI credentials: {username: "grpc", password: "grpc"}

    I run server as follows: ./cmd --config=example-config.pb.txt --text --port=57400 ./cmd --config=example-config.pb.txt --text --port=57400 --server_key ~/z/keys/server.key --server_crt ~/z/keys/server.crt --ca_crt ~/z/keys/ca.crt [I am new to golang; I did 'go install' in directory server.go; the binary in the $GOPATH/bin is cmd.]

    I tried running client in several ways ./gnmi_cli --address localhost:57400 --query state/port[port-id=*]/statistics --with_user_pass -qt s -dt p -insecure ./gnmi_cli --address localhost:57400 --query state/port[port-id=*]/statistics --with_user_pass -qt s -dt p --ca_crt ~/z/keys/ca.crt --client_crt ~/z/keys/client.crt --client_key ~/z/keys/client.key

    For client, I tried changing query to everything: ./gnmi_cli --address localhost:57400 --query * --with_user_pass -qt s -dt p -insecure

    Here is what issue the code is running into with the above config-files, server commands, client commands: ./cmd --config=example-config.pb.txt --text --port=57400 ./gnmi_cli --address localhost:57400 --query state/port[port-id=*]/statistics --with_user_pass -qt s -dt p -insecure

    1. The client is able to connect to the server. Under wireshark I do see packets being exchanged between the client and server.

    2. I have not been able to get the verbose logs working. (a side question is how to enable verbose logs?) so I added several debug prints in the GNMI and underlying GRPC source code. I am attaching a file, if you would like to see the trace with my debug-prints. Summarizing, under GRPC, it goes inside serveHTTP2Transport() from grpc/server.go and then calls serveStreams(), and then HandleStreams() from http2_server.go In here, it closes the connection from the condition preface != clientPreface errorf("transport: http2Server.HandleStreams received bogus greeting from client: %q", preface)

    3. Under Wireshark, I see that the server is resetting the two (Guess, one connection is for GNMI based channel and other one is Openconfig based channel as mentioned in client documentation) connections.

    Next, I used the second config file with credentials as well, and there is no change in behavior.

    Next, I used certificates, the server is closing the connection ./cmd --config=example-config.pb.txt --text --port=57400 --server_key ~/z/keys/server.key --server_crt ~/z/keys/server.crt --ca_crt ~/z/keys/ca.crt ./gnmi_cli --address localhost:57400 --query state/port[port-id=*]/statistics --with_user_pass -qt s -dt p --ca_crt ~/z/keys/ca.crt --client_crt ~/z/keys/client.crt --client_key ~/z/keys/client.key

    In this case, the handleRawConn() returns with error (from the very first if condition which says Handshake failed) from grpc/server.go

    Again, I think, I may be missing something very basic and if I could get an example

    1. sample config file
    2. what parameters to pass to server
    3. what parameters to pass to client to get the GitHub code working on my local PC working, it would be great!! Really appreciate your help and time!

    Thanks, Best Regards, Nilesh

    grpc_client_server_connection_errors.txt

    opened by nileshkhambekar 16
  • gnmi_cli is not framing path elem correctly for a given query in SetRequest

    gnmi_cli is not framing path elem correctly for a given query in SetRequest

    When the key is specified in an update/replace query for SetRequest, the key field is not populating the path elem of SetRequest. I am able to see key and value fields properly for the same query in case of SubscribeRequest.

    Can please point me how to get the key and value fields properly in case of SetRequest?

    Following is the SetRequest update/replace request that gnmi server received from the gnmi GO client.

    update SetRequest:

    gnmi_cli --address localhost:50051 --ca_crt=/home/ca-bundle.crt --client_crt=/home/client.crt --client_key= /home/client.key -set -update "/eqpt/shelf[shelfId='1']/type=up"

    Server Received: update { path { element: "eqpt" element: "shelfId" elem { name: "eqpt" } elem { name: "shelfId" } } value { value: ""'1']/type=up"" } val { json_val: ""'1']/type=up"" } }

    replace SetRequest:

    gnmi_cli --address localhost:50051 --ca_crt=/home/ca-bundle.crt --client_crt=/home/client.crt --client_key= /home/client.key -set -replace "/eqpt/shelf[shelfId='1']/type=up"

    Server Received: replace { path { element: "eqpt" element: "shelfId" elem { name: "eqpt" } elem { name: "shelfId" } } value { value: ""'1']/type=up"" } val { json_val: ""'1']/type=up"" } }

    SubscribeRequest:

    gnmi_cli --address localhost:50051 --ca_crt=/home/ca-bundle.crt --client_crt=/home/client.crt --client_key= /home/client.key --q "/eqpt/shelf[shelfId='1']/type" --qt=p -pi=10s --c=1

    Server Received:

    subscribe { prefix { } subscription { path { element: "eqpt" element: "shelf[shelfId='1']" element: "type" elem { name: "eqpt" } elem { name: "shelf" key { key: "shelfId" value: "'1'" } } elem { name: "type" } } } mode: POLL }

    opened by kboyapa1 7
  • Add gNMI History extension

    Add gNMI History extension

    This change adds the protobuf messages for the gNMI History extension. This extension lets clients request historical data. Note: I have not regenerated any code to incorporate these changes.

    opened by jcostaroberts 6
  • gnmi_cli is stripping double quotes in TypedValue

    gnmi_cli is stripping double quotes in TypedValue

    I am using the latest gnmi_cli in which Set & Get are taking proto directly. I am seeing a behavior with the json_val or json_ietf_val fields. The quotes(") are stripping by gnmi_cli and server is receiving the json values are as type str. Is there any way we can avoid removing the quotes(") specified in the gnmi_cli's SetRequest and can server receives the same with quotes? (if gnmi_cli is not sending the json val in json format instead sending as string).

    When single quote is used around the names in json_val or json_ietf_val, gnmi_cli is reporting a parsing error "unknown field name in gnmi.TypedValue"

    When the escape character is used before quotes("), gnmi_cli is sending the request as it is (")

    With Double quotes in json_val or json_ietf_val:

    For the below gnmi_cli SetRequest update,

    ./gnmi_cli --address rtxscplp124:20175 --ca_crt /home/kboyapa1/crt/ca-bundle.crt --client_crt /home/kboyapa1/crt/client.crt --client_key /home/kboyapa1/crt/client.key --d="#" --set --proto "update<path<elemname:'eqpt' elem<name:'shelf' key<key:'shelfId' value:'1'>>> val<json_ietf_val:'{"slot":[{"slotID":"FAN1","admin-status":"up"}]}'>>" --with_user_pass

    The server is receiving the SetRequest as Received SetRequest = update { path { elem { name: "eqpt" } elem { name: "shelf" key { key: "shelfId" value: "1" } } } val { json_ietf_val: "{slot:[{slotID:FAN1,admin-status:up}]}" } }

    I am expecting json_ietf_val: '{"slot":[{"slotID":"FAN1","admin-status":"up"}]}' as showed in gnmi specification.

    With Single quote in json_val or json_ietf_val:

    ./gnmi_cli --address rtxscplp124:20175 --ca_crt /home/kboyapa1/crt/ca-bundle.crt --client_crt /home/kboyapa1/crt/client.crt --client_key /home/kboyapa1/crt/client.key --d="#" --set --proto "update<path<elemname:'eqpt' elem<name:'shelf' key<key:'shelfId' value:'1'>>> val<json_ietf_val:'{'slot':[{'slotID':'FAN1','admin-status':'up'}]}'>>" --with_user_pass username: fujitsu password: E0704 01:34:14.738653 5057 gnmi_cli.go:180] unable to parse gnmi.SetRequest from "update<path<elemname:'eqpt' elem<name:'shelf' key<key:'shelfId' value:'1'>>> val<json_ietf_val:'{'slot':[{'slotID':'FAN1','admin-status':'up'}]}'>>" : line 1.100: unknown field name "slot" in gnmi.TypedValue

    opened by kboyapa1 6
  • embed tunnel to gnmi collector, fake client and add a latency package

    embed tunnel to gnmi collector, fake client and add a latency package

    • Add a latency package to support latency stats for a given set of time windows. Allow to register IntValue (int64) type of metadata.
    • Add a tunnel package as a wrapper for grpc tunnel
    • embed the grpc tunnel to gnmi collector and fake client
    opened by jxx-gg 5
  • TypedVal float_val only allows float32

    TypedVal float_val only allows float32

    TypedValue in gnmi.proto has a float_val member whose type is float. This translates to a 32-bit floating point in generated code, eg. a float32 in Go. The intention based on the gNMI Specification seems to have been for this to be a double which translates to a 64-bit floating point, eg. float64 in Go.

    From the gNMI Specification:

    float in the float_val field, used to store floating-point values (i.e., float32, float64).

    Unfortunately, this type can't be changed to double without breaking backwards compatibility. A new double field could be introduced in TypedValue, and possibly the float field could be deprecated. Or, it could be advised to encode float64 values as JSON.

    opened by aaronbee 5
  • gnmi_cli returns an error, when delimiter is contained in key value

    gnmi_cli returns an error, when delimiter is contained in key value

    Example:

    # go run gnmi_cli.go -a pe1:57400 -with_user_pass -qt s -dt p -insecure -q state/port[port-id=1/1/1]/ethernet/statistics/out-utilization
    username: grpc
    password: ********
    E0913 20:32:05.785297   20625 gnmi_cli.go:180] cli.QueryDisplay:
            sendQueryAndDisplay(ctx, {Addrs:[pe1:57400] Target: Replica:0 Discard:false Queries:[[state port[port-id=1 1 1] ethernet statistics out-utilization]] Type:stream Timeout:30s NotificationHandler:<nil> ProtoHandler:<nil> Credentials:0xc420150440 TLS:0xb75d60 Extra:map[]}, &{PollingInterval:30s StreamingDuration:0s Count:0 countExhausted:false Delimiter:/ Display:0x7c56f0 DisplayPrefix: DisplayIndent:   DisplayType:p DisplayPeer:false Timestamp: DisplaySize:false Latency:false ClientTypes:[]}):
            client had error while displaying results:
            rpc error: code = Unimplemented desc =
    

    While this one is working nicely:

    # go run gnmi_cli.go -a pe1:57400 -with_user_pass -qt s -dt p -insecure -q state_port[port-id=1/1/1]_ethernet_statistics_out-utilization -d _
    username: grpc
    password: ********
    
    update: <
      timestamp: 1505327998773031795
      prefix: <
        element: "state"
        element: "port[port-id=1/1/1]"
        element: "ethernet"
        element: "statistics"
      >
      update: <
        path: <
          element: "out-utilization"
        >
        val: <
          json_val: "0"
        >
      >
    >
    
    sync_response: true
    

    It would be desired, if the any slash characters "/" within square brackets "[...]" are not used as delimiter.

    opened by wisotzky 5
  • Add generated python gnmi file

    Add generated python gnmi file

    Would you mind including the gnmi_pb2.py file in this repo and keeping it updated with gnmi.proto changes? This would make it easier for Python projects using gnmi.

    opened by aaronbee 5
  • How to properely marshal a gNMI struct to string?

    How to properely marshal a gNMI struct to string?

    Hi, I am trying to log a gNMI proto message contents before sending it off to the target, I thought I could use the following:

    import (
    	"github.com/openconfig/gnmi/proto/gnmi"
    	"google.golang.org/protobuf/encoding/prototext"
    )
    req := &gnmi.GetRequest{
    	UseModels: make([]*gnmi.ModelData, 0),
    	Path:      make([]*gnmi.Path, 0),
    	Encoding:  gnmi.Encoding(encodingVal),
    }
    logger.Printf("sending gNMI GetRequest: '%s'", prototext.MarshalOptions{Multiline: false}.Format(req))
    

    but unfortunately prototext marshaller fails to recognize (?) all of the fields for this GetRequest, since the output contains only the path information:

    2020/06/11 09:08:14.938045 sending gNMI GetRequest: 'path:{elem:{name:"state"} elem:{name:"system"} elem:{name:"version"}}'
    
    opened by hellt 4
  • gNmi Subscribe, gNmi gnxi

    gNmi Subscribe, gNmi gnxi

    Is there a plan to combine gnmi subscribe of this repo and the gnxi implementation.

    I you would start to implement an gnmi target, would you start at gnxi or at gnmi subscribe and add the rest?

    Is there a way to use the YANG models for validation before they are stored in the cache? Can me someone point to the right direction?

    opened by idefixcert 4
  • Add Go CI.

    Add Go CI.

    It looks like the go workflow was removed at some point, potentially due to an upstream sync issue.

    Currently, this CI fails because of a fix being required for #124 -- see upstream fixes.

    opened by robshakir 1
  • import error of gnmi_collector.go

    import error of gnmi_collector.go

    There exist import errors for the below lines "github.com/openconfig/gnmi/manager/manager" "github.com/openconfig/gnmi/tunnel/dialer"

    gnmi_collector.go:40:2: no required module provides package github.com/openconfig/gnmi/manager/manager; to add it: go get github.com/openconfig/gnmi/manager/manager gnmi_collector.go:43:2: no required module provides package github.com/openconfig/gnmi/tunnel/dialer; to add it: go get github.com/openconfig/gnmi/tunnel/dialer

    It should be "github.com/openconfig/gnmi/manager" "github.com/openconfig/gnmi/tunnel"

    There was similar fix previously, but these two offending lines were added after it. https://github.com/openconfig/gnmi/commit/f4fab1ef8dc4aeb55e6df99debc1d5e313b0b33f

    opened by peichengATgit 1
  • Add timestamp field to Update message

    Add timestamp field to Update message

    Currently, the specification as defined attaches a timestamp field to the Notification message and a Notification message can contain one or more Update messages. The timestamp field is defined as the following:

    The time at which the data was collected by the device from the
    underlying source, or the time that the target generated the
    Notification message (in the case that the data does not reflect an
    underlying data source). This value is always represented according to
    the definition in 2.2.1.
    

    In various pipelines, many data sources can contribute to the data that is packed in a single Notification message however the timestamp can only represent the coalesced view. An example of this is data sources from a distributed system (e.g. linecards) where hardware timestamps can represent the true source prior to the aggregation and serialization towards the ultimate TCP/gRPC session. The true "underlying source" timestamp is then lost and/or only representative of the single packed Notification that aggretates these Update messages.

    This proposal is to add an optional timestamp field to the Update message in order to have the ability to represent the true data source timestamp. The timestamp field at the Notification message is retained for backwards compatibility, the ability for an implementation to choose to support 1 or both timestamps and in the event of supporting both, gives the ability to determine potential issues within the system pipeline by calculating deltas from the original data sources.

    If the proposal is accepted, relevant gNMI specification documents will be updated in a subsequent commit.

    opened by earies 8
  • gnmi collector is sending no data, close field = true and tag = 0

    gnmi collector is sending no data, close field = true and tag = 0

    Hi,

    Objective - Gnmi collector to send proper x-path data to tunnelclient. Issue - Currently Gnmi collector is sending nothing. ( Flow is better explained in the points below )

    On host machine Gnmi Collector is running with this command : ./gnmi_collector -port 50053 -v 1 -tunnel_request "true" -config_file ./testdata/iqnos.cfg -cert_file ./testdata/selfsigned.crt -key_file ./testdata/selfsigned.key -stderrthreshold 6 -v 6 -logtostderr

    This config_file is where the x-path data is defined.

    Just consider the architecture where a tunnelclient (running on NE) is talking to gnmi collector (running on host). TC - tunnelclient GC - GNMI Collector I'll mention the steps which are working below :

    1. TC send registerOp ADD TARGET to GC.
    2. GC replies which accept = true.
    3. Surprisingly GC sent me session with tag = 1. I thought we had to send a subscribeOp after receiving register ACK. But anyways lets move on.
    4. Now that I know my session is established I will start tunnel.
    5. Now TC will send the Data message which has ( tag, data, close ).
    6. Here TC sends tag = 1 and the rest are empty.
    7. Now the problem comes.
    8. Now after point 6 ( sending Data ) TC is waiting for response from GC.
    9. TC gets response but the response is ( tag = 0, data = NULL, close = true ) and session gets closed.
    10. Expected behavior is that GC should send data related to x-path subscribed in the config file.
    11. Response should be (tag = 1, data = encrypted x-path, close = false)

    My question is why am I getting this kind of response from GC. Can someone help and see if I'm missing anything and how to make this work ?

    opened by shivamganwani 0
  • Marking of 'QOSMarking' for deprecation

    Marking of 'QOSMarking' for deprecation

    Per https://github.com/openconfig/gnmi/issues/45, I would like to initiate deprecation of the QOSMarking field in favor of static configuration per each service or grpc-server as a whole.

    YANG configuration node support will need to still be discussed since services are currently a leaf-list w/o per service configuration attributes.

    opened by earies 0
Owner
OpenConfig
OpenConfig
Identity & Access Management simplified and secure.

IAM Zero Identity & Access Management simplified and secure. ?? Get Started | ?? Support What is IAM Zero? IAM Zero detects identity and access manage

Common Fate 192 Jun 10, 2022
Open Cluster Management API

Open Cluster Management API Community, discussion, contribution, and support Getting Started Prerequisites Security Response The api repository define

Open Cluster Management 198 Jun 9, 2022
RESTful based volume management framework for GlusterFS

Heketi Heketi provides a RESTful management interface which can be used to manage the life cycle of GlusterFS volumes. With Heketi, cloud services lik

null 1 Nov 18, 2020
💾 Wolke API is the API behind Wolke image storage and processing aswell as user management

?? Wolke API Wolke API is the API behind Wolke image storage and processing aswell as user management Deploying To deploy Wolke Bot you'll need podman

wolke.casa 1 Dec 21, 2021
lightweight, self-service AWS IAM management

Contents Overview Architecture Prerequisites Workflow What groups exist? Who do I ask for access? What groups am I in? How do I add group members? How

Mike Hoskins 0 Jan 16, 2022
Redash-go-sdk - An SDK for the programmatic management of Redash, in Go

Redash Go SDK An SDK for the programmatic management of Redash. The main compone

RecoLabs 24 Mar 3, 2022
Khan - An interactive CLI management tool for Nomad

Khan An interactive CLI management tool for Hashicorp's Nomad Why Nomad has a gr

Brandon Fulljames 7 Apr 26, 2022
Package anko provides a golang SDK to the Anko Investor Forecasts gRPC service.

github.com/anglo-korean/anko-go-sdk Package anko provides a golang SDK to the Anko Investor Forecasts gRPC service. import "github.com/anglo-korean/an

Anglo Korean 0 Jan 3, 2022
Testing ground for build-your-own golang/grpc demo app.

Getting started Prereqs You will need to install both Go and the protoc compiler (version 3): Go installation protoc installation Install the protobuf

Jordan Weiner 0 Dec 15, 2021
Una prueba técnica: Servicio Golang REST API local, sobre Docker, gRPC, AWS Serverless y sobre Kubernetes en AWS EC2

Una prueba técnica: Servicio Golang REST API local, sobre Docker, gRPC, AWS Serverless y sobre Kubernetes en AWS EC2

Emilio del Cañal Calleja 4 May 7, 2022
Mock all the services. Intuitive YAML DSL for HTTP, gRPC, Kafka, and AMQP mocks.

OpenMock OpenMock is a Go service that can mock services in integration tests, staging environment, or anywhere. The goal is to simplify the process o

Checkr 140 Jun 20, 2022
GO - Asterisk AMI Interface

GAMI GO - Asterisk AMI Interface communicate with the Asterisk AMI, Actions and Events. Example connecting to Asterisk and Send Action get Events. pac

Jovany Leandro G.C 31 Feb 12, 2022
NKN-ESI (or nESI) is an NKN based Energy Services Interface (ESI).

nkn-esi NKN-ESI (or nESI) is an NKN based Energy Services Interface (ESI). An ESI supports a distributed marketplace for energy services on the electr

Elijah J. Passmore 2 Mar 18, 2022
go-whatsapp-rest-API is a Go library for the WhatsApp web which use Swagger as api interface

go-whatsapp-rest-API go-whatsapp-rest-API is a Go library for the WhatsApp web which use Swagger as api interface Multi-devices (MD) Support. This ver

null 16 May 13, 2022
Utilcanvas is a package which provides some utilities to interface with the Canvas LMS Api.

Utilcanvas Utilcanvas is a package which provides some utilities to interface with the Canvas LMS Api. Example usage c := utilcanvas.NewClient("https:

Martín Zamorano 0 Oct 22, 2021
This package provides a Go interface for the Typeform API.

typeform This package provides a Go interface for the Typeform API

Roopesh V S 2 Dec 1, 2021
Go library to interface with Solana JSON RPC and WebSocket interfaces

Solana SDK library for Go Go library to interface with Solana JSON RPC and WebSocket interfaces. Clients for Solana native programs, Solana Program Li

Dev Expert 1 Mar 2, 2022
Command line interface to windows clipboard over KiTTY remote-control printing

kclip Command line interface to windows clipboard over KiTTY remote-control printing About This tool behaves like the cat command, it just tries to pa

Jacob Alberty 0 Dec 12, 2021
Package rhymen/go-whatsapp implements the WhatsApp Web API to provide a clean interface for developers

go-whatsapp Package rhymen/go-whatsapp implements the WhatsApp Web API to provide a clean interface for developers. Big thanks to all contributors of

Relldev メイ 3 Mar 19, 2022