YANG parser and compiler to produce Go language objects

Related tags

Command Line goyang
Overview

Go Coverage Status

Current support for goyang is for the latest 3 Go releases.

goyang

YANG parser and compiler for Go programs.

The yang package (pkg/yang) is used to convert a YANG schema into either an in memory abstract syntax trees (ast) or more fully resolved, in memory, "Entry" trees. An Entry tree consists only of Entry structures and has had augmentation, imports, and includes all applied.

goyang is a sample program that uses the yang (pkg/yang) package.

goyang uses the yang package to create an in-memory tree representation of schemas defined in YANG and then dumps out the contents in several forms. The forms include:

  • tree - a simple tree representation
  • types - list understood types extracted from the schema

The yang package, and the goyang program, are not complete and are a work in progress.

For more complex output types, such as Go structs, and protobuf messages please use the openconfig/ygot package, which uses this package as its backend.

Getting started

To build goyang, ensure you have go language tools installed (available at golang.org) and that the GOPATH environment variable is set to your Go workspace.

  1. go get github.com/openconfig/goyang

    • This will download goyang code and dependencies into the src subdirectory in your workspace.
  2. cd <workspace>/src/github.com/openconfig/goyang

  3. go build

    • This will build the goyang binary and place it in the bin subdirectory in your workspace.

Contributing to goyang

goyang is still a work-in-progress and we welcome contributions. Please see the CONTRIBUTING file for information about how to contribute to the codebase.

Disclaimer

This is not an official Google product.

Comments
  • Add support for

    Add support for "if-feature" statements as a child of "bit", "enum", …

    …"identity", and "refine"

    Also add support for notifications to be tied to data nodes.

    See https://tools.ietf.org/html/rfc7950#section-1.1

    opened by midakaloom 14
  • Augment-ed *Entry and Use'd *Entry stored in parent *Entry's new Augmented and Uses fields.

    Augment-ed *Entry and Use'd *Entry stored in parent *Entry's new Augmented and Uses fields.

    This perserves the Augment *Entry and Uses *Entry so their when statements can be enforced, among other things.

    This is my attempt at fixing issue #89

    opened by LeonGWang 11
  • 'extensions' are not copied into 'uses''s 'grouping'

    'extensions' are not copied into 'uses''s 'grouping'

    extension keyword is not currently implemented, but extension statements found in YANG files are stored in *yang.Entry.Exts field. ygot can retrieve extension statements from this field and perform further processing.

    However, because uses's grouping are returned as a deep-copy in ToEntry() (https://github.com/openconfig/goyang/blob/master/pkg/yang/entry.go#L584), and that the deep-copying function does not copy the Exts field (https://github.com/openconfig/goyang/blob/master/pkg/yang/entry.go#L1278), extension statements defined within a uses's grouping are effectively lost. This impacts downstream ygot and protogen usage.

    The comment at https://github.com/openconfig/goyang/blob/master/pkg/yang/entry.go#L1281 said there wasn't a clear use case of copying the Ext field, but I believe we do now.

    What's your opinion on this matter?

    opened by LeonGWang 10
  • Adding Extras to JSON - fixes #211

    Adding Extras to JSON - fixes #211

    In this Patch I'm doing 2 things

    1. Removing the exclusion of Extras from JSON, and excluding down the line the Parent and Source of Extra elements

    2. Improving how items are added to the Extras slice. When the item is a Slice add its elements individually.

    I can break this up in to 2 separate patches if necessary

    opened by SeanCondon 9
  • issues with resolving import-by-revision statements

    issues with resolving import-by-revision statements

    Hi @robshakir @wenovus,

    I am struggling to make goyang parse the modules which have import statements with a specified revision.

    To keep things real, lets take an example of Juniper yang modules, which come in the form of YANG files with [email protected] pattern - https://github.com/Juniper/yang/tree/master/20.2/20.2R1/junos/conf

    1. If the module imports another module with a specified revision (example)
      import junos-common-types {
       prefix jt;
       revision-date 2019-01-01;
      }
      

      the goyang doesn't seem to use the specified revision when it tries processes the imports. Instead it uses the "bare" name. In the linked code snippet name is the bare module name, without revision suffix.

    2. this results in ms.Read to operate on junos-common-types instead of [email protected] name.
    3. The ms.Read in its turn uses findFile that promises to pick the latest revision found, which is not what the import statement wanted.

    Shoudn't we instead read the module by revision by doing:

    	// Try to read it in.
    	if err := ms.Read(rev); err != nil {
    		return nil
    	}
    
    opened by hellt 7
  • releases builds and semantic versioning

    releases builds and semantic versioning

    Hi team, thank you all for making goyang a reality.

    I would like to know if the maintainers are up to adopt the semantic versioning for goyang? I think the common benefits of a versioned package are well-known. Aside from the common versioning benefits the adoption will also make it possible to create an automatic release pipeline powered by goreleaser and github actions (like we did for gnmic).

    Having pre-built goyang binaries ready available and downloadable (via github releases) will help users to leverage one of the best yang compilers out there. I am willing to contribute the release pipeline if the OC community is up for maintaining the versioning.

    opened by hellt 7
  • Add support for the 'reference' statement under import.

    Add support for the 'reference' statement under import.

    This change adds support for the YANG 1.1 construct:

    import foo {
      prefix "bar";
      reference "bat";
    }
    

    to goyang. The test added provides an extensible way to add tests for the contents of opaque Node structures to the goyang pkg.

    opened by robshakir 7
  • Flag-control whether new 'Uses' field is populated.

    Flag-control whether new 'Uses' field is populated.

     * (M) pkg/yang/{entry,entry_test,options}.go
      - Control whether Uses is populated based on an input flag. With
        large sets of schemas, the Uses data can cause a significant increase
        in the output size of the Entry data - which has memory footprint
        implications. Flag control the population of such fields.
    
    opened by robshakir 6
  • Detect missing closing brace from input

    Detect missing closing brace from input

    Consider the case of the following yang that is missing the final closing brace for module foo

    module foo {
      namespace foo;
      prefix f;
    
      container top {
         leaf f1 {
            type string;
         }
      }
    
    

    parsing that with ./goyang -f tree test.yang results in no error and no output.

    With this patch we output an error that says we are missing a closing brace.

    opened by wingeng 5
  • panic if the lexer items channel (would) overflow

    panic if the lexer items channel (would) overflow

    To avoid a deadlock when sending a *token to the lexer's items channel - which can occur if too many errors occur within a single token - instead panic when the items channel fills up.

    Non-error items (i.e., normal tokens) should not overflow the channel, so use a different panic message (indicating a library bug) if this occurs.

    Avoids a deadlock and addresses #116 .

    opened by andaru 5
  • Memory issue due to github.com/openconfig/goyang/pkg/yang#typeDictionary

    Memory issue due to github.com/openconfig/goyang/pkg/yang#typeDictionary

    Hello,

    I am using your great library while working on an implementation of a library that is parsing yang files and does some processing with them (in relation to this). Recently I realized parsing many models makes my library leak memory. After some debugging I discovered that it should be because of the use of typeDict that is growing with each model parsing. I would be willing to work on resolving this issue, but as I saw that there is already a TODO comment around it, I imagine you already have an idea what would be a good solution for you. The best solution for me would be to have reference to the typeDict in the Modules type, but it seems that some initialization is done for the default types, which would make that more difficult (we can have separate typeDict for the built-in types and the parsed ones). Do you have better ideas and would you be willing to review and possibly merge such a change?

    Thank you in advance!

    opened by ivajloip 5
  • Add ParseOption/DeviateOption to ignore not-supported statements

    Add ParseOption/DeviateOption to ignore not-supported statements

    // DeviateOptions contains options for how deviations are handled.
    type DeviateOptions struct {
    	// IgnoreDeviateNotSupported indicates to the parser to retain nodes
    	// that are marked with "deviate not-supported". An example use case is
    	// where the user wants to interact with different targets that have
    	// different support for a leaf without having to use a second instance
    	// of an AST.
    	IgnoreDeviateNotSupported bool
    }
    
    opened by wenovus 1
  • support for yang 1.1 `modifier` pattern substatement

    support for yang 1.1 `modifier` pattern substatement

    Hi all, this issues is a feature request to add support for modifier pattern substatement as per https://datatracker.ietf.org/doc/html/rfc7950#section-9.4.6

    opened by hellt 0
  • Cross-Module deviation statement doesn't properly check module prefixes?

    Cross-Module deviation statement doesn't properly check module prefixes?

    Need to write a reproducing YANG file first, but the reported problem is that a deviation without the proper prefix for an element in the path still compiles but shouldn't.

    opened by wenovus 1
Releases(v1.1.0)
  • v1.1.0(Jul 22, 2022)

    What's Changed

    • Add scheduled workflow. by @robshakir in https://github.com/openconfig/goyang/pull/228
    • Fix workflow syntax and run on every pull request by @wenovus in https://github.com/openconfig/goyang/pull/230
    • enum values are not exported in rawSchema JSON Fixes #227 by @SeanCondon in https://github.com/openconfig/goyang/pull/229
    • identities that have a base identity are not sorted Fixes #234 by @SeanCondon in https://github.com/openconfig/goyang/pull/235

    Full Changelog: https://github.com/openconfig/goyang/compare/v1.0.0...v1.1.0

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Feb 22, 2022)

    This is the v1.0.0 release of goyang with the aim of providing users with a better expectation of backwards-compatibility following the resolution of issues in this milestone.

    NOTE: This release contains several backwards-incompatible changes.

    What's Changed

    • added support for leaf-list default substatement #136 by @hellt in https://github.com/openconfig/goyang/pull/139
    • Remove global variable typeDict and exported function BuildAST. by @wenovus in https://github.com/openconfig/goyang/pull/207
    • Move global ParseOptions into Modules. by @wenovus in https://github.com/openconfig/goyang/pull/208
    • Move Path and pathMap under Modules to allow parsing of two different sets of files with different paths. by @wenovus in https://github.com/openconfig/goyang/pull/209
    • Rename util package to yangentry package. by @wenovus in https://github.com/openconfig/goyang/pull/220

    Full Changelog: https://github.com/openconfig/goyang/compare/v0.4.0...v1.0.0

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Jan 21, 2022)

    What's Changed

    • (BACKWARDS-INCOMPATIBLE) Inherit min/max in already-restricted range/length statements. by @wenovus in https://github.com/openconfig/goyang/pull/188
      • Reference PR for updating your dependent repository: https://github.com/openconfig/ygot/pull/617
    • YangType.Equal should check whether inputs are nil by @wenovus in https://github.com/openconfig/goyang/pull/215
    • Add unit tests for FindGrouping by @wenovus in https://github.com/openconfig/goyang/pull/219

    Full Changelog: https://github.com/openconfig/goyang/compare/v0.3.2...v0.4.0

    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Jan 4, 2022)

    What's Changed

    • factor out d *typeDictionary as a parameter to initTypes. by @wenovus in https://github.com/openconfig/goyang/pull/204
    • Move globals entryCache and mergedSubmodule into Modules. by @wenovus in https://github.com/openconfig/goyang/pull/205
    • Store global identities in typeDictionary. by @wenovus in https://github.com/openconfig/goyang/pull/206

    Full Changelog: https://github.com/openconfig/goyang/compare/v0.3.1...v0.3.2

    Source code(tar.gz)
    Source code(zip)
  • v0.3.1(Oct 26, 2021)

    What's Changed

    • Protect FindModuleByNamespace with locks by @kimmidi in https://github.com/openconfig/goyang/pull/216

    New Contributors

    • @kimmidi made their first contribution in https://github.com/openconfig/goyang/pull/216

    Full Changelog: https://github.com/openconfig/goyang/compare/v0.3.0...v0.3.1

    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Oct 21, 2021)

    What's Changed

    • Marshal Entry.Extra in JSON, and exclude down the line the Parent and Source of Extra elements (@SeanCondon).
    • When an Extra item is a slice, add its elements individually (@SeanCondon).
    • Modules.FindModuleByPrefix is removed since there is possibility for duplication. Use the FindModuleByPrefix function instead.
    • Fix deviate add and deviate delete on defaults statements.
    • Print error message when Entry.InstantiatingModule fails.
    • Allow arbitrary number of errors between each token.
    • Refactorings.

    Full Changelog: https://github.com/openconfig/goyang/compare/v0.2.9...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.9(Aug 16, 2021)

  • v0.2.8(Aug 12, 2021)

    • Fix augments from submodules (@nkitchen).
    • Allow registering multiple enums within a union type.
    • Export Modules.module.
    • Improve coverage to 81.6%.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.7(Jul 1, 2021)

  • v0.2.6(Jun 21, 2021)

  • v0.2.5(Jun 19, 2021)

  • v0.2.4(Dec 22, 2020)

    • Add POSIXPattern to YangType's Equal method such that strings within a union with different oc-ext:posix-pattern statements don't get deduped during parsing.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.3(Dec 16, 2020)

  • v0.2.2(Dec 2, 2020)

    • Error out on "max-elements 0".
    • Fix sortedErrors less method panic.
    • Handle mandatory attribute on leafs correctly.
    • Remove the redundant 'union' in the tree type output.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Oct 15, 2020)

  • v0.2.0(Oct 2, 2020)

  • v0.1.0(Oct 2, 2020)

Owner
OpenConfig
OpenConfig
Godbolt console wrapper for easily execute local file without any security risk and compiler.

Godbolt CLI Godbolt console wrapper for easily execute local file without any security risk and compiler. Install Compile the source code and add to y

aiocat 1 May 22, 2022
Tool to capture C/C++ compiler command lines for use by Sonargraph

ccspy Tool to capture C/C++ compiler command lines for use by Sonargraph-Architect. Purpose of the Tool When Sonargraph analyzes C/C++ code it must kn

Sonargraph 0 Dec 7, 2021
The runner project is to create an interface for users to run their code remotely without having to have any compiler on their machine

The runner project is to create an interface for users to run their code remotely without having to have any compiler on their machine. This is a work in progress project for TCSS 401X :)

cam 6 May 29, 2022
CONTRIBUTIONS ONLY: A Go (golang) command line and flag parser

CONTRIBUTIONS ONLY What does this mean? I do not have time to fix issues myself. The only way fixes or new features will be added is by people submitt

Alec Thomas 3.3k Dec 4, 2022
Brigodier is a command parser & dispatcher, designed and developed for command lines such as for Discord bots or Minecraft chat commands. It is a complete port from Mojang's "brigadier" into Go.

brigodier Brigodier is a command parser & dispatcher, designed and developed to provide a simple and flexible command framework. It can be used in man

Minekube 16 Jun 5, 2022
go command line option parser

go-flags: a go library for parsing command line arguments This library provides similar functionality to the builtin flag library of go, but provides

Jesse van den Kieboom 2.3k Dec 4, 2022
Fully featured Go (golang) command line option parser with built-in auto-completion support.

go-getoptions Go option parser inspired on the flexibility of Perl’s GetOpt::Long. Table of Contents Quick overview Examples Simple script Program wit

David Gamba 46 Sep 26, 2022
Kong is a command-line parser for Go

Kong is a command-line parser for Go Introduction Help Help as a user of a Kong application Defining help in Kong Command handling Switch on the comma

Alec Thomas 1.2k Dec 4, 2022
HAProxy configuration parser

HAProxy configuration parser autogenerated code if you change types/types.go you need to run go run generate/go-generate.go $(pwd) Contributing For co

haproxytech 73 Nov 20, 2022
A simple command line time description parser

Zeit Zeit is an extremely simple command line application to read a natural language time description and output it as a timestamp. The main usecase f

ElKowar 6 Aug 21, 2021
APFS parser written in pure Go

[WIP] go-apfs ?? APFS parser written in pure Go Originally from this ipsw branch Install go get github.com/blacktop/go-apfs apfs cli Install go instal

null 39 Nov 13, 2022
A golang tag key value parser

tag_parser A golang tag key value parser Installation go get github.com/gvassili/tag_parser Example package main import ( "fmt" "github.com/gvass

Gwenael 0 Nov 24, 2021
A Protobuf parser

A Protobuf parser for Go This package contains a cleanroom Protobuf parser for Go using Participle. This was originally an example within Participle.

Alec Thomas 13 Nov 9, 2022
Config File Parser

Config File Parser Speed It was Implemented by binary tree and only suitable for small project. Ignore Any line starting with specific prefix will be

AoXiang Li 1 May 20, 2022
Golisp-wtf - A lisp interpreter (still just a parser) implementation in golang. You may yell "What the fuck!?.." when you see the shitty code.

R6RS Scheme Lisp dialect interpreter This is an implementation of a subset of R6RS Scheme Lisp dialect in golang. The work is still in progress. At th

Vladimir Novikov 0 Jan 7, 2022
Ghissue - This repo contains a github issue parser, that is useful for Enterprise Github accounts.

Ghissue - This repo contains a github issue parser, that is useful for Enterprise Github accounts. Sometimes is needed to parse the content of the issue for some data extraction or statistics purposes.

niloofargheibi 1 Feb 6, 2022
News-parser-cli - Simple CLI which allows you to receive news depending on the parameters passed to it

news-parser-cli Simple CLI which allows you to receive news depending on the par

Maxym 0 Jan 4, 2022
Golang parser for Intuit Interchange Format (.IIF) files

Intuit Interchange Format (.IIF) Parser Install go get github.com/joshuaslate/iif Usage iiifile, err := os.Open("./transactions.iif") if err != nil {

Josh Slate 0 Jan 15, 2022
minigli is a tiny command argument parser for Go.

minigli is a tiny command argument parser for Go.

hitoshi44 0 Jan 29, 2022