CUE is an open source data constraint language which aims to simplify tasks involving defining and using data.

Overview

Go Reference Github GolangCI Go 1.14+ platforms

The CUE Data Constraint Language

Configure, Unify, Execute

CUE is an open source data constraint language which aims to simplify tasks involving defining and using data.

It is a superset of JSON, allowing users familiar with JSON to get started quickly.

What is it for?

You can use CUE to

  • define a detailed validation schema for your data (manually or automatically from data)
  • reduce boilerplate in your data (manually or automatically from schema)
  • extract a schema from code
  • generate type definitions and validation code
  • merge JSON in a principled way
  • define and run declarative scripts

How?

CUE merges the notion of schema and data. The same CUE definition can simultaneously be used for validating data and act as a template to reduce boilerplate. Schema definition is enriched with fine-grained value definitions and default values. At the same time, data can be simplified by removing values implied by such detailed definitions. The merging of these two concepts enables many tasks to be handled in a principled way.

Constraints provide a simple and well-defined, yet powerful, alternative to inheritance, a common source of complexity with configuration languages.

CUE Scripting

The CUE scripting layer defines declarative scripting, expressed in CUE, on top of data. This solves three problems: working around the closedness of CUE definitions (we say CUE is hermetic), providing an easy way to share common scripts and workflows for using data, and giving CUE the knowledge of how data is used to optimize validation.

There are many tools that interpret data or use a specialized language for a specific domain (Kustomize, Ksonnet). This solves dealing with data on one level, but the problem it solves may repeat itself at a higher level when integrating other systems in a workflow. CUE scripting is generic and allows users to define any workflow.

Tooling

CUE is designed for automation. Some aspects of this are:

  • convert existing YAML and JSON
  • automatically simplify configurations
  • rich APIs designed for automated tooling
  • formatter
  • arbitrary-precision arithmetic
  • generate CUE templates from source code
  • generate source code from CUE definitions (TODO)

Download and Install

Release builds

Download the latest release from GitHub.

Install using Homebrew

Using Homebrew, you can install using the CUE Homebrew tap:

brew install cuelang/tap/cue

Install from Source

If you already have Go installed, the short version is:

GO111MODULE=on go get cuelang.org/go/cmd/cue

Or, if you are using Go 1.16:

go install cuelang.org/go/cmd/[email protected]

This will install the cue command line tool.

For more details see Installing CUE.

Learning CUE

The fastest way to learn the basics is to follow the tutorial on basic language constructs.

A more elaborate tutorial demonstrating of how to convert and restructure an existing set of Kubernetes configurations is available in written form.

References

Contributing

Our canonical Git repository is located at https://review.gerrithub.io/q/project:cue-lang%252Fcue.

To contribute, please read the Contribution Guide.

To report issues or make a feature request, use the issue tracker.

Changes can be contributed using Gerrit or Github pull requests.

Code of Conduct

Guidelines for participating in CUE community spaces and a reporting process for handling issues can be found in the Code of Conduct.

Contact

You can get in touch with the cuelang community in the following ways:


Unless otherwise noted, the CUE source files are distributed under the Apache 2.0 license found in the LICENSE file.

This is not an officially supported Google product.

Comments
  • cue: use dots as a separator for folding instead of spaces?

    cue: use dots as a separator for folding instead of spaces?

    Originally opened by @ghost in https://github.com/cuelang/cue/issues/60

    This input:

    outer middle inner: 3
    

    produces this with CUE

    {
       "outer": {
          "middle": {
             "inner": 3
          }
       }
    }
    

    and this with YAML:

    {
       "outer middle inner": 3
    }
    

    this seems like too much sugar to be worth it

    https://github.com/cuelang/cue/blob/master/doc/tutorial/basics/fold.md

    FeedbackWanted 
    opened by cueckoo 32
  • Add an option to quote all strings in YAML

    Add an option to quote all strings in YAML

    Currently, exporting to YAML is a mine field, because most things well defined as strings in a CUE file will be exported unquoted in YAML.

    Of course, because YAML interprets magically most unquoted things, this leads to plenty of issues. E.G:

    • dates can be wrongly interpreted: https://github.com/cue-lang/cue/issues/1076
    • the norway problem: https://hitchdev.com/strictyaml/why/implicit-typing-removed/
    • the python 3.10 problem: https://github.com/actions/setup-python/issues/160

    I think quoting all the strings in the resulting YAML should be the default behavior. However, this would probably cause a lot of troubles now that the old behavior is used everywhere.

    Alternatively, a nice solution would be to provide an option to force cue to quote everything. This option could be turned on with either:

    • a cli --flag
    • an env var
    • a directive in the cue file
    FeatureRequest FeedbackWanted NeedsDesign encoding 
    opened by ksamuel 22
  • cmd/cue: reference error in value imported from a package lost during export

    cmd/cue: reference error in value imported from a package lost during export

    Originally opened by @uhthomas in https://github.com/cuelang/cue/issues/854

    What version of CUE are you using (cue version)?

    $ cue version
    cue version v0.3.0-beta.7 linux/amd64
    

    Does this issue reproduce with the latest release?

    Yes.

    What did you do?

    ! exec cue export
    cmp stderr stderr.golden
    
    -- a.cue --
    package a
    
    import "mod.com/b"
    
    theb: b.name
    -- b/b.cue --
    package b
    
    import "mod.com/c"
    
    b: c.c & {
    	other: "name"
    }
    
    name: b.other
    -- c/c.cue --
    package c
    -- cue.mod/module.cue --
    module: "mod.com"
    -- stderr.golden --
    b: undefined field c:
        ./b/b.cue:5:6
    

    With beta.3 this test passes; with beta.4 (or indeed any commit including bedb047a988217bab040c29eeb8a51988ce23b76) fails with:

    > ! exec cue export
    [stdout]
    {
        "theb": "name"
    }
    FAIL: /tmp/testscript235319619/repro.txt/script.txt:1: unexpected command success
    

    What did you expect to see?

    A passing test.

    What did you see instead?

    Per above; a failing test, because cue export succeeds where it should not.

    Edit: updated description to the contents of https://github.com/cuelang/cue/issues/854#issuecomment-808683933 for ease of reference.

    NeedsFix 
    opened by cueckoo 22
  • How to set up per-environment configs

    How to set up per-environment configs

    Originally opened by @vikstrous in https://github.com/cuelang/cue/issues/190

    I'm trying to set up per-environment configs with cue, but I'm running into a bug.

    ... in paths doesn't work with symlinks.

    Repro:

    tree
    .
    ├── cue.mod
    │   ├── module.cue
    │   ├── pkg
    │   └── usr
    ├── environments
    │   ├── development
    │   │   ├── development.cue
    │   │   └── services -> ../../services/
    │   └── production
    │       ├── production.cue
    │       └── services -> ../../services/
    └── services
        └── frontend
            └── frontend.cue
    
    10 directories, 4 files
    
    cat environments/development/development.cue
    package service
    
    Environment:: "development"
    
    cat environments/production/production.cue
    package service
    
    Environment:: "production"
    
    cat services/frontend/frontend.cue
    package service
    
    main: "hello \(Environment)"
    
    cue eval ./environments/development/services/frontend/
    Environment :: "development"
    main:          "hello development"
    

    Expected:

    cue eval ./environments/development/services/...
    Environment :: "development"
    main:          "hello development"
    

    Actual:

    cue eval ./environments/development/services/...
    cue: "./environments/development/services/..." matched no packages
    

    If there's any info about how to customize configs for different environments or any best practices, please let me know. I've read all of the docs and I didn't see anything mentioning this issue.

    FeedbackWanted roadmap/cli 
    opened by cueckoo 19
  • doc: update README to explain how CUE differs from jsonnet?

    doc: update README to explain how CUE differs from jsonnet?

    Originally opened by @ngrilly in https://github.com/cuelang/cue/issues/33

    How is CUE different from jsonnet, both coming from Google, and the latter being largely promoted especially in the context of Kubernetes?

    opened by cueckoo 16
  • ci: update Go versions

    ci: update Go versions

    Update Release go version to at least 1.16 so that darwin arm64 target binaries can be released

    Issue for PR #1223 that sparked more conversation

    @myitcv commented:

    There are a few things we probably want to address here:    
    - expanding the build matrix to include Go 1.17
    - building CUE release artefacts with the latest version of Go
    - expanding the release matrix to include new Go ports
    - testing all the above (somewhat related to goreleaser test goreleaser/goreleaser#2355))
    

    @myitcv to @verdverm:

    we used to build with the latest version of Go. Looks like when I updated our build matrix from 
    1.16.0-rc1 to 1.16 I didn't flip to define 1.16 (at the time) as the latest stable version of Go. We 
    can fix that in whatever changes come from this discussion.
    

    @verdverm:

    another update in 1.17 I was thinking might be helpful for API users is the dependency pruning. 
    That would require updating the Go directive in go.mod. Thoughts?
    
    Another thought, is there preference for including (or not) the patch version in the workflows? (i.e. 1.16 vs 1.16.5)
    Any implications on reproducibility, like a new patch version released between final commit testing and tagging for a release?
    
    NeedsFix 
    opened by hrvolapeter 15
  • Memory leak in core/adt

    Memory leak in core/adt

    Originally opened by @samalba in https://github.com/cuelang/cue/issues/661

    What version of CUE are you using (cue version)?

    $ cue version
    0.3.0-beta.2
    

    Does this issue reproduce with the latest release?

    Yes. I also reproduced it as well with master (c58e0ffc4c74d1b3cffe6467b7f144b9dde631b7) - same result than beta.2.

    What did you do?

    Cannot give a repro as it's happening with blocklayer's runtime, using the Cue Go API. In few seconds, the binary is consuming several gigs of RAM (stays stable at around 34MB with 0.3.0-alpha6).

    As @myitcv suggested, I enabled pprof and gathered this attached heap profile.

    heap.out.gz

    Here are the top consumers:

    (pprof) top
    Showing nodes accounting for 1023.36MB, 87.95%!o(MISSING)f 1163.57MB total
    Dropped 82 nodes (cum <= 5.82MB)
    Showing top 10 nodes out of 84
          flat  flat%!s(MISSING)um%!c(MISSING)um   cum%!
    (MISSING)
      322.54MB 27.72%   322.54MB 27.72%!c(MISSING)uelang.org/go/internal/core/adt.(*Vertex).GetArc
      158.51MB 13.62%   158.51MB 13.62%!c(MISSING)uelang.org/go/internal/core/adt.(*Vertex).addConjunct
      136.02MB 11.69%   136.02MB 11.69%!c(MISSING)uelang.org/go/internal/core/adt.(*nodeContext).createDisjunct
      113.01MB  9.71%   666.57MB 57.29%!c(MISSING)uelang.org/go/internal/core/adt.(*nodeContext).addStruct
       91.51MB  7.86%    91.51MB  7.86%!c(MISSING)uelang.org/go/internal/core/adt.updateCyclic
       75.50MB  6.49%    75.50MB  6.49%!c(MISSING)uelang.org/go/internal/core/adt.(*Vertex).AddStruct
       44.76MB  3.85%    50.27MB  4.32%!c(MISSING)uelang.org/go/pkg/encoding/json.Marshal
          39MB  3.35%    78.51MB  6.75%!c(MISSING)uelang.org/go/internal/core/adt.(*StructInfo).MatchAndInsert
          24MB  2.06%       24MB  2.06%!c(MISSING)uelang.org/go/internal/core/adt.clone
       18.50MB  1.59%    18.50MB  1.59%!c(MISSING)uelang.org/go/internal/core/adt.CloseInfo.SpawnRef (inline)
    
    NeedsFix 
    opened by cueckoo 15
  • Panic during cue eval

    Panic during cue eval

    Originally opened by @verdverm in https://github.com/cuelang/cue/issues/598

    What version of CUE are you using (cue version)?

    master @ 6c49cf0

    the previous commit(s) do not have this panic

    panic: runtime error: invalid memory address or nil pointer dereference [recovered]
            panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x6d7017]
    
    goroutine 1 [running]:
    cuelang.org/go/cmd/cue/cmd.recoverError(0xc000723ec0)
            /home/tony/cue/cue/cmd/cue/cmd/root.go:221 +0x95
    panic(0xbcfac0, 0x11a5f00)
            /usr/local/go/src/runtime/panic.go:969 +0x1b9
    cuelang.org/go/internal/core/adt.(*Vertex).Default(0x0, 0xd9e0e0)
            /home/tony/cue/cue/internal/core/adt/default.go:46 +0x37
    cuelang.org/go/internal/core/adt.(*OpContext).node(0xc0002cd6c0, 0xd99420, 0xc0003d0cd0, 0x40bc01, 0xc000020000)
            /home/tony/cue/cue/internal/core/adt/context.go:676 +0x1cc
    cuelang.org/go/internal/core/adt.(*SelectorExpr).resolve(0xc00044f0e0, 0xc0002cd6c0, 0xc000390c30)
            /home/tony/cue/cue/internal/core/adt/expr.go:588 +0x54
    cuelang.org/go/internal/core/adt.(*OpContext).Resolve(0xc0002cd6c0, 0xc0001df9f0, 0xd93660, 0xc00044f0e0, 0xc00044f0e0, 0x1)
            /home/tony/cue/cue/internal/core/adt/context.go:331 +0xdb
    cuelang.org/go/internal/core/eval.(*nodeContext).evalExpr(0xc0000cf980, 0xc0001df9f0, 0xd8bc40, 0xc00044f100, 0x0)
            /home/tony/cue/cue/internal/core/eval/eval.go:1144 +0x49e
    cuelang.org/go/internal/core/eval.(*nodeContext).addExprConjunct(0xc0000cf980, 0xc0001df9f0, 0xd8bc40, 0xc00044f100, 0x7ea800000000)
            /home/tony/cue/cue/internal/core/eval/eval.go:1114 +0xaa
    cuelang.org/go/internal/core/eval.(*Evaluator).evalVertex(0xc000345e60, 0xc0002cd6c0, 0xc000340360, 0x4, 0x0, 0x0, 0x0)
            /home/tony/cue/cue/internal/core/eval/eval.go:408 +0x2d8
    cuelang.org/go/internal/core/eval.(*Evaluator).UnifyAccept(0xc000345e60, 0xc0002cd6c0, 0xc000340360, 0x719804, 0x0, 0x0)
            /home/tony/cue/cue/internal/core/eval/eval.go:283 +0xa7
    cuelang.org/go/internal/core/eval.(*Evaluator).Unify(0xc000345e60, 0xc0002cd6c0, 0xc000340360, 0xc0001d4a04)
            /home/tony/cue/cue/internal/core/eval/eval.go:257 +0x50
    cuelang.org/go/internal/core/eval.(*nodeContext).completeArcs(0xc0000cf380, 0xc000130004)
            /home/tony/cue/cue/internal/core/eval/eval.go:620 +0xc8
    cuelang.org/go/internal/core/eval.(*nodeContext).postDisjunct(0xc0000cf380, 0xbdb004)
            /home/tony/cue/cue/internal/core/eval/eval.go:599 +0x470
    cuelang.org/go/internal/core/eval.(*nodeContext).updateResult(0xc0000cf380, 0xc0005b6904, 0x3)
            /home/tony/cue/cue/internal/core/eval/disjunct.go:132 +0x4c
    cuelang.org/go/internal/core/eval.(*nodeContext).tryDisjuncts(0xc0000cf380, 0xc000510704, 0xd8bc40)
            /home/tony/cue/cue/internal/core/eval/disjunct.go:203 +0x1a7
    cuelang.org/go/internal/core/eval.(*Evaluator).evalVertex(0xc000345e60, 0xc0002cd6c0, 0xc000132240, 0xc000785704, 0x0, 0x0, 0x0)
            /home/tony/cue/cue/internal/core/eval/eval.go:438 +0x3b5
    ...
    

    fmt.Printf("nil node: %!v(MISSING)\n", v)

    ...
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.Bottom{Src:(*ast.SelectorExpr)( 0xc000343260), Err:errors.list{(*adt.ValueError)( 0xc000674d20), (*adt.ValueError)( 0xc000674d90)}, Code: 4, HasRecursive:false, ChildError:false, Value:adt.Value(nil)}
    nil node: &adt.Bottom{Src:ast.Node(nil), Err:(*errors.posError)( 0xc000094660), Code: 4, HasRecursive:false, ChildError:false, Value:adt.Value(nil)}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.Bottom{Src:(*ast.SelectorExpr)( 0xc000343260), Err:errors.list{(*adt.ValueError)( 0xc000675340), (*adt.ValueError)( 0xc0006753b0)}, Code: 4, HasRecursive:false, ChildError:false, Value:adt.Value(nil)}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.Bottom{Src:(*ast.SelectorExpr)( 0xc000343260), Err:errors.list{(*adt.ValueError)( 0xc0006758f0), (*adt.ValueError)( 0xc000675960)}, Code: 4, HasRecursive:false, ChildError:false, Value:adt.Value(nil)}
    nil node: &adt.Bottom{Src:ast.Node(nil), Err:(*errors.posError)( 0xc000094660), Code: 4, HasRecursive:false, ChildError:false, Value:adt.Value(nil)}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.Bottom{Src:(*ast.SelectorExpr)( 0xc000343260), Err:errors.list{(*adt.ValueError)( 0xc000675f10), (*adt.ValueError)( 0xc000675f80)}, Code: 4, HasRecursive:false, ChildError:false, Value:adt.Value(nil)}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.BasicType{Src:(*ast.Ident)(nil), K: 0xff}
    nil node: &adt.Bottom{Src:(*ast.SelectorExpr)( 0xc000343260), Err:errors.list{(*adt.ValueError)( 0xc0006804d0), (*adt.ValueError)( 0xc000680540)}, Code: 4, HasRecursive:false, ChildError:false, Value:adt.Value(nil)}
    nil node: &adt.StructMarker{NeedClose:false}
    
    NeedsVerification 
    opened by cueckoo 15
  • Document usage of intstr.IntOrString in the k8s tutorial

    Document usage of intstr.IntOrString in the k8s tutorial

    Originally opened by @grantzvolsky in https://github.com/cuelang/cue/issues/148

    The kubernetes tutorial shows how to Extract CUE templates directly from Go source.

    The generated Cue templates include intstr.

    The question is how to use this struct with Cue.

    How could one e.g. configure a service targetPort? Neither of the configurations below work.

     service "backend" spec ports: [{
       port:       80
       targetPort: "backend-http" // Error 1
       targetPort: intstr.FromString("backend-http") // Error 2
       targetPort: { Type: 1, IntVal: 0, StrVal: "backend-http" } // Yields invalid output
     }]
    

    Error 1:

    service."backend".spec.ports.0.targetPort: conflicting values IntOrString and "backend-http" (mismatched types struct and string):
        ./cue/backend-overlays-local.cue:10:15
        ./pkg/k8s.io/api/core/v1/types_go_gen.cue:4389:15
    terminating because of errors
    

    Error 2:

    service."backend".spec.ports.0.targetPort: undefined field "FromString":
        ./cue/backend-overlays-local.cue:10:15
    a
    

    Invalid output (yaml dump):

      ports:
      - name: 80-http
        port: 80
        protocol: TCP
        targetPort:
          IntVal: 0
          StrVal: backend-http
          Type: 1
    

    A possible workaround would be to define IntOrString as IntOrString :: int | string, but that would go against the instructions in the template ("// Code generated by cue get go. DO NOT EDIT.").

    Documentation NeedsInvestigation get go 
    opened by cueckoo 15
  • panic: getNodeContext: nodeContext out of sync

    panic: getNodeContext: nodeContext out of sync

    Originally opened by @binarylogic in https://github.com/cuelang/cue/issues/633

    What version of CUE are you using (cue version)?

    ➜ cue version
    cue version 0.3.0-beta.1 darwin/amd64
    

    Does this issue reproduce with the latest release?

    Yes

    What did you do?

    We use Cue over at Vector to maintain project metadata (such as configuration) for our docs. Up until 0.3.0-beta.1 everything has worked fine.

    1. git clone [email protected]:timberio/vector.git
    2. cd vector
    3. cue export docs/**/*.cue

    What did you expect to see?

    The normal JSON document generated from our cue files.

    What did you see instead?

    panic: getNodeContext: nodeContext out of sync
      panic: freeNode: nodeContext out of sync [recovered]
      panic: freeNode: nodeContext out of sync
    
    goroutine 1 [running]:
    cuelang.org/go/cmd/cue/cmd.recoverError(0xc00221fec0)
      /Users/mpvl/Documents/dev/release/cue/cmd/cue/cmd/root.go:221 +0x95
    panic(0x17996c0, 0x19760a0)
      /usr/local/go/src/runtime/panic.go:969 +0x1b9
    cuelang.org/go/internal/core/adt.(*Vertex).freeNode(0xc0043726c0, 0xc004337180)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:799 +0x132
    panic(0x17996c0, 0x1976080)
      /usr/local/go/src/runtime/panic.go:975 +0x47a
    cuelang.org/go/internal/core/adt.(*Vertex).getNodeContext(0xc00438c2d0, 0xc000329450, 0xc00438c240)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:785 +0xd6
    cuelang.org/go/internal/core/adt.(*Unifier).Unify(0xc000329468, 0xc000329450, 0xc00438c2d0, 0x180e905)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:207 +0x92
    cuelang.org/go/internal/core/adt.(*nodeContext).completeArcs(0xc004337880, 0x5)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:577 +0x136
    cuelang.org/go/internal/core/adt.(*nodeContext).postDisjunct(0xc004337880, 0x605)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:548 +0x4fb
    cuelang.org/go/internal/core/adt.(*nodeContext).expandDisjuncts(0xc004337880, 0xc0038dbe05, 0xc004337180, 0x0, 0x1)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/disjunct.go:137 +0x1008
    cuelang.org/go/internal/core/adt.(*nodeContext).expandDisjuncts(0xc004337180, 0xc004372603, 0xc004337180, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/disjunct.go:213 +0x408
    cuelang.org/go/internal/core/adt.(*Unifier).Unify(0xc000329468, 0xc000329450, 0xc0043726c0, 0x12ccb03)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:315 +0x4a8
    cuelang.org/go/internal/core/adt.(*Unifier).evaluate(0xc000329468, 0xc000329450, 0xc0043726c0, 0x810a303, 0xc0015a64b0, 0x1)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:139 +0x8c
    cuelang.org/go/internal/core/adt.(*OpContext).evalState(0xc000329450, 0x199a300, 0xc0015a64b0, 0xc00389d603, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/context.go:585 +0x247
    cuelang.org/go/internal/core/adt.(*OpContext).node(0xc000329450, 0x198d340, 0xc001aad4a0, 0x199a300, 0xc0015a64b0, 0x180e801, 0x1)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/context.go:725 +0x5f
    cuelang.org/go/internal/core/adt.(*SelectorExpr).resolve(0xc001aad4a0, 0xc000329450, 0xc001aad4a0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/expr.go:698 +0x65
    cuelang.org/go/internal/core/adt.(*OpContext).evalState(0xc000329450, 0x199a5c0, 0xc001aad4a0, 0x1807002, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/context.go:577 +0x1fd
    cuelang.org/go/internal/core/adt.(*OpContext).Evaluate(0xc000329450, 0xc0038db1d0, 0x199a5c0, 0xc001aad4a0, 0xc001a93280, 0x199ecc0, 0xc001a93280)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/context.go:507 +0x105
    cuelang.org/go/internal/core/adt.(*OpContext).Concrete(0xc000329450, 0xc0038db1d0, 0x199a5c0, 0xc001aad4a0, 0x17cce20, 0x1d6fe78, 0x1009eff, 0xc00001e000, 0x17e5a40)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/context.go:445 +0x53
    cuelang.org/go/internal/core/adt.(*BinaryExpr).evaluate(0xc0019bb3e0, 0xc000329450, 0xc0019bb3e0, 0x86731d8)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/expr.go:961 +0x285
    cuelang.org/go/internal/core/adt.(*OpContext).evalState(0xc000329450, 0x1999fc0, 0xc0019bb3e0, 0x2, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/context.go:573 +0x38c
    cuelang.org/go/internal/core/adt.(*OpContext).value(0xc000329450, 0x1999fc0, 0xc0019bb3e0, 0xc003d2d500, 0xc0019bb470)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/context.go:537 +0x48
    cuelang.org/go/internal/core/adt.(*IfClause).yield(0xc0019bb410, 0xc000329450, 0xc003d2d500)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/expr.go:1417 +0x46
    cuelang.org/go/internal/core/adt.(*OpContext).Yield(0xc000329450, 0xc0038db1d0, 0x19944c0, 0xc0019bb410, 0xc003d2d500, 0x19944c0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/context.go:433 +0xf9
    cuelang.org/go/internal/core/adt.(*nodeContext).injectEmbedded(0xc004328000, 0xc004328200, 0x0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:1799 +0x14f
    cuelang.org/go/internal/core/adt.(*nodeContext).expandOne(0xc004328000, 0xc004348360)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:1722 +0xc5
    cuelang.org/go/internal/core/adt.(*Unifier).Unify(0xc000329468, 0xc000329450, 0xc004348360, 0x180e905)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:277 +0x3c5
    cuelang.org/go/internal/core/adt.(*nodeContext).completeArcs(0xc0041d7880, 0x5)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:577 +0x136
    cuelang.org/go/internal/core/adt.(*nodeContext).postDisjunct(0xc0041d7880, 0x5)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:548 +0x4fb
    cuelang.org/go/internal/core/adt.(*nodeContext).expandDisjuncts(0xc0041d7880, 0xc00431f805, 0xc0041d7880, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/disjunct.go:137 +0x1008
    cuelang.org/go/internal/core/adt.(*Unifier).Unify(0xc000329468, 0xc000329450, 0xc00431f8c0, 0x180e905)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:315 +0x4a8
    cuelang.org/go/internal/core/adt.(*nodeContext).completeArcs(0xc000037c00, 0x5)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:577 +0x136
    cuelang.org/go/internal/core/adt.(*nodeContext).postDisjunct(0xc000037c00, 0x1829805)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:548 +0x4fb
    cuelang.org/go/internal/core/adt.(*nodeContext).expandDisjuncts(0xc000037c00, 0xc0014ab305, 0xc000037c00, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/disjunct.go:137 +0x1008
    cuelang.org/go/internal/core/adt.(*Unifier).Unify(0xc000329468, 0xc000329450, 0xc0014ab320, 0x180e905)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:315 +0x4a8
    cuelang.org/go/internal/core/adt.(*nodeContext).completeArcs(0xc000282a80, 0x5)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:577 +0x136
    cuelang.org/go/internal/core/adt.(*nodeContext).postDisjunct(0xc000282a80, 0x5)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:548 +0x4fb
    cuelang.org/go/internal/core/adt.(*nodeContext).expandDisjuncts(0xc000282a80, 0xc001448605, 0xc000282a80, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/disjunct.go:137 +0x1008
    cuelang.org/go/internal/core/adt.(*Unifier).Unify(0xc000329468, 0xc000329450, 0xc001448630, 0x180e905)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:315 +0x4a8
    cuelang.org/go/internal/core/adt.(*nodeContext).completeArcs(0xc000282380, 0x5)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:577 +0x136
    cuelang.org/go/internal/core/adt.(*nodeContext).postDisjunct(0xc000282380, 0xc00088bc05)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:548 +0x4fb
    cuelang.org/go/internal/core/adt.(*nodeContext).expandDisjuncts(0xc000282380, 0xc000fabb05, 0xc000282380, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/disjunct.go:137 +0x1008
    cuelang.org/go/internal/core/adt.(*Unifier).Unify(0xc000329468, 0xc000329450, 0xc000fabb00, 0x180e905)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:315 +0x4a8
    cuelang.org/go/internal/core/adt.(*nodeContext).completeArcs(0xc000282000, 0x5)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:577 +0x136
    cuelang.org/go/internal/core/adt.(*nodeContext).postDisjunct(0xc000282000, 0x5)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:548 +0x4fb
    cuelang.org/go/internal/core/adt.(*nodeContext).expandDisjuncts(0xc000282000, 0xc00182e505, 0xc000282000, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/disjunct.go:137 +0x1008
    cuelang.org/go/internal/core/adt.(*Unifier).Unify(0xc000329468, 0xc000329450, 0xc00182e510, 0xc000329405)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/eval.go:315 +0x4a8
    cuelang.org/go/internal/core/adt.(*Vertex).Finalize(...)
      /Users/mpvl/Documents/dev/release/cue/internal/core/adt/composite.go:401
    cuelang.org/go/cue.(*Instance).Value(0xc00188a700, 0x6, 0xc00031fea0)
      /Users/mpvl/Documents/dev/release/cue/cue/instance.go:271 +0x6c
    cuelang.org/go/cmd/cue/cmd.buildInstances(0xc00031fea0, 0xc0017e7898, 0x1, 0x1, 0x17c8180, 0xc000321d40, 0xc0002c3b58)
      /Users/mpvl/Documents/dev/release/cue/cmd/cue/cmd/common.go:552 +0x131
    cuelang.org/go/cmd/cue/cmd.(*buildPlan).instances(0xc000178180, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/cmd/cue/cmd/common.go:156 +0x58
    cuelang.org/go/cmd/cue/cmd.runExport(0xc00031fea0, 0xc00036c600, 0x10d, 0x10d, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/cmd/cue/cmd/export.go:114 +0x1be
    cuelang.org/go/cmd/cue/cmd.mkRunE.func1(0xc000344000, 0xc00036c600, 0x10d, 0x10d, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/cmd/cue/cmd/root.go:46 +0x6c
    github.com/spf13/cobra.(*Command).execute(0xc000344000, 0xc00036b300, 0x10d, 0x130, 0xc000344000, 0xc00036b300)
      /Users/mpvl/go/pkg/mod/github.com/spf13/[email protected]/command.go:842 +0x47c
    github.com/spf13/cobra.(*Command).ExecuteC(0xc000281080, 0x0, 0x0, 0x0)
      /Users/mpvl/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x375
    github.com/spf13/cobra.(*Command).Execute(...)
      /Users/mpvl/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
    cuelang.org/go/cmd/cue/cmd.(*Command).Run(0xc00031fea0, 0x1997f00, 0xc00003a0b8, 0x0, 0x0)
      /Users/mpvl/Documents/dev/release/cue/cmd/cue/cmd/root.go:206 +0x65
    cuelang.org/go/cmd/cue/cmd.mainErr(0x1997f00, 0xc00003a0b8, 0xc000031310, 0x10e, 0x12f, 0x18da528, 0xc0002c3f48)
      /Users/mpvl/Documents/dev/release/cue/cmd/cue/cmd/root.go:145 +0x8a
    cuelang.org/go/cmd/cue/cmd.Main(0xc000100058)
      /Users/mpvl/Documents/dev/release/cue/cmd/cue/cmd/root.go:127 +0x9c
    main.main()
      /Users/mpvl/Documents/dev/release/cue/cmd/cue/main.go:24 +0x25
    
    NeedsFix roadmap/performance 
    opened by cueckoo 14
  • cue get go does not work consistently

    cue get go does not work consistently

    Originally opened by @verdverm in https://github.com/cuelang/cue/issues/621

    ceu get go does not load its own interfaces when a go vendor directory is present and CUE is not in the dependencies.

    The error is in initInterfaces() (https://github.com/cuelang/cue/blob/master/cmd/cue/cmd/get_go.go#L287)

    packages.Load does not return an error, but there are errors in p[0].Errors

    We should be checking for these errors where this function is used.


    repro:

    mkdir intstr && cd intstr
    go mod init foo.bar/intstr
    cue mod init foo.bar/intstr
    cat << EOF > repro.go
    package intstr
    import (
            "k8s.io/apimachinery/pkg/util/intstr"
    )
    _ = intstr.IntOrString
    EOF
    go mod vendor
    cue get go k8s.io/apimachinery/pkg/util/intstr
    

    possible resolutions:

    1. check for the errors and return better messages
    2. modify the Config passed to packages.Load
    3. embed and load the toTop and toText interfaces

    (1) I'd generally prefer to have a solution that does not require the user to have Cue as a dependency. This seems like unpleasant DX, essentially telling them they need to have an artificial dependency via overly complex instructions.

    (2) could be an easy solution. However, I'm leery of modifying the Config so that Cue code is fetched automatically. This might make some SecOps people unhappy / break in corp intranets.

    (3) Config has an Overlay we could use to embed these files as strings. This seems like the best way to go.

    We will also want to catch the errors in the other locations and provide an error message there.

    What do others think?

    NeedsFix get go 
    opened by cueckoo 14
  • Can I write the Fibonacci sequence in cue?

    Can I write the Fibonacci sequence in cue?

    What version of CUE are you using (cue version)?

    $ cue version
    cue version v0.4.3 darwin/amd64
    

    Does this issue reproduce with the latest release?

    Yes

    What did you do?

    For the following code snippet (fib.cue), I run with cue export fib.cue

    #Fib: {
        n: int,
        if n <= 2 {
            value: 1
        }
        if n > 2 {
            _n1: #Fib & {
                n: n - 1
            },
            _n2: #Fib & {
                n: n - 2
            }
            value: _n1.value + _n2.value
        }
    }
    fib8: #Fib & {
        n: 8
    }
    
    

    What did you expect to see?

    {
        "fib8": {
            "n": 8,
            "value": 21
        }
    }
    

    What did you see instead?

    cycle error:
        ../fib.cue:3:8
    cycle error:
        ../fib.cue:6:8
    

    It seems that for #Fib in different positions, cue treats them as the same instance?

    NeedsInvestigation Triage 
    opened by Peefy 5
  • Enable the OpenSSF Scorecard Github Action

    Enable the OpenSSF Scorecard Github Action

    Hello, I am Joyce and I'm working on behalf of Google and the Open Source Security Foundation to help essential open-source projects improve their supply-chain security. Given the CUE Language relevance and impact in many projects, the OpenSSF has identified it as one of the 100 most critical open source projects.

    Is your feature request related to a problem? Please describe. According to Open Source Security and Risk Analysis Report, 84% of all codebases have at least one vulnerability, with an average of 158 per codebase. The majority have been in the code for more than 2 years and have documented solutions available.

    Even in large tech companies, the tedious process of reviewing code for vulnerabilities falls down the priority list, and there is little insight into known vulnerabilities and solutions that companies can draw on.

    That’s where the OpenSSF tool called Scorecards is helping. Its focus is to understand the security posture of a project and assess the risks that the dependencies could introduce.

    Describe the solution you'd like

    Scorecards runs dozens of automated security checks to help maintainers better understand their project's supply-chain security posture. It is developed by the OpenSSF, in partnership with GitHub.

    To simplify maintainers' lives, the OpenSSF has also developed the Scorecard GitHub Action. It is very lightweight and runs on every change to the repository's main branch. The results of its checks are available on the project's security dashboard, and include suggestions on how to solve any issues (see examples in the Additional context). The Action does not run or interact with any workflows, but merely parses them to identify possible vulnerabilities. This Action has been adopted by 1800+ projects already.

    This way, the OpenSSF Scorecard Github Action could help you to increase the security of the repository, providing some guarantee that it is (mostly) safe from malicious sabotage.

    Would you be interested in a PR which adds this Action? Optionally, it can also publish your results to the OpenSSF REST API, which allows a badge with the project's score to be added to its README, which is a great way to show off your hard work to improve security best practices and to help raising the collective level of open source security.

    Any concerns or doubts please let me know.

    Additional context Code scanning dashboard with multiple alerts, including Code-Review and Token-Permissions

    Detail of a Token-Permissions alert, indicating the specific file and remediation steps

    FeatureRequest Triage 
    opened by joycebrum 0
  • Introducing pattern constraint causes

    Introducing pattern constraint causes "cue export" to spin indefinitely

    What version of CUE are you using (cue version)?

    cue version v0.4.4-0.20220915174651-ad253ed099e9
    
           -compiler gc
         CGO_ENABLED 1
              GOARCH amd64
                GOOS darwin
             GOAMD64 v1

    Does this issue reproduce with the latest release?

    Yes, both in version 0.4.3 and the tip as of today (ad253ed099e94ddcffd07e6f73a76735a96cfaf8).

    What did you do?

    I wrote a few definitions as described in preceding discussion in the "language" channel of the "CUE" Slack workspace, involving a recursive definition as part of describing Terraform's JSON output format. Building up from there, I wrote definitions for types, values of any type, values of a particular type, and documents with fields bearing values of those types.

    Most of this works fine, up until when I try to use a pattern constraint to define that general document schema, where each of the field must bear one of these values. When I try, the cue export command spins using 2.5 CPUs for as long as twenty minutes. That's the longest I've let it run so far before killing it.

    I'll share the definitions that I've written so far, together with a small JSON file that shows input data against which I'd unify these definitions.

    CUE definitions
    #ValueType:
    	// See https://github.com/zclconf/go-cty/blob/3792a7b5328c364b10c6d642e3ea0d9761338740/cty/json.go#L16-L89.
    	"bool" | // Primitive types
    	"number" |
    	"string" |
    	["list", #ValueType] |
    	["map", #ValueType] | // Complex types
    	["object", {[string]: #ValueType}] |
    	["object", {[string]: #ValueType}, [...string]] |
    	["set", #ValueType] |
    	["tuple", [...#ValueType]] |
    	"dynamic" // Pseudo-type
    
    #OutputValue: {
    	type:         #ValueType
    	sensitive:    bool
    	description?: string
    	value:        _
    }
    
    // Make defining schemas more convenient.
    #BoolTypedOutputValue: #OutputValue & {
    	type:  "bool"
    	value: bool
    }
    #NumberTypedOutputValue: #OutputValue & {
    	type:  "number"
    	value: number
    }
    #StringTypedOutputValue: #OutputValue & {
    	type:  "string"
    	value: string
    }
    let compositeTypedOutputValue = #OutputValue & {
    	#collectionType: "list" | "map" | "object" | "set" | "tuple"
    	{
    		type: [#collectionType, ...]
    	}
    }
    #ListTypedOutputValue: compositeTypedOutputValue & {
    	#collectionType: "list"
    	value: [...]
    }
    #MapTypedOutputValue: compositeTypedOutputValue & {
    	#collectionType: "map"
    	value: [string]: _
    }
    #ObjectTypedOutputValue: compositeTypedOutputValue & {
    	#collectionType: "object"
    	value: [string]: _
    }
    #SetTypedOutputValue: compositeTypedOutputValue & {
    	#collectionType: "set"
    	value: [...]
    }
    #TupleTypedOutputValue: compositeTypedOutputValue & {
    	#collectionType: "tuple"
    	value: [...]
    }
    
    // Output defines the schema for data emitted by the "terraform output -json" command.
    // THIS IS THE PROBLEM.
    //#Output: [string]: #OutputValue
    // Loosen the schema just to compare the behavior.
    #Output: [string]: _
    
    _#AWSTFOutput: #Output & {
    	let hostedZones = #TupleTypedOutputValue & {
    		value: [...{
    			id:   string // More elaborate in actual code.
    			name: string // More elaborate in actual code.
    		}]
    	}
    
    	acme_dns_challenge_prover_iam_role:   #ObjectTypedOutputValue
    	assume_dns_record_publisher_iam_role: #ObjectTypedOutputValue
    	external_dns_route_53_hosted_zones:   hostedZones
    	iam_users:                            #ObjectTypedOutputValue
    	iam_users: value: [string]: {
    		arn:       string // More elaborate in actual code.
    		unique_id: string // More elaborate in actual code.
    	}
    	route_53_hosted_zones: hostedZones
    }
    
    // Example use of these definitions unified with real input data
    example: _#AWSTFOutput & {
    	// NB: This is a JSON object pasted in, redacted in places.
    	"acme_dns_challenge_prover_iam_role": {
    		"sensitive": false
    		"type": [
    			"object",
    			{
    				"arn":       "string"
    				"name":      "string"
    				"unique_id": "string"
    			},
    		]
    		"value": {
    			"arn":       "arn:aws:iam::123456789012:role/some/path/acme-dns-challenge-prover"
    			"name":      "acme-dns-challenge-prover"
    			"unique_id": "AROAZWV55UJHHXTZOPWHI"
    		}
    	}
    	"assume_dns_record_publisher_iam_role": {
    		"sensitive": false
    		"type": [
    			"object",
    			{
    				"arn":       "string"
    				"name":      "string"
    				"unique_id": "string"
    			},
    		]
    		"value": {
    			"arn":       "arn:aws:iam::123456789012:role/some/path/dns-record-publisher"
    			"name":      "dns-record-publisher"
    			"unique_id": "AROAZWV55UJHO5BPDUMOB"
    		}
    	}
    	"external_dns_route_53_hosted_zones": {
    		"sensitive": false
    		"type": [
    			"tuple",
    			[
    				[
    					"object",
    					{
    						"id":   "string"
    						"name": "string"
    					},
    				],
    				[
    					"object",
    					{
    						"id":   "string"
    						"name": "string"
    					},
    				],
    			],
    		]
    		"value": [
    			{
    				"id":   "ZBIUQ9J8XHK3M"
    				"name": "example.com"
    			},
    			{
    				"id":   "Z08962273OP4H9PT589NQ"
    				"name": "example-1.com"
    			},
    		]
    	}
    	"iam_users": {
    		"sensitive": false
    		"type": [
    			"object",
    			{
    				"ExternalDNS": [
    					"object",
    					{
    						"arn":       "string"
    						"unique_id": "string"
    					},
    				]
    				"cert-manager": [
    					"object",
    					{
    						"arn":       "string"
    						"unique_id": "string"
    					},
    				]
    			},
    		]
    		"value": {
    			"daExternalDNS": {
    				"arn":       "arn:aws:iam::123456789012:user/some/path/ExternalDNS"
    				"unique_id": "AIDAZWV55UJ0123456789"
    			}
    			"dacert-manager": {
    				"arn":       "arn:aws:iam::123456789012:user/some/path/cert-manager"
    				"unique_id": "AIDAZWV55UJ0123456789"
    			}
    		}
    	}
    	"route_53_hosted_zones": {
    		"sensitive": false
    		"type": [
    			"tuple",
    			[
    				[
    					"object",
    					{
    						"id":   "string"
    						"name": "string"
    					},
    				],
    			],
    		]
    		"value": [
    			{
    				"id":   "ZBI0123456789"
    				"name": "example.com"
    			},
    		]
    	}
    }
    

    Note the comment in the source that reads "THIS IS THE PROBLEM." If I uncomment the line after it to read as follows, cue export gets stuck:

    #Output: [string]: #OutputValue
    

    The same problem arises if I write the following:

    _#AWSTFOutput: [string]: #OutputValue
    

    That is, I'd like to express that _#AWSTFOutput is a JSON document produced by terraform output -json with a particular set of output values, such that each field in the JSON object must bear a value that conforms to #OutputValue.

    Relaxing that pattern constraint to the following allows cue export to proceed:

    #Output: [string]: _
    

    However, that fails to satisfy my goal of describing a valid JSON object, as these values could be of any type.

    What did you expect to see?

    cue export should emit the JSON document as it was, very quickly, without complaining about any schema violations.

    What did you see instead?

    cue export consumes several CPUs for as long as I'll allow it to run, while emitting no output.

    cycle disjunctions 
    opened by seh 4
  • Error importing gitlab json schema

    Error importing gitlab json schema

    What version of CUE are you using (cue version)?

    $ cue version
    cue version v0.4.3 darwin/arm64
    

    Does this issue reproduce with the latest release?

    Yep

    What did you do?

    curl -JLO https://gitlab.com/gitlab-org/gitlab/-/raw/master/app/assets/javascripts/editor/schema/ci.json
    cue import -f -l "#CISpec:" -p gitlab ci.json
    

    Permalink to latest commit: https://gitlab.com/gitlab-org/gitlab/-/blob/7cac80c74d52cce38faa8a0068305a370a417ebd/app/assets/javascripts/editor/schema/ci.json

    What did you expect to see?

    The schema converted to CUE

    What did you see instead?

    constraint not allowed because type string is excluded:
        ./ci.json:116:33
    constraint not allowed because type string is excluded:
        ./ci.json:703:25
    

    Lines 116 and 703 for reference

    NeedsInvestigation Triage 
    opened by benmoss 0
  • cmd/cue: cue fmt silently fails to work when there are missing imports

    cmd/cue: cue fmt silently fails to work when there are missing imports

    What version of CUE are you using (cue version)?

    $ cue version
    cue version v0.0.0-20220915174651-ad253ed099e9
    
           -compiler gc
         CGO_ENABLED 1
              GOARCH amd64
                GOOS linux
             GOAMD64 v1
                 vcs git
        vcs.revision ad253ed099e94ddcffd07e6f73a76735a96cfaf8
            vcs.time 2022-09-15T17:46:51Z
        vcs.modified false
    
    

    Does this issue reproduce with the latest release?

    No, the error isn't silent under v0.4.3.

    What did you do?

    I ran this testscript file:

    exec cue fmt x1/x1.cue x2/x2.cue
    cmp x1/x1.cue expectx1.cue
    cmp x2/x2.cue expectx2.cue
    
    -- x1/x1.cue --
    package x
    
    import _ "github.com/foo/bar"
    -- x2/x2.cue --
    package y
    
    	needsFormat: int
    
    -- expectx1.cue --
    package x
    
    import _ "github.com/foo/bar"
    -- expectx2.cue --
    package y
    
    needsFormat: int
    

    What did you expect to see?

    A passing test.

    What did you see instead?

    > exec cue fmt x1/x1.cue x2/x2.cue
    > cmp x1/x1.cue expectx1.cue
    > cmp x2/x2.cue expectx2.cue
    --- x2/x2.cue
    +++ expectx2.cue
    @@ -1,4 +1,3 @@
     package y
     
    -	needsFormat: int
    -
    +needsFormat: int
    
    FAIL: /tmp/testscript1726192439/x.txtar/script.txtar:3: x2/x2.cue and expectx2.cue differ
    

    That is, the cue fmt command appears to succeed but doesn't actually format all the files.

    Under v0.4.3 I see this instead:

    found packages "x" (x1.cue) and y (x2.cue) in "$WORK"
    import failed: cannot find package "github.com/foo/bar":
        ./x1/x1.cue:3:8
    

    I would argue that both these failures are spurious. If I give cue fmt an explicit list of CUE files, I think it should probably format them independently and regardless of unresolved imports.

    NeedsInvestigation Triage 
    opened by rogpeppe 0
  • evaluator: comprehension cycle reported where previously this was allowed

    evaluator: comprehension cycle reported where previously this was allowed

    What version of CUE are you using (cue version)?

    $ cue version
    cue version v0.0.0-20220915134846-df148ee6b613
    
           -compiler gc
         CGO_ENABLED 1
              GOARCH arm64
                GOOS linux
                 vcs git
        vcs.revision df148ee6b6133e0466f7964eec027f2e0349cdbe
            vcs.time 2022-09-15T13:48:46Z
        vcs.modified true
    

    Does this issue reproduce with the latest release?

    Yes

    What did you do?

    exec cue export x.cue
    cmp stdout stdout.golden
    
    -- x.cue --
    package preguide
    
    #Guide: {
    	// Scenarios defines the set of scenarios this guide should cover.
    	Scenarios: [name=string]: #Scenario & {
    		Name: name
    	}
    
    	// Terminals declares the required terminals for this guide.
    	Terminals: [name=string]: #Terminal & {
    		Name: name
    	}
    
    	// This for loop templates to ensure that terminal declares an
    	// image for every scenario.
    	for scenario, _ in Scenarios for terminal, _ in Terminals {
    		Terminals: "\(terminal)": Scenarios: "\(scenario)": #TerminalScenario
    	}
    
    }
    
    #Terminal: {
    	Name: string
    	Scenarios: [string]: #TerminalScenario
    }
    
    #TerminalScenario: {
    	Image: string
    }
    
    #Scenario: {
    	Name:        string
    	Description: string
    }
    
    g1: #Guide & {
    	Terminals: term1: #Terminal & {
    		Scenarios: go115: Image: "this_will_never_be_used"
    	}
    
    	Scenarios: go115: {
    		Description: "Go 1.15"
    	}
    }
    -- stdout.golden --
    {
        "g1": {
            "Scenarios": {
                "go115": {
                    "Name": "go115",
                    "Description": "Go 1.15"
                }
            },
            "Terminals": {
                "term1": {
                    "Name": "term1",
                    "Scenarios": {
                        "go115": {
                            "Image": "this_will_never_be_used"
                        }
                    }
                }
            }
        }
    }
    

    What did you expect to see?

    Passing test. This passes with v0.4.1.

    What did you see instead?

    With tip (df148ee6):

    > exec cue export x.cue
    [stderr]
    cycle error:
        ./x.cue:16:50
    [exit status 1]
    FAIL: /tmp/testscript1377975653/repro.txtar/script.txtar:1: unexpected command failure
    

    What's interesting is that this got broken in be2ee9b62d2d9326fe025bed40215d7b5104a662. However the output against that commit is different to tip:

    > exec cue export x.cue
    [stderr]
    g1: cannot add field Terminals: was already used:
        ./x.cue:17:3
    [exit status 1]
    FAIL: /tmp/testscript2841319868/repro.txtar/script.txtar:1: unexpected command failure
    
    NeedsInvestigation cycle comprehensions 
    opened by myitcv 3
Releases(v0.4.3)
  • v0.4.3(Apr 21, 2022)

    This release includes various bug fixes, including fixing some crashes. It also fixes a memory leak that would cause long-running servers to OOM (see #1418). It also now supports Go 1.18.

    It also fixes various builtin extensions.

    This release also makes a lot of preparations for the upcoming comprehension rework, which will allow us to do all kinds improvements and performance enhancements.

    As a reminder: users can register their projects with unity, our regression and performance testing setup. unity is used to ensure that a project's CUE evaluations do not unexpectedly stop working, or regress in terms of performance. unity continues to catch multiple issues with each release. Adding your project to unity not only guarantees that we will not break your tests (if we do, we will work with you to fix your CUE code), but it also helps to improve the quality of each CUE release. We are in the process of adding support for adding private projects to unity.

    Thank you to @benmoss, @cuichenli, @danbison, @emcfarlane, @eonpatapon, @jared-gs, @jdreaver, @jlongtine, @mattmoor, @mvdan, @oncilla, @roidelapluie, @slewiskelly, @xinau, and @yannk for contributing to this release!

    Language changes

    This release removes all remaining support for quoted identifiers. Although this hasn't been supported for a while, there were still parts of the code that handled them, resulting in cryptic error messages.

    Core evaluator

    This release sees an overall improvement in error location reporting. These changes are quite substantial but largely, aside from a few resulting bug fixes, do not alter behavior.

    Builtins

    net

    Add CIDR range verification function.

    regexp

    Add ReplaceAll and ReplaceAllLiteral

    time

    Fixes a hermeticity issues where a local time zone may be selected. Added FormatString to ultimately replace Format, where Format will first have to be deprecated.

    Add FormatString, FormatDuration and Split

    tool/file

    Add MkdirTemp, RemoveAll

    tool/http

    Add Support for TLS settings.

    Changelog

    • 5c43a87a all: apply Go 1.18's gofmt -s
    • 41ad3a08 all: bump x/text
    • e74624be all: drop golang.org/x/xerrors
    • b6cdc743 all: format .cue files in txtar archives
    • 054cc312 all: move to go1.17-based go.mod
    • f396ea88 all: upgrade github.com/spf13/cobra to v1.4.0
    • f26cc29d all: upgrade go-internals
    • 83a17005 build: fix typo inclduded -> included
    • dc2c9e09 ci: add Go 1.18 and bump action and Go versions
    • f44242c2 ci: optimize CI build runtimes
    • 256f1f9e ci: update to the latest goreleaser version
    • 935a926c cmd/cue/cmd: make get go work with go 1.18
    • 37cc8ac3 cmd/cue/cmd: support byte in get go
    • 5127136c cmd/cue/cmd: support uintptr in get go
    • 69e2dcc8 cmd/cue: make long tests pass with Go 1.18
    • aca00a06 cmd/cue: thread cue.Context throughout calls
    • 583c11e6 cmd/import: fix typo in doc
    • 89d6078f crypto/ed25519: support signature verification
    • 6b138e45 cue/load: remove some unused code
    • 41364815 cue/tools/flow: allow tasks in hidden fields
    • 699ce659 cue: allow incomplete errors in Fields method
    • 71b43a8a cue: fix doc string for CompileBytes and references
    • 84d3cad2 cue: fix IsClosed
    • 24c91170 cue: fix TextUnmarshal bug in Decode
    • b1edc14d cue: fix nil pointer deref in BuildInstances
    • a7eda136 cue: remove all support for quoted identifiers
    • 49e295bc cue: try harder to find a position for Pos
    • 2eb7d1ef doc/ref/spec.md: fix hello world example in spec
    • 1c52d6a7 doc/spec: specify default constraint is not implemented
    • ba164f52 doc: delete whitespace in README to test master build
    • a6995f5e doc: fix "click" typo
    • eaaf5f02 doc: fix up k8s tutorial
    • 7627c936 doc: mention the commit message hook for --signoff
    • 20202083 doc: remove duplicate GerritHub clone instructions
    • b67dc4db interal/core/adt: improve error locations
    • fa141c28 internal/cmd/qgo: make go 1.18 compatibility
    • 569fcbd3 internal/core/adt: add nested logging support
    • 09c1ee0b internal/core/adt: adjust error positions
    • 0a4d48d4 internal/core/adt: change to incomplete error
    • b995f5b2 internal/core/adt: debug options for sorting arcs
    • 448b94c2 internal/core/adt: don't yield struct on comprehension
    • 1e45d060 internal/core/adt: fix closedness for API disjunctions
    • 6b37a01f internal/core/adt: get rid of NotExistError
    • e70db24a internal/core/adt: hoist code to prep for comprehension change
    • 8f3c71b3 internal/core/adt: introduce Comprehension type
    • f045a5a9 internal/core/adt: make comprehension value an Expr
    • b2d9e16c internal/core/adt: merge if and for comprehension lists
    • e499daed internal/core/adt: pass Conjunct to addVertexConjuncts
    • 28f8d476 internal/core/adt: performance: improve lists disambiguation
    • 49fdab87 internal/core/adt: prepare for field conflict message
    • e0577bbd internal/core/adt: prevent state erasure
    • be2ee9b6 internal/core/adt: prevent state inversion
    • 7cdcc70b internal/core/adt: reintroduce Conjunct.Expr
    • 0e634cf0 internal/core/adt: report error for integer fields
    • 975f57c3 internal/core/adt: store Ellipsis instead of Expr in Additional
    • 2421553a internal/core/adt: track positions for incomplete types
    • 8dee602c internal/core/adt: use partial lookup
    • 19782a63 internal/core/dep: prevent crash
    • 195cdf7e internal/core/runtime: don't share import cache
    • c561f1fa internal/core: automated rename
    • 28c1adb1 internal/cuetxar: provide CUE_FORMAT_TXTAR to format .cue files
    • 4cbd2e6c pkg/crypto/ed25519: format generated test case
    • 189de92d pkg/net: add CIDR range verification function
    • 37eb22b4 pkg/regexp: add ReplaceAll and ReplaceAllLiteral
    • 53a108d6 pkg/regexp: bring docs more inline with Go equivalents
    • fe201325 pkg/regexp: delete duplicate test case in txtar
    • 08fa1c72 pkg/regexp: reorder for ease of maintenance
    • a15720bc pkg/time: add FormatDuration
    • 71e9d033 pkg/time: add FormatString and Split
    • 65ff0f46 pkg/time: fix hermeticity issue
    • e32817ad pkg/time: use time.UTC for ParseInLocation
    • 977d3532 pkg/tool/cli: Fix Ask task
    • 8dee5496 pkg/tool/file: add MkdirTemp, RemoveAll
    • da75cdf3 pkg/tool/http: add tls settings
    • ffb044f3 pkg/tool/http: check tls.verify exists
    • 05f74a89 pkg: remove usage of qgo in go generate
    • 04ac6664 tools/flow: add Value() to retrieve controller value
    • a03817fc tools/flow: fix test race
    • 1b4b7e97 tools/trim: appropriately label vertex
    • 8a2e8322 tools/trim: don't remove structs with comprehensions

    Full Changelog: https://github.com/cue-lang/cue/compare/v0.4.2...v0.4.3

    Docker images

    • docker pull docker.io/cuelang/cue:0.4.3
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(576 bytes)
    cue_v0.4.3_darwin_amd64.tar.gz(5.95 MB)
    cue_v0.4.3_darwin_arm64.tar.gz(5.71 MB)
    cue_v0.4.3_linux_amd64.tar.gz(5.75 MB)
    cue_v0.4.3_linux_arm64.tar.gz(5.29 MB)
    cue_v0.4.3_windows_amd64.zip(5.84 MB)
    cue_v0.4.3_windows_arm64.zip(5.38 MB)
  • v0.4.3-beta.2(Apr 5, 2022)

    This release fixes a couple of bugs and slight incompatibilities introduced with the last release and adds various documentation fixes.

    Changelog

    • 41ad3a08 all: bump x/text
    • f396ea88 all: upgrade github.com/spf13/cobra to v1.4.0
    • 83a17005 build: fix typo inclduded -> included
    • aca00a06 cmd/cue: thread cue.Context throughout calls
    • 583c11e6 cmd/import: fix typo in doc
    • 2eb7d1ef doc/ref/spec.md: fix hello world example in spec
    • ba164f52 doc: delete whitespace in README to test master build
    • ffb044f3 pkg/tool/http: check tls.verify exists
    • a03817fc tools/flow: fix test race

    Docker images

    • docker pull docker.io/cuelang/cue:0.4.3-beta.2
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(618 bytes)
    cue_v0.4.3-beta.2_darwin_amd64.tar.gz(5.62 MB)
    cue_v0.4.3-beta.2_darwin_arm64.tar.gz(5.53 MB)
    cue_v0.4.3-beta.2_linux_amd64.tar.gz(5.42 MB)
    cue_v0.4.3-beta.2_linux_arm64.tar.gz(5.01 MB)
    cue_v0.4.3-beta.2_windows_amd64.zip(5.55 MB)
    cue_v0.4.3-beta.2_windows_arm64.zip(5.13 MB)
  • v0.4.3-beta.1(Mar 17, 2022)

    This release includes various bug fixes, including fixing some crashes. It also fixes a memory leak that would cause long-running servers to OOM (see #1418).

    It also fixes various builtin extensions.

    This release also makes a lot of preparations for the upcoming comprehension rework, which will allow us to do all kinds improvements and performance enhancements.

    As a reminder: users can register their projects with unity, our regression and performance testing setup. unity is used to ensure that a project's CUE evaluations do not unexpectedly stop working, or regress in terms of performance. unity continues to catch multiple issues with each release. Adding your project to unity not only guarantees that we will not break your tests (if we do, we will work with you to fix your CUE code), but it also helps to improve the quality of each CUE release. We are in the process of adding support for adding private projects to unity.

    Language changes

    This release removes all remaining support for quoted identifiers. Although this hasn't been supported for a while, there were still parts of the code that handled them, resulting in cryptic error messages.

    Core evaluator

    This release sees an overall improvement in error location reporting. These changes are quite substantial but largely, aside from a few resulting bug fixes, do not alter behavior.

    Builtins

    net

    Add CIDR range verification function.

    regexp

    Add ReplaceAll and ReplaceAllLiteral

    time

    Fixes a hermeticity issues where a local time zone may be selected. Added FormatString to ultimately replace Format, where Format will first have to be deprecated.

    Add FormatString, FormatDuration and Split

    tool/file

    Add MkdirTemp, RemoveAll

    tool/http

    Add Support for TLS settings.

    Full Changelog: https://github.com/cue-lang/cue/compare/v0.4.2...v0.4.3-beta.1

    Changelog

    • 71b43a8a Fix doc string for CompileBytes and references
    • b6cdc743 all: format .cue files in txtar archives
    • 054cc312 all: move to go1.17-based go.mod
    • f44242c2 ci: optimize CI build runtimes
    • 89d6078f crypto/ed25519: support signature verification
    • 41364815 cue/tools/flow: allow tasks in hidden fields
    • 699ce659 cue: allow incomplete errors in Fields method
    • 84d3cad2 cue: fix IsClosed
    • 24c91170 cue: fix TextUnmarshal bug in Decode
    • b1edc14d cue: fix nil pointer deref in BuildInstances
    • a7eda136 cue: remove all support for quoted identifiers
    • 49e295bc cue: try harder to find a position for Pos
    • 1c52d6a7 doc/spec: specify default constraint is not implemented
    • a6995f5e doc: fix "click" typo
    • eaaf5f02 doc: fix up k8s tutorial
    • 7627c936 doc: mention the commit message hook for --signoff
    • 20202083 doc: remove duplicate GerritHub clone instructions
    • b67dc4db interal/core/adt: improve error locations
    • 569fcbd3 internal/core/adt: add nested logging support
    • 09c1ee0b internal/core/adt: adjust error positions
    • 0a4d48d4 internal/core/adt: change to incomplete error
    • b995f5b2 internal/core/adt: debug options for sorting arcs
    • 448b94c2 internal/core/adt: don't yield struct on comprehension
    • 1e45d060 internal/core/adt: fix closedness for API disjunctions
    • 6b37a01f internal/core/adt: get rid of NotExistError
    • e70db24a internal/core/adt: hoist code to prep for comprehension change
    • 8f3c71b3 internal/core/adt: introduce Comprehension type
    • f045a5a9 internal/core/adt: make comprehension value an Expr
    • b2d9e16c internal/core/adt: merge if and for comprehension lists
    • e499daed internal/core/adt: pass Conjunct to addVertexConjuncts
    • 28f8d476 internal/core/adt: performance: improve lists disambiguation
    • 49fdab87 internal/core/adt: prepare for field conflict message
    • e0577bbd internal/core/adt: prevent state erasure
    • be2ee9b6 internal/core/adt: prevent state inversion
    • 7cdcc70b internal/core/adt: reintroduce Conjunct.Expr
    • 0e634cf0 internal/core/adt: report error for integer fields
    • 975f57c3 internal/core/adt: store Ellipsis instead of Expr in Additional
    • 2421553a internal/core/adt: track positions for incomplete types
    • 8dee602c internal/core/adt: use partial lookup
    • 19782a63 internal/core/dep: prevent crash
    • 195cdf7e internal/core/runtime: don't share import cache
    • c561f1fa internal/core: automated rename
    • 28c1adb1 internal/cuetxar: provide CUE_FORMAT_TXTAR to format .cue files
    • 4cbd2e6c pkg/crypto/ed25519: format generated test case
    • 189de92d pkg/net: add CIDR range verification function
    • 37eb22b4 pkg/regexp: add ReplaceAll and ReplaceAllLiteral
    • 53a108d6 pkg/regexp: bring docs more inline with Go equivalents
    • fe201325 pkg/regexp: delete duplicate test case in txtar
    • 08fa1c72 pkg/regexp: reorder for ease of maintenance
    • a15720bc pkg/time: add FormatDuration
    • 71e9d033 pkg/time: add FormatString and Split
    • 65ff0f46 pkg/time: fix hermeticity issue
    • e32817ad pkg/time: use time.UTC for ParseInLocation
    • 977d3532 pkg/tool/cli: Fix Ask task
    • 8dee5496 pkg/tool/file: add MkdirTemp, RemoveAll
    • da75cdf3 pkg/tool/http: add tls settings
    • 04ac6664 tools/flow: add Value() to retrieve controller value
    • 1b4b7e97 tools/trim: appropriately label vertex
    • 8a2e8322 tools/trim: don't remove structs with comprehensions

    Docker images

    • docker pull docker.io/cuelang/cue:0.4.3-beta.1
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(618 bytes)
    cue_v0.4.3-beta.1_darwin_amd64.tar.gz(5.66 MB)
    cue_v0.4.3-beta.1_darwin_arm64.tar.gz(5.57 MB)
    cue_v0.4.3-beta.1_linux_amd64.tar.gz(5.46 MB)
    cue_v0.4.3-beta.1_linux_arm64.tar.gz(5.05 MB)
    cue_v0.4.3-beta.1_windows_amd64.zip(5.59 MB)
    cue_v0.4.3-beta.1_windows_arm64.zip(5.17 MB)
  • v0.4.2(Feb 4, 2022)

    This patch release largely focuses on bug fixes, but also includes some additions to the cmd/cue cmd tooling layer and builtins. Some of the bug fixes were identified as part of preparing for an upcoming CUE talk at FOSDEM.

    Thank you to @brandonbloom, @eonpatapon, @obowersa and @seh for contributing to this release!

    Core evaluator

    A number of bug fixes relating to edge cases of CUE evaluation. Notably, 1374dc99007937166e204302792df34c46138df0 fixes some subtle bugs where fields can be missed if patterns are inserted too late.

    0aaf4c69be6394bcb20c0a659382f48b9cffec33 disallows _ as a field label, fixing a bug where the following CUE was previously accepted as valid:

    _: 5
    

    This now results in:

    cannot use _ as label:
        ./x.cue:1:1
    

    Tooling layer (cmd/cue cmd)

    @eonpatapon added Mkdir, MkdirAll to pkg/tool/file. https://github.com/cue-lang/cue/issues/1502 was raised as a follow-up, a wider review of how the pkg/… API needs to change pre v1.

    Spec

    @seh contributed to the "Declarations and scope" section, clarifying that top-level let identifiers are scoped to the containing file block.

    cmd/cue

    @slewiskelly highlighted a regression introduced as far back as aaf6e846d8450663dd4fd265021a969d02a2bf4e where cmd/cue is used to validate data using a schema. In this regression, CUE inadvertently marks the failures as "fatal", instead of "incomplete" (that is, could be satisfied by making a value more concrete). 04812bfb515de3300ac21ca181542f8ea02c9ea1 disables schema checking for now whilst we consider a more precise solution.

    Builtins

    Both pkg/encoding/yaml and pkg/encoding/json now support UnmarshalStream.

    Changelog

    • 6bc922c8 tools/trim: prevent nil-pointer panic
    • 9aeaf704 pkg/encoding/json: implement UnmarshalStream
    • 480b28b1 pkg/encoding/yaml: implement UnmarshalStream
    • 880863af encoing/json: made independent of pkg/encoding/json
    • 802a8520 cmd/cue/cmd: only consider "kind" field for known tasks
    • bd3b6eaf cmd/cue: add newline between imported objects
    • 04812bfb cmd/cue: fix spurious errors
    • 2fe5251e doc/ref/spec.md: clarify scope of top-level let ID
    • 556f57f5 doc/tutorial: fix typo in 55_defs.txt
    • f0d03f37 cue/ast: fix typo in docstring
    • f29b460e cmd/cue/cmd: remove stale reference to ::
    • 50d69c90 all: prepare for private repository support in unity
    • 5581d643 doc: fix heading level for div et al builtins
    • 45c54f85 pkg/tool/file: add Mkdir, MkdirAll
    • 34c4f9cc Revert "pkg/tool/file: add Mkdir, MkdirAll"
    • 23cc102d pkg/tool/file: add Mkdir, MkdirAll
    • 0aaf4c69 internal/core: disallow _ as field label
    • c1bf5504 internal/core/eval: dereference indirections earlier
    • 9f6a40ef internal/core/adt: zero-value feature means "invalid"
    • 1374dc99 internal/core/adt: single place for inserting matched patterns

    Full Changelog: https://github.com/cue-lang/cue/compare/v0.4.1...v0.4.2

    Docker images

    • docker pull docker.io/cuelang/cue:0.4.2
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(576 bytes)
    cue_v0.4.2_darwin_amd64.tar.gz(5.65 MB)
    cue_v0.4.2_darwin_arm64.tar.gz(5.56 MB)
    cue_v0.4.2_linux_amd64.tar.gz(5.45 MB)
    cue_v0.4.2_linux_arm64.tar.gz(5.04 MB)
    cue_v0.4.2_windows_amd64.zip(5.58 MB)
    cue_v0.4.2_windows_arm64.zip(5.16 MB)
  • v0.4.2-rc.1(Feb 1, 2022)

    This patch release largely focuses on bug fixes, but also includes some additions to the cmd/cue cmd tooling layer and builtins. Some of the bug fixes were identified as part of preparing for an upcoming CUE talk at FOSDEM.

    Thank you to @brandonbloom, @eonpatapon, @obowersa and @seh for contributing to this release!

    Core evaluator

    A number of bug fixes relating to edge cases of CUE evaluation. Notably, 1374dc99007937166e204302792df34c46138df0 fixes some subtle bugs where fields can be missed if patterns are inserted too late.

    0aaf4c69be6394bcb20c0a659382f48b9cffec33 disallows _ as a field label, fixing a bug where the following CUE was previously accepted as valid:

    _: 5
    

    This now results in:

    cannot use _ as label:
        ./x.cue:1:1
    

    Tooling layer (cmd/cue cmd)

    @eonpatapon added Mkdir, MkdirAll to pkg/tool/file. https://github.com/cue-lang/cue/issues/1502 was raised as a follow-up, a wider review of how the pkg/… API needs to change pre v1.

    Spec

    @seh contributed to the "Declarations and scope" section, clarifying that top-level let identifiers are scoped to the containing file block.

    cmd/cue

    @slewiskelly highlighted a regression introduced as far back as aaf6e846d8450663dd4fd265021a969d02a2bf4e where cmd/cue is used to validate data using a schema. In this regression, CUE inadvertently marks the failures as "fatal", instead of "incomplete" (that is, could be satisfied by making a value more concrete). 04812bfb515de3300ac21ca181542f8ea02c9ea1 disables schema checking for now whilst we consider a more precise solution.

    Builtins

    Both pkg/encoding/yaml and pkg/encoding/json now support UnmarshalStream.

    Changelog

    • 6bc922c8 tools/trim: prevent nil-pointer panic
    • 9aeaf704 pkg/encoding/json: implement UnmarshalStream
    • 480b28b1 pkg/encoding/yaml: implement UnmarshalStream
    • 880863af encoing/json: made independent of pkg/encoding/json
    • 802a8520 cmd/cue/cmd: only consider "kind" field for known tasks
    • bd3b6eaf cmd/cue: add newline between imported objects
    • 04812bfb cmd/cue: fix spurious errors
    • 2fe5251e doc/ref/spec.md: clarify scope of top-level let ID
    • 556f57f5 doc/tutorial: fix typo in 55_defs.txt
    • f0d03f37 cue/ast: fix typo in docstring
    • f29b460e cmd/cue/cmd: remove stale reference to ::
    • 50d69c90 all: prepare for private repository support in unity
    • 5581d643 doc: fix heading level for div et al builtins
    • 45c54f85 pkg/tool/file: add Mkdir, MkdirAll
    • 34c4f9cc Revert "pkg/tool/file: add Mkdir, MkdirAll"
    • 23cc102d pkg/tool/file: add Mkdir, MkdirAll
    • 0aaf4c69 internal/core: disallow _ as field label
    • c1bf5504 internal/core/eval: dereference indirections earlier
    • 9f6a40ef internal/core/adt: zero-value feature means "invalid"
    • 1374dc99 internal/core/adt: single place for inserting matched patterns

    Full Changelog: https://github.com/cue-lang/cue/compare/v0.4.1...v0.4.2-rc.1

    Docker images

    • docker pull docker.io/cuelang/cue:0.4.2-rc.1
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(606 bytes)
    cue_v0.4.2-rc.1_darwin_amd64.tar.gz(5.65 MB)
    cue_v0.4.2-rc.1_darwin_arm64.tar.gz(5.56 MB)
    cue_v0.4.2-rc.1_linux_amd64.tar.gz(5.45 MB)
    cue_v0.4.2-rc.1_linux_arm64.tar.gz(5.04 MB)
    cue_v0.4.2-rc.1_windows_amd64.zip(5.58 MB)
    cue_v0.4.2-rc.1_windows_arm64.zip(5.16 MB)
  • v0.4.1(Jan 12, 2022)

    After an extended break following CUE's migration to cue-lang/cue and multiple pre-releases, the v0.4.1 release mainly focuses on bug fixes. We also renew our pledge to in future release smaller changes more frequently on our way to language stability.

    Thank you to @chai2010, @yujunz, @caarlos0, @hrvolapeter, @slewiskelly, @antong, @mvdan, @ameowlia and @palmamartin for contributing to this release!

    Project-level changes

    • Create amd64 and arm64 release assets for linux, windows and darwin, with accompanying amd64 and arm64 Docker images and manifests: v0.4.1-beta.6
    • Remove go1.15 from build matrix: v0.4.1-beta.6
    • Improve contribution project contribution guide: v0.4.1-beta.6

    Encoders

    cmd/cue

    • Support working directory in tool/exec.Run: v0.4.1-beta.6
    • Numerous bug fixes relating to cue eval exporting of let expressions (#1116): v0.4.1-beta.6
    • Fix for cue def bug where multiple list conjuncts resulted in an output containing a {} conjunct: v0.4.1-beta.6

    API

    Evaluator

    • Fix a number of issues related to the use of aliases in lists by introducing a new scope with list values: v0.4.1-beta.6
    • Fix for issues related to defaults and string and number lists (#1025, #1404): v0.4.1-beta.6
    • Fix for a bug where comprehensions inside a definition resulted in an error (#1404): v0.4.1-beta.6
    • Important fix for bug related to the handling of default values that result in error (#1304 and #1257): v0.4.1-beta.6
    • Various unexpected panics have been fixed, including a panic in the case of circular let declarations (#1042): v0.4.1-beta.6
    • Fix matching of "_" label: v0.4.1-rc.2

    Language

    This release makes no changes to the language.

    Changelog

    • internal/core/runtime: check non-existing builtin packages earlier d4d0f142
    • cue: remove duplicate test code 8fbaf81e
    • internal/core/adt: use single return for LoadImport e5c71761
    • internal/core/adt: fix matching of "_" label 3d3f721d
    • readme: fix Go version badge 60d6503d
    • cue/cuecontext: allow concurrent use 25cfb208
    • gitiles: change logo from SVG to PNG 8a40ef07
    • gitiles: customize navigation bar 28b42576
    • doc: ensure all min Go versions refer to 1.16 79303b6d
    • ci: configure GoReleaser to mark prereleases as such 547e5cc2
    • ci: only trigger new version events after successful new version 6e68ca4c
    • ci: fix GoReleaser arm64 template details 795719e3
    • ci: fix target for homebrew tap b68f4c3c
    • ci: fix up GoReleaser image template names 02556bc2
    • ci: use GoReleaser v1.0.0 e7da62fa
    • cue: support ResolveReferences f68e000e
    • internal/core/export: keep open lists open d413bb83
    • internal/core/export: fix let exporting 9a040dcf
    • pkg: fix default issue for string and numbers lists 910ff4d5
    • internal/core/adt: fix regression of dynamic closed fields in definitions 6c306785
    • internal/core/export: treat empty data vertices as structs 9e7d4d63
    • cmd/cue: confirm issues 1152 and 1153 are fixed 4d505e0d
    • cmd/cue: get go: ignore anonymous enums eb53c4e3
    • internal/core/adt: close open lists for Default b99ce0ad
    • cue: auto-unwrap single embedded value in Expr 061e661a
    • pkg/internal: compute combined error severity 7922f9ba
    • internal/core/export: fix bug with API-generated top-level close c490d4cf
    • ci: remove go1.15 from the build matrix e62a95ab
    • cue: fix output in ExampleContext f8804677
    • internal/core/adt: don't cache incomplete errors for let 99825265
    • cue/testdata: hoist let cycle errors 4f3988fa
    • internal/core: use scope for lists 91476ab2
    • ci: correct location of unity 718c9a07
    • internal/core/compile: remove unnecessary code f65dd0f7
    • cue: fix subsumption bug for disjuntion filtering b89ad85f
    • internal/encoding/yaml: upgrade go-yaml.v3 a8b721a8
    • encoding/openapi: fix crash in reference 15adb6c4
    • cmd/cue/cmd: improve shell arg parsing in tests 4e76bb47
    • internal/core/export: don't add empty temp struct 0e56b828
    • doc/ref/spec.md: include $ in identifier 75c3d6b7
    • internal/encoding/yaml: canonicalize non-string map keys c1c2cdaa
    • encoding/yaml: quote strings more aggressively a83c6c6b
    • doc/tutorial/kubernetes: update types b9ac1eab
    • internal/core/adt: fix bound simplification for floats 59bdbb50
    • internal/core/adt: fix disjunction bug 62e876d1
    • cue: Expr handles empty lists to or/and builtins 9bb874c9
    • cue: define all Op constant values explicitly with that type 72936e43
    • simplify install instructions assuming Go 1.16+ 29a686d8
    • doc: revamp contribution guide ahead of contributing live-stream dbf59922
    • ci: update to use go1.17.3 and go1.16.10 9217c4d0
    • doc/ref/spec: fix typos and heading anchors 437920ea
    • cue: define all Kind constant values explicitly with that type fca61f5b
    • spec: Update example with regards to pattern constraints 8031fe7c
    • ci: latest Go versions in build matrix ddd96c66
    • feat: update go build matrix 5cf1aa16
    • doc: fix repeated word typos in spec 35c2f3c5
    • doc: remove useless whitespace in main README e8550b89
    • doc/contrib: update git codereview change to use -s flag 0f53054d
    • cmd/cue: revert version string change ecb17c94
    • ci: improve goreleaser workflow 370fac9b
    • doc/tutorial: fix errant use of list operators 38c8f7d7
    • ci: tidy up encoding of JSON args to curl a4a38ed8
    • ci: re-enable full CI workflows 7d25ee23
    • ci: tidy up quoting of JSON args in workflow commands 59079b1a
    • readme: fix top-level CONTRIBUTING guide 5f28bb49
    • readme: link to CoC from main readme and contributing guide 5ab75b8d
    • general: use cuelang.org/issues as the general issue link 35c4960f
    • github: fix up empty issue template a68df18d
    • github: tidy up issue templates 723a7fe4
    • general: update unity location 3d9cd71b
    • ci: fix Docker workflow 6da50d98
    • all: change references from cuelang/cue to cue-lang/cue 06484a39
    • internal/core/compile: fix let cycle error 20ed9abd
    • ci: fix up new version trigger 0070bb10
    • doc/tutorial/kubernetes: fix invalid user commands aa61ee7f
    • doc/tutorial/kubernetes: update k8s version 67bf363b
    • cmd/cue: fix up get go docs around "enum" types ec427f98
    • Allow setting working directory for exec.Run 8e053d40
    • interanal/core/runtime: remove commented out fields e8de4b1f

    Full Changelog: https://github.com/cue-lang/cue/compare/v0.4.0...v0.4.1

    Docker images

    • docker pull docker.io/cuelang/cue:0.4.1
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(576 bytes)
    cue_v0.4.1_darwin_amd64.tar.gz(5.64 MB)
    cue_v0.4.1_darwin_arm64.tar.gz(5.56 MB)
    cue_v0.4.1_linux_amd64.tar.gz(5.45 MB)
    cue_v0.4.1_linux_arm64.tar.gz(5.04 MB)
    cue_v0.4.1_windows_amd64.zip(5.57 MB)
    cue_v0.4.1_windows_arm64.zip(5.16 MB)
  • v0.4.1-rc.2(Jan 11, 2022)

    Changelog

    • 3d3f721d internal/core/adt: fix matching of "_" label
    • 60d6503d readme: fix Go version badge

    Docker images

    • docker pull docker.io/cuelang/cue:0.4.1-rc.2

    Full Changelog: https://github.com/cue-lang/cue/compare/v0.4.1-rc.1...v0.4.1-rc.2

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(606 bytes)
    cue_v0.4.1-rc.2_darwin_amd64.tar.gz(5.64 MB)
    cue_v0.4.1-rc.2_darwin_arm64.tar.gz(5.56 MB)
    cue_v0.4.1-rc.2_linux_amd64.tar.gz(5.45 MB)
    cue_v0.4.1-rc.2_linux_arm64.tar.gz(5.04 MB)
    cue_v0.4.1-rc.2_windows_amd64.zip(5.57 MB)
    cue_v0.4.1-rc.2_windows_arm64.zip(5.16 MB)
  • v0.4.1-rc.1(Jan 6, 2022)

    Changelog

    • 25cfb208 cue/cuecontext: allow concurrent use
    • 79303b6d doc: ensure all min Go versions refer to 1.16
    • 8a40ef07 gitiles: change logo from SVG to PNG
    • 28b42576 gitiles: customize navigation bar

    Docker images

    • docker pull docker.io/cuelang/cue:0.4.1-rc.1

    Full Changelog: https://github.com/cue-lang/cue/compare/v0.4.1-beta.6...v0.4.1-rc.1

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(606 bytes)
    cue_v0.4.1-rc.1_darwin_amd64.tar.gz(5.64 MB)
    cue_v0.4.1-rc.1_darwin_arm64.tar.gz(5.56 MB)
    cue_v0.4.1-rc.1_linux_amd64.tar.gz(5.45 MB)
    cue_v0.4.1-rc.1_linux_arm64.tar.gz(5.04 MB)
    cue_v0.4.1-rc.1_windows_amd64.zip(5.57 MB)
    cue_v0.4.1-rc.1_windows_arm64.zip(5.16 MB)
  • v0.4.1-beta.6(Dec 2, 2021)

    Bug fixes

    After an extended summer break following CUE's migration to cue-lang/cue, this release mainly focuses on bug fixes.

    Thank you to @mvdan, @ameowlia, @antong, @slewiskelly, @hrvolapeter, @caarlos0, @yujunz, and @chai2010 for contributing to this release!

    Project-level changes

    A number of release-related issues that were missed in the migration from cuelang/cue to cue-lang/cue have been fixed. GoReleaser now correctly marks prereleases as such.

    The CUE project now uses v1 of GoReleaser and as such we produce both amd64 and arm64 release assets for linux, windows and darwin, with accompanying amd64 and arm64 Docker images and manifests. The homebrew-tap recipe has also been updated to avoid an error message reported via https://github.com/cue-lang/homebrew-tap/issues/1.

    As we near the halfway point to go1.18, go1.15 has been removed from the build matrix. The project's policy for previous versions of Go will be formalized in the near future, but is likely to be that each major Go release is supported until there are two newer major releases, with a 3 month grace period following the release of a new major version (3 months being half a 6-month release cycle) before support for the oldest version is dropped. Correspondingly, go install instructions in the README have been updated to drop support for the old-style GO111MODULE form, instead preferring the go install cuelang.org/[email protected]$version form.

    Ahead of a live stream focussing on contributing to the CUE project back in November, the contribution guide was updated to provide more detail around the PR workflow. Further improvements will be made to reflect feedback following the livestream.

    Encoders

    An upgrade to the version of gopkg.in/yaml.v3 used by the CUE project may result in some formatting changes for Yaml output. This new version now also supports better string escaping for YAML v1.1 backwards compatibility support. However, it does not seem to apply these escapes consistently, so for now we added our own implementation (see also #1368 and #1076)

    For instance, some unquoted strings might, when consumed by some old YAML parsers, be interpreted as values other than strings (float 60, bool, date-time...). Now:

    service: "2222:22" 
    

    is consistently exported to Yaml as:

    service: "2222:22" // before RHS was 2222:22
    

    If this more aggressive quoting still proves insufficient, we can consider quoting all string values by default.

    cmd/cue

    Thanks to @chai2010, a working directory can be set for an exec.Run command in the tooling layer:

    import "tool/exec"
    
    command: ls: exec.Run & {
    	cmd: "ls"
    	dir: "/tmp"
    }
    

    cue eval exporting previously resulted in some strange results that involved let expressions (#1116). These bugs have been fixed and corresponding updates made to test expectations in unity. Similarly, a cue def bug where multiple list conjuncts resulted in an output containing a {} conjunct is now fixed (#1284).

    API

    cue.ResolveReferences has been fixed to work for the KubeVela project, although it is anticipated a better long-term solution is required.

    @ameowlia helped fix an issue with a test example for cue.Context.

    Various other bug fixes.

    Evaluator

    91476ab2151490e11c044fa48a95c713cf019b0b fixes a number of issues related to the use of aliases in lists (#1308), by introducing a new scope with list values. A side effect of this change is that value aliases now work with list values:

    y: X = [1, 2, X[0]]   // y: [1, 2, 1]
    

    A long-standing issue with defaults and string and number lists (#1025, #1404) has been fixed such that the following example no longer results in an incomplete-error:

    exec cue export
    
    -- x.cue --
    package x
    
    import "strings"
    
    #something: {
    	image:    string | *"some default"
    	toString: strings.Join([image], "")
    }
    
    x: #something
    

    A bug where comprehensions inside a definition resulted in an error (#1404) has now been fixed, meaning the following is correctly allowed:

    #D: {
    	a: foo: 123
    	b: {for k, v in a {(k): v}}
    }
    

    An important bug related to the handling of default values that result in error has been fixed (#1304 and #1257).

    Various unexpected panics have been fixed, including a panic in the case of circular let declarations (#1042).

    Language

    This release makes no changes to the language.

    The CUE spec now correctly defines $ as an identifier (#1264).

    Changelog

    • 547e5cc2 ci: configure GoReleaser to mark prereleases as such
    • 6e68ca4c ci: only trigger new version events after successful new version
    • 795719e3 ci: fix GoReleaser arm64 template details
    • b68f4c3c ci: fix target for homebrew tap
    • 02556bc2 ci: fix up GoReleaser image template names
    • e7da62fa ci: use GoReleaser v1.0.0
    • f68e000e cue: support ResolveReferences
    • d413bb83 internal/core/export: keep open lists open
    • 9a040dcf internal/core/export: fix let exporting
    • 910ff4d5 pkg: fix default issue for string and numbers lists
    • 6c306785 internal/core/adt: fix regression of dynamic closed fields in definitions
    • 9e7d4d63 internal/core/export: treat empty data vertices as structs
    • 4d505e0d cmd/cue: confirm issues 1152 and 1153 are fixed
    • eb53c4e3 cmd/cue: get go: ignore anonymous enums
    • b99ce0ad internal/core/adt: close open lists for Default
    • 061e661a cue: auto-unwrap single embedded value in Expr
    • 7922f9ba pkg/internal: compute combined error severity
    • c490d4cf internal/core/export: fix bug with API-generated top-level close
    • e62a95ab ci: remove go1.15 from the build matrix
    • f8804677 cue: fix output in ExampleContext
    • 99825265 internal/core/adt: don't cache incomplete errors for let
    • 4f3988fa cue/testdata: hoist let cycle errors
    • 91476ab2 internal/core: use scope for lists
    • 718c9a07 ci: correct location of unity
    • f65dd0f7 internal/core/compile: remove unnecessary code
    • b89ad85f cue: fix subsumption bug for disjuntion filtering
    • a8b721a8 internal/encoding/yaml: upgrade go-yaml.v3
    • 15adb6c4 encoding/openapi: fix crash in reference
    • 4e76bb47 cmd/cue/cmd: improve shell arg parsing in tests
    • 0e56b828 internal/core/export: don't add empty temp struct
    • 75c3d6b7 doc/ref/spec.md: include $ in identifier
    • c1c2cdaa internal/encoding/yaml: canonicalize non-string map keys
    • a83c6c6b encoding/yaml: quote strings more aggressively
    • b9ac1eab doc/tutorial/kubernetes: update types
    • 59bdbb50 internal/core/adt: fix bound simplification for floats
    • 62e876d1 internal/core/adt: fix disjunction bug
    • 9bb874c9 cue: Expr handles empty lists to or/and builtins
    • 72936e43 cue: define all Op constant values explicitly with that type
    • 29a686d8 simplify install instructions assuming Go 1.16+
    • dbf59922 doc: revamp contribution guide ahead of contributing live-stream
    • 9217c4d0 ci: update to use go1.17.3 and go1.16.10
    • 437920ea doc/ref/spec: fix typos and heading anchors
    • fca61f5b cue: define all Kind constant values explicitly with that type
    • 8031fe7c spec: Update example with regards to pattern constraints
    • ddd96c66 ci: latest Go versions in build matrix
    • 5cf1aa16 feat: update go build matrix
    • 35c2f3c5 doc: fix repeated word typos in spec
    • e8550b89 doc: remove useless whitespace in main README
    • 0f53054d doc/contrib: update git codereview change to use -s flag
    • ecb17c94 cmd/cue: revert version string change
    • 370fac9b ci: improve goreleaser workflow
    • 38c8f7d7 doc/tutorial: fix errant use of list operators
    • a4a38ed8 ci: tidy up encoding of JSON args to curl
    • 7d25ee23 ci: re-enable full CI workflows
    • 59079b1a ci: tidy up quoting of JSON args in workflow commands
    • 5f28bb49 readme: fix top-level CONTRIBUTING guide
    • 5ab75b8d readme: link to CoC from main readme and contributing guide
    • 35c4960f general: use cuelang.org/issues as the general issue link
    • a68df18d github: fix up empty issue template
    • 723a7fe4 github: tidy up issue templates
    • 3d9cd71b general: update unity location
    • 6da50d98 ci: fix Docker workflow
    • 06484a39 all: change references from cuelang/cue to cue-lang/cue
    • 20ed9abd internal/core/compile: fix let cycle error
    • 0070bb10 ci: fix up new version trigger
    • aa61ee7f doc/tutorial/kubernetes: fix invalid user commands
    • 67bf363b doc/tutorial/kubernetes: update k8s version
    • ec427f98 cmd/cue: fix up get go docs around "enum" types
    • 8e053d40 Allow setting working directory for exec.Run
    • e8de4b1f interanal/core/runtime: remove commented out fields

    Docker images

    • docker pull docker.io/cuelang/cue:0.4.1-beta.6
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(618 bytes)
    cue_v0.4.1-beta.6_darwin_amd64.tar.gz(5.64 MB)
    cue_v0.4.1-beta.6_darwin_arm64.tar.gz(5.56 MB)
    cue_v0.4.1-beta.6_linux_amd64.tar.gz(5.45 MB)
    cue_v0.4.1-beta.6_linux_arm64.tar.gz(5.04 MB)
    cue_v0.4.1-beta.6_windows_amd64.zip(5.57 MB)
    cue_v0.4.1-beta.6_windows_arm64.zip(5.16 MB)
  • v0.4.0(Jul 3, 2021)

    In line with our aim to release smaller changes more frequently on our way to language stability, the v0.4.0 release comes just under two months after v0.3.0. It includes one language addition, a fairly significant reshaping of the API and tooling improvements, as well as the usual host of bug fixes, and performance and error message enhancements.

    This release includes a more comprehensive implementation of protobuf, supporting textproto and JSON protobuf mappings, and paves the way for binary protobuf support. See the v0.4.0-alpha.1 release notes for more details.

    This release also sets the direction for the CUE API, and brings the overall API much closer to where we would like it to go in the run up to v1.0.0. It also prepares the API for the query extension. See the v0.4.0-alpha.2 release notes for more details.

    The v0.4.0 minor release follows the pre-v1.0.0 release policy described in the v0.3.0 release notes: it contains backwards incompatible changes discussed below.

    As a reminder: users can register their projects with unity. unity is used to ensure that a project's CUE evaluations do not unexpectedly stop working, or regress in terms of performance. We are in the process of adding support for API-based tests, as well as private instances.

    Language additions

    Tooling

    API additions

    CUE package additions

    Performance and error handling

    Backwards incompatible changes

    Changelog

    b39a2d00 cmd/cue/cmd: avoid roundtrip when printing non-CUE in eval 3b0a537a cmd/cue/cmd: fix bug in resolving builtin package shorthands cd944260 cue: don't extract embedded values 37bf801b cue: keep sane references for embedded disjunctions in Expr a4e0f525 encoding/openapi: detect cycles when expanding references

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(384 bytes)
    cue_v0.4.0_darwin_amd64.tar.gz(6.01 MB)
    cue_v0.4.0_linux_amd64.tar.gz(5.78 MB)
    cue_v0.4.0_linux_arm64.tar.gz(5.29 MB)
    cue_v0.4.0_windows_amd64.zip(5.89 MB)
  • v0.4.0-rc.1(Jul 3, 2021)

    This release includes various bug fixes.

    Changelog

    393ec282 cue/cmd/cue: fix regression for -H flag 13a4d3c5 encoding/openapi: finalise when constructing openapi AST 287d43c6 internal/core/export: don't wrap embedded scalars in _#def 1e14710c internal/third_party/yaml: fix faulty decoding of hidden fields

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(404 bytes)
    cue_v0.4.0-rc.1_darwin_amd64.tar.gz(6.01 MB)
    cue_v0.4.0-rc.1_linux_amd64.tar.gz(5.78 MB)
    cue_v0.4.0-rc.1_linux_arm64.tar.gz(5.29 MB)
    cue_v0.4.0-rc.1_windows_amd64.zip(5.89 MB)
  • v0.4.0-beta.2(Jul 3, 2021)

    This release includes various bug fixes and error message improvements.

    Changelog

    d5041a1d cmd/cue/cmd: better checks for incorrect flags 858efa92 cue/ast/astutil: fix resolution bugs 5b8ab47c cue: elide comma after ... 3fdad163 cue: move field name to end of error message dd188a60 cue: move more labels and values to end of error message 48ddb2b7 doc/ref/spec.md: fix typos 30abd663 internal/core/adt: fix typo cd286a8e internal/core/export: fix export of disjunction with eliminated defaults 7180c7da pkg/tool/http: fill in status and statusCode 073c6aad pkg/tool/http: make request.body optional

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(412 bytes)
    cue_v0.4.0-beta.2_darwin_amd64.tar.gz(6.01 MB)
    cue_v0.4.0-beta.2_linux_amd64.tar.gz(5.78 MB)
    cue_v0.4.0-beta.2_linux_arm64.tar.gz(5.29 MB)
    cue_v0.4.0-beta.2_windows_amd64.zip(5.89 MB)
  • v0.4.0-beta.1(Jul 3, 2021)

    This release fixes some bugs, improves some errors and adds API. This version also drops support for some old constructs that have not been supported in the CUE tool for some time now. The main reason to start being more “aggressive” about this is to avoid surprises as we’re inching towards a full backwards compatibility guarantee. Note that “aggressive” is probably a strong word, these constructs have not been supported for over a year.

    Language additions

    Value aliases

    CUE now allows value aliases, as suggested in various proposals and remarks. This allows, for instance, users to rewrite:

    foo: {
        a: b + c
        b: int
        c: int
    }
    bar: foo & {b: 1, c: 2}
    

    as

    foo: X={
        a: X.b + X.c
    }
    bar: foo & {b: 1, c: 2}
    

    In this case the benefit may be small, but there are various cases where not having this ability results in significant inconvenience. Also the query proposal, the UX design around the usage of the proposed must builtin, as well as various other patterns, rely on this ability.

    Note the subtle but important difference between this and using field aliases. Using a field alias here, say X=foo: a: X.b + X.c would be equivalent to foo: a: foo.b + foo.c. As a result, in bar: foo, X.b would still be bound to the original location and not resolve to any b specified in bar.

    Which one to use depends on the application. As a general rule, field aliases should be used when defining “types”, whereas value aliases should be used when defining “values”. An example of where one wants to use field aliases:

    List="my-list-type": {
        value: _
        next:  List
    }
    

    Here, using a value alias would result in a list where every value must be the same. This is probably the most subtle aspect of CUE. The guideline of using values aliases when defining values, and field aliases when defining types, however, should give users a good steer on the matter.

    Tooling

    Tag variables to inject contextual system values

    The injection mechanism now allows injecting specific variables, such as current time, username, or hostname, or a random number, into fields marked with a @tag attribute. For instance:

    import "path"
    
    _os: string @tag(os,var=os)
    
    path.Base(dir, _os)
    

    allows for an OS-specific Base operation. Except for cue cmd, the var will only be injected when the -T is explicitly specified on the command line.

    This approach 1) allows packages to remain hermetic, 2) allows using such functionality without having to resort to cue cmd, and 3) avoids introducing another injection point.

    See cue help injection for more details.

    Binary file type

    The cue tool now supports binary files. These are very much like the supported “text” files, but translate the file contents to a CUE binary literal, rather than a text literal.

    There are no default file extensions to recognize a binary file. The binary: file qualifier can be used to read a file as binary. See cue help filetypes for more info.

    To make loading binary files in bulk more useful when importing, the import command now supports the --ext flag, allowing certain file types to be interpreted as a certain format. This also works for other types. See cue help import for more info.

    --force/-f

    This flag is now allowed for any command that outputs files.

    API additions

    Context.NewList

    Uses can now construct a new list cue.Value with Context.NewList. This is defined on Context to allow the creation of empty lists, which still need to be associated with a Context.

    Value.FillPath

    FillPath now accepts list indices. The Go-to-CUE are now documented in more detail.

    Fillpath now also accepts the selectors AnyString and AnyIndex, which allows setting pattern constraints. For instance, v.FillPath(cue.MakePath(cue.Str("a"), cue.AnyString, x), adds a: [string]: x to v.

    It also allows setting optional fields. For instance, v.FillPath(cue.MakePath(cue.Str("b").Optional()), 1) will add b?: 1 to v, using the usual rules for unifying such fields.

    Context.Encode

    The documentation of this method has been significantly improved. This also fixes some bugs that were uncovered in the process.

    These are now also two additional options: InferBuiltins and ImportPath. These were exposed to support the cue command in using the new API, but thus also allows other tool writers to use the same kind of functionality.

    CUE package additions

    crypto/hmac

    The new package hmac implements the Keyed-Hash Message Authentication Code (HMAC) as defined in U.S. Federal Information Processing Standards Publication 198.

    uuid

    The new package uuid defines functionality for creating UUIDs as defined in RFC 4122.

    Currently only Version 5 (SHA1) and Version 3 (MD5) are supported. The new tag variable injection mechanism that was introduced in this release, however, also opens up the possibility to include other versions, if needed.

    Backwards incompatible changes

    Dropped support for old-style aliases

    Aliases of the style X = 4 (now let X = 4) have been deprecated since v0.1.4. However, by setting the version or in API use, these aliases would still work. Support for them has been fully removed. Down the line, this syntax can be reused as value aliases as well, relaxing the places where value aliases can be used.

    For the time being, cue fix can still be used to rewrite old-style aliases to their let equivalent. There was a bug that missed the detection of some forms of the deprecated alias type. This has now been fixed.

    Dropped support for old-style comprehensions

    Support for the old-style comprehensions (e.g. [ x+1 for x in foo]) has now been fully removed. Use an older version of CUE to rewrite such old-style comprehensions using cue fmt or cue fix.

    Removed ast.TemplateLabel type from API

    This has not been parsed, supported, or used for ages. The type has now been removed from the API.

    Value.Expr

    Value.Expr now translates the or and and builtins as if they were native | or & constructs. This fixes a bug in encoding/openapi and is likely to prevent many issues. If one was relying on detecting the use of these builtins, however, this now will no longer work.

    Value.Decode

    This method was reimplemented to not use Value.MarshalJSON. This fixes many bugs, but may cause issues if users were relying on old behavior. The new implementation still strictly adheres to the JSON encoding spec when this makes sense, but deviates in some important aspects. For instance, numbers like 1 are now integers in CUE, rather than float. Also binary values are now handled correctly.

    Bug fixes

    • hidden references used on the command line now resolve to hidden fields in the package under evaluation
    • equality of null value

    Changelog

    5b14995e ci: debug tip trigger failures 24ce27e8 ci: fix broken tip and new version triggers a8369ae8 ci: fix tip and new version curl and env problems c143a3ab ci: fix tip triggers with proper quoting 746e02ec ci: more debugging of tip triggers 6c6b4e7c ci: trigger unity and cuelang.org builds/equivalent on tip/new versions 97cac927 cmd/cue/cmd: do not silently ignore unknown filetypes 89abc181 cmd/cue/cmd: fix resolution of hidden values across tools 4f7caea9 cmd/cue/cmd: move tag flags handling to central location b9e7d907 cmd/cue/cmd: support binary file type b449c0fd cmd/cue: support --force for all commands that allow output flags aa703992 cue/ast: fully remove support for old-style comprehensions b73ab0bc cue/ast: remove support for TemplateLabel 1f1c3f6e cue/load: provide reason for exclusion of files 975ba50b cue/load: remove unused code bcdf277b cue/load: support injection of system variables c2a68a90 cue/parser: allow pattern constraints unconditionally 3e109189 cue/parser: better error messages for use of deprecated constructs 3ef90b32 cue/testdata, doc, pkg: fix some it's/its typos f60c88a6 cue: Value.Expr: process or and and builtins semantically. d9af603e cue: add InferBuiltins EncodeOption b49cbbd3 cue: add NewList 1f618f06 cue: align import with top-level package for -e 66efc67c cue: allow setting list elements in FillPath 67c6b6f9 cue: better document Context.Encode and friends 9d044f07 cue: hoist attribute-related functionality to separate file ed5cdf0e cue: more path cleanup 33473028 cue: move Decode to separate file c365513b cue: prepare API implementation for structure sharing 04573564 cue: properly decode "any" labels 819cf955 cue: reimplement Decode to not use MarshalJSON 5481b414 cue: remove use of isDef 2e1ac0fb cue: support optional fields for FillPath ac7e9921 cue: undeprecate Iterator.Label 33169053 doc/ref/spec.md: introduction of value aliases a31dd016 doc/ref/spec.md: remove "dead code" 2198ac36 docs: add link to Code of Conduct to CONTRIBUTING guide f5213bea github: add bug report template link to repro advice 1ad66ab1 internal/core/adt: fix null values equality 3cdf845c internal/core/adt: use Closed for checking recursive closedness 5cd76bb2 internal/core/comple: drop support for old-style aliases 8823e2a6 internal/core: support field value aliases 6d67b48a pkg/crypto/hmac: add crypto/hmac support 21189212 pkg/crypto/hmac: fix package documentation a7d39870 pkg/time: fix documentation bug in Unix fd05bf4d pkg/uuid: implementation of hermetic UUID functions

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(412 bytes)
    cue_v0.4.0-beta.1_darwin_amd64.tar.gz(6.01 MB)
    cue_v0.4.0-beta.1_linux_amd64.tar.gz(5.78 MB)
    cue_v0.4.0-beta.1_linux_arm64.tar.gz(5.29 MB)
    cue_v0.4.0-beta.1_windows_amd64.zip(5.89 MB)
  • v0.4.0-alpha.2(Jul 3, 2021)

    This release focuses on setting direction for the CUE API. This brings the overall API much closer to where we would like it to go. It also prepares the API for the query extension.

    Note that this is deliberately marked as an alpha release to allow people to give feedback in the case we need to make some tweaks to the API.

    A central piece in that are the cue.Path and cue.Selector types. Another key part is getting rid of the Instance type. This type proves to be unnecessary and not using it results in a considerably nicer UX.

    The old functionality is marked as deprecated and will stay around until we provide a way to automatically rewrite them. We will just no longer support them and we use a trick mentioned in this tweet to hide most of the deprecated methods from the Go docs in the meantime (pending a "proper" fix in pkg.go.dev).

    Other improvements include:

    • another big round of tuning error messages, most notably adding more line information.
    • a performance bug fix that caused significant slowdown when doing repeated calls of Fill or Unify on the same values.

    API additions

    The cue.Path model

    The old API was designed before the existence of definitions. The fact that definitions live in a different namespace, broke the API at various levels, and patching this up resulted in a brittle and complicated API.

    The cue.Path-centred API addresses these issues, resulting in a much smaller API surface while at the same time being less prone to error.

    This new API also paves the way for the query extension.

    Some deprecated types and methods:

    The following types and methods all get replaced by LookupPath:

    • FieldInfo
    • Instance.Lookup
    • Instance.LookupDef
    • Instance.LookupField
    • Value.Elem
    • Value.FieldByName
    • Value.Lookup
    • Value.LookupDef
    • Value.LookupField
    • Value.Struct
    • Value.Template
    • Struct

    Similarly, FillPath and ReferencePath replace (Value|Instance).Fill and Reference. The latter also is instrumental in getting rid of Instance. In the long run we will want to repurpose Value.Lookup and Value.Fill, but that will be a long way out.

    The new Iterator.Selector method replaces:

    • Iterator.Label
    • Iterator.IsLabel
    • Iterator.IsOptional
    • Iterator.IsDefinition

    Using Selector also makes it more explicit what kind of labels one has and is thus less error prone. The selectors can be used to construct paths for lookup methods without loss of information, making the API more precise.

    Phasing out Instance

    The Instance type was initially intended to allow enforcing certain constraints. The data model of the new evaluator allows doing so without the need for Instance.

    Getting rid of Instance has some big benefits. Most notably, the need to link Value to Instance makes it very hard to avoid memory retention, even when a user believes a Value is no longer needed. This causes long-running CUE services to grow in memory usage over time.

    Another issue is usability: aside for duplication of methods, it may be hard for a user to know when to use a Value or Instance.

    Context to replace Runtime

    There are a few reasons for this change. As part of removing Instance, we needed a new way to build Values. The Context type now defines a new set of Value constructors.

    Another reason was to have a cleaner way to break the dependency cycle that existed for linking in the core builtins. Previously, the user had to add a import _ "cuelang.org/go/pkg" somewhere in the code, or use a package that already did so, to make the builtins available. Now the user is expected to create a Context using cuecontext.New(), which takes care of this.

    Finally, the name Runtime was confusing to some. The Context maintains indices and tables that are shared between creations of values, but there is no inherent “running” state.

    The Context now also makes it easier to resolve identifiers in another Value’s scope or to directly encode Go values and types.

    Value.Allows

    This method now enables querying whether a Value would support a field of a specific type if it were to be added. This also uses the Selector type to specify the kind of field.

    This replaces IsClosed.

    Backwards incompatible changes

    The new APIs are just additions. In many cases, the old API has been implemented in terms of the new API, but should still function as usual. This did result in some bug fixes, however, so one may observe changes.

    Value.Format

    The one change that may cause backwards incompatibility is the standard fmt.Formatter implementation of Value, which now has a more principled implementation. The standard %v formatter now prints it as a value, but allows incomplete values. The %+v formatter, as before, prints the fully evaluated value and is similar to cue export. The %#v formatter, which previously printed an esoteric debug format, now prints an equivalent of cue def.

    Many of the standard Go formatting verbs will now be interpreted as such if the Value is of a compatible Go type. See the documentation of Value.Format for more details.

    There have been various bug fixes in the exporter code as part of this change.

    cue/encoding

    This package has been removed. It really didn’t do anything except from being a distraction. In the off chance that anybody was using this package, just deleting that code would probably solve it.

    Changelog

    9e34a41e cue/ast/astutil: export ImportPathName 50c137a5 cue/encoding: removed unused package. dcb2a1fb cue/errors: add Wrap f044ad1c cue/format: expose indent option 68224334 cue: add Iterator.Selector f14c9a4d cue: add Selector.PkgPath c5c9125d cue: add test for filling empty path 17d4e16e cue: clean up Format 618c10c5 cue: deprecate Instance.(Doc|Fill) 790bed32 cue: get rid of NewRuntime 4937cb9a cue: hide deprecated methods from docs 421ead31 cue: introduce Context to replace Runtime d76e2ccd cue: remove MakeValue 908614eb internal/core/adt: dedup errors b8ce660c internal/core/adt: improve performance for repeated calls to Unify/Fill 276ce26a internal/core/adt: record more error positions 14ec6e26 internal/core/export: add real Final option b5b04294 internal/core/export: bug fixes for exporting API-generated values c62b750e internal/core/export: bug fixes in definitions 64ede631 internal/core/export: extract docs from root b937727c internal/core/export: fix definition wrapping c2907727 internal/value: implement interface type that is both value and instance 1f78a8db internal: replace untyped functions with typed ones d5ff6727 pkg: clean up builtin errors

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(416 bytes)
    cue_v0.4.0-alpha.2_darwin_amd64.tar.gz(5.94 MB)
    cue_v0.4.0-alpha.2_linux_amd64.tar.gz(5.72 MB)
    cue_v0.4.0-alpha.2_linux_arm64.tar.gz(5.24 MB)
    cue_v0.4.0-alpha.2_windows_amd64.zip(5.83 MB)
  • v0.4.0-alpha.1(Jul 3, 2021)

    This release introduces a more comprehensive implementation of protobuf, supporting textproto and JSON protobuf mappings, and paving the way for binary protobuf support as well.

    This release required some significant changes to the cue command logic.

    A hallmark property of protobuf data formats is that they cannot be parsed without a schema (unlike JSON, for instance). This is true to various degrees for the different formats, but it holds true in some shape or form for all of these. This required some changes in the CUE tooling to make this possible. It also required some backwards incompatible changes.

    Protobuf changes

    Filetype *.textproto

    CUE now supports interpreting and writing text proto files. Text proto files cannot be interpreted, or even parsed, without a schema. This means that .textproto files can only be read with an explicit schema using the --schema/-d flag. Moreover, the schema must have @protobuf for certain types, such as maps, to be interpreted properly.

    Note that the available .textproto parsing libraries are incredibly buggy. So there will be some rough edges that are kind out of CUE’s hands.

    JSON conversion: package jsonpb and json+pb

    The Protobuf documentation has a recommendation on how Proto messages should map to JSON. Package jsonpb now supports this mapping in both directions. It implements this by rewriting a CUE AST based on a given schema, a cue.Value. This allows this mapping to also be combined in conjunction with Yaml, for instance.

    On the command line this feature can be used through the pb “interpretation”, for instance as json+pb or yaml+pb. Both input and output are supported.

    Note that interpretations to CUE require a schema for the interpretation. This can be an explicitly specified schema using the --schema/-d flag or an implicit only by unifying it with a schema value or playing it within a schema using the placement flags (see cue help flags).

    Backwards incompatible changes

    @protobuf tag

    The protobuf previously had only one required argument: the numeric value of the enum. The type was optional and only included if different in name from the CUE type. As it turned out, though, the CUE type was not always sufficient information to be able to represent proto values. Most notably, integer values are encoded differently in JSON depending on the type of integer in the proto specification (this is not a typo!!).

    The new format includes the type unconditionally as the second argument. CUE does its best to recognize old formats for backwards compatibility purposes, but this may cause issues.

    @protobuf(<tag num>,<type>, ...options)
    

    Protobuf options are still represented as string literals, allowing CUE options, such as alternative names (name=x) to be represented as the usual <key>=<value> format.

    Another change is the representation for map types. Previously, the Protobuf map type was included verbatim (map<T,U>). This was somewhat inconvenient for parsing attributes, though: angular brackets are, unlike (), [], and {} not matched. So the comma in the map type commanded some escaping. To avoid this, maps are now represented as map[T]U. We contemplated using [T]:U, but opted for the map prefix for clarity and future extendibility.

    JSON mappings

    An initial design decision for the Proto to CUE mapping was to have CUE follow the Proto to JSON mapping. By hindsight, though, this did not make much sense. The inconsistency of having integers represented as strings does not make sense for a language that supports expressions on such integers (unless we want to give up typing, perhaps).

    The stance now is to take the representation that makes sense, and have a protobuf-specific CUE to/from JSON converters. This is akin to the JSON schema and OpenAPI converters, which map CUE to some data format, using JSON or YAML, for instance, as a transport layer.

    Luckily, the Protobuf to CUE mapping already deviated from the recommended mapping, always defaulting to int for integer types. So there are no changes there, other than that there is now support for following the recommended mappings for import and export.

    Enum mappings

    The most noteworthy backwards incompatible change is how enum types are mapped. Previously, CUE followed the recommended JSON mapping of using strings. This turned out to be a bad idea for various reasons. Firstly, the source of truth are integer values, really, not the names. There may be multiple names per number, making comparing names somewhat meaningless. Finally, most other languages used integers as representations, making the CUE representation not interoperate well with such languages.

    Although the old mapping is still supported, the integer-based mapping is now recommended.

    The default enum representation when using cue import proto is now to represent enums as integers in CUE. The --proto_enum=json flag can be used to invoke the old conversion behavior.

    The API will keep converting using the JSON format, but now has a Config.EnumMode option for selecting the alternative behavior.

    The Protobuf to JSON interpretation (filetype json+pb or package jsonpb) supports converting back and forth in either format.

    Changelog

    aaf6e846 cmd/cue/cmd: compute schema before values 0b5084a8 cmd/cue/cmd: hook up protobuf encodings types 8e5eeab2 cmd/cue/cmd: parseArgs: split values and schemas early f2282364 cmd/cue/cmd: preparse command line path expressions 75d0180d cmd/cue/cmd: simplify parseArgs in preparation of proto support dea3c5d0 cue/build: organized Instance fields 660b0906 cue/build: remove support for file lists 38ad7c3a cue: add ReferencePath af509c6c cue: eliminate context type 52db572c cue: get rid of internal index type c0fe9ce6 cue: refactor index 362e3a57 encoding/protobuf/jsonpb: add encoder 4a288d52 encoding/protobuf/textproto: add decoder implementation a0035ded encoding/protobuf/textproto: add encoder 3bdfa5d1 encoding/protobuf: always include type as second argument dcfff000 encoding/protobuf: support integer enums 8ba98eeb internal/encoding: pass schema to config 80a0a6e8 internal: move DebugStr to new astinternal package 22abdad5 internal: replace internal.CoreValue with more type-safe variant

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(416 bytes)
    cue_v0.4.0-alpha.1_darwin_amd64.tar.gz(5.94 MB)
    cue_v0.4.0-alpha.1_linux_amd64.tar.gz(5.72 MB)
    cue_v0.4.0-alpha.1_linux_arm64.tar.gz(5.23 MB)
    cue_v0.4.0-alpha.1_windows_amd64.zip(5.83 MB)
  • v0.3.2(Jul 3, 2021)

  • v0.3.1(Jul 3, 2021)

    This release contains several bug fixes, API additions, some API deprecations, and a language addition. One of the bug fixes may lead to unexpected behavior (see below).

    Backwards Incompatible Bug Fix

    One common use case of cmd/cue involves combining CUE and non-CUE files. Consider the following example:

    // check.cue
    package check
    
    map: {for n in nodes {"\(n.name)": n}}
    
    # data.yaml
    nodes:
    - name: bar
      parent: foo
    
    - name: baz
      parent: foo
    

    With v0.3.0 and earlier, cue eval would happily combine these two files:

    $ cue eval data.yaml check.cue
    map: {
        bar: {
            name:   "bar"
            parent: "foo"
        }
        baz: {
            name:   "baz"
            parent: "foo"
        }
    }
    nodes: [{
        name:   "bar"
        parent: "foo"
    }, {
        name:   "baz"
        parent: "foo"
    }]
    

    But cue vet would complain:

    $ cue vet data.yaml check.cue
    map: reference "nodes" not found:
        ./check.cue:3:16
    

    cue vet is actually correct here. Identifier resolution should only happen across files that belong to the same package: non-CUE files are equivalent to a CUE file without a package clause or an anonymous package clause (package _), hence cue eval should really fail in this instance.

    v0.3.1 makes cue eval and other commands consistent with cue vet. This was deemed a sufficiently breaking change to warrant its own release.

    The fix in this case is to define nodes in the scope of package check:

    // check.cue
    
    package check
    
    nodes: _
    map: {for n in nodes {"\(n.name)": n}}
    

    at which point cue eval succeeds as before.

    An upcoming change will also ensure this condition of identifier resolution is satisfied for all non-package CUE file arguments.

    API changes

    The cue.Selector model has been extended to allow querying optional fields and arbitrary elements.

    Deprecations

    The following methods of Value have now been deprecated: FieldByName Elem Template

    Really, you should only need LookupPath for your lookup needs. For the most part, these old methods have been reimplemented using the new API.

    Also, updated the doc that one really, really should not use Merge.

    If you are a user of the API we strongly recommend running staticcheck: it will raise errors where you are using deprecated parts of the API.

    Language Additions

    CUE now allows parenthesized expressions as labels. This is not (yet) published in the spec, but part of the query proposal. So "\(x)": foo, where x is a string, may now be written as (x): foo.

    Changelog

    a8ae7d1c cmd/cue/cmd: allow "exec" in commands for TestX 460357bd cue/ast: allow parentheses as labels e70a1db5 cue/build: remove unused Scope field f0adb4eb cue: deprecate Value.Elem and Value.Template f063a613 cue: deprecate Value.FieldByName 6a1ae9c3 cue: move LookupPath to query e4401832 cue: remove error type aliases 4459434a cue: resolve identifiers in an ast.Expr for FillPath c505c195 cue: separate closedness for lists and structs 957003c0 cue: support optional field lookup in LookupPath ad4d1a13 cue: update Merge comment to REALLY NOT USE IT 5c2b2815 internal/core: don't resolve values into anonymous packages 72e8fb43 internal/diff: fix printing and top-level diffs

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(384 bytes)
    cue_v0.3.1_darwin_amd64.tar.gz(5.85 MB)
    cue_v0.3.1_linux_amd64.tar.gz(5.63 MB)
    cue_v0.3.1_linux_arm64.tar.gz(5.15 MB)
    cue_v0.3.1_windows_amd64.zip(5.73 MB)
  • v0.3.0(Jul 3, 2021)

    After a long road, we have now an official v0.3.0 release. Aside from implementing some important changes to the language (see release notes), it prepares CUE for exciting new features such as the query and policy extensions and other language extensions as well as much better performance.

    The focus will now be to get to language stability and a v1.0.0 release.

    With this release will also come a different approach to versioning leading up to the v1.0.0 release. The goal is to get to the stability guarantee for the language first, then the API and tooling. Up to v1.0.0, version increments will have the following meanings:

    • minor (_.x._)
      • backwards incompatible changes
      • bug fixes with notable backwards incompatible changes
      • large new features
    • patch (_._.x)
      • bug and spec conformance fixes
      • new features

    We plan to get the few small remaining backwards incompatible languages chances out as soon as possible. To make the transition as simple as possible, we aim to put possible impactful bug fixes in small releases, so that users get a chance to get used to them. And, of course, we plan to make any necessary transitions as smooth as possible using cue fix aids when possible.

    Note that to smooth out any transition, users can register their repos at https://github.com/cue-sh/unity. This is used to ensure that their CUE evaluations will not unexpectedly stop working.

    API

    encoding/protobuf/jsonpb

    Protobuf defines its own, somewhat peculiar, mapping to JSON. This package provides a way to interpret JSON for such mappings, given a CUE schema. This is a first step towards broader Protobuf value support.

    cue.Selector

    Several new features have been added to Selector, including the possibility to query whether a selector is a regular string field and the ability to create a Selector form an ast.Label.

    Value.Attributes

    Value has been extended with a more extensive API to retrieve attributes.

    Value.Subsume

    Now allows raw (CUE-native) subsumption, instead of just the "Schema" or "Data" interpretations.

    Value.FillPath

    For completeness, note that beta.8 already introduced FillPath.

    Changelog

    44760604 cmd/cue/cmd: load import dependencies of tool files 276e164a cue/literal: expose some internal data c24a2819 cue/load: drop out-of-date comment about type of value in Overlay map 792da39a cue: add function for Value.Attributes() 1ea47e09 cue: allow raw subsumption, taking defaults into account 20a4878e cue: expose some path utilities 48f2a221 doc/cmd: refer people to cue help for now aa994145 doc: fix various typos 2115955a encoding/openapi: correctly extract type in the presence of null a1903ca3 encoding/openapi: dedup conjuncts and disjuncts 3701beff encoding/protobuf/jsonpb: add Rewrite* for interpreting JSON in PB terms 96e84eb1 tools/flow/testdata: add test with package dependency

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(384 bytes)
    cue_v0.3.0_darwin_amd64.tar.gz(5.84 MB)
    cue_v0.3.0_linux_amd64.tar.gz(5.62 MB)
    cue_v0.3.0_linux_arm64.tar.gz(5.14 MB)
    cue_v0.3.0_windows_amd64.zip(5.73 MB)
  • v0.3.0-beta.8(Jul 3, 2021)

    This release really is what beta.7 should have been as we actually forgot to include what was indicated as the most important change in beta.7 🤦‍♂️.

    In the meantime there are a few more fixes. If nothing serious comes in, we’ll cut v0.3.0 in about a week!

    Changelog

    2e934c00 ci: do not unset Code-Review label b00c91fb cmd/cue/cmd: disallow commands in non-tool files 27d305c2 cmd: fix -h handling f9164c6f cue/ast: sort CommentGroups by position when adding 7ca39680 cue: add FillPath 4ca1a5d6 cue: allow hidden fields in Path ccca558d cue: allow spaces around attribute args efd22f6b cue: allow string value as first element in ParsePath a1551b0f doc/ref: fix HTML anchor link in spec 61951712 doc: fix running Kubernetes test with CUE_UPDATE=1 cbda0d34 internal/core/adt: fix omission of closedness check 9b263eb3 internal/core/adt: track more incomplete errors a6e16272 internal/core/adt: update comments for Subsume e3e11e30 internal: store Body in Attr type b1730b6f tools/trim: optional fields should not remove non-optional

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(412 bytes)
    cue_v0.3.0-beta.8_darwin_amd64.tar.gz(5.84 MB)
    cue_v0.3.0-beta.8_linux_amd64.tar.gz(5.62 MB)
    cue_v0.3.0-beta.8_linux_arm64.tar.gz(5.14 MB)
    cue_v0.3.0-beta.8_windows_amd64.zip(5.72 MB)
  • v0.3.0-beta.7(Jul 3, 2021)

    This release contains several bug fixes.

    The most important bug fix, perhaps, is that it fixes a long-standing regression where commands for cue cmd were allowed to be specified in non-tool files. The current implementation will require a command to be define in at least one *_tool.cue file.

    Performance fixes have been put on hold for the moment to allow some stabilization before the final v0.3.0 release.

    Changelog

    35e19b2d cmd/cue: fix bug in -o, which would not write files for some commands 7c5d28e6 cmd/cue: fix race in testscript tests that use go/packages 81aa8cd6 cmd/export: fix help d4cb2c8d cue: add locations for disallowed field eb7b60da doc/ref/spec.md: fix ListLit production fec7a9c8 encoding/openapi: fix title handling regression bbe68e1a internal/core/adt: allow pattern constraints alongside fields 5049ab79 internal/core/adt: fix nil interface panic 1f9300d7 internal/core/adt: handle error properly for failed pattern constraint 2a937a95 internal/core/adt: improve error message dd5083f7 tools/flow: add clarifications in the API comments

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(412 bytes)
    cue_v0.3.0-beta.7_darwin_amd64.tar.gz(5.84 MB)
    cue_v0.3.0-beta.7_linux_amd64.tar.gz(5.62 MB)
    cue_v0.3.0-beta.7_linux_arm64.tar.gz(5.14 MB)
    cue_v0.3.0-beta.7_windows_amd64.zip(5.72 MB)
  • v0.3.0-beta.6(Jul 3, 2021)

    This release has some important bug fixes, fixing some regressions in disjunctions and trim, mostly, among some other bug fixes.

    Some of the regressions in this release slipped in because a swat of tests was accidentally disabled in the move to the new evaluator. All former tests are now reenabled and a few more corresponding issues have been resolved.

    There are some performance issues remaining. We are considering leaving those in and do a v0.3.0 release, addressing these in a subsequent release. In this case, we are getting fairly close to a v0.3.0.

    There will still be a beta.7 release, which addresses a regression, allowing commands in non-tool files, that we would like to address in a separate release.

    Changelog

    91abe0de ci: fix goreleaser config a113048d ci: move to using go1.16 f014c14c ci: set a "trybot" tag for trybot repository dispatch flow 5d3afa97 cmd/cue/cmd: correct documentation for -d flag de1f80c5 cmd/cue/cmd: remove skip tests 304a3e0e cmd/cue/cmd: use unify semantics merging across packages ae438126 cmd/cue: make get go operate on a best-efforts basis cf5c48e3 cmd/cue: only ignore type and parse errors with get go 7326cfa6 cue: Value.Equal should not only consider values 830cfbc5 cue: remove build.Context reference 17ea1d5a doc/ref/spec.md: a few fixes f5a19ef0 doc/ref/spec.md: fix escaping 3ca86804 doc/ref/spec.md: fix int_lit 0 e4d0d134 doc/ref/spec.md: remove faulty use of \x in double-quoted string 7df9fa47 doc/tutorial/basics: fix example b51368ea doc/tutorial/kubernetes: add required go mod init step 430c3ddf encoding/jsonschema: fix dropping of struct for additionalProperties: true 1347dd40 general: add .unity-bin to gitignore a0e19707 general: add unity configuration to codereview.cfg 351a77e4 internal/core/adt: add more error positions 79644c3b internal/core/adt: allow hidden fields alongside embedded lists 70e2f2ec internal/core/adt: discard errors in field verification 452c7bd9 internal/core/adt: don't count optional fields as proof of struct 3d926afd internal/core/adt: don't eliminate incomplete disjuncts prematurely 1589a074 internal/core/adt: get default when comparing ot bottom e12e26e9 internal/core/compile: fix let resolution bug 77e6b51a internal/core/compile: reenable disabled tests 2c86835c internal/core/eval: minor performance improvements for disjunctions e1e70315 internal/core/runtime: decouple cue API from Runtime c6b7ab2e internal/cuetest: add support for issue-based conditions/checkers 38f0f634 pkg/internal: fix decimal and string list errors af3c9dce pkg/list: fix list.Unique dedupping dcf932f3 tools/trim: fix too aggressive elimination for disjunctions

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(412 bytes)
    cue_v0.3.0-beta.6_darwin_amd64.tar.gz(5.84 MB)
    cue_v0.3.0-beta.6_linux_amd64.tar.gz(5.62 MB)
    cue_v0.3.0-beta.6_linux_arm64.tar.gz(5.14 MB)
    cue_v0.3.0-beta.6_windows_amd64.zip(5.72 MB)
  • v0.3.0-beta.5(Jul 3, 2021)

    This release has some important bug fixes. It also is a bit stricter in considering some errors that were previously evaluation-time errors to be compile time errors.

    There is also a slight change in default handling, which should not affect most configurations.

    New compile-time errors

    The expression >10 <20 is syntactically valid CUE, but can never meaningfully evaluate to something. It parses as >(10 < 20) and thus evaluates to >true, which is undefined. The error messages associated with this were often confusing to the user. Also, leaving this to be detected during runtime would sometimes hide the error away.

    Consistent with the “fail early” mantra, these cases are now detected and reported by the compiler.

    Backwards compatibility

    Defaults

    Previously, for

    #def: {
       *{} | {a: int} | {b: int}
       *{} | {c: int} | {d: int}
    }
    x: #def & {a: int}
    

    would officially resolve to

    x: {a: int} | {a: int, c: int} | {a: int, d: int}
    

    Thanks to a hack in the current v0.3 implementation, the first disjunct was made the default and all resolved fine. We have now slightly changed the definition of defaults to handle this case. Basically, now if during a unification all terms of a disjunction that are marked as default are eliminated, the remaining terms will behave as if they originated from an unmarked disjunction. In practice this means defaults behave as users typically expect them to behave. This may also cover cases that were not previously handed

    Note that v0.2 solved things differently: if in the result of a disjunction one term subsumes another, then the latter could be dropped. So in v0.2, the answer would be x: {a: int}, even in the absence of defaults. The problem with this approach is that it could lead to confusing answers and that the default could not always be accurately determined. A goal of v0.3 was to have a simpler story for disambiguation and let simplifications be merely a matter of optimization.

    There is still a slight discrepancy in the current implementation and the spec that may cause unexpected results. We expect this to be sufficiently rare and we intend to address this in a rework of disjunction computation, aimed at further performance optimizations.

    Changelog

    1b03c9fa ci: move to non-Docker version of goreleaser ae51469c ci: skip known flakey test on go1.15 windows 500e431d doc/ref/spec.md: add missing binary_digits 89cada6f doc/tutorial: address cuelang/cuelang.org#126 77b475f8 doc: fix install instructions in main README and contributing 25ba1d94 internal/core/adt: comment out leak test b25147c3 internal/core/adt: control fixes for if clauses 267379b6 internal/core/adt: fix trim bug 48d255a1 internal/core/adt: some optimizations for pattern constraints 87d43b74 internal/core/adt: strip "notDefault" status for disjunctions with unused defaults 760aa115 internal/core/compile: make >10 <20 a compile error b20ac5a2 internal/core/convert: fix conversion from float bcd752ad internal/core/eval: handle embedded top correctly a5daa168 internal/core/export: fix quoting of definitions cb4fae90 internal/cuetest: consolidate testscript/cuetxtar helpers bf6c3fce internal/cuetest: move Kubernetes related helper to where they are used 15934d98 tools/trim: add tracer 1d9aa173 tools/trim: don't trim shared nodes af5ea978 tools/trim: fix defaults bug

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(411 bytes)
    cue_0.3.0-beta.5_Darwin_x86_64.tar.gz(5.84 MB)
    cue_0.3.0-beta.5_Linux_arm64.tar.gz(5.13 MB)
    cue_0.3.0-beta.5_Linux_x86_64.tar.gz(5.62 MB)
    cue_0.3.0-beta.5_Windows_x86_64.zip(5.72 MB)
  • v0.3.0-beta.4(Jul 3, 2021)

    This release has some important bug fixes related to performance, disjunctions, and performance of disjunctions.

    Backwards compatibility

    As a result of the disjunction rework, we introduced another internal workaround for dealing with the fact that there is no good way to represent proto oneOf fields in CUE. Right now this is done as

    *{} | { a: int } | { b: int }
    

    or

    *{} | { a?: int } | { b?: int }
    

    The need for the default is because one cannot say that a field must exist. So the default is used to disambiguate. This works well for one set of oneOfs, but not in the presence of multiple oneOfs.

    A previous hack would unmark a disjunction as having a default if all but one of the defaults remains. After the critical performance improvements, this still worked, but only because these changes introduced a bug. Now this is fixed we can no longer rely on it. To keep protobuf working, there is now another hack that recognizes this pattern and adjusts defaults in a similar manner.

    This is an unacceptable solution, of course. The idea is to allow specifying required fields instead, somewhat like this

    {} | { a!: int } | { b!: int }
    

    and transition to this model.

    The required field specification has many benefits and solves several modeling limitations of CUE we’ve encountered since its inception. It can also almost, if not entirely, eliminate the need for ?, provides a solution for some limitations of the proposed query syntax, and generally provides better alternatives for cases where one finds the need to abuse defaults.

    We plan to officially introduce this construct in a proposal soon.

    Changelog

    09cd07bb all: add tests for the GitHub workflow process f0d1fa4e ci: add first version of copybara workflow 10a48e90 ci: drop go1.13 from build matrix; add go1.16rc1 e097927d ci: fix broken build 34333ce1 ci: fix job dependency in test workflow 726d93e4 ci: fix start message of CI jobs for CL builds 9b7fc4b5 ci: fix workflow definitions 3088513d ci: only go test -race on latest Go+linux, and master 5b3d5514 ci: refactor CI setup 2338b593 ci: rename repository dispatch workflow ef59f61a ci: update version of Go for go generate to 1.14.14 257486c8 cmd/cue/cmd: allow fmt to pass with errors dfce25c5 cmd/cue: add goproxytest instance for script tests b8d99727 cmd/cue: allow CUE_UPDATE to update cmd/cue/cmd testscript tests a6169255 cmd/cue: embed packages required for get go checking 22aafc2a cmd/cue: fix typo in cmd documentation headline 37c16cc5 cmd/cue: use gotooltest within testscript tests d9d04872 cue/scanner: fix JSON conformance ba8eb371 cue: allow LookupDef on embedded scalars c6f1287a cue: deprecate Merge f73e78f8 doc/../43_schema.txt: remove unnecessary word 6c3be7f8 doc: fix typo in types tutorial (closed -> close) 2df92cb5 doc: remove duplicate command from Kubernetes tutorial b360fe43 encoding/gocode: use FieldByName in generated code 7110020e general: move from godoc.org to pkg.go.dev for documentation d5d5fd15 general: remove junk files 63104c84 internal/core/adt: "fix" disjunction resolution 2ca6c5e1 internal/core/adt: cache evaluation results for dynamic fields c866a515 internal/core/adt: don't check cycles for inlined arcs 6b960014 internal/core/adt: fix benign memory leak b8b48925 internal/core/adt: fix bottom cycle error c0912749 internal/core/adt: fix bug that left errors of incompleted expressions unreported 9c5489fc internal/core/adt: fix memory leak ee78ec3a internal/core/adt: fix nil panic 8d537aac internal/core/adt: fix regressions in disjunct elimination e6a2fb02 internal/core/adt: fix special cycle condition bedb047a internal/core/adt: more fixes to cycle handling 8334a9e1 internal/core/adt: more merging of adt and former eval d0124e2a internal/core/adt: refactor node lookup 0ad02fdf internal/core/adt: restrict notification d2aa995f internal/core/export: don't embed proper conjuncts 76a72c4f internal/core/export: fix optional output bug fdccfbc5 internal/core/export: retain top in embedded scalars 34587b9c internal/core/export: support package and struct attributes dfaea59a internal/filetypes: fix test breakage 10180b64 pkg: consistently use 0666 as perm when creating regular files f55f17ea tools/fix: eliminate x & _ and unnecessary parens 611d6227 tools/flow: replace package doc example with real example

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(411 bytes)
    cue_0.3.0-beta.4_Darwin_x86_64.tar.gz(9.83 MB)
    cue_0.3.0-beta.4_Linux_arm64.tar.gz(9.31 MB)
    cue_0.3.0-beta.4_Linux_x86_64.tar.gz(9.98 MB)
    cue_0.3.0-beta.4_Windows_x86_64.zip(10.06 MB)
  • v0.3.0-beta.3(Jul 3, 2021)

    This release addresses some important bug fixes, including a performance regression. It also sports a complete reimplementation of trim, utilizing the possibilities of the data structures of the new evaluator.

    cue trim

    The biggest new feature of trim is that definitions now can remove fields of non-definitions. Other than that, it mostly is just bug fixes. The data structures of the new evaluator make it considerably easier to implement, leaving less than a quarter of the code.

    encoding/protobuf

    This package now links in the full set of builtins, obviating the need for API users to have to do this.

    Changelog

    fdc7010f cmd/cue/cmd: fix cue cmd crash a8de61c6 cue/testdata/cycle: fix another cycle bug 28cfa742 cue: exempt hidden fields also in Unify and UnifyAccept ca5d2db5 doc/tutorial/kubernetes: update files of Kubernetes demo f0c025c1 encoding/protobuf: enforce loading builting packages 08e814f8 internal/core/adt: add support info for trim 8e6e7959 internal/core/adt: change closeInfo.embedded to a flag field 0e401d66 internal/core/adt: fix performance issue with disjunctions 82bda840 internal/core/adt: treat files as belonging to same struct c58e0ffc internal/core/eval: strip arcs for nodes with structural cycles 0f570a90 internal/core/validate: pick up non-concrete values in defaults f03928d2 tools/fix: allow fixing non-essential simplification 68988976 tools/trim: implement using adt

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(411 bytes)
    cue_0.3.0-beta.3_Darwin_x86_64.tar.gz(9.79 MB)
    cue_0.3.0-beta.3_Linux_arm64.tar.gz(9.28 MB)
    cue_0.3.0-beta.3_Linux_x86_64.tar.gz(9.93 MB)
    cue_0.3.0-beta.3_Windows_x86_64.zip(10.02 MB)
  • v0.3.0-beta.2(Jul 3, 2021)

    This release predominantly addresses some regressions and bug fixes. There are also some additional small performance fixes.

    Backwards compatibility

    There are a couple of bug fixes that may lead to backwards incompatibilities.

    Disjunction disambiguation and “incomplete” errors

    There were some bug fixes where disjunctions were discarded as duplicates prematurely. This was already an issue since v0.3.0-alpha1. As a result, you may now get an “incomplete value” error where before you didn’t. For instance, {a: int, c?: int} | {a: int, b?: int} now will remain as two separate disjuncts, whereas previously it was assumed evaluation was done too early that the optional fields were irrelevant.

    In v0.2.2 this would also not be disambiguated. There are, however, still some differences between v0.2 and v0.3. Pre-v0.3 CUE used subsumption to disambiguate disjuncts. Subsumption is inherently imprecise and allows false negatives. This would inherently make it hard to predict when values were disambiguated. As subsumption was used at no other place during evaluation, v0.3 takes a simpler approach to only check for equality, whereby optional fields are only considered equal if they originate from the same original structs.

    The general idea is that one should typically not have disjunctions of structs without those having discriminator fields (i.e. the disjuncts would fail if unified with each other, for instance with a kind field with a specific value, like K8s does). If one still needs this, then one of the structs should be marked as default to disambiguate.

    In other words, v0.3 may give an “incomplete error” where v0.2 did not. For instance: #Fields: _ | [string]: #Fields would normalize away with v0.2, but won’t with v0.3. The simple solution is to use _ or {...}. The v0.3.0-alpha releases had a bug that would prevent these alternatives from working properly, but that has been fixed as of beta.1.

    We may still consider allowing more disambiguation at the point of making values concrete. This will need to be considered carefully, though. We may also allow some compile-time optimizations, like translating _ | X to _. We have not done so yet, as a set of rules for such translations has the danger of making disambiguation arbitrary; we would like If people have trouble migrating to v0.3.0-beta.2, we can prioritize such simplification rules for beta.3.

    We intend to come up with a set of clear and predictable rules before v0.3.0 on when CUE may disambiguate disjunctions. Until that time, please ensure that disjunctions with structs are disambiguated using defaults and discriminator fields. This should generally be considered to be proper style and highly recommendable anyway. Note that any protocol buffer and all K8s structs, for instance, follow a pattern that can be disambiguated easily.

    Stricter closedness as a result of a bug fix

    The value #B: {#A} & {a: 1} is now interpreted as #B: #A & {a: 1}. Previously, this was interpreted as #B: {#A, a: 1}, allowing field a regardless of whether #A allows it or not. This was a leftover of the old closedness rules of pre-v0.3.0. The old rules were somewhat convoluted and based on user feedback and experience we simplified the rules and made them more intuitive for v0.3.0.

    API

    We have added the list.Concat and list.Repeat builtins in anticipation of removing the * and + operators for lists in CUE. These operators are quite unintuitive as it is not clear how they interact with closedness, which has raised repeated questions. They also complicate the implementation. This has been mitigated by, internally, translating list addition to list of two list comprehensions. They also complicated automated analysis, which is harder to work around. Better to get rid of them altogether.

    The next steps for the transition is to let cue fix automatically rewrite list addition and multiplications in terms of their builtins. Ultimately list addition could be replaced with a query shorthand, like [a] + [b] -> [ a.*, b.* ], using a.* as the hypothetical notation for all values of a.

    Performance

    We have seen some considerable performance improvements for some configurations. There is still plenty of room for improvement though, both algorithmically, which can bring long-running evaluations down to below a second or two, and by reducing the constant overhead, by itself allowing a running time reduction of at least another 90% in many cases. So if there are any pressing performance issues, we would love to hear about them.

    Changelog

    ed7d254d ci: delete CL build branches once workflow is complete 8c416223 ci: fix dispatch workflow ccbfbd2e ci: move to build branch model 237a4aef ci: use the latest GitHub workflow definition f1b18040 cmd/cue/cmd/getgo: don't be clever ff48658e doc/ref/spec.md: disallow new definitions in closed structs 172f0060 doc/ref/spec.md: get rid of list operators 5691606b internal/core/adt: add test for incomplete references in lets bef7b263 internal/core/adt: fix cycle bug a086f74c internal/core/adt: fix disjunct dedupping regression ae1203cd internal/core/adt: fix overzealous disambiguation 826bd7bd internal/core/adt: fix regression in close builtin 34a91b9a internal/core/adt: fix some regressions 663939eb internal/core/adt: improve errors for validator conflicts 13b68c81 internal/core/adt: minor cleanup 355cccd2 internal/core/adt: precompute optional field type info 9eee235e internal/core/adt: remove special "tentative" mode 2ef72d85 internal/core/eval: fix result of closed embedding 03aab971 internal/core: re-allow new definitions in closed structs 0ffde15b pkg/internal: fix error codes for decimal and string lists 867f71fe pkg/list: add builtins for list concatenation and repetition

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(411 bytes)
    cue_0.3.0-beta.2_Darwin_x86_64.tar.gz(9.79 MB)
    cue_0.3.0-beta.2_Linux_arm64.tar.gz(9.28 MB)
    cue_0.3.0-beta.2_Linux_x86_64.tar.gz(9.95 MB)
    cue_0.3.0-beta.2_Windows_x86_64.zip(10.03 MB)
  • v0.3.0-beta.1(Jul 3, 2021)

    This release is focussed predominantly on performance. It includes several algorithmic improvements that eliminate possible exponential processing and in general takes more care to avoid recomputation.

    Aside from algorithmic changes, these changes also prepare for numerous further optimizations. The algorithmic changes already already give enough of an improvement (especially for some large files) to warrant another release.

    Language change

    This release also introduces a slight language change regarding disjunctions. Consider

    x: *1 | a
    a: *2 | 3
    

    Previously, the result of x would be *1 | *2 | 3, in other words, a default value of 1 | 2. With the new semantics, the default in this case will be 1. In general, with the new semantics, if a disjunction has a term with a default, the other terms can never result in a default value.

    Superficially, this makes it easier to see where defaults may come from and eliminates the need to check all terms of a disjunction. But this change also prevents some unintuitive behavior and even cases where making values more specific may lead to a less specific result.

    Backwards compatibility

    The changed disjunction rule may cause backwards incompatibility.

    There are some minor bug fixes as a result of these changes that may change the behavior For instance, [...] | [] was previously normalized to [].

    There are also some changes to error messages as a result of these changes.

    Changelog

    cc418539 doc/ref/spec.md: change disjunction rule d5b4ca85 internal/core/adt: add logging utilities for debugging 0c3791ea internal/core/adt: add memory management 008d37d2 internal/core/adt: catch errors for circular failures b619e0ca internal/core/adt: consider closed status for equality 396202cc internal/core/adt: don't pre-expand disjunctions 69e0c96f internal/core/adt: introduce HasDefaults for Disjunction 514082ac internal/core/adt: keep nodeContext in Vertex a595db28 internal/core/adt: skip partial unify if unnencessary 02ff29bf internal/core/convert: fix faulty label update 778719ed internal/core/eval: breadth-first disjunction resolution 2be20f84 internal/core/eval: do not precompute matchers 399ba762 internal/core/eval: export fieldSet fields 0ef6caa8 internal/core/eval: make arcMap a list 1fa69d59 internal/core/eval: recursive disjunctions 9ca9e043 internal/core/eval: separate out dynamic fields 0e1e7b5a internal/core/eval: small cleanup of evaluation code 062b3d88 internal/core/eval: sort empty disjunction message to top of group 359685af internal/core/eval: use env from fieldSet d4c600fb internal/core/export: extract docs from nested Vertex values c62d8a05 internal/core: merge adt with eval package cea55b28 internal/core: reimplementation of closedness algorithm 93c79a34 tools/flow: record dependencies per value based on path

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(411 bytes)
    cue_0.3.0-beta.1_Darwin_x86_64.tar.gz(9.78 MB)
    cue_0.3.0-beta.1_Linux_arm64.tar.gz(9.27 MB)
    cue_0.3.0-beta.1_Linux_x86_64.tar.gz(9.93 MB)
    cue_0.3.0-beta.1_Windows_x86_64.zip(10.01 MB)
  • v0.3.0-alpha6(Jul 3, 2021)

    This release is characterized by closing some significant remaining gaps in with the language specification.

    There are the usual bug fixes and improvement to error messages, though.

    Language

    The language has now been brought closer to the spec, implementing some of the remaining gaps. There have also been some simplifications that will allow further generalizing the language down the road more easily and without introducing backwards incompatibility.

    Multiple comprehensions per list

    As the spec already allowed, it is now possible to include multiple comprehensions per list.

    For instance:

    src:    [ "John", "Mary" ]
    cond: true
    a: [
        for x in src { x },
    
        if cond { "Sasha" }
    ]
    

    The list operators (concatenation and multiplication) are now also implemented in terms of comprehensions. There is a great deal of confusion about list operators (is the result open, closed, when do overlapping constraints apply?). This confusion is eliminated by writing the equivalent as list of multiple comprehensions. Our intention is to remove these operators and have comprehensions (and later queries) be the only way in which lists can be concatenated or multiplied. Of course, cue fmt will help in the transition there as usual.

    Embedded scalars

    Just as it is possible to write

    [ #a, #b ]
    
    #a: 1
    #b: 2
    

    at the top of a CUE file, it is now also possible to have an embedded scalar along with definitions in any struct:

    x: {
        [ #a, #b ]
    
        #a: 1
        #b: 2
    }
    

    Originally, the top-level construct was allowed to make CUE strictly compatible with JSON. Generalizing this to any struct makes the use of imported file objects consistent with normal structs. Moreover, it turns out that this construct can be quite useful overall.

    Old CUE code is not affected by this change.

    Semantics of [pattern]: value fields (no change after all)

    The spec recently included a change that proposed to use the JSON schema semantics for pattern constraints ([k]: v), rather than the old CUE semantics. To aid in the transition, cue fmt rewrote these fields to be enclosed in curly braces, which mimics the old semantics under these planned rules.

    It has now been decided to keep the old semantics and stop rewriting old uses.

    As it turns out, the old semantics was used quite a bit and was actually the majority use case. The argument previously was that the JSON schema semantics was not easily expressible in terms of the old semantics, but that the old semantics is easy to simulate with the JSON schema semantics.

    Instead we now intend to allow the JSON schema semantics additionally by writing [...k]: v. The nice thing here is that any pattern which is to be matched additionally, is now always preceded by a .... It also is a nicer generalization of the syntax.

    But above all, it removes the need for a potentially painful transition.

    Removed integer division operators

    CUE allowed integer division and remainder per the div, mod, quo, and rem operators. These were relatively rarely used and complicated the CUE syntax, formatting and other parts of various subsystems.

    Moreover, they made it lexically impossible to allow indexing of the form a.3 (instead of a[3]) down the road. We are not certain if we will allow this, but allowing it makes the syntax considerably more regular and consistent. With this change, we have the option to do so in a backwards compatible way.

    These operators have now been replaced with builtins. As usual cue fmt will rewrite old uses to use the builtin for some time to come. It will use the __foo variant to avoid shadowing issues. These can be manually rewritten to remove the leading underscores if necessary.

    Variable argument builtins

    The model for builtins, which are still underspecified in the spec, is narrowed down more. It is now possible to define variable argument builtins.

    The main reasons:

    1. Variable argument builtins are quite useful in being able to extend APIs in a backwards compatible way,
    2. It allows introducing some environment dependence in a clean, intuitive and nonetheless hermetic way.

    The first target was pkg/path which now supports multiple OSes (see below).

    Bug fixes

    Errors in nested comprehension clauses were sometimes dropped, causing a failure in a nested if clause to be silently ignored. This has now been fixed.

    Other fixes includes crashes related to let clauses and bytes interpolation.

    Error messages

    There are various areas where error messages have been improved.

    cue fmt

    There are various printing bugs that were fixed. Overall the stability of fmt should have been improved notably.

    API

    The API got a few fixes related to backwards compatibility and should now be more like v0.2.0 where this makes sense.

    pkg/path

    This package has now been extended to support multiple operating systems simultaneously. Most builtins have been extended to allow an additional optional argument to indicate the OS for which the builtin is to be applied. For instance:

    path.Split("foo\bar", os)
    

    will now return a different value depending on whether os is path.Windows or path.Unix. The os argument can also be omitted, in which case it defaults the existing behavior, which is identical to path.Unix.

    Note that os will not default to the currently running OS. This would make the behavior non-hermetic. The idea is that the existing injection mechanism can be used (or a small extension of it to make this specific case easier) to inject an environment-specific value.

    Backwards compatibility

    The cue tool is now a bit more aggressive to panic if it encounters an internal inconsistency error. In return, it now allows the user to define a CUE_DEBUG=0 environment variable to disable this. In many cases the crashes can be ignored while still getting useful results.

    Not using this option and filing bugs can help improve the stability of CUE though.

    Main backwards incompatibility to note:

    • removed integer division operators (now builtins). This can be fixed by running cue fmt.
    • some of the bug fixes may cause CUE files that previously succeeded erroneously to fail now.

    Changelog

    e77ccb1c cmd/cue/cmd: don't bail for non-matching files fecdd834 cmd/cue/cmd: ignore possible top-level tasks bbe493a7 cmd/cue/cmd: more position information for field errors fe73e8f9 cue/format: format let clause 110d0bff cue/format: pad floats with leading or trailing periods fac5d2e3 cue/load: allow ./... to load all packages 555fb73d cue/parser: fix comment placement 1c904cc2 cue: allow multiple comprehensions in lists 8872b98f cue: fix PathCorrection tests 6c49cf0f cue: implement embedded scalars d5177fd1 cue: map API values to more native ADT b687b7f7 cue: remove aliases used for refactoring 19f6f3f6 doc/ref/spec.md: perpare for allowing a.4.f f52a0ed8 doc/ref/spec.md: revert to the old semantics of [k]: v 76ea22cb internal/core/adt: add Assert for debugging 37293f95 internal/core/adt: add methods for concreteness 41afc87e internal/core/adt: automatic renaming 2d568d30 internal/core/adt: better message for permanently non-concrete values f62bfed9 internal/core/adt: clean up Builtin and Validator semantics 4db8ffb1 internal/core/adt: do list artihmetic with comprehensions cbcb701f internal/core/adt: error message improvements d0dd8887 internal/core/adt: improve error message for optional fields 08a1652d internal/core/adt: introduce base value type 6de877a9 internal/core/adt: make progression check an assert 8b137527 internal/core/adt: move decimal logic into one place b8c852d4 internal/core/adt: pass errors up in nested comprehensions 36f20510 internal/core/compile: add integer division builtins 9c6ded8d internal/core/compile: don't panic for incorrect labels 13e4af16 internal/core/compile: fix issue in and builtin ff3e32f3 internal/core/convert: allow results of math to be integer 31896af4 internal/core/eval: add more tests for embedding d49777fe internal/core/eval: better incomplete error messages 55602d94 internal/core/eval: fix closedness issue for lists 9f1fec6f internal/core/eval: hoist vertex processing logic cefe38b1 internal/core/eval: keep semantics for [K]: T acc35f11 internal/core/eval: track fieldSets for data values e7e27e00 internal/core/export: allow showing recursive errors. 039fb590 internal/core/export: fix bytes printing bug 2ea30a22 internal/core/export: fix crash with let comprehension export 16809ead internal/core/export: support byte interpolations 63594eff internal/core/runtime: assign first label to _. 08116621 internal/core/subsume: tests embeded scalars a4d09289 internal/core/subsume: various bug fixes 136e51b4 internal/core: allow variable arg functions e841714b internal/core: move equality checks from eval to adt d174bc09 internal/cue/adt: remove use of Default in node fa6308f2 internal/encoding: don't clobber files on crash ec6f95d8 internal/filetpes: improve error messages fb3b02ed pkg/internal/builtin: use normal formatting for builtin test results a50960c8 pkg/path/testdata: adapt Go implementation a2692ef4 pkg/path/testdata: move files to minimize diffs c77b9b0d pkg/path/testdata: stage filepath files from Go 85f8b466 pkg/path: activate OS-dependent version 64434c46 tools/fix: rewrite integer division d5ced741 tools/flow: ignore faulty tasks outside the root

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(411 bytes)
    cue_0.3.0-alpha6_Darwin_x86_64.tar.gz(5.82 MB)
    cue_0.3.0-alpha6_Linux_arm64.tar.gz(5.12 MB)
    cue_0.3.0-alpha6_Linux_x86_64.tar.gz(5.60 MB)
    cue_0.3.0-alpha6_Windows_x86_64.zip(5.71 MB)
  • v0.3.0-alpha5(Jul 3, 2021)

    This release is mainly focused on various bug fixes and error message improvements. This change also continues to factor out code from cmd/cue to make the functionality available outside of this tool.

    Below is a selection of the fixes.

    Although this is officially alpha, it is probably a more solid release to use than v0.2.2. For this reason the "Pre-release" tag has been left off from this release.

    Language

    Cycle handling

    Various bugs related to cycle handing were fixed, including hangs and spurious detection of cycles.

    Closing gaps with the specs

    Hidden fields are now scoped per package.

    Package pkg/sort

    Sort now always uses stable so that reproducible outcomes between different compiles of CUE can be guaranteed.

    API

    cue.Path

    The introduction of #-style definitions are not handled well by the old API. This is a consequence of them living in a separate namespace, which is not expressible in the old API.

    The new cue.Path type is introduced to start addressing this. The first method that supports this is LookupPath.

    This API is designed to be compatible with the planned query extension.

    Package tools/flow

    This new package allows arbitrary user-defined CUE schema to be interpreted as task definitions, automatically handling dependency analysis and scheduling tasks based on these dependencies.

    This was written from scratch and makes use of a new native dependency analysis package, allowing for more precise analysis.

    cue command

    Go bridge

    The translation of CUE to Go is now brought in line with Go’s encoding/json, fixing some bugs that caused the interpretation to differ.

    cue cmd

    This now uses the new tools/flow package, fixing a bunch of outstanding bugs. Most notably, it now allows dynamic task creation, for instance by using comprehension, and depending on such dynamic tasks: $after should now only be needed if there are really no dependencies at the CUE level.

    Changelog

    b4aa96d7 all: implement hidden identifiers scoped per-package aa878875 ci: drop use of set-env in CI scripts 4cda4a3c cmd/cue/cmd: allow inlined embedding of pointer types c715b94f cmd/cue/cmd: use tools/flow e05eee7d cue/errors: correct handling of wrapping list errors 54b13dbe cue/literal: fix multiline quotes 6165736e cue: allow access to Selectors of Path f4f2a02f cue: dig up wrapped error in valueError 74329a13 cue: doc improvements 56eb4b16 cue: generate error node rather than panic on faulty output. 03abe876 cue: remove most uses of xerrors 409dacfa cue: support getting and looking up paths fcd96de9 doc/ref/spec.md: define "regular"; fix typos 86612555 encoding/protobuf: fix trailing comment in oneof bug b5821dc2 interanal/core/runtime: move building logic from cue package b886b0f1 internal/core/compile: fix mutual dependent let clauses 3585705a internal/core/convert: fix for embedded structs are not honored b99fd082 internal/core/cue: fix hang on cycles in embeddings in comprehensions c60e1153 internal/core/dep: first stab at precise dependency analyzer 7463d11d internal/core/dep: support dynamic dependencies 1888d65b internal/core/eval: fix bug in cycle handling 1d8c688c internal/core/eval: fix bug with nested embeded comprehensions aee9955c internal/core/eval: fix cycle bug for comprehensions 1a2105eb internal/core/eval: fix hang in cyclic in comprehension evaluation 3cfb4ab3 internal/core/eval: fix irregular dynamic fields d1cbe10a internal/core/eval: fix spurious cycle detection 635fbdd4 internal/core/eval: fix spurious cycle for bulk constraints 7980ec5d internal/core/eval: handle disjunction failure more properly ab0a2f39 internal/core/eval: re-allow ... with embeddings 30704a7c internal/core/eval: reduce per-node allocations 1e0faf0f internal/core/export: dedup let clauses for merged structs 187c7343 internal/core/export: handle alias and lets cd621ffa internal/cue/eval: fix faulty status in tentative result dbfa73b0 internal/cuetxtar: allow multiple golden files f2a4a42b internal/filetypes: hard-wire common case 737a103f internal: hoist a single embedding in ToExpr 99d18dcf pkg/list/sort: always use stable sort 00f345b8 tools/flow: API for CUE-based workflow engine d3ff4a17 tools/flow: add IgnoreConcrete option

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(411 bytes)
    cue_0.3.0-alpha5_Darwin_x86_64.tar.gz(5.78 MB)
    cue_0.3.0-alpha5_Linux_arm64.tar.gz(5.09 MB)
    cue_0.3.0-alpha5_Linux_x86_64.tar.gz(5.56 MB)
    cue_0.3.0-alpha5_Windows_x86_64.zip(5.67 MB)
  • v0.3.0-alpha4(Jul 3, 2021)

    This is a relatively small release. But given that it contains some significant bug fixes and performance improvements, it seemed useful to get these out sooner.

    This also

    Closedness refinements

    The rules surrounding closedness have been further clarified in the spec where this was previously ambiguous. Most notably, it was previously unclear whether B.b in

    #A: {a: int}
    
    B: {
        #A
        b: c: int
    }
    

    should be closed or not. This is now clarified to it not being closed. In other words, fields added alongside an embedded definition that were not part of the original embedding are not closed automatically.

    This does not change the current behaviour.

    There was also a bug fix related to closing structs that could leave struct open if they were referenced through a selector path like #A.b, where the reference to #A failed to trigger a close.

    There remains one known gap in the implementations compared to the spec: embedding a closed struct will currently cause the field to be closed, whereas the spec says that the struct in which it is included should already be closed. This will be a backwards-incompatible change.

    field not allowed error

    As far as is known, aside from the spec deviation mentioned above (which is somewhat esoteric), there are no more known deviations that result in a spurious error of this kind.

    There is, however, one known case where this error seems to appear erroneously. Consider the following:

    #a: {b: int} | {}
    b: #a & {b: string}
    

    This may still result in a “field b not allowed” error. The problem is that the error reporting around disjunctions is still lacking. What happens is that only the error of one of the failed disjuncts is reported. In this case it fails on the first disjunct (as int != string) and the subsequently fails on the second, which doesn’t allow b. As only one of the errors is reported, it may appear that it fails on the first disjunct, which is just confusing.

    The intention is to considerably improve disjunction errors soon. In the meantime, the added position information for conflict errors will hopefully help a hand to disambiguate such errors.

    let clauses are now allowed in comprehensions

    This has been a part of the spec for a long time, but now has finally been implemented.

    ... anywhere in struct

    This was already allowed by the spec, compiler, and evaluator, but now is also allowed by the parser.

    Simple string literals now allowed as selector

    Another thing that has been allowed by the spec for a while is the use of string literals as selectors. So a.”foo”.bar is now allowed. Only single-line strings without interpolations are allowed at the moment (as per the spec).

    Command line tool

    cli.Ask

    This is now supported.

    Error reporting

    Error reporting has been improved further. Most notably, positions for conflict errors (as resulting from 3 & 4), now have positions added and should be on par with v0.2.2. This also adds the fundamentals for adding richer position notation.

    Performance

    This release fixes one big performance issue, which could lead to exponential runtime for disjunctions. This can have a big impact for many configurations. For one user, it reduced the running time from 5 hours to less than 2 seconds.

    As part of this fix, we now introduced a benchmark suite.

    There are tons of significant performance improvements still possible, so this is not the last of it.

    Notable Bug fixes

    String handling

    CUE used to piggyback on Gosstrconv.Quote` for quoting strings. This was not ideal, as CUE and Go strings are not the same. It also resulted in some bugs. This is now fixed. This results in better string formatting overall.

    Handling of files named -

    The cue command line tool sometimes hung when a directory contained a file named -. This is now handled.

    Yaml version

    Until now, CUE would output YAML in version 1.2 and read it in version 1.1. And it seems that the “right thing to do” is to implement version 1.2 these days. This has now provisionally been fixed to read it mostly as version 1.2. A more proper implementation would be to replace internal/third_party/yaml to use the yaml.v3 package.

    This means, for instance, that the value no is now treated as a string instead of the boolean value false. (And thus the demo on cuelang.org will have to be replaced.)

    Message-level options in Protobuf

    Handling of message options in .proto files would previously fail. They are now translated into declaration options in CUE with the format: @protobuf(optionname=value).

    Backwards compatibility

    YAML

    As YAML 1.2 is not backwards compatible with YAML 1.1, inputs to CUE that relied on a YAML 1.1 interpretation may now yield different results.

    Closedness rules

    The refinement of the rules of closedness are largely backwards compatible (and what is not is not yet implemented), but some of the bug fixes may give slightly different results nonetheless.

    Changelog

    8ad07fa Readme.md: remove Go Report Card 2ac4d85 cue/format: indent multiline string literals based on context. c886094 cue/literal: implement CUE-specific quoting 11b507c cue/load: don't hang on files named - 6354c40 cue/parser: allow ... anywhere in struct 241c5bf cue/parser: allow let clause in comprehension 80f0bee cue/parser: better error message for old-style definitions 732b6d5 cue/parser: support string selector labels e9286f0 doc/ref/spec.md: fix builtin func ff306b7 doc/ref/spec.md: fix spec bug and broken link 15e8a05 doc/ref/spec.md: refinement of closedness 65468d5 doc/tutorial/kubernetes: update import 3af2683 encoding/protobuf: parse message options d23402c internal/core/compile: fix alias resolution bug 742593f internal/core/compile: fix hidden definition bug da69dc0 internal/core/eval: ... does not apply to defintions 0eb7cc5 internal/core/eval: add more positions to conflict errors 3914ef8 internal/core/eval: check for nil pointer 03fa5a8 internal/core/eval: fully evaluate nested unification f0df4df internal/core/eval: performance: unify an arc only once per node 1fdc02a internal/core/eval: simplify validators upon evaluator 1c29042 internal/third_party/yaml: drop support for non-standard scalars 5bfe37d pkg/tool/cli: support ask task

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(411 bytes)
    cue_0.3.0-alpha4_Darwin_x86_64.tar.gz(5.75 MB)
    cue_0.3.0-alpha4_Linux_arm64.tar.gz(5.06 MB)
    cue_0.3.0-alpha4_Linux_x86_64.tar.gz(5.54 MB)
    cue_0.3.0-alpha4_Windows_x86_64.zip(5.64 MB)
  • v0.3.0-alpha3(Jul 3, 2021)

    This release mainly introduces per-file inclusion based on tags, bug fixes for bugs introduced in the new evaluator, and tests to close issues raised on the old evaluator that are now verified to be fixed.

    Language extensions

    Attribute declarations in preamble

    Attribute declarations are now allowed before the package clause and import declarations. This is used to implement file-wide inclusion.

    Any scalar now allowed in interpolations

    This includes boolean and byte values. Boolean values are represented as their JSON value. In string interpolations, bytes are converted to valid UTF-8 by replacing the maximal subpart of ill-formed subsequences with a single replacement character (W3C encoding standard).

    Whole file inclusion

    CUE now allows files to be conditionally included in a build. To conditionally include a file in a CUE program, one can now write:

    @if(prod)
    
    package acmecorp
    

    and use the -t prod flag to cause this file to be included. The expression can be any valid CUE expression using only boolean operators and boolean values.

    See cue help injection for more details.

    Closedness rewrite

    A large number of bug fixes related to an incorrect “field not allowed” error are now in. The algorithm for closedness has been significantly simplified and streamlined based on the benefit of hindsight.

    Note that one may still have a misleading “field not allowed” error if a field is disallowed in one disjunct and allowed in another, but where there is another field in conflict. Right now only one of the errors is reported.

    CUE API

    New evaluator

    The CUE API is now moved to the new evaluator.

    Package dependencies

    Builtins now live in their own package and are no longer copied into the cue API. This was part of the effort to break the implementation into logical pieces, instead of having one large package.

    This has the additional advantage that each builtin package now only lives in its own directory and is not copied. This makes maintenance easier and also removes some of the limitations imposed by the old implementation.

    Unfortunately, this means that custom loader implementations (ones that do not use cue/load for loading packages) will have to include

    import 	_ "cuelang.org/go/pkg"
    

    somewhere in the package to trigger linking in the builtin packages.

    cue tool

    This release also marks the first steps of moving functionality from the cue command line tool to the CUE APIs. The intention is that the cue tool will be mostly a wrapper around core functionality present in the API. Other tooling could provide a different “view” without being second-class citizens in the ecosystem and while providing consistency between such tools.

    The first step towards this is the injection mechanism. The existing parts of this were moved to the cue/load package. The new file inclusion functionality, meanwhile, was directly implemented in cue/load.

    The next step will be to move non-CUE file support and packaging mechanisms to cue/load and friends. Then next up is the tooling layer, starting with an API for dependency analysis.

    Next steps

    There are two outstanding issues on the table to come to a beta release:

    Disjunctions

    Although there are no known semantic bugs related to disjunctions, the code needs some restructuring to improve error messages and to fix some performance issues. This is slated for the alpha4 release.

    Structural cycles

    There are still some bugs outstanding related to structural cycles. The plan is to address all of these in an alpha5 (or possibly beta) release.

    Backwards compatibility

    Go version

    Support for go1.12 was dropped. In exchange, this release is verified to build with Go’s tip (1.16).

    CUE API

    The CUE API has now fully been moved over to use the new evaluator. The old evaluator code has been removed.

    • cue.Merge: the purpose of Merge is to unify the underlying instances as data. It now no longer also unifies definitions. Bug fixes in closedness exposed this previously unnoticed oversight.

    Custom loader implementations (ones that do not use cue/load for loading packages) will have to include

    import 	_ "cuelang.org/go/pkg"
    

    somewhere in the package to trigger linking in the builtin packages.

    Language changes

    CUE code that previously expected a boolean or bytes value substituted in a string interpolation to fail my now break.

    cue cmd

    Commands that operate on multiple instances simultaneously (e.g. cue cmd foo ./...) can no longer reference definitions in the underlying commands. For more info see cue.Merge in the API section.

    Changelog

    d69319c all: add missing copyright 4c10692 all: fix four previously ignored errors 6c27cef ci: drop go1.12 add go1.15 c3f30d8 ci: name dispatch workflow differently to default branch push workflow 1c54297 cmd/cue/cmd: fix get go package handling 1e8906a cmd/cue/cmd: move injection mechanism to cue/load faf617c cmd/cue/cmd: verify issue 236 is fixed e71970c cmd/cue/cmd: verify issue 322 is fixed 8776561 cmd/cue/cmd: verify issue 425 is fixed ba5708b cmd/cue: get go: add --package flag 1a9b88d cmd/cue: get go: use constant.Value.ExactString to extract constant f25c04d cmd/help: fix example in doc 089f461 cue/load: add support for build tags bc101b3 cue/load: add tool and test files to BuildFiles if requested 20c637a cue/load: relax places where @tag is allowed bd3dd75 cue/load: rewrite pre-resolved files after injection c264475 cue/parser: allow attributes before package and import clauses 4cce6c4 cue: apply wrapping to all errors in list d2fdbf0 cue: clean up handling of preamble declarations 80e70b9 cue: fix instance error handling c76a530 cue: implement fill in terms of Value.Fill 845df05 cue: make internal cue port main API implementation abce145 cue: move to external builtin packages ce28b20 cue: prepare to hoist builtin code 9acd624 cue: prepare to hoist builtins, step 2 40a6bcd cue: remove dead code 6f84d11 deps: upgrade to github.com/rogpeppe/go-internal v1.6.1 edea6f3 doc/ref/spec.md: fix len count for structs 043c534 doc/ref/spec: clarification on cycles dd0fa88 doc/ref/spec: consider definitions for closedness c7c14fd doc/tutorial/kubernetes: remove a duplicate kind field e3a03a1 internal/core/adt: exclude definitions in data conversion db18b37 internal/core/adt: prevent nil interface bug for Source f5fa009 internal/core/adt: remove all closedness IDs in ToData 30ca062 internal/core/adt: support interpolation of bytes and bool df01042 internal/core/compile: better error message for user-defined errors d55e2b5 internal/core/eval: allow lists resulting from dependent expressions c72ce5d internal/core/eval: fix empty data vertex bug 0b57e43 internal/core/eval: fix interpolation of numbers e0c19d1 internal/core/eval: fix issue 507 304b02e internal/core/eval: nested comprehension fix 62528c3 internal/core/eval: rewrite of closedness algorithm dc2401c internal/core/eval: verify issue 258 is fixed ddd99d5 internal/core/eval: verify issue 293 is fixed ea5d964 internal/core/eval: verify issue 295 is fixed 480e1a1 internal/core/eval: verify issue 299 if fixed 541bf03 internal/core/eval: verify issue 306 is fixed 5905c22 internal/core/eval: verify issue 332 is fixed 121ace7 internal/core/eval: verify issue 342 is fixed 07ce2f6 internal/core/eval: verify issue 353 is fixed 4ecec98 internal/core/eval: verify issue 370 is fixed fbf9de3 internal/core/eval: verify issue 398 is fixed c6ade67 internal/core/export: fix bug 473 b9b2c54 internal/core/export: better handling of generated CUE 1e1fe6f internal/core/export: handle scalar values in adt.Vertex 67d1d42 internal/core/export: suport default picking df9468e internal/core/export: verify issue 349 is fixed 60c207f internal/core/runtime: collect errors for all files in resolve a6ce869 internal/core: finalize list in slice 148f67a internal/core: fix incorrect error handling 3d863fc internal/core: move CloseID from Environment to Conjunct bf882b0 pkg/internal: copy cue builtin code to separate package dda7de4 pkg/math: fix issue 418 4f4b7a3 pkg/tool/file/append: create file if none exists. c8a4514 pkg/tool/http: fix handling of errors 248794e pkg: generate per-directory tests from builtin_test.go 1bcb73a update go-internal to pull in fix for Go 1.16

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(411 bytes)
    cue_0.3.0-alpha3_Darwin_x86_64.tar.gz(5.73 MB)
    cue_0.3.0-alpha3_Linux_arm64.tar.gz(5.04 MB)
    cue_0.3.0-alpha3_Linux_x86_64.tar.gz(5.51 MB)
    cue_0.3.0-alpha3_Windows_x86_64.zip(5.61 MB)
A library for performing data pipeline / ETL tasks in Go.

Ratchet A library for performing data pipeline / ETL tasks in Go. The Go programming language's simplicity, execution speed, and concurrency support m

Daily Burn 385 Jan 19, 2022
Declarative streaming ETL for mundane tasks, written in Go

Benthos is a high performance and resilient stream processor, able to connect various sources and sinks in a range of brokering patterns and perform hydration, enrichments, transformations and filters on payloads.

Ashley Jeffs 4.8k Sep 23, 2022
Dud is a lightweight tool for versioning data alongside source code and building data pipelines.

Dud Website | Install | Getting Started | Source Code Dud is a lightweight tool for versioning data alongside source code and building data pipelines.

Kevin Hanselman 111 Sep 2, 2022
xyr is a very lightweight, simple and powerful data ETL platform that helps you to query available data sources using SQL.

xyr [WIP] xyr is a very lightweight, simple and powerful data ETL platform that helps you to query available data sources using SQL. Supported Drivers

Mohammed Al Ashaal 56 Jul 14, 2022
Prometheus Common Data Exporter can parse JSON, XML, yaml or other format data from various sources (such as HTTP response message, local file, TCP response message and UDP response message) into Prometheus metric data.

Prometheus Common Data Exporter Prometheus Common Data Exporter 用于将多种来源(如http响应报文、本地文件、TCP响应报文、UDP响应报文)的Json、xml、yaml或其它格式的数据,解析为Prometheus metric数据。

null 7 May 18, 2022
Gonum is a set of numeric libraries for the Go programming language. It contains libraries for matrices, statistics, optimization, and more

Gonum Installation The core packages of the Gonum suite are written in pure Go with some assembly. Installation is done using go get. go get -u gonum.

null 6.1k Sep 26, 2022
Baker is a high performance, composable and extendable data-processing pipeline for the big data era

Baker is a high performance, composable and extendable data-processing pipeline for the big data era. It shines at converting, processing, extracting or storing records (structured data), applying whatever transformation between input and output through easy-to-write filters.

AdRoll 152 Sep 7, 2022
Stream data into Google BigQuery concurrently using InsertAll() or BQ Storage.

bqwriter A Go package to write data into Google BigQuery concurrently with a high throughput. By default the InsertAll() API is used (REST API under t

null 11 Sep 1, 2022
Map, Reduce, Filter, De/Multiplex, etc. for the Go language.

proto proto gives Go operations like Map, Reduce, Filter, De/Multiplex, etc. without sacrificing idiomatic harmony or speed. It also introduces a conv

Erich Blume 60 Feb 20, 2022
indodate is a plugin for golang programming language for date convertion on indonesian format

indodate is a package for golang programming language for date conversion on indonesian format

Eko Kurniadi 1 Oct 23, 2021
Simple CRUD application using CockroachDB and Go

Simple CRUD application using CockroachDB and Go

Martín Montes 2 Feb 20, 2022
DEPRECATED: Data collection and processing made easy.

This project is deprecated. Please see this email for more details. Heka Data Acquisition and Processing Made Easy Heka is a tool for collecting and c

Mozilla Services 3.4k Sep 16, 2022
Kanzi is a modern, modular, expendable and efficient lossless data compressor implemented in Go.

kanzi Kanzi is a modern, modular, expendable and efficient lossless data compressor implemented in Go. modern: state-of-the-art algorithms are impleme

null 422 Aug 15, 2022
churro is a cloud-native Extract-Transform-Load (ETL) application designed to build, scale, and manage data pipeline applications.

Churro - ETL for Kubernetes churro is a cloud-native Extract-Transform-Load (ETL) application designed to build, scale, and manage data pipeline appli

churrodata 13 Mar 10, 2022
Dev Lake is the one-stop solution that integrates, analyzes, and visualizes software development data

Dev Lake is the one-stop solution that integrates, analyzes, and visualizes software development data throughout the software development life cycle (SDLC) for engineering teams.

Merico 57 Sep 23, 2022
A distributed, fault-tolerant pipeline for observability data

Table of Contents What Is Veneur? Use Case See Also Status Features Vendor And Backend Agnostic Modern Metrics Format (Or Others!) Global Aggregation

Stripe 1.6k Sep 6, 2022
Data syncing in golang for ClickHouse.

ClickHouse Data Synchromesh Data syncing in golang for ClickHouse. based on go-zero ARCH A typical data warehouse architecture design of data sync Aut

好未来技术 851 Sep 28, 2022
sq is a command line tool that provides jq-style access to structured data sources such as SQL databases, or document formats like CSV or Excel.

sq: swiss-army knife for data sq is a command line tool that provides jq-style access to structured data sources such as SQL databases, or document fo

Neil O'Toole 388 Sep 25, 2022