What exactly did you do?

Tried to use <<UNORDERED>> in side a payload with an array complains length doesn't match

ja.Assertf(string(body), `
	{
		"code":200,
		"revenue":[
			"<<UNORDERED>>",
			{"date":"2020-10-01","value":100.01},
			{"date":"2020-10-02","value":200.01}
		]
	}`)

What did you expect would happen?

It would work and validate the json

What actually happened?

    overall_structs_test.go:57: length of arrays at '$.revenue' were different. Expected array to be of length 3, but contained 2 element(s)
    overall_structs_test.go:57: actual JSON at '$.revenue' was:
        [{"date":"2020-10-01","value":100.01},{"date":"2020-10-02","value":200.01}]
        but expected JSON was:
        ["\u003c\u003cUNORDERED\u003e\u003e",{"date":"2020-10-01","value":100.01},{"date":"2020-10-02","value":200.01}]

Additional info

• Output from go version: [e.g. go version go1.14.5 darwin/amd64] go version go1.17.4 darwin/amd64
• Version of this package: [e.g. v.1.0.2] v1.0.2

What exactly did you do?

Compare <<UNORDERED>> JSON arrays where the elements are objects and the expectation for some fields only checks for <<PRESENCE>>.

Test example:

func TestUNORDEREDwithPRESENCE(t *testing.T) {
	act := `{
		"data": [
			{ "foo": 1, "bar": 2 },
			{ "foo": 11, "bar": 22 }
		]
	}`
	exp := `{
		"data": [
			"<<UNORDERED>>",
			{ "foo": 11, "bar": "<<PRESENCE>>" },
			{ "foo": 1, "bar": "<<PRESENCE>>" }
		]
	}`
	ja := jsonassert.New(t)
	ja.Assertf(act, exp)
}

See test examples with some sanity in this Go Playground.

What did you expect would happen?

The test should pass - I should be able to compare <<UNORDERED>> arrays having elements with <<PRESENCE>> in them.

What actually happened?

=== RUN   TestUNORDEREDwithPRESENCE
    prog.go:81: actual JSON at '$.data[0]' contained an unexpected element: {"bar":2,"foo":1}
    prog.go:81: actual JSON at '$.data[1]' contained an unexpected element: {"bar":22,"foo":11}
    prog.go:81: expected JSON at '$.data[0]':
        {"bar":"\u003c\u003cPRESENCE\u003e\u003e","foo":11}
        was missing from actual payload
    prog.go:81: expected JSON at '$.data[1]':
        {"bar":"\u003c\u003cPRESENCE\u003e\u003e","foo":1}
        was missing from actual payload
--- FAIL: TestUNORDEREDwithPRESENCE (0.00s)
FAIL

Additional info

• Output from go version: go version go1.18.2 darwin/arm64
• Version of this package: v.1.1.0

As can be seen in the Go Playground example, I did some sanity tests, and it seems that <<UNORDERED>> and <<PRESENCE>> work well for the example above, except for when they are mixed together. Looking in the code, it seems to be caused by. the difference between Asserter.checkArrayUnordered and Asserter.checkArrayOrdered. The unordered version uses Asserter.deepEqual, whereas the ordered version uses Asserter.pathassertf.

Hi, thanks for the package. I had a particular use case using a testing library and did a quick and messy fork to prove the concept on a fork. This might be outside the scope or your vision of the project which is completely fine. I could see others having a similar issue with any testing framework, and adding this flexibility might increase adoption

What problem are you trying to solve?

Use JSON comparison features without involving testing.T
I'm using a testing/assertion library that does not check JSON structure well (Go Convey). I'd like to use the reporter of the testing library instead fo directly on testing.T

Describe how you wish this package would help you solve the problem

I'd like access to a comparison function that takes the same signature as the Assertf() method and returns an error or nil

Example

I have done a quick proof-of-concept for my needs at this fork: ryanlelek/jsonassert.
This involved a lot of copy/paste and removal of the method on (a *Asserter) You could probably split the comparison out similar to how I did and then wrap in a method for (a *Asserter) Here's how I'm able to use within testing framework

err := jsonassert.Assert(res.BodyString, `{
    "format": "json",
    "data": "example"
}`)
So(err, ShouldBeNil)

Then I can check the value of err within the testing framework

What exactly did you do?

ja.Assertf(`{"test":"A%AAA%BBBB%CC"}`, `{"test":"A%AAA%BBBB%CC"}`)

I think this happens because the fmt.Sprintf used in the package

What did you expect would happen?

I expect assert pass in this situation my guess is on the fmt.Sprintf used in Assertf function, It identifies the %char as some unsupported formatter

What actually happened?

Assertf format %A has unknown verb A

Additional info

• Output from go version: go version go1.15.13 darwin/amd64
• Version of this package: v1.0.1

Welcome!

Thanks for checking out this PR: it is an eternal review of the project from its humble beginnings. Feel free to have a look through the diff and identify any locations where you feel there's room for improvement. I really do appreciate these fly-by reviews.

Setting up the repository

This document serves as both a list of all the rules and steps to set up a very good open source Go project, and also the issue contents of the GitHub issue generated by github.com/kinbiko/go-template's astroturf.sh script that automates a bunch of these very steps.

CI

• [x] Builds against one of the 2 latest Go versions
• [x] Has golangci-lint v1.45.2 passing
• [x] Runs tests with the race detector
• [x] Runs integration tests, if applicable
• [x] Reports coverage to coveralls
• [x] Code scanning with CodeQL

Community Standards

• [x] Description w/ tags
• [x] README
• [x] Code of conduct
• [x] Contributing guideline
• [x] License
• [x] Issue template(s)
• [x] Pull request template
• [x] CODEOWNERS

GitHub settings

• [x] Wiki pages disabled
• [x] Sponsorship enabled
• [x] Projects disabled (my OSS work is managed in a central project board)
• [x] GitHub archive program enabled
• [x] Discussions disabled
• [x] Merge commits disabled
• [x] Squash merging enabled
• [x] Rebase merging disabled
• [x] Suggest updating branches enabled
• [x] Auto-merge disabled
• [x] Auto-delete head branches enabled
• [x] Limit "approve" or "request changes" to users with explicit permissions
• [x] Default branch is main, with the following branch protection rules enabled (everything else disabled):
  • [x] Require PRs before merging (no approvals required, don't dismiss PR approvals on new commits, don't require codeowner review)
  • [x] Linear history enabled (assumption: only enabling squash merges globally means linear history in default branch)
• [x] Allow kinbiko and select non-kinbiko actions only:
  • [x] Allow actions created by GitHub enabled
  • [x] Allow actions by Marketplace verified creators enabled
  • [x] shogo82148/[email protected] added as allowed action
• [x] Require approval before running actions for first-time contributors
• [x] GitHub actions approving PRs disabled
• [x] Dependabot alerts enabled for security vulns
• [x] Dependabot updates enabled for security vulns
• [x] Code scanning (CodeQL) enabled
• [x] Consistent labels:
  • bug
  • dependencies
  • documentation
  • enhancement
  • good first issue
  • help wanted
  • question

README

• [x] Has tags:
  • [x] Build status
  • [x] Coverage percentage
  • [x] Go report link
  • [x] Latest version
  • [x] Godoc with link to pkg.go.dev
  • [x] License tag
• [x] Explains the why and then the what.
• [x] Usage information incl any installation guidelines
• [x] Link to docs

What problem are you trying to solve?

It's not possible to use this package to make assertions against payloads that have arrays that aren't deterministic in their ordering.

Describe how you wish this package would help you solve the problem

Expose a directive similar to <<PRESENCE>> that can be added as the first element of the array in order to ignore ordering.

// should pass
ja.Assertf(`["foo", "bar"]`, `["<<UNORDERED>>", "bar", "foo"]`)

References

This issue was created based on the request from @blaskovicz on #25.

• Use the "<<UNORDERED>>" directive as the first string in the unordered array to ignore ordering when comparing arrays.
• Adds unit tests for this new logic.
• Adds a big real-world payload integration test that should cover most of the features of this package in combination

Closes #31

To do

There are still a few more changes that should be implemented before cutting a new minor release.

• [x] Documentation in README
• [x] Documentation in Go Docs
• [x] Example test documentation
• [ ] Take this feature out for a spin on a real project.
• [x] Self-review of the code

Add missing test cases that became apparent after reorganizing the integration test structure into subtests.

Hoping that it will become easier to implement and gain confidence in other features such as <<UNORDERED>> and <<PARTIAL>> after this change.

Instead of checking if the printer adheres to the testing.tt interface throughout, wrap any non-matching types in a utility with a NOOP Helper() implementation.

Removes the unnecessary exposure of the provided Printer in the Asserter. In the strictest sense, this change isn't backwards compatible, in that Asserter no longer exposes a Printer field. If for some obscure reason this affects you, you can replace all calls to ja.Printer with the type that you passed in to New().

This means that the line number at which a test assertion appeared will show up at the right line of code relative to what the user wrote, as opposed to where in this package it appeared.

CC: @CodeHex - beat you to it

What problem are you trying to solve?

I understand that this is an assertion tool and not a snapshotting tool, but it's the best choice for asserting that the JSON returned from an API call is correct, and in general it works fantastically for this.

However, it is slightly clunky when you are first writing a test. Unless you know exactly what the JSON should look like, you either need to generate it manually and paste into the test or else go through one value at a time fixing up errors until the test passes. Both of which are awkward.

(And yes, I'm aware this is slightly lazy, but it's super convenient especially when there's a lot of large JSON responses to check)

Describe how you wish this package would help you solve the problem

It would be fantastic if there were some way that the library could output the full actual JSON in some cases. Maybe if the expected JSON were just the blank string, for example.

That way you could write a test of

	ja.Assertf(string(body), ``)

Run the test, copy the output JSON into the test and know that it's correct.

Closes #20

• Adds Containsf API
• Removes silly panic.
• Implements an "<<UNORDERED>>" feature to ignore ordering in arrays.

TODO:

• [ ] Buggy: Commented-out test still fails -- will return to fixing it at some point.
• [ ] Self-review
• [ ] Consider releasing an alpha/beta version.
• [ ] Tidy up commits.
• [ ] Judge whether to commit to introducing this new feature, or to drop it for simplicity.

Hello. First of all, great package!
It really reduces boilerplate code in tests and makes everything more readable.

I have a feature request/consideration I hope would get included to this project and I may help implement it if that's something that makes sense for this library: Partial JSON matching.

It can be behind a flag or in another method, but it basically means a JSON with 4 properties, when checked against a JSON with 2 properties, would not return the following error:

expected 2 keys at '$' but got 4 keys unexpected object key(s) ["status","cause"] found at '$'

More often than not, we need to check for the overall structure and main fields and not the whole thing, and having to check for presence of every single field can take a lot of time.

Is this something that makes sense for you as the author of this library?
As much as I like creating my own fork, I think it's best for the community if we keep things centralized.

Thanks.