Decode / encode XML to/from map[string]interface{} (or JSON); extract values with dot-notation paths and wildcards. Replaces x2j and j2x packages.

Related tags

Text Processing mxj
Overview

mxj - to/from maps, XML and JSON

Decode/encode XML to/from map[string]interface{} (or JSON) values, and extract/modify values from maps by key or key-path, including wildcards.

mxj supplants the legacy x2j and j2x packages. If you want the old syntax, use mxj/x2j and mxj/j2x packages.

Installation

Using go.mod:
go get github.com/clbanning/mxj/[email protected]
import "github.com/clbanning/mxj/v2"

... or just vendor the package.

Related Packages

https://github.com/clbanning/checkxml provides functions for validating XML data.

Refactor Encoder - 2020.05.01

Issue #70 highlighted that encoding large maps does not scale well, since the original logic used string appends operations. Using bytes.Buffer results in linear scaling for very large XML docs. (Metrics based on MacBook Pro i7 w/ 16 GB.)
Nodes      m.XML() time
54809       12.53708ms
109780      32.403183ms
164678      59.826412ms
482598     109.358007ms

Refactor Decoder - 2015.11.15

For over a year I've wanted to refactor the XML-to-map[string]interface{} decoder to make it more performant. I recently took the time to do that, since we were using github.com/clbanning/mxj in a production system that could be deployed on a Raspberry Pi. Now the decoder is comparable to the stdlib JSON-to-map[string]interface{} decoder in terms of its additional processing overhead relative to decoding to a structure value. As shown by:
BenchmarkNewMapXml-4         	  100000	     18043 ns/op
BenchmarkNewStructXml-4      	  100000	     14892 ns/op
BenchmarkNewMapJson-4        	  300000	      4633 ns/op
BenchmarkNewStructJson-4     	  300000	      3427 ns/op
BenchmarkNewMapXmlBooks-4    	   20000	     82850 ns/op
BenchmarkNewStructXmlBooks-4 	   20000	     67822 ns/op
BenchmarkNewMapJsonBooks-4   	  100000	     17222 ns/op
BenchmarkNewStructJsonBooks-4	  100000	     15309 ns/op

Notices

2021.02.02: v2.5 - add XmlCheckIsValid toggle to force checking that the encoded XML is valid
2020.12.14: v2.4 - add XMLEscapeCharsDecoder to preserve XML escaped characters in Map values
2020.10.28: v2.3 - add TrimWhiteSpace option
2020.05.01: v2.2 - optimize map to XML encoding for large XML docs.
2019.07.04: v2.0 - remove unnecessary methods - mv.XmlWriterRaw, mv.XmlIndentWriterRaw - for Map and MapSeq.
2019.07.04: Add MapSeq type and move associated functions and methods from Map to MapSeq.
2019.01.21: DecodeSimpleValuesAsMap - decode to map[<tag>:map["#text":<value>]] rather than map[<tag>:<value>]
2018.04.18: mv.Xml/mv.XmlIndent encodes non-map[string]interface{} map values - map[string]string, map[int]uint, etc.
2018.03.29: mv.Gob/NewMapGob support gob encoding/decoding of Maps.
2018.03.26: Added mxj/x2j-wrapper sub-package for migrating from legacy x2j package.
2017.02.22: LeafNode paths can use ".N" syntax rather than "[N]" for list member indexing.
2017.02.10: SetFieldSeparator changes field separator for args in UpdateValuesForPath, ValuesFor... methods.
2017.02.06: Support XMPP stream processing - HandleXMPPStreamTag().
2016.11.07: Preserve name space prefix syntax in XmlSeq parser - NewMapXmlSeq(), etc.
2016.06.25: Support overriding default XML attribute prefix, "-", in Map keys - SetAttrPrefix().
2016.05.26: Support customization of xml.Decoder by exposing CustomDecoder variable.
2016.03.19: Escape invalid chars when encoding XML attribute and element values - XMLEscapeChars().
2016.03.02: By default decoding XML with float64 and bool value casting will not cast "NaN", "Inf", and "-Inf".
            To cast them to float64, first set flag with CastNanInf(true).
2016.02.22: New mv.Root(), mv.Elements(), mv.Attributes methods let you examine XML document structure.
2016.02.16: Add CoerceKeysToLower() option to handle tags with mixed capitalization.
2016.02.12: Seek for first xml.StartElement token; only return error if io.EOF is reached first (handles BOM).
2015.12.02: XML decoding/encoding that preserves original structure of document. See NewMapXmlSeq()
            and mv.XmlSeq() / mv.XmlSeqIndent().
2015-05-20: New: mv.StringIndentNoTypeInfo().
            Also, alphabetically sort map[string]interface{} values by key to prettify output for mv.Xml(),
            mv.XmlIndent(), mv.StringIndent(), mv.StringIndentNoTypeInfo().
2014-11-09: IncludeTagSeqNum() adds "_seq" key with XML doc positional information.
            (NOTE: PreserveXmlList() is similar and will be here soon.)
2014-09-18: inspired by NYTimes fork, added PrependAttrWithHyphen() to allow stripping hyphen from attribute tag.
2014-08-02: AnyXml() and AnyXmlIndent() will try to marshal arbitrary values to XML.
2014-04-28: ValuesForPath() and NewMap() now accept path with indexed array references.

Basic Unmarshal XML to map[string]interface{}

type Map map[string]interface{}

Create a Map value, 'mv', from any map[string]interface{} value, 'v':

mv := Map(v)

Unmarshal / marshal XML as a Map value, 'mv':

mv, err := NewMapXml(xmlValue) // unmarshal
xmlValue, err := mv.Xml()      // marshal

Unmarshal XML from an io.Reader as a Map value, 'mv':

mv, err := NewMapXmlReader(xmlReader)         // repeated calls, as with an os.File Reader, will process stream
mv, raw, err := NewMapXmlReaderRaw(xmlReader) // 'raw' is the raw XML that was decoded

Marshal Map value, 'mv', to an XML Writer (io.Writer):

err := mv.XmlWriter(xmlWriter)
raw, err := mv.XmlWriterRaw(xmlWriter) // 'raw' is the raw XML that was written on xmlWriter

Also, for prettified output:

xmlValue, err := mv.XmlIndent(prefix, indent, ...)
err := mv.XmlIndentWriter(xmlWriter, prefix, indent, ...)
raw, err := mv.XmlIndentWriterRaw(xmlWriter, prefix, indent, ...)

Bulk process XML with error handling (note: handlers must return a boolean value):

err := HandleXmlReader(xmlReader, mapHandler(Map), errHandler(error))
err := HandleXmlReaderRaw(xmlReader, mapHandler(Map, []byte), errHandler(error, []byte))

Converting XML to JSON: see Examples for NewMapXml and HandleXmlReader.

There are comparable functions and methods for JSON processing.

Arbitrary structure values can be decoded to / encoded from Map values:

mv, err := NewMapStruct(structVal)
err := mv.Struct(structPointer)

Extract / modify Map values

To work with XML tag values, JSON or Map key values or structure field values, decode the XML, JSON or structure to a `Map` value, 'mv', or cast a `map[string]interface{}` value to a `Map` value, 'mv', then:
paths := mv.PathsForKey(key)
path := mv.PathForKeyShortest(key)
values, err := mv.ValuesForKey(key, subkeys)
values, err := mv.ValuesForPath(path, subkeys)
count, err := mv.UpdateValuesForPath(newVal, path, subkeys)

Get everything at once, irrespective of path depth:

leafnodes := mv.LeafNodes()
leafvalues := mv.LeafValues()

A new Map with whatever keys are desired can be created from the current Map and then encoded in XML or JSON. (Note: keys can use dot-notation.)

newMap, err := mv.NewMap("oldKey_1:newKey_1", "oldKey_2:newKey_2", ..., "oldKey_N:newKey_N")
newMap, err := mv.NewMap("oldKey1", "oldKey3", "oldKey5") // a subset of 'mv'; see "examples/partial.go"
newXml, err := newMap.Xml()   // for example
newJson, err := newMap.Json() // ditto

Usage

The package is fairly well self-documented with examples.

Also, the subdirectory "examples" contains a wide range of examples, several taken from golang-nuts discussions.

XML parsing conventions

Using NewMapXml()

  • Attributes are parsed to map[string]interface{} values by prefixing a hyphen, -, to the attribute label. (Unless overridden by PrependAttrWithHyphen(false) or SetAttrPrefix().)
  • If the element is a simple element and has attributes, the element value is given the key #text for its map[string]interface{} representation. (See the 'atomFeedString.xml' test data, below.)
  • XML comments, directives, and process instructions are ignored.
  • If CoerceKeysToLower() has been called, then the resultant keys will be lower case.

Using NewMapXmlSeq()

  • Attributes are parsed to map["#attr"]map[<attr_label>]map[string]interface{}values where the <attr_label> value has "#text" and "#seq" keys - the "#text" key holds the value for <attr_label>.
  • All elements, except for the root, have a "#seq" key.
  • Comments, directives, and process instructions are unmarshalled into the Map using the keys "#comment", "#directive", and "#procinst", respectively. (See documentation for more specifics.)
  • Name space syntax is preserved:
    • <ns:key>something</ns.key> parses to map["ns:key"]interface{}{"something"}
    • xmlns:ns="http://myns.com/ns" parses to map["xmlns:ns"]interface{}{"http://myns.com/ns"}

Both

  • By default, "Nan", "Inf", and "-Inf" values are not cast to float64. If you want them to be cast, set a flag to cast them using CastNanInf(true).

XML encoding conventions

  • 'nil' Map values, which may represent 'null' JSON values, are encoded as <tag/>. NOTE: the operation is not symmetric as <tag/> elements are decoded as tag:"" Map values, which, then, encode in JSON as "tag":"" values.
  • ALSO: there is no guarantee that the encoded XML doc will be the same as the decoded one. (Go randomizes the walk through map[string]interface{} values.) If you plan to re-encode the Map value to XML and want the same sequencing of elements look at NewMapXmlSeq() and mv.XmlSeq() - these try to preserve the element sequencing but with added complexity when working with the Map representation.

Running "go test"

Because there are no guarantees on the sequence map elements are retrieved, the tests have been written for visual verification in most cases. One advantage is that you can easily use the output from running "go test" as examples of calling the various functions and methods.

Motivation

I make extensive use of JSON for messaging and typically unmarshal the messages into map[string]interface{} values. This is easily done using json.Unmarshal from the standard Go libraries. Unfortunately, many legacy solutions use structured XML messages; in those environments the applications would have to be refactored to interoperate with my components.

The better solution is to just provide an alternative HTTP handler that receives XML messages and parses it into a map[string]interface{} value and then reuse all the JSON-based code. The Go xml.Unmarshal() function does not provide the same option of unmarshaling XML messages into map[string]interface{} values. So I wrote a couple of small functions to fill this gap and released them as the x2j package.

Over the next year and a half additional features were added, and the companion j2x package was released to address XML encoding of arbitrary JSON and map[string]interface{} values. As part of a refactoring of our production system and looking at how we had been using the x2j and j2x packages we found that we rarely performed direct XML-to-JSON or JSON-to_XML conversion and that working with the XML or JSON as map[string]interface{} values was the primary value. Thus, everything was refactored into the mxj package.

Comments
  • XmlToMapParser nil error

    XmlToMapParser nil error

    I've encountered a case where 'skey' seems to be empty and golang throws 'panic: assignment to entry in nil map'. Adding a check for 'skey' fixes that.

    opened by adrianlungu 19
  • Reading directly from reader will fail parse, but writing and reading again succeed

    Reading directly from reader will fail parse, but writing and reading again succeed

    When getting the following perfectly valid XML (a response from PANW Wildfire component): ` 2.0 <file_info> yes b918d3b98f590d99ac3694b373958017256a22b4 PE dd8e64991f68eeebb52c9947ea56f39a776185d1a076741533aa6843022c03b0 7f638f13d0797ef9b1a393808dc93b94 55296 </file_info> <task_info> 3.0 2 Windows XP, Adobe Reader 9.4.0, Flash 10, Office 2007 dd8e64991f68eeebb52c9947ea56f39a776185d1a076741533aa6843022c03b0 7f638f13d0797ef9b1a393808dc93b94 55296 yes

    This is a WildFire test sample Created or modified a file in the Windows system folder Created or modified a file Modified the Windows Registry Created Process c:\documents and settings\administrator\sample.exe Set key \REGISTRY\MACHINE\SOFTWARE\PaloAlto\PanCar to value 1 3.0 5 Windows 7 x64 SP1, Adobe Reader 11, Flash 11, Office 2010 dd8e64991f68eeebb52c9947ea56f39a776185d1a076741533aa6843022c03b0 7f638f13d0797ef9b1a393808dc93b94 55296 yes This is a WildFire test sample Created or modified a file Modified the Windows Registry Created Process C:\Users\Administrator\sample.exe Set key \REGISTRY\MACHINE\SOFTWARE\Wow6432Node\PaloAlto\PanCar to value 1 3.0 204 PE Static Analyzer dd8e64991f68eeebb52c9947ea56f39a776185d1a076741533aa6843022c03b0 7f638f13d0797ef9b1a393808dc93b94 no Contains an invalid checksum Contains sections with size discrepancies ` And using the following function: `func ParseXMLResponseToJSON(reader io.Reader, resEntry *domain.Entry) error { b, err := ioutil.ReadAll(reader) if err != nil { return err } fmt.Println(string(b)) reader = bytes.NewReader(b) m, err := mxj.NewMapXmlReader(reader) if err != nil { return err } content, err := json.Marshal(m) if err != nil { return err } return string(content) }` I get the response in json. However, when I read direcly from the reader: `func ParseXMLResponseToJSON(reader io.Reader, resEntry *domain.Entry) error { /*b, err := ioutil.ReadAll(reader) if err != nil { return err } fmt.Println(string(b))*/ reader = bytes.NewReader(b) m, err := mxj.NewMapXmlReader(reader) if err != nil { return err } content, err := json.Marshal(m) if err != nil { return err } return string(content)` I get the following error: `xml.Decoder.Token() - XML syntax error on line 135: unexpected EOF`
    opened by Spivaka 11
  • Xml serialization time grows exponentially with size of map

    Xml serialization time grows exponentially with size of map

    map.Xml() time grows exponentially for large map. For example if we read 1MB XML and serialize it back it take almost 20s and if we double the size (duplicate the xml nodes from same input) then time taken is almost 90s.

    I tested with this xml (http://aiweb.cs.washington.edu/research/projects/xmltk/xmldata/data/mondial/mondial-3.0.xml) with following code.

    package main
    
    import (
    	"fmt"
    	"io/ioutil"
    	"time"
    
    	mxj "github.com/clbanning/mxj"
    )
    
    func main() {
    	b, _ := ioutil.ReadFile("/tmp/large.xml")
    	start := time.Now()
    	m, _ := mxj.NewMapXml(b)
    	fmt.Printf("XML Read time %v\n", time.Now().Sub(start))
    	start = time.Now()
    	b, _ = m.Xml()
    	fmt.Printf("XML Write time %v\n", time.Now().Sub(start))
    }
    

    I debugged further and I think it is because of string concatenation (*s += ). I created a simple code to string concatenation and found that it is very slow as string grows. When I replaced it by bytes.Buffer.Read then there was significant improvement.

    opened by nadddy 10
  • getSubKeyMap

    getSubKeyMap

    There seems to be a couple issues with this function.

    Getting a runtime error: index out of range [recovered] First the switch is on the length of the split string. If the length of the string array is 3 (case 3), then you can't reference vv[3] which would be the 4th element right? It must be vv[2] to access the third string.

    Second, if your text has any ":" in it, it will get confused. My text had a "http://blah/blah/blah" in it. So, after I made the above change, I got the error: "unknown subkey conversion spec:..."

    Basically I was trying to read in a kml file, change the description and name fields and then print it back out. I was trying to use the paths= PathsForKey("description") values =ValuesForPath(paths[0]).
    UpdateValuesForPath(newVal, path[0], "description:"+value[0]) with a double loop in there to get all the paths and values. But, because description can have any kind of text, this will not be possible. Is there a better way of doing this?

    Thanks, Lorie

    opened by lorie131 10
  • NewMapXmlSeq process instructions

    NewMapXmlSeq process instructions

    hi,

    I'm dealing with some XML files that have <?xml version="1.0" encoding="utf-8"?> at top of the file by using the NewMapXmlSeq cause I want to preserve the format and some comments in the file.

    But I could only get the result map[#procinst:map[#target:xml #inst:version="1.0" encoding="utf-8"]], and the returned error is no root key

    As you mentioned in the doc:

    comments, directives, and procinsts that are NOT part of a document with a root key will be returned as
      map[string]interface{} and the error value 'NoRoot'.
    

    How can I get the whole map of the XML file? I thought it's quite common that <?xml version="1.0" encoding="utf-8"?> not belong to any root

    opened by s850042002 8
  • [Bug]integer will be transformed to double

    [Bug]integer will be transformed to double

    Input json: "ipv4": { "address-mode": "manual", "address": { "ip": "10.10.10.10", "prefix-length": 30 }

    Output xml: <ipv4><address><ip>10.0.22.14</ip><prefix-length>30.000000</prefix-length></address><address-mode>manual</address-mode><enabled>true</enabled><mtu>1500.000000</mtu><vti_peer>10.10.10.10</vti_peer></ipv4>

    In this case, prefix-length:30 became 30.000000 Do anyone have the same problem with me?

    opened by chenYunHsin-maker 7
  • XmlWriter produces formally incorrect XML

    XmlWriter produces formally incorrect XML

    But it should not, of course. E.g. serializing this mxj-map

            m[""] = "empty"
    	m["$invalid"] = "hex$"
    	m["entities"] = "<>&"
    	m["nil"] = nil
    

    produces this string, which is formally incorrect XML:

    <root>
      <>empty</>
      <$invalid>hex$</$invalid>
      <entities><>&</entities>
      <nil><nil></nil>
    </root>
    

    Errors are:

    • <> is not a valid XML element
    • $ is not valid in XML element names
    • the three basic XML entities <, > and & must be escaped as &lt;, &gt; and &amp;
    • the nil interface{} value is serialized as <nil> without escaping < and >

    Additionally there are some decisions to be drawn for how to mitigate these problems. Probably some strategies need to be implemented for a client to customize the behavior of the API

    • empty element names should either be omitted or given a customizeable name
    • invalid element name characters could be either dropped completely or somehow encoded by valid entity characters like a hex string representation decorated with the valid _ char
    • nil valued elements could be serialized as <element-name/>, <element-name></element-name>, omitted, or marked with xsi:nil="true" (see e.g. https://www.oreilly.com/library/view/xml-in-a/0596007647/re166.html)

    test fixture (example)

    package web
    
    import (
    	"bytes"
    	"testing"
    
    	"github.com/clbanning/mxj/v2"
    )
    
    func TestJSON2XMLConverter(t *testing.T) {
    	m := mxj.New()
    
    	m[""] = "empty"
    	m["$invalid"] = "hex$"
    	m["entities"] = "<>&"
    	m["nil"] = nil
    
    	buf := new(bytes.Buffer)
    	err := m.XmlWriter(buf, "root")
    	if err != nil {
    		t.Fatalf("writer: %v", err)
    	}
    
    	if got, want := buf.String(), `<root><_24_invalid>hex$</_24_invalid><entities>&lt;&gt;&amp;</entities><nil></nil></root>`; got != want {
    		// want <root><_>empty</_><_24_invalid>hex$</_24_invalid><entities>&lt;&gt;&amp;</entities><nil></nil></root>
    		// got <root><>empty</><$invalid>hex$</$invalid><entities><>&</entities><nil><nil></nil></root>
    		t.Errorf("want %v, got %v", want, got)
    	}
    }
    
    opened by Lercher 6
  • When marshal with map[string]string, it gave me UNKNOWN/>"">

    When marshal with map[string]string, it gave me ">UNKNOWN/>"

    Hi there, mxj is a great project and I've been using it in my gf project in XML marshal/unmarshal. It worked quite well with map[string]interface{} type, but when the parameter contains other map type (eg: map[string]string/map[string]int etc...), it failed working.

    Here's the simple codes to reproduce this issue:

    package main
    
    import (
        "fmt"
        "github.com/clbanning/mxj"
    )
    
    func main() {
        m := make(map[string]interface{})
        m["m"] = map[string]string {
            "k" : "v",
        }
        b, _ := mxj.Map(m).Xml()
        fmt.Println(string(b))
    }
    

    It's supposed to give me {"m":{"k":"v"}}, but actually I got >UNKNOWN/>.

    Would you please take a look, and kindly give me a nice feedback, thanks!

    opened by gqcn 6
  • Inner Tag Ordering

    Inner Tag Ordering

    Is there any way to get the ordering of inner tags, if they are variable in their tag name?

    For example:

    Obj Attr3 Attr1 Attr2 Obj

    This ends up a map

    {Obj:{Attr1:"",Attr2:"",Attr3:""}}, and there is no way to tell that the order was 3,1,2. I've run into a situation where I need to know the order, is there some flag or something that would allow me to turn the attributes into an ordered array, or to write out the order the parser ran into them as an attribute or something?

    opened by murphysean 6
  • NewMapXml truncate html tag content

    NewMapXml truncate html tag content

    Using NewMapXml with data that contains HTML tags results in truncated output.

    package main
    
    import (
     "fmt"
    
     "github.com/clbanning/mxj"
    )
    
    func main() {
     bytes := []byte(`<?xml version="1.0"?>
    <article>
    	<body>
        		<p>hello world<b>truncate me</b></p>
    	</body>
    </article>`)
    
     m, err := mxj.NewMapXml(bytes)
     if err != nil {
      fmt.Println("err:", err)
      return
     }
     fmt.Println("mxj.Map:\n",m.StringIndent(3))
    
     mxj.XMLEscapeChars(false)
     v, err := m.Xml()
     if err != nil {
      fmt.Println("err:", err)
      return
     }
     fmt.Println("\nm.Xml:\n", string(v))
    }
    
    mxj.Map:
           article : 
            body : 
              p : [string] hello world
    
    m.Xml:
     <article><body><p>hello world</p></body></article>
    

    The expected behaviour could be to leave the content as it is without the truncation. @clbanning what do you think?

    opened by salimane 5
  • Unexpected handling of entities

    Unexpected handling of entities

    Using NewMapXml with data that contains entities (e.g. &lt;) results in unexpected output.

    bytes := []byte(`<?xml version="1.0"?>
    <systemList>
      <system>
        <name>retropie</name>
        <fullname>RetroPie</fullname>
        <path>/home/fozz/RetroPie/retropiemenu</path>
        <extension>.rp .sh</extension>
        <command>sudo /home/fozz/RetroPie-Setup/retropie_packages.sh retropiemenu launch %ROM% &lt;/dev/tty &gt;/dev/tty</command>
        <platform/>
        <theme>retropie</theme>
      </system>
    </systemList>`)
    
    val, _ := mxj.NewMapXml(bytes)
    
    // Print val["systemList"]["system"]["command"]
    
    fmt.Println(val["systemList"].(map[string]interface{})["system"].(map[string]interface{})["command"])
    
    //  sudo /home/fozz/RetroPie-Setup/retropie_packages.sh retropiemenu launch %ROM% </dev/tty >/dev/tty
    

    I expected the value of systemList > system > command to be: sudo /home/fozz/RetroPie-Setup/retropie_packages.sh retropiemenu launch %ROM% &lt;/dev/tty &gt;/dev/tty

    But I can see that it's actual value is: sudo /home/fozz/RetroPie-Setup/retropie_packages.sh retropiemenu launch %ROM% </dev/tty >/dev/tty

    opened by TomWright 5
  • * fix unescaped double quotes (

    * fix unescaped double quotes (") placed in values of XML attributes

    If value of XML attribute contains double quotes (") and mxj.XMLEscapeChars is false, the library produces invalid XML through XML() function (unescaped double quotes are placed into value of attributes).

    opened by miketobler 1
  • What's the best way to compute

    What's the best way to compute "#seq" when inserting new data into a MapSeq?

    I'm currently using mxj.MapSeq to modify XML documents, but I find adding new nodes cumbersome. It's likely that I'm doing something wrong, so I come to you for advice :sweat_smile:

    Right now, when I want to add a new child node to some XML element, I assume that I need to figure out manually what the sequence number of that child node should be. To do so, I walk the MapSeq to find the largest number, then I add one to that. Is there a better way to do this?

    In code, that looks like:

    
    // mapSeqNumber returns the sequence number associated with this map
    func mapSeqNumber(m map[string]interface{}) (int, bool) {
    	if seq, ok := m["#seq"].(int); ok {
    		return seq, true
    	}
    
    	if seq, ok := m["#seq"].(float64); ok {
    		return int(seq), true
    	}
    
    	return 0, false
    }
    
    // mapSeqMaxSeqNumber returns the maximum sequence number of all child maps
    func mapSeqMaxSeqNumber(m map[string]interface{}) int {
    	maxSeqNumber := -1
    	for _, val := range m {
    		switch data := val.(type) {
    		case map[string]interface{}:
    			seqNumber, ok := mapSeqNumber(data)
    			if !ok {
    				continue
    			}
    			if seqNumber <= maxSeqNumber {
    				continue
    			}
    			maxSeqNumber = seqNumber
    		case []interface{}:
    			for _, v := range data {
    				vMap, ok := v.(map[string]interface{})
    				if !ok {
    					continue
    				}
    				seqNumber, ok := mapSeqNumber(vMap)
    				if !ok {
    					continue
    				}
    				if seqNumber <= maxSeqNumber {
    					continue
    				}
    				maxSeqNumber = seqNumber
    			}
    		}
    	}
    	return maxSeqNumber
    }
    
    // Load some XML
    m, _ := mxj.NewMapXmlSeq(/*... XML DATA ...*/)
    // Get the maximum child sequence number
    maxSeqNumber := mapSeqMaxSeqNumber(m)
    // Create a new child with the next sequence number
    m["NewChildNode"] = map[string]interface{
        "#seq": maxSeqNumber + 1,
    }
    
    opened by davidkrauser 6
  • Integration with Google oss-fuzz fuzzing service

    Integration with Google oss-fuzz fuzzing service

    Hi, I would like to help integrate this project into Google oss-fuzz.

    • As an initial step for integration I have created this PR: https://github.com/google/oss-fuzz/pull/8155, it contains necessary logic from an OSS-Fuzz perspective to integrate mxj. This includes developing initial fuzzers as well as integrating into OSS-Fuzz.

    • Essentially, OSS-Fuzz is a free service run by Google that performs continuous fuzzing of important open source projects.

    • If you would like to integrate, could I please have an email(s), it must be associated with a Google account like gmail (why?). by doing that, the provided email(s) will get access to the data produced by OSS-Fuzz, such as bug reports, coverage reports and more stats.

    • Notice the email(s) affiliated with the project will be public in the OSS-Fuzz repo, as they will be part of a configuration file.

    opened by manunio 1
Releases(2.5.7)
Owner
Charles Banning
Charles Banning
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

omniparser Omniparser is a native Golang ETL parser that ingests input data of various formats (CSV, txt, fixed length/width, XML, EDI/X12/EDIFACT, JS

JF Technology 494 Sep 27, 2022
Convert xml and json to go struct

xj2go The goal is to convert xml or json file to go struct file. Usage Download and install it: $ go get -u -v github.com/wk30/xj2go/cmd/... $ xj [-t

null 28 Sep 27, 2022
wikipedia-jsonl is a CLI that converts Wikipedia dump XML to JSON Lines format.

wikipedia-jsonl wikipedia-jsonl is a CLI that converts Wikipedia dump XML to JSON Lines format. How to use At first, download the XML dump from Wikime

Minoru Osuka 2 Feb 13, 2022
Parses the Graphviz DOT language in golang

Parses the Graphviz DOT language and creates an interface, in golang, with which to easily create new and manipulate existing graphs which can be writ

Walter Schulze 495 Sep 24, 2022
Extract structured data from web sites. Web sites scraping.

Dataflow kit Dataflow kit ("DFK") is a Web Scraping framework for Gophers. It extracts data from web pages, following the specified CSS Selectors. You

Dmitry Narizhnykh 543 Sep 17, 2022
Extract urls from text

xurls Extract urls from text using regular expressions. Requires Go 1.13 or later. import "mvdan.cc/xurls/v2" func main() { rxRelaxed := xurls.Relax

Daniel Martí 962 Sep 24, 2022
parse and generate XML easily in go

etree The etree package is a lightweight, pure go package that expresses XML in the form of an element tree. Its design was inspired by the Python Ele

Brett Vickers 991 Sep 21, 2022
'go test' runner with output optimized for humans, JUnit XML for CI integration, and a summary of the test results.

gotestsum gotestsum runs tests using go test --json, prints formatted test output, and a summary of the test run. It is designed to work well for both

null 1.1k Sep 23, 2022
Go XML sitemap and sitemap index generator

Install go get github.com/turk/go-sitemap Example for sitemapindex func () main(c *gin.Context) { s := sitemap.NewSitemapIndex(c.Writer, true)

Suleyman Yilmaz 3 Jun 29, 2022
This package provides Go (golang) types and helper functions to do some basic but useful things with mxGraph diagrams in XML, which is most famously used by app.diagrams.net, the new name of draw.io.

Go Draw - Golang MX This package provides types and helper functions to do some basic but useful things with mxGraph diagrams in XML, which is most fa

null 2 Aug 30, 2022
Quick and simple parser for PFSense XML configuration files, good for auditing firewall rules

pfcfg-parser version 0.0.1 : 13 January 2022 A quick and simple parser for PFSense XML configuration files to generate a plain text file of the main c

Rory Campbell-Lange 0 Jan 13, 2022
Sqly - An easy-to-use extension for sqlx, base on xml files and named query/exec

sqly An easy-to-use extension for sqlx ,base on xml files and named query/exec t

nvac 1 Jun 12, 2022
xmlquery is Golang XPath package for XML query.

xmlquery Overview xmlquery is an XPath query package for XML documents, allowing you to extract data or evaluate from XML documents with an XPath expr

null 307 Sep 11, 2022
Extraction politique de conformité : xlsx (fichier de suivi) -> xml (format AlgoSec)

go_policyExtractor Extraction politique de conformité : xlsx (fichier de suivi) -> xml (format AlgoSec). Le programme suivant se base sur les intitulé

Nokeni 0 Nov 4, 2021
axmlfmt is an opinionated formatter for Android XML resources

axmlfmt axmlfmt is an opinionated formatter for Android XML resources. It takes XML that looks like <?xml version="1.0" encoding="utf-8"?> <LinearLayo

Rashad Sookram 2 May 14, 2022
Freestyle xml parser with golang

fxml - FreeStyle XML Parser This package provides a simple parser which reads a XML document and output a tree structure, which does not need a pre-de

null 8 Jul 1, 2022
🧑‍💻 Go XML generator without Structs™

exml ??‍?? Go XML generator without Structs™ Package exml allows XML documents to be generated without the usage of structs or maps. It is not intende

Victor 2 May 16, 2022
Unified text diffing in Go (copy of the internal diffing packages the officlal Go language server uses)

gotextdiff - unified text diffing in Go This is a copy of the Go text diffing packages that the official Go language server gopls uses internally to g

Hexops 89 Sep 8, 2022
Package strit introduces a new type of string iterator, along with a number of iterator constructors, wrappers and combinators.

strit Package strit (STRing ITerator) assists in development of string processing pipelines by providing a simple iteration model that allows for easy

Maxim 84 Jun 21, 2022