Java properties scanner for Go

Overview

Travis CI Status License GoDoc

Overview

Please run git pull --tags to update the tags. See below why.

properties is a Go library for reading and writing properties files.

It supports reading from multiple files or URLs and Spring style recursive property expansion of expressions like ${key} to their corresponding value. Value expressions can refer to other keys like in ${key} or to environment variables like in ${USER}. Filenames can also contain environment variables like in /home/${USER}/myapp.properties.

Properties can be decoded into structs, maps, arrays and values through struct tags.

Comments and the order of keys are preserved. Comments can be modified and can be written to the output.

The properties library supports both ISO-8859-1 and UTF-8 encoded data.

Starting from version 1.3.0 the behavior of the MustXXX() functions is configurable by providing a custom ErrorHandler function. The default has changed from panic to log.Fatal but this is configurable and custom error handling functions can be provided. See the package documentation for details.

Read the full documentation on GoDoc

Getting Started

import (
	"flag"
	"github.com/magiconair/properties"
)

func main() {
	// init from a file
	p := properties.MustLoadFile("${HOME}/config.properties", properties.UTF8)

	// or multiple files
	p = properties.MustLoadFiles([]string{
			"${HOME}/config.properties",
			"${HOME}/config-${USER}.properties",
		}, properties.UTF8, true)

	// or from a map
	p = properties.LoadMap(map[string]string{"key": "value", "abc": "def"})

	// or from a string
	p = properties.MustLoadString("key=value\nabc=def")

	// or from a URL
	p = properties.MustLoadURL("http://host/path")

	// or from multiple URLs
	p = properties.MustLoadURL([]string{
			"http://host/config",
			"http://host/config-${USER}",
		}, true)

	// or from flags
	p.MustFlag(flag.CommandLine)

	// get values through getters
	host := p.MustGetString("host")
	port := p.GetInt("port", 8080)

	// or through Decode
	type Config struct {
		Host    string        `properties:"host"`
		Port    int           `properties:"port,default=9000"`
		Accept  []string      `properties:"accept,default=image/png;image;gif"`
		Timeout time.Duration `properties:"timeout,default=5s"`
	}
	var cfg Config
	if err := p.Decode(&cfg); err != nil {
		log.Fatal(err)
	}
}

Installation and Upgrade

$ go get -u github.com/magiconair/properties

License

2 clause BSD license. See LICENSE file for details.

ToDo

  • Dump contents with passwords and secrets obscured

Updated Git tags

13 Feb 2018

I realized that all of the git tags I had pushed before v1.7.5 were lightweight tags and I've only recently learned that this doesn't play well with git describe 😞

I have replaced all lightweight tags with signed tags using this script which should retain the commit date, name and email address. Please run git pull --tags to update them.

Worst case you have to reclone the repo.

#!/bin/bash
tag=$1
echo "Updating $tag"
date=$(git show ${tag}^0 --format=%aD | head -1)
email=$(git show ${tag}^0 --format=%aE | head -1)
name=$(git show ${tag}^0 --format=%aN | head -1)
GIT_COMMITTER_DATE="$date" GIT_COMMITTER_NAME="$name" GIT_COMMITTER_EMAIL="$email" git tag -s -f ${tag} ${tag}^0 -m ${tag}

I apologize for the inconvenience.

Frank

Issues
  • FTBFS: test failure:

    FTBFS: test failure: "circular reference"

    As reported in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830727 magiconair/properties fails test on x86_64 as follows:

      === RUN   Test
      2016/07/10 22:26:08 circular reference
      exit status 1
      FAIL  github.com/magiconair/properties    0.004s
    
    opened by onlyjob 12
  • backslashes must be escaped on write

    backslashes must be escaped on write

    according to https://docs.oracle.com/cd/E23095_01/Platform.93/ATGProgGuide/html/s0204propertiesfileformat01.html The backslash character must be escaped as a double backslash

    opened by apesternikov 9
  • Make TestMustGetParsedDuration backwards compatible

    Make TestMustGetParsedDuration backwards compatible

    Fails:

    % go version go version go1.18beta2 linux/amd64

    As per:

    https://go.dev/doc/devel/release#policy
    

    Supported versions are today 1.16 and 1.17, so we can drop this brittle check.

    opened by maage 6
  • enhance error message for circular references

    enhance error message for circular references

    Note: This is an improvement over #49 for circular reference error message.

    Context

    When there are properties with circular references, the error message thrown contains one of the key in the cycle. This can be a bit annoying to fix, since it does not give a trace. Given this library supports loading properties from multiple sources, grep etc can fail.

    Input:

    foo = depends on ${bar}
    
    bar = which depends on ${baz}
    
    baz = ${foo} completes a cyclic reference
    
    

    The error message thrown when properties.MustLoadFile is called with the above file:

    circular reference in "foo = ${foo}"
    

    This pull request is an attempt to make it look better, my take:

    circular reference in:
    foo=depends on ${bar}
    bar=which depends on ${baz}
    baz=${foo} completes a cyclic reference
    

    This is not perfect, it still does not highlight the exact source of these properties, but I felt that the overhead of carrying that context to properties.expand may be a bit too much, since the method is currently stateless.

    Thanks for the library!

    opened by sriv 6
  • have symbol equal in the value property

    have symbol equal in the value property

    Hi I ran into this issue from eBay fabio, i want to set property 'registry.consul.register.tags'. i.e. "key1=val1" as one of the tag, but it has this kind of error: [FATAL] 1.1.5. circular reference

    is that a way to escape equal symbol. thanks

    invalid 
    opened by wsf1991 6
  • Support typed setter functions

    Support typed setter functions

    I'm not sure if I'm missing something, but it seems you can only set properties to strings. I know you can read other types, but what if I was to SetInt()?

    enhancement 
    opened by theherk 5
  • Bug(writeComments): Changes comments if \ is included

    Bug(writeComments): Changes comments if \ is included

    When reading comments \ are loaded correctly, but when writing they are then replaced by \\. This leads to wrong comments when writing and reading multiple times.

    I assume that it is not necessary to encode the comments.

    opened by doxsch 4
  • Support duplicate expansions in the same value

    Support duplicate expansions in the same value

    This enables properties like this:

    a = b
    c = ${a} ${a}
    

    Previously, this would incorrectly throw a circular reference error. Now instead this will correctly evaluate the above as "c = b b".

    opened by yobert 4
  • Make expansion of expressions optional per load

    Make expansion of expressions optional per load

    As part of our configuration service, we need to store configurations as is in key-value records without substituing the ${..} expressions. Application that retrieve properties from this service will do the expansions.

    My suggestion is to provide another Load function that takes the extra boolean parameter expandExpressions

    opened by emicklei 4
  • Do not fail on blank lines with whitespace only

    Do not fail on blank lines with whitespace only

    It's pretty easy to leave a single tab on a line after editing a large properties file. I think the parser should simply ignore such a line instead of failing.

    opened by homiak 3
  • added ToMap and ToMap with filters so that users may be able to trans…

    added ToMap and ToMap with filters so that users may be able to trans…

    I have added to things that might be useful -- I am building a library which many of other projects are using, my problem is I do not want them exposed to properties.Properties data structure directly, please let me know if my code is acceptable. Thought this would be useful to others.

    opened by AnirudhVyas 3
  • Skip ignored private fields

    Skip ignored private fields

    This pull request allows adding some private fields on properties structs without having problems, only marking this field as skipped with a specific tag as the test example below.:

    	type S struct {
    		p     string `properties:"-"`
    		X     string `properties:"-"`
    		Undef string `properties:",default=some value"`
    	}
    

    This change will avoid an error cannot set p using the struct above.

    opened by rafaelhl 1
  • Equal sign is incorrectly escaped in key

    Equal sign is incorrectly escaped in key

    I'm doing the following test:

    // Set a key to "a=b" and value "c"
    p := properties.NewProperties()
    p.Set("a=b", "c")
    buf := new(bytes.Buffer)
    p.Write(buf, properties.UTF8)
    println(buf.String())
    

    I'd espect the serialized version to be:

    a\=b = c
    

    Since the \ should be used to escape the first equal sign, but the program prints:

    a=b = c
    

    Which seems wrong. If saved in a file and read again, the key becomes "a" (while, when reading from the correct file, the key is correctly interpreted as "a=b").

    opened by nicolaferraro 0
  • Ability Decode to Multiple structs

    Ability Decode to Multiple structs

    In case the properties file has a logical division of the configurations, I'd like to decode to several dedicated structs.

    For example: Given the following config.properties file

    app.port=3000
    db.host=127.0.0.1
    db.port=8000
    db.username=username123
    db.password=password123
    pglistener.min_reconn=10 * time.Second
    pglistener.max_reconn=1 * time.Minute
    ...
    ...
    

    I'd like to decode to the following structs:

    type Config struct {
    	AppPort string `properties:"app.port,default=3000"`
    }
    
    type DBConfig struct {
    	Host          string  `properties:"db.host,default=127.0.0.1"`
    	Port           int       `properties:"db.port,default=5432"`
    	Username string `properties:"db.username"`
    	Password  string `properties:"db.password"`
    }
    
    
    type PGListener struct {
    	MinReconnectInterval time.Duration `properties:"pglistener.min_reconn"`
    	MaxReconnectInterval time.Duration `properties:"pglistener.max_reconn"`
    }
    
    opened by BoraBak 4
  • Unable to parse an array of values

    Unable to parse an array of values

    Given a properties file consisting of:

    Test = foo,bar,baz
    

    A config struct of

    type Config struct {
        Test []string `properties:"Test"`
    }
    

    And the following decoding

        var cfg Config
        if err := p.Decode(&cfg); err != nil {
            log.Fatal(err)
        }
        fmt.Printf("got test values: %s\n", cfg.Test[1])
    ```
    
    My program crashes as cfg.Test is of length 1.
    opened by ahshah 0
  • decode into map[string]interface{}

    decode into map[string]interface{}

    Hello, would be possible to add support to decode into map[string]interface{}. I need to decode a properties file into a generic interface{} but below code is returning an error. Is there a reason to only allow it to be a struct?

    	if t.Kind() != reflect.Ptr || v.Elem().Type().Kind() != reflect.Struct {
    		return fmt.Errorf("not a pointer to struct: %s", t)
    	}
    
    opened by fugkco 0
Owner
Frank Schröder
Distributed Systems Engineer, Go Developer, @gopcua and [email protected] maintainer. Working @northvolt building better batteries.
Frank Schröder
Go -> Haxe -> JS Java C# C++ C Python Lua

go2hx Compile: Go -> Haxe -> Js, Lua, C#, C++, Java, C, Python warning: heavily experimental still a ways to go before an alpha. Come give feedback on

go2hx 59 Jul 19, 2022
Aes for go and java; build go fo wasm and use wasm parse java response.

aes_go_wasm_java aes for go and java; build go fo wasm and use wasm parse java response. vscode setting config settings.json { "go.toolsEnvVars":

忆年 0 Dec 14, 2021
Update-java-ca-certificates - Small utility to convert the system trust store to a system Java KeyStore

update-java-ca-certificates This small utility takes care of creating a system-w

Swisscom 4 Jul 13, 2022
Super Java Vulnerability Scanner

XiuScan 不完善,正在开发中 介绍 一个纯Golang编写基于命令行的Java框架漏洞扫描工具 致力于参考xray打造一款高效方便的漏扫神器 计划支持Fastjson、Shiro、Struts2、Spring、WebLogic等框架 PS: 取名为XiuScan因为带我入安全的大哥是修君 特点

4ra1n 116 Dec 30, 2021
Tool to support the estimation for true sales prices for Danish properties.

Hjem Dette værktøj er designet til at støtte huskøbere til at kunne danne sig et overblik historiske købspriser for nærområdet givet man har udset sig

Thomas Kobber Panum 114 Jul 11, 2022
Golang library for reading properties from configuration files in JSON and YAML format or from environment variables.

go-config Golang library for reading properties from configuration files in JSON and YAML format or from environment variables. Usage Create config in

null 3 Feb 22, 2022
A protoc plugin that generates fieldmask paths as static type properties for proto messages

protoc-gen-fieldmask A protoc plugin that generates fieldmask paths as static ty

null 8 Jun 20, 2022
Rental-api - A RESTful-API that allows developers to connect to data about rental properties

Rentals-API is a RESTful-API that allows developers to connect to data about rental properties.

null 0 Jan 24, 2022
Gbu-scanner - Go Blog Updates (Scanner service)

Go Blog Updates - Scanner This service scans go blog (go.dev) and publishes new posts to message broker (rabbitmq). It uses mongodb as a storage for a

null 1 Jan 10, 2022
GONET-Scanner - Golang network scanner with arp discovery and own parser

GO/NET Scanner ScreenShots Install chmod +x install.sh ./install.sh [as root] U

Luis Javier 59 Jul 24, 2022
bluemonday: a fast golang HTML sanitizer (inspired by the OWASP Java HTML Sanitizer) to scrub user generated content of XSS

bluemonday bluemonday is a HTML sanitizer implemented in Go. It is fast and highly configurable. bluemonday takes untrusted user generated content as

Microcosm 2.4k Aug 1, 2022
XSD (XML Schema Definition) parser and Go/C/Java/Rust/TypeScript code generator

xgen Introduction xgen is a library written in pure Go providing a set of functions that allow you to parse XSD (XML schema definition) files. This li

null 173 Jul 22, 2022
Golang->Haxe->CPP/CSharp/Java/JavaScript transpiler

TARDIS Go -> Haxe transpiler Haxe -> C++ / C# / Java / JavaScript Project status: a non-working curiosity, development currently on-ice The advent of

TARDIS Go 418 Jul 13, 2022
Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots

Tinode Instant Messaging Server Instant messaging server. Backend in pure Go (license GPL 3.0), client-side binding in Java, Javascript, and Swift, as

Tinode 9.1k Aug 12, 2022
Go Stream, like Java 8 Stream.

Go Stream, like Java 8 Stream.

Youth.霖 63 Jul 30, 2022
minectl 🗺 is a cli for creating Minecraft (java or bedrock) server on different cloud provider.

minectl ?? minectl️️ is a cli for creating Minecraft (java or bedrock) server on different cloud provider. It is a private side project of me, to lear

Engin Diri 85 Jul 29, 2022
jacobin - A more than minimal JVM written in Go and capable of running Java 11 bytecode.

This overview gives the background on this project, including its aspirations and the features that it supports. The remaining pages discuss the basics of JVM operation and, where applicable, how Jacobin implements the various steps, noting any items that would be of particular interest to JVM cognoscenti.

Andrew Binstock 37 Aug 4, 2022
Cake is a lightweight HTTP client library for GO, inspired by Java Open-Feign.

Cake is a lightweight HTTP client library for GO, inspired by Java Open-Feign. Installation # With Go Modules, recommanded with go version > 1.16

snown 4 Mar 4, 2022
convert curl commands to Python, JavaScript, Go, PHP, R, Dart, Java, MATLAB, Rust, Elixir and more

curlconverter curlconverter transpiles curl commands into programs in other programming languages. $ curlconverter --data "Hello, world!" example.com

null 5.7k Aug 1, 2022
A Cloud Native Buildpack that contributes SDKMAN and uses it to install dependencies like the Java Virtual Machine

gcr.io/paketo-buildpacks/sdkman A Cloud Native Buildpack that contributes SDKMAN and uses it to install dependencies like the Java Virtual Machine. Be

Daniel Mikusa 1 Jan 8, 2022
A library designed for hosting Minecraft: Java Edition listeners.

expresso A library designed for hosting Minecraft: Java Edition listeners. Features Hosting listeners. All handshake, status, and login state packets.

Tal 3 Jul 29, 2022
Go Collection Stream API, inspired in Java 8 Stream.

GoStream gostream 是一个数据流式处理库。它可以声明式地对数据进行转换、过滤、排序、分组、收集,而无需关心操作细节。 Changelog 2021-11-18 add ToSet() collector Roadmap 移除go-linq依赖 Get GoStream go get

null 1 Jan 10, 2022
Chaosblade executor for chaos experiments on Java applications

Chaosblade-exec-jvm: Chaosblade executor for chaos experiments on Java applications Introduction The project is a chaosblade executor based on jvm-san

null 334 Aug 3, 2022
grab busy Java thread

BusyJavaThreadGraber 抓取最繁忙的 N 个 Java 线程,快速排查 java 程序的 cpu 问题,看看是哪个线程在作妖。 使用 安装 go git clone {ssh/http}, cd main, go build ./main -pid {java pid} -tick

gwk_2 3 Jan 19, 2022
A small server for verifing if a given java program is succeptibel to CVE-2021-44228

CVE-2021-44228-Test-Server A small server for verifing if a given java program is succeptibel to CVE-2021-44228 Usage Build the program using go build

Rasmus Hansen 5 Jun 19, 2022
Go -> Haxe -> JS Java C# C++ C Python Lua

go2hx Compile: Go -> Haxe -> Js, Lua, C#, C++, Java, C, Python warning: heavily experimental still a ways to go before an alpha. Come give feedback on

go2hx 59 Jul 19, 2022
A minimalistic LDAP server that is meant for test vulnerability to JNDI+LDAP injection attacks in Java, especially CVE-2021-44228.

jndi-ldap-test-server This is a minimalistic LDAP server that is meant for test vulnerability to JNDI+LDAP injection attacks in Java, especially CVE-2

Rakuten Group, Inc. 10 Jul 9, 2022
Java with Go interoperability?

Java with Go interoperability? Say, we have a java classes with a lot of code that can not reasonably be ported to golang. What do we do? We call java

Alexey Pozdnyakov 2 Dec 16, 2021
null 8 Dec 20, 2021