Go-linq - A powerful language integrated query (LINQ) library for Golang

Overview

go-linq GoDoc Build Status Coverage Status Go Report Card

A powerful language integrated query (LINQ) library for Go.

  • Written in vanilla Go, no dependencies!
  • Complete lazy evaluation with iterator pattern
  • Safe for concurrent use
  • Supports generic functions to make your code cleaner and free of type assertions
  • Supports arrays, slices, maps, strings, channels and custom collections

Installation

When used with Go modules, use the following import path:

go get github.com/ahmetb/go-linq/v3

Older versions of Go using different dependency management tools can use the following import path to prevent breaking API changes:

go get gopkg.in/ahmetb/go-linq.v3

Quickstart

Usage is as easy as chaining methods like:

From(slice) .Where(predicate) .Select(selector) .Union(data)

Example 1: Find all owners of cars manufactured after 2015

import . "github.com/ahmetb/go-linq/v3"

type Car struct {
    year int
    owner, model string
}

...


var owners []string

From(cars).Where(func(c interface{}) bool {
	return c.(Car).year >= 2015
}).Select(func(c interface{}) interface{} {
	return c.(Car).owner
}).ToSlice(&owners)

Or, you can use generic functions, like WhereT and SelectT to simplify your code (at a performance penalty):

var owners []string

From(cars).WhereT(func(c Car) bool {
	return c.year >= 2015
}).SelectT(func(c Car) string {
	return c.owner
}).ToSlice(&owners)

Example 2: Find the author who has written the most books

import . "github.com/ahmetb/go-linq/v3"

type Book struct {
	id      int
	title   string
	authors []string
}

author := From(books).SelectMany( // make a flat array of authors
	func(book interface{}) Query {
		return From(book.(Book).authors)
	}).GroupBy( // group by author
	func(author interface{}) interface{} {
		return author // author as key
	}, func(author interface{}) interface{} {
		return author // author as value
	}).OrderByDescending( // sort groups by its length
	func(group interface{}) interface{} {
		return len(group.(Group).Group)
	}).Select( // get authors out of groups
	func(group interface{}) interface{} {
		return group.(Group).Key
	}).First() // take the first author

Example 3: Implement a custom method that leaves only values greater than the specified threshold

type MyQuery Query

func (q MyQuery) GreaterThan(threshold int) Query {
	return Query{
		Iterate: func() Iterator {
			next := q.Iterate()

			return func() (item interface{}, ok bool) {
				for item, ok = next(); ok; item, ok = next() {
					if item.(int) > threshold {
						return
					}
				}

				return
			}
		},
	}
}

result := MyQuery(Range(1,10)).GreaterThan(5).Results()

Generic Functions

Although Go doesn't implement generics, with some reflection tricks, you can use go-linq without typing interface{}s and type assertions. This will introduce a performance penalty (5x-10x slower) but will yield in a cleaner and more readable code.

Methods with T suffix (such as WhereT) accept functions with generic types. So instead of

.Select(func(v interface{}) interface{} {...})

you can type:

.SelectT(func(v YourType) YourOtherType {...})

This will make your code free of interface{} and type assertions.

Example 4: "MapReduce" in a slice of string sentences to list the top 5 most used words using generic functions

var results []string

From(sentences).
	// split sentences to words
	SelectManyT(func(sentence string) Query {
		return From(strings.Split(sentence, " "))
	}).
	// group the words
	GroupByT(
		func(word string) string { return word },
		func(word string) string { return word },
	).
	// order by count
	OrderByDescendingT(func(wordGroup Group) int {
		return len(wordGroup.Group)
	}).
	// order by the word
	ThenByT(func(wordGroup Group) string {
		return wordGroup.Key.(string)
	}).
	Take(5).  // take the top 5
	// project the words using the index as rank
	SelectIndexedT(func(index int, wordGroup Group) string {
		return fmt.Sprintf("Rank: #%d, Word: %s, Counts: %d", index+1, wordGroup.Key, len(wordGroup.Group))
	}).
	ToSlice(&results)

More examples can be found in the documentation.

Release Notes

v3.2.0 (2020-12-29)
* Added FromChannelT().
* Added DefaultIfEmpty().

v3.1.0 (2019-07-09)
* Support for Go modules
* Added IndexOf()/IndexOfT().

v3.0.0 (2017-01-10)
* Breaking change: ToSlice() now overwrites existing slice starting
  from index 0 and grows/reslices it as needed.
* Generic methods support (thanks @cleitonmarx!)
  - Accepting parametrized functions was originally proposed in #26
  - You can now avoid type assertions and interface{}s
  - Functions with generic methods are named as "MethodNameT" and
    signature for the existing LINQ methods are unchanged.
* Added ForEach(), ForEachIndexed() and AggregateWithSeedBy().

v2.0.0 (2016-09-02)
* IMPORTANT: This release is a BREAKING CHANGE. The old version
  is archived at the 'archive/0.9' branch or the 0.9 tags.
* A COMPLETE REWRITE of go-linq with better performance and memory
  efficiency. (thanks @kalaninja!)
* API has significantly changed. Most notably:
  - linq.T removed in favor of interface{}
  - library methods no longer return errors
  - PLINQ removed for now (see channels support)
  - support for channels, custom collections and comparables

v0.9-rc4
* GroupBy()

v0.9-rc3.2
* bugfix: All() iterating over values instead of indices

v0.9-rc3.1
* bugfix: modifying result slice affects subsequent query methods

v0.9-rc3
* removed FirstOrNil, LastOrNil, ElementAtOrNil methods

v0.9-rc2.5
* slice-accepting methods accept slices of any type with reflections

v0.9-rc2
* parallel linq (plinq) implemented
* Queryable separated into Query & ParallelQuery
* fixed early termination for All

v0.9-rc1
* many linq methods are implemented
* methods have error handling support
* type assertion limitations are unresolved
* travis-ci.org build integrated
* open sourced on github, master & dev branches
Issues
  • [WIP] Generic functions proposal

    [WIP] Generic functions proposal

    Hey guys,

    This is just a proposal, I'd love to hear some feedback.

    Cheers,

    Cleiton

    var r []person
    From(listPerson).WhereT(func(i person) bool {
          return i.Age > 18
    }).ToSlice(&r)
    
    opened by cleitonmarx 34
  • Production use?

    Production use?

    Really a huge fan of LINQ. I use it all the time in the front-end (map, reduce are basically the same ideas). Would love to use this package but the one question I have is---"Is this used in production by anyone?"

    question 
    opened by frankandrobot 16
  • ToSlice dose not re-empty the dist slice

    ToSlice dose not re-empty the dist slice

    Codes:

    func main() {
    	l := []string{"foo", "bar", "baz"}
    	linq.From(l).ToSlice(&l)
    	fmt.Println(l)
    }
    

    Result:

    [foo bar baz foo bar baz]
    

    go-linq version:

    - name: github.com/ahmetalpbalkan/go-linq
      version: 8985ec99e11a8bff7eb19dd0a0b2187770dab23a
    

    I don't know whether this is a bug or a feature, but if this is a feature, I think it's better to have this behavior documented on ToSlice's API doc.

    question 
    opened by aisk 14
  • Updated travis and added go mod.

    Updated travis and added go mod.

    This is an attempt at adding go mod to this library.

    @ahmetb I'm not sure if you had any plans on adding this, but I've started the process to migrate some of my projects from dep to go mod and thought I'd add this too.

    Should resolve #77

    opened by elliotcourant 13
  • runtime error: hash of unhashable type map[string]interface {}

    runtime error: hash of unhashable type map[string]interface {}

    Unable to perform group by query on the following dataset

    rows := []map[string]interface{}{
    	{"name": "Mostain", "salary": 30000, "ShipmentTag": "b001"},
    	{"name": "Sanzida", "salary": 20000, "ShipmentTag": "b001"},
    	{"name": "Riaz", "salary": 21000, "ShipmentTag": "b002"},
    	{"name": "Pallobi", "salary": 30000, "ShipmentTag": "b001"},
    	{"name": "Moaz", "salary": 20000, "ShipmentTag": "b001"},
    	{"name": "Tareq", "salary": 21000, "ShipmentTag": "b002"},
    }
    
    	data := From(rows).Where(func(c interface{}) bool {
    		cmap := c.(map[string]interface{})
    		return len(cmap["name"].(string)) > 0
    
    	}).GroupBy(func(ShipmentTag interface{}) interface{} {
    		return ShipmentTag
    
    	}, func(ShipmentTag interface{}) interface{} {
    		return ShipmentTag
    
    	}).First()
    
    	fmt.Println(data)
    

    Error details:

    goroutine 1 [running]: github.com/ahmetb/go-linq/v3.Query.GroupBy.func1() E:/GOLANG/pkg/mod/github.com/ahmetb/go-linq/[email protected]/groupby.go:21 +0x15f github.com/ahmetb/go-linq/v3.Query.First({0x5f0aa0?}) E:/GOLANG/pkg/mod/github.com/ahmetb/go-linq/[email protected]/result.go:188 +0x1a main.main()

    opened by mateors 9
  • v2.0

    v2.0

    Hi there, I've managed to deal with all my "busy man" problems. So, back online and ready for some opensource.

    Here is the PR for v2.0. It doesn't have a readme file yet, because I am unaware about the links to godoc, travis, etc. I assume that if we tag the new commit with as "v2.0", the link to godoc would be smth like https://godoc.org/gopkg.in/ahmetalpbalkan/go-linq.v2.0 and it seems that tha branch for travis and coveralls can be specified at their urls. Is it correct? Should I tag it myself?

    opened by kalaninja 9
  • question: usage with KV store.

    question: usage with KV store.

    this sounds like a good match with boltdb, in terms of doing queries in memory.

    i currently use boltdb and also riak. i kind of hate riak but am stuck with it still. i was wondering if you think that linq would help me for writing many queries i need to do on top of boltdb ?

    question 
    opened by joeblew99 8
  • bug in WhereIndexed

    bug in WhereIndexed

    I believe WhereIndexed doesn't work as design:

    https://play.golang.org/p/HD5-rxIkpx0

    package main
    
    import (
    	"fmt"
    
    	"github.com/ahmetb/go-linq/v3"
    )
    
    func main() {
    
    	r := linq.Range(1, 10).
    		WhereIndexed(func(i int, _ interface{}) bool {
    			return i%3 == 0
    		}).
    		Results()
    	fmt.Println(r)
    }
    

    output of the code above is [1 2 3 4 5 6 7 8 9 10] instead of [3 6 9]

    maybe the bug is here

    https://github.com/ahmetb/go-linq/blob/master/where.go#L61

    thanks!

    opened by quexer 7
  • How to implement complex SQL functions

    How to implement complex SQL functions

    How to implement complex SQL functions similar to the following

    type TransactionRecord struct {
    	ID        uint64
    	AccountID string
    	TCode     string
    	TDate     string
    	TType     uint8
    	Amount    float64
    }
    
    var tds []TransactionRecord
    t1 := TransactionRecord{ID: 1, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 17, Amount: 23.0}
    t2 := TransactionRecord{ID: 2, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 17, Amount: 99.0}
    t3 := TransactionRecord{ID: 3, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 17, Amount: 25.0}
    
    t4 := TransactionRecord{ID: 1, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 24, Amount: 13.0}
    t5 := TransactionRecord{ID: 2, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 24, Amount: 69.0}
    t6 := TransactionRecord{ID: 3, AccountID: "A001", TCode: "161700", TDate: "2020-08-01", TType: 24, Amount: 85.0}
    		
    t7 := TransactionRecord{ID: 4, AccountID: "A001", TCode: "161700", TDate: "2020-08-02", TType: 17, Amount: 11.0}
    t8 := TransactionRecord{ID: 5, AccountID: "A001", TCode: "161700", TDate: "2020-08-02", TType: 17, Amount: 33.0}
    t9 := TransactionRecord{ID: 6, AccountID: "A001", TCode: "161700", TDate: "2020-08-02", TType: 17, Amount: 57.0}
    		
    tds = append(tds, t1)
    tds = append(tds, t2)
    tds = append(tds, t3)
    tds = append(tds, t4)
    tds = append(tds, t5)
    tds = append(tds, t6)
    tds = append(tds, t7)
    tds = append(tds, t8)
    tds = append(tds, t9)
    

    I want to implement the following SQL functions, how to write code with go-linq package

    select AccountID,TCode, TDate,TType, sum(Amount)  as am from TransactionDetails where TType = 17 or TType = 24 GROUP BY AccountID, TCode,TDate, TType ORDER BY AccountID, TCode,TDate, TType;
    
    opened by vinerr 5
  • gopkg.in/ahmetb/go-linq.v3: gopkg.in/ahmetb/go-linq.v3@v3.1.0: parsing go.mod:         module declares its path as: github.com/ahmetb/go-linq/v3 but was required as: gopkg.in/ahmetb/go-linq.v3

    gopkg.in/ahmetb/go-linq.v3: gopkg.in/ahmetb/[email protected]: parsing go.mod: module declares its path as: github.com/ahmetb/go-linq/v3 but was required as: gopkg.in/ahmetb/go-linq.v3

    I am trying to do create go.mod in my project. And this error arrives what should I do. I have imported gopkg.in/ahmetb/go-linq.v3 this package in my project files

    opened by aparnag07 5
  • Typed channels support

    Typed channels support

    • introduced a new method FromTypedChannel()
    • From() calls FromChannel() if channel is of type chan interface{} and FromTypedChannel for all other channels
    • benchmark for FromTypedChannel() and FromChannel() performance comparison
    opened by kalaninja 5
  • Try out the new Go generics proposal with go-linq

    Try out the new Go generics proposal with go-linq

    Go now has a prototype of generics implementation. Here are some resources:

    • https://blog.golang.org/generics-next-step
    • https://go.googlesource.com/proposal/+/refs/heads/master/design/go2draft-type-parameters.md
    • https://rakyll.org/generics-proposal/

    This potentially could give a severe performance boost to go-linq, as well as actually make this library useful.

    I haven't taken a closer look at how we would do this yet. For example, we might still end up having to do some type assertions. However, it might help us redesign the package and release a v4 if the generics proposal is adopted.

    enhancement 
    opened by ahmetb 13
Releases(v3.2.0)
  • v3.2.0(Dec 29, 2020)

  • v3.0.0(Jan 10, 2017)

    • Breaking change: ToSlice() now overwrites existing slice starting from index 0 and grows/reslices it as needed.
    • Generic/parametrized methods support (Thanks @cleitonmarx!)
      • Accepting parametrized functions was originally proposed in #26
      • You can now avoid type assertions and interface{}s
      • Functions with generic methods are named as MethodNameT
      • Method signatures for the existing LINQ methods are unchanged.
    • Added ForEach(), ForEachIndexed() and AggregateWithSeedBy().

    Also thanks @kalaninja for working on this release.

    Source code(tar.gz)
    Source code(zip)
  • v2.0.0-rc0(Sep 11, 2016)

    IMPORTANT: Breaking Changes

    v2.0 release contains BREAKING changes to the API.

    The old version is archived at the archive/0.9 branch or the 0.9 tags.

    Changelog

    • v2.0 is A COMPLETE REWRITE of go-linq with better performance and memory efficiency.
      (thanks @kalaninja!)
    • API has significantly changed. Most notably:
      • linq.T removed in favor of interface{}
      • library methods no longer return errors
      • PLINQ removed for now (see channels support)
      • support for channels, custom collections and comparables
    Source code(tar.gz)
    Source code(zip)
  • v0.9-rc5(Apr 9, 2015)

Owner
Ahmet Alp Balkan
Software Engineer at Google Cloud on cloud-native technologies and serverless platforms like Kubernetes/GKE and Cloud Run.
Ahmet Alp Balkan
Antch, a fast, powerful and extensible web crawling & scraping framework for Go

Antch Antch, inspired by Scrapy. If you're familiar with scrapy, you can quickly get started. Antch is a fast, powerful and extensible web crawling &

null 233 Jun 4, 2022
DSV Parallel Processor takes input files and query specification via a spec file

DSV Parallel Processor Spec file DSV Parallel Processor takes input files and query specification via a spec file (conventionally named "spec.toml").

Wattanit Hotrakool 0 Oct 9, 2021
FreeSWITCH Event Socket library for the Go programming language.

eventsocket FreeSWITCH Event Socket library for the Go programming language. It supports both inbound and outbound event socket connections, acting ei

Alexandre Fiori 104 Jun 20, 2022
Some utilities for Persian language in Go (Golang)

persian Some utilities for Persian language in Go (Golang). Installation go get github.com/mavihq/persian API .ToPersianDigits Converts all English d

هلو | آموزش زبان با بازی 64 Jun 7, 2022
A modern programming language written in Golang.

MangoScript A modern programming language written in Golang. Here is what I want MangoScript to look like: struct Rectangle { width: number he

PlebusSupremus1234 3 Nov 12, 2021
A toy language parser, lexer and interpreter written in Golang

Monkey - A toy programming language Monkey is a toy programming language used to learn how to write a lexer, parser and interpreter. The language is i

Adam Petrovic 1 Nov 16, 2021
Unit tests generator for Go programming language

GoUnit GoUnit is a commandline tool that generates tests stubs based on source function or method signature. There are plugins for Vim Emacs Atom Subl

Max Chechel 60 Jun 9, 2022
Simple interface to libmagic for Go Programming Language

File Magic in Go Introduction Provides simple interface to libmagic for Go Programming Language. Table of Contents Contributing Versioning Author Copy

Krzysztof Wilczyński 12 Dec 22, 2021
Go language interface to the PAPI performance API

go-papi Description go-papi provides a Go interface to PAPI, the Performance Application Programming Interface. PAPI provides convenient access to har

Los Alamos National Laboratory 16 Mar 19, 2020
go language generics system

Gotgo This document describes the third iteration of my attempt at a reasonable implementation of generics for go based on the idea of template packag

David Roundy 119 Dec 4, 2021
The Gorilla Programming Language

Gorilla Programming Language Gorilla is a tiny, dynamically typed, multi-engine programming language It has flexible syntax, a compiler, as well as an

null 29 Apr 16, 2022
Elastic is an Elasticsearch client for the Go programming language.

Elastic is an Elasticsearch client for the Go programming language.

Oliver Eilhard 6.9k Jun 26, 2022
CodePlayground is a playground tool for go and rust language.

CodePlayground CodePlayground is a playground tool for go and rust language. Installation Use homebrews to install code-playground. brew tap trendyol/

Trendyol Open Source 47 Mar 5, 2022
Assembly syntax that makes you feel like you're writing code in a high-level language.

shasm Assembly syntax that makes you feel like you're writing code in a high-level language. Shasm is not an Assembler. Shasm simply compiles Shasm sy

Shoyaaa 14 Jun 5, 2021
The new home of the CUE language! Validate and define text-based and dynamic configuration

The CUE Data Constraint Language Configure, Unify, Execute CUE is an open source data constraint language which aims to simplify tasks involving defin

null 2.7k Jun 30, 2022
👩🏼‍💻A simple compiled programming language

The language is written in Go and the target language is C. The built-in library is written in C too

paco 26 May 26, 2022
feedme project porting in Go language

newsapi-go This project has the intent to provide a valid interface for newsapi (https://newsapi.org/). usage The project is still under mantainance a

Fulvio 5 Oct 28, 2021
Go specs implemented as a scripting language in Rust.

Goscript A script language like Python or Lua written in Rust, with exactly the same syntax as Go's. The Goal Runs most pure Go code, probably add som

null 1.3k Jun 20, 2022
Lithia is an experimental functional programming language with an implicit but strong and dynamic type system.

Lithia is an experimental functional programming language with an implicit but strong and dynamic type system. Lithia is designed around a few core concepts in mind all language features contribute to.

Valentin Knabel 6 Jun 14, 2022