Natural language detection package in pure Go

Overview

getlang

GoDoc Go Report Card Build Status cover.run go

getlang provides fast natural language detection in Go.

Features

  • Offline -- no internet connection required
  • Supports 29 languages
  • Provides ISO 639 language codes
  • Fast

Getting started

Installation:

    go get -u github.com/rylans/getlang

example:

package main

import (
	"fmt"
	"github.com/rylans/getlang"
)

func main(){
  info := getlang.FromString("Wszyscy ludzie rodzą się wolni i równi w swojej godności i prawach")
  fmt.Println(info.LanguageCode(), info.Confidence())
}

Documentation

getlang on godoc

License

MIT

Acknowledgements and Citations

  • Thanks to abadojack for the trigram generation logic in whatlanggo
  • Cavnar, William B., and John M. Trenkle. "N-gram-based text categorization." Ann arbor mi 48113.2 (1994): 161-175.
Comments
  • Looking for the text case explaination or documentation.

    Looking for the text case explaination or documentation.

    I added Punjabi language support to this package and run on my local machine. When I submitted the pull request there was a build failed indicated by Tarvis. I want to contribute to this Repo and want to fix the error of my code. If I could get any documentation or guidance it will help me a lot. Thank you Bharat Sewani

    opened by bharatsewani1993 2
  • Adding Persian language detection

    Adding Persian language detection

    Also Arabic detection mechanism has been changed, because Arabic and Persian script is the same, so I had added Arabic profile as well in the profiles.go file

    opened by hojabri 0
  • Add support for CJK languages

    Add support for CJK languages

    Support language detection for Chinese, Japanese and Korean.

    The 3gram character approach will not work for these languages. Consider detecting these languages based on which part of unicode the CJK runes map to.

    enhancement help wanted 
    opened by rylans 0
  • czech language support, introduced dynamic weight for unicode blocks

    czech language support, introduced dynamic weight for unicode blocks

    Hello, I've added support for a czech language as well as support for dynamic weight for scripts (scriptCountFactor).

    The reason for that is that in a czech alphabet, there are few characters like ř, š, ů which you won't find in any other language.

    I hope you'll like my take on that.

    opened by zdebra 2
  • Add language detection for Kannada

    Add language detection for Kannada

    Can add support for this language by its script. Add this to the scripts variable (defined on line 42 of getlang.go) "kn": []*unicode.RangeTable{unicode.Kannada},

    enhancement good first issue 
    opened by rylans 2
Owner
Rylan
Rylan
Selected Machine Learning algorithms for natural language processing and semantic analysis in Golang

Natural Language Processing Implementations of selected machine learning algorithms for natural language processing in golang. The primary focus for t

James Bowman 374 Sep 27, 2022
Self-contained Machine Learning and Natural Language Processing library in Go

If you like the project, please ★ star this repository to show your support! ?? A Machine Learning library written in pure Go designed to support rele

NLP Odyssey 1.2k Sep 22, 2022
A natural language date/time parser with pluggable rules

when when is a natural language date/time parser with pluggable rules and merge strategies Examples tonight at 11:10 pm at Friday afternoon the deadli

Oleg Lebedev 1.2k Sep 12, 2022
Cross platform locale detection for Golang

go-locale go-locale is a Golang lib for cross platform locale detection. OS Support Support all OS that Golang supported, except android: aix: IBM AIX

Xuanwo 86 Aug 20, 2022
Self-contained Japanese Morphological Analyzer written in pure Go

Kagome v2 Kagome is an open source Japanese morphological analyzer written in pure golang. The dictionary/statistical models such as MeCab-IPADIC, Uni

ikawaha 670 Sep 24, 2022
Stemmer packages for Go programming language. Includes English, German and Dutch stemmers.

Stemmer package for Go Stemmer package provides an interface for stemmers and includes English, German and Dutch stemmers as sub-packages: porter2 sub

Dmitry Chestnykh 52 Sep 27, 2022
Gopher-translator - A HTTP API that accepts english word or sentences and translates them to Gopher language

Gopher Translator Service An interview assignment project. To see the full assig

Teodor Draganov 0 Jan 25, 2022
Complete Translation - translate a document to another language

Complete Translation This project is to translate a document to another language. The initial target is English to Korean. Consider this project is no

코딩냄비 4 Feb 25, 2022
The shamoji (杓文字) is a word filtering package

shamoji About The shamoji (杓文字) is word filtering package. Install $ go get -u github.com/osamingo/shamoji Usage package main import ( "fmt" "sync

Osamu TONOMORI 12 May 4, 2022
A Go package for n-gram based text categorization, with support for utf-8 and raw text

A Go package for n-gram based text categorization, with support for utf-8 and raw text. To do: write documentation make it faster Keywords: text categ

Peter Kleiweg 67 Feb 13, 2022
Package i18n provides internationalization and localization for your Go applications.

i18n Package i18n provides internationalization and localization for your Go applications. Installation The minimum requirement of Go is 1.16. go get

null 53 Aug 8, 2022
A high-performance concurrent scanner written by go, which can be used for survival detection, tcp port detection, and web service detection.

aScan A high-performance concurrent scanner written by go, which can be used for survival detection, tcp port detection, and web service detection. Fu

seventeen 16 Aug 15, 2022
Fast face detection, pupil/eyes localization and facial landmark points detection library in pure Go.

Pigo is a pure Go face detection, pupil/eyes localization and facial landmark points detection library based on Pixel Intensity Comparison-based Objec

Endre Simo 3.8k Sep 27, 2022
Natural language detection library for Go

Whatlanggo Natural language detection for Go. Features Supports 84 languages 100% written in Go No external dependencies Fast Recognizes not only a la

Abado Jack Mtulla 555 Sep 25, 2022
👄 The most accurate natural language detection library in the Go ecosystem, suitable for long and short text alike

?? The most accurate natural language detection library in the Go ecosystem, suitable for long and short text alike

Peter M. Stahl 693 Sep 16, 2022
👄 The most accurate natural language detection library in the Go ecosystem, suitable for long and short text alike

Its task is simple: It tells you which language some provided textual data is written in. This is very useful as a preprocessing step for linguistic data in natural language processing applications such as text classification and spell checking. Other use cases, for instance, might include routing e-mails to the right geographically located customer service department, based on the e-mails' languages.

Peter M. Stahl 696 Sep 25, 2022
Natural-deploy - A natural and simple way to deploy workloads or anything on other machines.

Natural Deploy Its Go way of doing Ansibles: Motivation: Have you ever felt when using ansible or any declarative type of program that is used for dep

Akilan Selvacoumar 0 Jan 3, 2022
Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Bytedance Inc. 1.4k Sep 27, 2022
Selected Machine Learning algorithms for natural language processing and semantic analysis in Golang

Natural Language Processing Implementations of selected machine learning algorithms for natural language processing in golang. The primary focus for t

James Bowman 373 Sep 15, 2022
Selected Machine Learning algorithms for natural language processing and semantic analysis in Golang

Natural Language Processing Implementations of selected machine learning algorithms for natural language processing in golang. The primary focus for t

James Bowman 374 Sep 27, 2022