User agent string parser in golang

Overview

User agent parsing

useragent is a library written in golang to parse user agent strings.

Usage

First install the library with:

go get xojoc.pw/useragent

useragent is simple to use. First parse a string with useragent.Parse and then access the fields of useragent.UserAgent for the required information. Example:

see godoc for the complete documentation.

How it works?

      Lasciate ogne speranza, voi ch'intrate. -Dante

Parsing user agent strings is a hell. There is no standard for user agent strings, so useragent must use some heuristics. The site http://www.useragentstring.com/ has been invaluable during development. Some relevant links are also:

for the supported user agents see:

If you think useragent doesn't parse correctly a particular user agent string, just open an issue :).

Why this library?

useragent doesn't just split the user agent string and look for specific strings like other parsers, but it has specific parser for the most common browsers/crawlers and falls back to a generic parser for everything else. Its main features are:

  • Simple and stable API.
  • High precision in detection of the most common browsers/crawlers.
  • Detects mobile/tablet devices.
  • OS detection.
  • URL with more information about the user agent (usually it's the home page).
  • Security level detection when reported by browsers.

Who?

useragent was written by Alexandru Cojocaru and uses blang/semver to parse versions.

Thanks a lot to @brendanwalters (from http://pendo.io) for the contributions.

Donate!

License

useragent is released under the GPLv3 or later, see COPYING.

Issues
  • Misidentify microsoft edge with Chrome

    Misidentify microsoft edge with Chrome

    I've got this error when using Microsoft Edge or using this below user agent string:

    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.10240

    opened by DxTa 2
  • Support for Kindle Fire and discrimination between iPad/iPhone

    Support for Kindle Fire and discrimination between iPad/iPhone

    We've found your tool quite useful and would like to contribute some enhancements.

    This patch recognizes the Silk browser on modern Kindle Fire tablets (but not 1st generation), as well as discriminating between mobile and tablet iOS devices.

    opened by brendanwalters 2
  • Googlebot is identified as a browser

    Googlebot is identified as a browser

    For instance, the user-agent Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) has a Type of browser. Google maintains a list of user agents for Googelbot.

    opened by oschwald 1
  • Support for some libraries and Firefox on iOS

    Support for some libraries and Firefox on iOS

    UIWebView (iOS library) support is coming soon, but it's uglier and more of a change (have to derive a version number as it isn't given directly), so it'll be a separate PR once I finish it.

    The parseUnixLike break-out is due to needing it in multiple places now.

    opened by brendanwalters 1
  • support for curl and HTTPie

    support for curl and HTTPie

    The first user-agent I reached for to test xojoc/useragent in my service was curl. I'm also a fan of HTTPie.

    I extracted a generic.go file that parallels browser.go and crawler.go, each with their own parser and supporting data. This allowed me to more easily see where to add additional libraries. If you'd rather I approach it another way, please let me know.

    opened by cainlevy 0
  • not detect this user agent

    not detect this user agent

    "Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_3 like Mac OS X) AppleWebKit/603.3.8 (KHTML, like Gecko) Mobile/14G60 [FBAN/FBIOS;FBAV/138.0.0.49.91;FBBV/69441604;FBDV/iPhone8,4;FBMD/iPhone;FBSN/iOS;FBSV/10.3.3;FBSS/2;FBCR/BouyguesTelecom;FBID/phone;FBLC/fr_FR;FBOP/5;FBRV/0]"

    opened by dvrkps 0
  • Does not parse googlebot correctly

    Does not parse googlebot correctly

    Hi. Thanks for your great module.

    There is only one issue. This User-Agent is not parsed correctly: "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

    opened by gugu 0
  • Cannot get useragent

    Cannot get useragent

    Seems like SSl certificate is broken @xojoc

    go get xojoc.pw/useragent: unrecognized import path "xojoc.pw/useragent": https fetch: Get "https://xojoc.pw/useragent?go-get=1": x509: certificate is not valid for any names, but wanted to match xojoc.pw

    opened by switchelven-admin 0
  • Fails to parse many permutations of User Agent strings

    Fails to parse many permutations of User Agent strings

    I dropped this into a little project parsing a lot of strings and find that the fail (nil) output is quite high, about 60%:

    Screen Shot 2020-12-02 at 2 19 59 PM

    Some common example strings that did not parse but seem to have sufficient structure:

    "Mozilla/5.0 (iPhone; CPU iPhone OS 13_6_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 Indeed App 60.0",
    "Mozilla/5.0 (iPad; CPU OS 13_5_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 Version/12.1 Safari/604.1",
    "Mozilla/5.0 (iPhone; CPU iPhone OS 13_1_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148",
    "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0; .NET4.0C; .NET4.0E),1181023",
    "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/7.0),1201147",
    "\"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)  Chrome/74.0.3729.0 Safari/537.36\",1319071",
    "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 2.0.50727),1057949",
    
    opened by benjaminapetersen 0
  • OS release name based on OS version

    OS release name based on OS version

    This is actually a feature request. Maybe we can also get the OS release name like Windows 6.1.0 can have Windows 7.

    Maybe?

    type UserAgent struct {
        // The original user agent string.
        Original string
        Type     Type
        // The browser/crawler/etc. name. For example:
        //  Firefox, IceCat, Iceweasel
        //  Dillo
        //  Chrome
        //  MSIE
        //  Googlebot
        //   etc.
        // If the name is not known, Name will be `unknown'.
        Name    string
        Version semver.Version
        // The OS name. Can be one of:
        //  GNU/Linux
        //  FreeBSD
        //  OpenBSD
        //  NetBSD
        //  Windows
        //  Mac OS X
        //  Android
        //  Firefox OS
        //  CrOS
        //   etc.
        // If the os is not known, OS will be `unknown'.
        OS        string
        OSVersion semver.Version
        // THIS ONE HERE!!! :D
        OSReleaseName string 
        Security  Security
        // URL with more information about the user agent (in most cases it's the home page).
        // If unknown is nil.
        URL *url.URL
        // Is it a phone device?
        Mobile bool
        // Is it a tablet device?
        Tablet bool
    }
    
    opened by majimboo 0
  • Does not detect Windows Phone as mobile

    Does not detect Windows Phone as mobile

    Expected behaviour

    Windows Phone based devices should be detected as “mobile” (ua.Mobile == true).

    Actual behaviour

    1. Windows Phone based smart phones (such as Lumia) are not detected as expected with ua.Mobile == true.
    2. WP 8.0 and WP 10 UA strings do get parsed with ua := useragent.Parse(userAgentHeader), but for WP 8.1 ua == nil

    Examples of UA strings

    Examples of Windows Phone 7.x UA strings:

    • currently I do not have WP 7.x UA info, but will update this when I get one

    Examples of Windows Phone 8.0 UA strings:

    • Mozilla/5.0 (compatible; MSIE 10.0; Windows Phone 8.0; Trident/6.0; IEMobile/10.0; ARM; Touch; NOKIA; Lumia 520)
    • Mozilla/5.0 (compatible; MSIE 10.0; Windows Phone 8.0; Trident/6.0; IEMobile/10.0; ARM; Touch; NOKIA; Lumia 635)

    Examples of Windows Phone 8.1 UA strings:

    • Mozilla/5.0 (Mobile; Windows Phone 8.1; Android 4.0; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 630) like iPhone OS 7_0_3 Mac OS X AppleWebKit/537 (KHTML, like Gecko) Mobile Safari/537
    • Mozilla/5.0 (Mobile; Windows Phone 8.1; Android 4.0; ARM; Trident/7.0; Touch; rv:11.0; IEMobile/11.0; NOKIA; Lumia 930) like iPhone OS 7_0_3 Mac OS X AppleWebKit/537 (KHTML, like Gecko) Mobile Safari/537

    Examples of Windows 10 Mobile (“Windows Phone 10”) UA strings:

    • Mozilla/5.0 (Windows Phone 10.0; Android 6.0.1; Microsoft; Lumia 640 LTE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Mobile Safari/537.36 Edge/14.14393
    • Mozilla/5.0 (Windows Phone 10.0; Android 6.0.1; Microsoft; Lumia 950) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Mobile Safari/537.36 Edge/14.14393

    Example screenshot

    Example screenshot with Nokia Lumia 930 Windows Phone 8.1 device via BrowserStack:

    image

    opened by aripalo 0
Owner
Alexandru Cojocaru
I'm a programmer and problem solver.
Alexandru Cojocaru
String-matching in Golang using the Knuth–Morris–Pratt algorithm (KMP)

gokmp String-matching in Golang using the Knuth–Morris–Pratt algorithm (KMP). Disclaimer This library was written as part of my Master's Thesis and sh

Patrick-Ranjit D. Madsen 38 Dec 16, 2021
Decode / encode XML to/from map[string]interface{} (or JSON); extract values with dot-notation paths and wildcards. Replaces x2j and j2x packages.

mxj - to/from maps, XML and JSON Decode/encode XML to/from map[string]interface{} (or JSON) values, and extract/modify values from maps by key or key-

Charles Banning 523 Aug 7, 2022
A Go slugify application that handles string

slugify A Go slugify application that handles string Example: package main import ( "fmt" "github.com/avelino/slugify" ) func main() { text := "E

Avelino 31 Jan 24, 2022
A fast string sorting algorithm (MSD radix sort)

Your basic radix sort A fast string sorting algorithm This is an optimized sorting algorithm equivalent to sort.Strings in the Go standard library. Fo

Algorithms to Go 177 Jul 27, 2022
A collection of well-known string hash functions, implemented in Go

This library is a collection of "well-known" 32-bit string hashes, implemented in Go. It includes: Java string hash ELF-32 Jenkins' One-A

Damian Gryski 64 Mar 3, 2022
String i18n utilities for the Go Programming Language

About polyglot polyglot is a String translation package and tool for Go. Setup Make sure you have a working Go installation. See Getting Started Now r

Alexander Neumann 38 Mar 19, 2022
Package strit introduces a new type of string iterator, along with a number of iterator constructors, wrappers and combinators.

strit Package strit (STRing ITerator) assists in development of string processing pipelines by providing a simple iteration model that allows for easy

Maxim 84 Jun 21, 2022
bluemonday: a fast golang HTML sanitizer (inspired by the OWASP Java HTML Sanitizer) to scrub user generated content of XSS

bluemonday bluemonday is a HTML sanitizer implemented in Go. It is fast and highly configurable. bluemonday takes untrusted user generated content as

Microcosm 2.4k Aug 1, 2022
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

omniparser Omniparser is a native Golang ETL parser that ingests input data of various formats (CSV, txt, fixed length/width, XML, EDI/X12/EDIFACT, JS

JF Technology 482 Aug 1, 2022
TOML parser for Golang with reflection.

THIS PROJECT IS UNMAINTAINED The last commit to this repo before writing this message occurred over two years ago. While it was never my intention to

Andrew Gallant 3.9k Aug 6, 2022
Freestyle xml parser with golang

fxml - FreeStyle XML Parser This package provides a simple parser which reads a XML document and output a tree structure, which does not need a pre-de

null 8 Jul 1, 2022
A simple json parser built using golang

jsonparser A simple json parser built using golang Installation: go get -u githu

Krisna Pranav 1 Dec 29, 2021
A NMEA parser library in pure Go

go-nmea This is a NMEA library for the Go programming language (Golang). Features Parse individual NMEA 0183 sentences Support for sentences with NMEA

Adrián Moreno 180 Jul 9, 2022
A shell parser, formatter, and interpreter with bash support; includes shfmt

sh A shell parser, formatter, and interpreter. Supports POSIX Shell, Bash, and mksh. Requires Go 1.14 or later. Quick start To parse shell scripts, in

Daniel Martí 5k Aug 7, 2022
A simple CSS parser and inliner in Go

douceur A simple CSS parser and inliner in Golang. Parser is vaguely inspired by CSS Syntax Module Level 3 and corresponding JS parser. Inliner only p

Aymerick 219 Jul 15, 2022
Simple HCL (HashiCorp Configuration Language) parser for your vars.

HCL to Markdown About To write a good documentation for terraform module, quite often we just need to print all our input variables as a fancy table.

Dmytro Shamenko 15 Dec 14, 2021
A markdown parser written in Go. Easy to extend, standard(CommonMark) compliant, well structured.

goldmark A Markdown parser written in Go. Easy to extend, standards-compliant, well-structured. goldmark is compliant with CommonMark 0.29. Motivation

Yusuke Inuzuka 2.3k Jul 30, 2022
Unified diff parser and printer for Go

go-diff Diff parser and printer for Go. Installing go get -u github.com/sourcegraph/go-diff/diff Usage It doesn't actually compute a diff. It only rea

Sourcegraph 367 Aug 5, 2022
A PDF renderer for the goldmark markdown parser.

goldmark-pdf goldmark-pdf is a renderer for goldmark that allows rendering to PDF. Reference See https://pkg.go.dev/github.com/stephenafamo/goldmark-p

Stephen Afam-Osemene 86 Jul 6, 2022