Official Go Client for OpenSearch

Overview

Go Reference Build Unit Integration codecov Chat PRs welcome!

OpenSearch logo

OpenSearch Go Client

Welcome!

opensearch-go is a community-driven, open source fork of go-elasticsearch licensed under the Apache v2.0 License. For more information, see opensearch.org.

Project Resources

Code of Conduct

This project has adopted the Amazon Open Source Code of Conduct. For more information see the Code of Conduct FAQ, or contact [email protected] with any additional questions or comments.

License

This project is licensed under the Apache v2.0 License.

Copyright

Copyright OpenSearch Contributors. See NOTICE for details.

Issues
  • Adding support for AWS SDK V2 request signer

    Adding support for AWS SDK V2 request signer

    Description

    This PR introduces support for a new package that allows to use the new AWS SDK V2 to sign the OpenSearch Go client's HTTP request messages according to the AWS Signature spec V4 .

    I've also updated all project dependencies to latest versions for bonus maintenance.

    Issues Resolved

    No issues found in tracker relating to this.

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

    opened by matelang 15
  • [BUG] Tag version with

    [BUG] Tag version with "v" prefix to make it work with go.mod

    Hi, go mod requires tags to start with a "v" prefix. Otherwise, it's invalid. I've noticed this when trying to update a go.mod to use the latest release.

    Could you please retag it to be v1.0.0?

    Context: https://github.com/golang/go/issues/30146#issuecomment-492669348 https://github.com/golang/go/issues/32945

    bug 
    opened by henvic 14
  • Updates to bulk indexer.

    Updates to bulk indexer.

    Description

    Add version and routing fields, optimize the memory usage.

    Issues Resolved

    #68

    Check List

    • [ ] Commits are signed per the DCO using --signoff

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

    opened by nevins-b 12
  • [FEATURE] Add IAM authentication option

    [FEATURE] Add IAM authentication option

    If people want to connect to an Amazon OpenSearch Service cluster, they have to figure out how to use this client alongside some third-party signing library and/or the AWS SDK, which is non-trivial due to how we (by design) conceal the underlying HTTP requests that the client makes. We should offer IAM signing as an option, just like the OpenSearch CLI does.

    Is your feature request related to a problem? If you want to use this client with an Amazon OpenSearch Service cluster that has IAM authentication rather than basic authentication, good luck.

    What solution would you like?

    When initializing the client, an AuthType option. If basic (or null), accept username and password. If IAM, accept options for access key, secret key, session token, region, and service.

    What alternatives have you considered?

    An additional Go IAM signing library for use on top of the client. But given that it would only work with the client, it seems better and easier to just add it to the client.

    Do you have any additional context?

    https://github.com/opensearch-project/OpenSearch/issues/1400

    enhancement 1.1.0 
    opened by aetter 9
  • Update module path to use /v2

    Update module path to use /v2

    Signed-off-by: Vijayan Balasubramanian [email protected]

    Description

    If the module is version v2, the major version of the module must be included as a /v2 at the end of the module paths used in go.mod files e.g module github.com/opensearch-project/opensearch-go/v2 require github.com/opensearch-project/opensearch-go/v2 v2.0.0 and in the package import path e.g import "github.com/opensearch-project/opensearch-go/v2/opensearchapi

    Issues Resolved

    #115

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

    backport 2.0 
    opened by VijayanB 8
  • golangci-lint integration

    golangci-lint integration

    Description

    golangci-lint is a Go linters aggregator. It helps improve the code quality and prevents bugs. I'd recomede to integrate golangci-lint and then enable linter one by one.

    Issues Resolved

    There are many issues with formatting, misspellings and even potentially dangerous bugs.

    Check List

    • [x] Commits are signed per the DCO using --signoff

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

    opened by dzyanis 5
  • opensearchutil: add missing bulk indexer response item fields

    opensearchutil: add missing bulk indexer response item fields

    Description

    If a bulk request contains any script error, Opensearch server points to what is wrong with the request. This information is valuable to those debug what is going on under the hood.

    This PR adds missing bulk indexer response item fields.

    Reproducer script

    #!/bin/bash
    
    INDEX="test"
    OSURL="http://localhost:9200"
    
    
    curl -X POST "$OSURL/$INDEX/_bulk?pretty&refresh=true" -H 'Content-Type: application/json' -d '
    {"update":{"_id":"1","_index":"test"}}
    {"script":{"source":"ctx._source.x = params.x","params":{"x":2}},"upsert":{"x":"1"}}
    '
    
    # null pointer exception: ctx._source.y 
    curl -X POST "$OSURL/$INDEX/_bulk?pretty&refresh=true" -H 'Content-Type: application/json' -d '
    {"update":{"_id":"1","_index":"test"}}
    {"script":{"source":"if (ctx._source.y < params.x) {ctx._source.y = params.x}","params":{"x": 2}},"upsert":{"x":"1"}}
    '
    

    Opensearch response

    {
      "took" : 287,
      "errors" : false,
      "items" : [
        {
          "update" : {
            "_index" : "test",
            "_type" : "_doc",
            "_id" : "1",
            "_version" : 1,
            "result" : "created",
            "forced_refresh" : true,
            "_shards" : {
              "total" : 2,
              "successful" : 2,
              "failed" : 0
            },
            "_seq_no" : 0,
            "_primary_term" : 1,
            "status" : 201
          }
        }
      ]
    }
    {
      "took" : 3,
      "errors" : true,
      "items" : [
        {
          "update" : {
            "_index" : "test",
            "_type" : "_doc",
            "_id" : "1",
            "status" : 400,
            "error" : {
              "type" : "illegal_argument_exception",
              "reason" : "failed to execute script",
              "caused_by" : {
                "type" : "script_exception",
                "reason" : "runtime error",
                "script_stack" : [
                  "if (ctx._source.y < params.x) {",
                  "                          ^---- HERE"
                ],
                "script" : "if (ctx._source.y < params.x) {ctx._source.y = params.x}",
                "lang" : "painless",
                "position" : {
                  "offset" : 26,
                  "start" : 0,
                  "end" : 31
                },
                "caused_by" : {
                  "type" : "null_pointer_exception",
                  "reason" : null
                }
              }
            }
          }
        }
      ]
    }
    
    

    Check List

    • [x] Commits are signed per the DCO using --signoff

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

    opened by igungor 5
  • [FEATURE] Support version and routing fields in bulk indexer

    [FEATURE] Support version and routing fields in bulk indexer

    Is your feature request related to a problem? Currently the bulk index request structs do not support the version or routing options.

    What solution would you like? Let's add these fields to the structs and update the writer to build the json correctly.

    What alternatives have you considered? I'm not sure there are any, with how the current implementation of the bulk indexer works we're unable to set these fields unless they are available on the structs.

    Do you have any additional context? https://opensearch.org/docs/opensearch/rest-api/document-apis/index-document/ the version, routing, and version_type fields are the ones requested

    enhancement 
    opened by nevins-b 5
  • Add example for IAM authentication

    Add example for IAM authentication

    Description

    Add an example explaining how to perform authentication using IAMs.

    Issues Resolved

    #75

    Check List

    • [x] Commits are signed per the DCO using --signoff

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

    opened by rmasclef 4
  • opensearchutil: fix dropped test error

    opensearchutil: fix dropped test error

    This fixes a spot in the tests where an err was being passed into a function but then dropped.

    Check List

    • [X] Commits are signed per the DCO using --signoff

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

    opened by alrs 3
  • Accept either ELASTICSEARCH_URL or OPENSEARCH_URL

    Accept either ELASTICSEARCH_URL or OPENSEARCH_URL

    To support backward compatibility, if addresses are not specified in config during new client api, we will look for addresses either from ELASTICSEARCH_URL or OPENSEARCH_URL environment variable. if we find both, then will exit with error message that both can't be set .

    opened by VijayanB 3
  • CVE-2022-27191 (High) detected in golang.org/x/crypto-0c34fe9e7dc2486962ef9867e3edb3503537209f

    CVE-2022-27191 (High) detected in golang.org/x/crypto-0c34fe9e7dc2486962ef9867e3edb3503537209f

    CVE-2022-27191 - High Severity Vulnerability

    Vulnerable Library - golang.org/x/crypto-0c34fe9e7dc2486962ef9867e3edb3503537209f

    [mirror] Go supplementary cryptography libraries

    Dependency Hierarchy:

    • :x: golang.org/x/crypto-0c34fe9e7dc2486962ef9867e3edb3503537209f (Vulnerable Library)

    Found in HEAD commit: 3f3ab782590b20cf59f880bfdbd70556a225006e

    Found in base branch: main

    Vulnerability Details

    The golang.org/x/crypto/ssh package before 0.0.0-20220314234659-1baeb1ce4c0b for Go allows an attacker to crash a server in certain circumstances involving AddHostKey.

    Publish Date: 2022-03-18

    URL: CVE-2022-27191

    CVSS 3 Score Details (7.5)

    Base Score Metrics:

    • Exploitability Metrics:
      • Attack Vector: Network
      • Attack Complexity: Low
      • Privileges Required: None
      • User Interaction: None
      • Scope: Unchanged
    • Impact Metrics:
      • Confidentiality Impact: None
      • Integrity Impact: None
      • Availability Impact: High

    For more information on CVSS3 Scores, click here.

    Suggested Fix

    Type: Upgrade version

    Origin: https://nvd.nist.gov/vuln/detail/CVE-2022-27191

    Release Date: 2022-03-18

    Fix Resolution: golang-golang-x-crypto-dev - 1:0.0~git20220315.3147a52-1;golang-go.crypto-dev - 1:0.0~git20220315.3147a52-1

    security vulnerability 
    opened by mend-for-github-com[bot] 0
  • [FEATURE] Add support for inclusive naming

    [FEATURE] Add support for inclusive naming

    Add support for inclusive naming by deprecating the master nomenclature.

    See meta issue https://github.com/opensearch-project/opensearch-clients/issues/16 for details.

    enhancement 
    opened by VachaShah 0
  • Using standardized templates from .github

    Using standardized templates from .github

    Signed-off-by: Vacha Shah [email protected]

    Description

    Using standardized templates from .github to make them consistent.

    Issues Resolved

    List any issues this PR will resolve, e.g. Closes [...].

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

    opened by VachaShah 0
  • [BUG] Extreme latency in BulkIndexer

    [BUG] Extreme latency in BulkIndexer

    What is the bug? It seems that with a BulkIndexer with 2 workers, I am getting unexpected latency on BulkIndexer.Add(). It seems that somehow the workers are not consuming the queue within any reasonable sort of timeframe, I'm seeing delays of over 20s!

    For example, in the last hour I've 53 cases of >1s latency on just Add() out of a total of 174 calls.

    How can one reproduce the bug? With 2 workers running, adding items from different goroutines and a relatively busy search cluster.

    What is the expected behavior? Sub-millisecond latencies, basically the time it takes to shove something into a channel.

    What is your host/environment?

    • OS: Ubuntu 20.02
    • Version: 1.1.0 (but nothing has changed to the bulkindexer since the fork from ES)

    Do you have any screenshots? image

    bug 
    opened by dokterbob 2
  • Add setting for allow_leading_wildcard to search

    Add setting for allow_leading_wildcard to search

    Description

    Allow for setting, disabling, allow_leading_wildcard on a search

    Issues Resolved

    List any issues this PR will resolve, e.g. Closes [...].

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

    opened by nevins-b 3
  • Correct CurlLogger's URL to actual destination

    Correct CurlLogger's URL to actual destination

    Description

    The CurlLogger hardcodes http://localhost:9200. This change instead uses the given configuration values.

    Issues Resolved

    n/a

    Check List

    • [x] Commits are signed per the DCO using --signoff

    By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

    opened by dekimsey 1
Releases(v2.0.0)
  • v2.0.0(Jun 16, 2022)

    Version 2.0.0 Release Notes

    Compatible with OpenSearch 2.0

    Features

    • Type mapping removals for OpenSearch 2.0 (#109)
    • Support service configuration (#102)
    • Fix dropped test error (#95)
    • Add example for IAM authentication (#93)
    • Upgrading tools and text dependencies (#105)
    • Update module path (#118, #120)

    Infrastructure

    • golangci-lint integration (#96)
    • Add auto backport functionality (#104)
    • Add support to test against multiple versions of OpenSearch for compatiblity (#106)
    • Add support to test against unreleased OpenSearch (#107)
    • Test against OpenSearch 2.0 (#111)

    Full Changelog: https://github.com/opensearch-project/opensearch-go/compare/v1.1.0...v2.0.0

    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Feb 16, 2022)

    Version 1.1.0 Release Notes

    Compatible with OpenSearch 1.2.0

    Features

    • Updates to bulk indexer. (#67)
    • Support Signing Request feature(#81)
    • opensearchutil: add missing bulk indexer response item fields(#90)

    Infrastructure

    • Configure WhiteSource for GitHub.com(#79)
    • Format files using goimport (#80)
    • Include dco workflow check and fix invalid url path(#72)
    Source code(tar.gz)
    Source code(zip)
  • 1.0.0(Sep 20, 2021)

  • v1.0.0(Oct 11, 2021)

A simple non-official client for qvapay service with go, for our comunity

qvapay-go A simple non-official client for qvapay service with go, for our comunity Setup You can install this package by using the go get tool and in

Kenrique Ortega 8 Dec 26, 2021
Iaphub-go - Non-official Go client for Iaphub

Iaphub-go iaphub-go is an unofficial Go client for iaphub.com REST API. Installation go get github.com/n10ty/iaphub-go Usage package main import ( "

Andrey 1 Mar 15, 2022
Official codebase and releases for the DEXbot Client application

The sidecar to your cryptocurrency wallet Create an account on DEXbot.io » Read the Docs to Get Started » Introduction DEXbot is a decentralized appli

DEXbot 7 Apr 16, 2022
Official IVPN Desktop app

IVPN for Desktop (Windows/macOS/Linux) IVPN for Desktop is the official IVPN app for desktop platforms. Some of the features include: multiple protoco

IVPN 130 Jul 1, 2022
Official Go implementation of the Fixcoin protocol

XFSGO Official Go implementation of the XFS protocol. Usage To learn more about the available xfsgo commands, use xfsgo help or type a command followe

XFS Network 0 May 18, 2022
🤖 Automatically scrape PortableApps.com (or official release page) and convert into Edgeless plugin package

Edgeless 自动插件机器人 2 简介 该项目是为了使用 Golang 重新实现 Edgeless 自动插件机器人 特性 (WIP) 完全兼容 Edgeless 自动插件机器人,包括 Tasks,以实现无缝迁移 更快的构建速度 更好的代码结构 更高的拓展性 工作进度 截止至 2021/11/28

Hydrogen 2 Dec 7, 2021
The official repository of the Gravity Bridge Blockchain

Gravity bridge is Cosmos <-> Ethereum bridge designed to run on the Cosmos SDK blockchains like the Cosmos Hub focused on maximum design simplicity an

Gravity Bridge Foundation 80 Jun 30, 2022
Official Go implementation of the Klaytn protocol

Klaytn Official golang implementation of the Klaytn protocol. Please visit KlaytnDocs for more details on Klaytn design, node operation guides and app

ProtoFire 0 Dec 5, 2021
Official Golang implementation of the PANO protocol

Panoptis Official Golang implementation of the PANO protocol Building the source Building pano requires both a Go (version 1.14 or later) and a C comp

PanoptesDev 1 Dec 14, 2021
Prisma Client Go is an auto-generated and fully type-safe database client

Prisma Client Go Typesafe database access for Go Quickstart • Website • Docs • API reference • Blog • Slack • Twitter Prisma Client Go is an auto-gene

Prisma 1.2k Jul 2, 2022
The Dual-Stack Dynamic DNS client, the world's first dynamic DNS client built for IPv6.

dsddns DsDDNS is the Dual-Stack Dynamic DNS client. A dynamic DNS client keeps your DNS records in sync with the IP addresses associated with your hom

Ryan Young 10 Jun 20, 2022
Go Substrate RPC Client (GSRPC)Go Substrate RPC Client (GSRPC)

Go Substrate RPC Client (GSRPC) Substrate RPC client in Go. It provides APIs and types around Polkadot and any Substrate-based chain RPC calls. This c

Chino Chang 1 Nov 11, 2021
Server and client implementation of the grpc go libraries to perform unary, client streaming, server streaming and full duplex RPCs from gRPC go introduction

Description This is an implementation of a gRPC client and server that provides route guidance from gRPC Basics: Go tutorial. It demonstrates how to u

Joram Wambugu 0 Nov 24, 2021
Godaddy-domains-client-go - Godaddy domains api Client golang - Write automaticly from swagger codegen

Go API client for swagger Overview This API client was generated by the swagger-codegen project. By using the swagger-spec from a remote server, you c

Mickael Stanislas 0 Jan 9, 2022
Tailscale-client-go - A client implementation for the Tailscale HTTP API

tailscale-client-go A client implementation for the Tailscale HTTP API Example p

David Bond 9 Jun 8, 2022
Comunicación de envios de archivos entres cliente-servidor, client-client.

Client - Server - Client Estes es un proyecto simple de comunicacion de envios de archivos del cliente al servidor y viceversamente, y de cliente a cl

Melvin RB 0 Feb 11, 2022
CoAP Client/Server implementing RFC 7252 for the Go Language

Canopus Canopus is a client/server implementation of the Constrained Application Protocol (CoAP) Updates 25.11.2016 I've added basic dTLS Support base

Zubair Hamed 148 Jan 23, 2022
FTP client package for Go

goftp A FTP client package for Go Install go get -u github.com/jlaffaye/ftp Documentation https://pkg.go.dev/github.com/jlaffaye/ftp?tab=doc Example

Julien Laffaye 947 Jun 20, 2022
Go PowerDNS 4.x API Client

PowerDNS 4.x API bindings for Golang This community project provides bindings for the currently latest version of PowerDNS Authoritative Server. Featu

Johannes Eiglsperger 57 Jun 26, 2022