MongoDB generic REST server in Go

Related tags

DevOps Tools mora
Overview

Mora - Mongo Rest API

REST server for accessing MongoDB documents and meta data

Documents

When querying on collections those parameters are available:

query  - use mongo shell syntax, e.g. {"size":42}
limit  - maximum number of documents in the result
skip   - offset in the result set
fields - comma separated list of (path-dotted) field names
sort   - comma separated list of (path-dotted) field names
extended_json - set to "true" to return responses in [MongoDB Extended JSON](http://docs.mongodb.org/manual/reference/mongodb-extended-json/) format
Examples
Listing aliases
$ curl 'http://127.0.0.1:8181/docs/' \
>   -D - \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:06:30 GMT
Content-Length: 61

{
  "success": true,
  "data": [
   "test",
   "local"
  ]
}
Listing databases
$ curl 'http://127.0.0.1:8181/docs/local/' \
>   -D - \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:07:11 GMT
Content-Length: 61

{
  "success": true,
  "data": [
   "local",
   "use1"
  ]
}
Listing collections
$ curl 'http://127.0.0.1:8181/docs/local/local' \
>   -D - \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:24:10 GMT
Content-Length: 98

{
  "success": true,
  "data": [
   "new-collection",
   "startup_log",
   "system.indexes"
  ]
}
Inserting document
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection/document-id' \
-D - \
-X POST \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
--data '{"title": "Some title", "content": "document content"}'

HTTP/1.1 201 Created
Content-Location: /docs/local/local/new-collection/document-id
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:23:33 GMT
Content-Length: 116

{
  "success": true,
  "data": {
   "created": true,
   "url": "/docs/local/local/new-collection/document-id"
  }
}
Finding document
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection/document-id' \
>   -D - \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:32:33 GMT
Content-Length: 123

{
  "success": true,
  "data": {
   "_id": "document-id",
   "content": "document content",
   "title": "Some title"
  }
}
Finding documents
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection?limit=1&skip=1' \
>    -D - \
>    -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Wed, 23 Apr 2014 23:18:39 GMT
Content-Length: 387

{
  "success": true,
  "prev_url": "/docs/local/local/new-collection?limit=1\u0026skip=0",
  "next_url": "/docs/local/local/new-collection?limit=1\u0026skip=2",
  "data": [
   {
    "_id": "535849cfb734f91cdc000002",
    "content": "document content",
    "title": "Some title"
   }
  ]
}
Updating document
$ curl 'http://127.0.0.1:8181/docs/local/database/new-collection/document-id' \
>  -D - \
>  -X PUT \
>  -H 'Content-Type: application/json' \
>  -H 'Accept: application/json' \
>  --data '{"title": "New title"}'
HTTP/1.1 200 OK
Content-Location: /docs/local/database/new-collection/document-id
Content-Type: application/json
Date: Tue, 22 Apr 2014 06:37:02 GMT
Content-Length: 133

{
  "success": true,
  "data": {
   "created": false,
   "url": "/docs/local/database/new-collection/document-id"
  }
}
Updating documents
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection' \
>   -D - \
>   -X PUT \
>   -H 'Content-Type: application/json' \
>   -H 'Accept: application/json' \
>   --data '{"$set": {"title": "New title"}}'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Wed, 23 Apr 2014 23:33:11 GMT
Content-Length: 22

{
  "success": true
}
Removing document
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection/document-id'  \
>   -D - \
>   -X DELETE \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:42:47 GMT
Content-Length: 22

{
  "success": true
}
Removing collection
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection'  \
>   -D - \
>   -X DELETE \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:43:24 GMT
Content-Length: 22

{
  "success": true
}
Statistics
Database statistics
$ curl http://127.0.0.1:8181/stats/local/local -D -
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 08:17:46 GMT
Content-Length: 341

{
  "success": true,
  "data": {
   "avgObjSize": 595.6,
   "collections": 3,
   "dataFileVersion": {
    "major": 4,
    "minor": 5
   },
   "dataSize": 5956,
   "db": "local",
   "fileSize": 67108864,
   "indexSize": 0,
   "indexes": 0,
   "nsSizeMB": 16,
   "numExtents": 3,
   "objects": 10,
   "ok": 1,
   "storageSize": 10502144
  }
}
Collection statistics
$ curl http://127.0.0.1:8181/stats/local/local/startup_log -D -
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 08:18:16 GMT
Content-Length: 389

{
  "success": true,
  "data": {
   "avgObjSize": 728,
   "capped": true,
   "count": 8,
   "indexSizes": {},
   "lastExtentSize": 10485760,
   "max": 9223372036854775807,
   "nindexes": 0,
   "ns": "local.startup_log",
   "numExtents": 1,
   "ok": 1,
   "paddingFactor": 1,
   "size": 5824,
   "storageSize": 10485760,
   "systemFlags": 0,
   "totalIndexSize": 0,
   "userFlags": 0
  }
}

Install from source

go get -u github.com/emicklei/mora

Create a release

sh release.sh

Configuration

Mora uses a simple properties file to specify host,port,aliases and other options

# listener info is required
http.server.host=localhost
http.server.port=8181

# enable cross site requests
http.server.cors=true

# for swagger support (optional)
swagger.path=/apidocs/
swagger.file.path=./swagger-ui/dist

# mongo instances are listed here; specify an alias for each
mongod.{alias}.host=localhost
mongod.{alias}.port=27017
# initial and operational timeout in seconds
mongod.{alias}.timeout=5
# optional authentication
mongod.{alias}.username=
mongod.{alias}.password=
mongod.{alias}.database=
# read preference mode
# supported options (case-insensitive): https://godoc.org/gopkg.in/mgo.v2#Mode
mongod.{alias}.mode=primary
# alternatively, a mongodb connection string uri can be used instead
# supported options: https://godoc.org/gopkg.in/mgo.v2#Dial
mongod.{alias}.uri=mongodb://myuser:[email protected]:40001,otherhost:40001/mydb

# enable /stats/ endpoint
mora.statistics.enable=true

Run

$ mora -config mora.properties

Swagger

Swagger UI is displaying automatically generated API documentation and playground.

Swagger UI

© 2013, http://ernestmicklei.com. MIT License

Issues
  • Mora Authentication

    Mora Authentication

    Hey, I'd like to have some authentication options for mora API usage. I think it should be some out-of mongo, distributable authentication service, maybe Keystone (api docs) which is common choice in public and private clouds today as it's part of popular and strong OpenStack. I would like to hear other options and opinions about them, and maybe some voices against my word because Keystone was just a shot.

    opened by crackcomm 7
  • Fix all numbers inserted or updated becoming strings

    Fix all numbers inserted or updated becoming strings

    The parsing of the JSON body was being handled by go-restful, which uses the JSON decoder.UseNumber() method. This parses all numbers as strings instead of integers or floats, which makes it impossible to insert or update numeric values with mora. By handling the JSON parsing locally, we can skip UseNumber, so integers and floats remain as those native types.

    See: https://github.com/emicklei/go-restful/blob/v1.1.3/request.go#L112-L114

    opened by GUI 4
  • resp.WriteHeader(status) in func WriteStatus is not required anymore.

    resp.WriteHeader(status) in func WriteStatus is not required anymore.

    resp.WriteHeader(status) in WriteStatus (response.go) is not required anymore as WriteEntity in go-restful support WriteHeaderAndEntity. So, we can use only resp.WriteHeaderAndEntity(status, r)

    That creates "multiple response.WriteHeader calls" in logs.

    opened by sapamja 3
  • Can't authenticate with Mora

    Can't authenticate with Mora

    I am trying to bring up an API for my mongoDB instance.

    I currently have a user with read only rules that I can connect with so my mora.properties file looks like this:

    #listener info is required
    http.server.host=0.0.0.0
    http.server.port=9123
    
    mongo.db.uri=mongodb://user:[email protected]:27017/mydb
    

    I have also tried the following

    #listener info is required
    http.server.host=0.0.0.0
    http.server.port=9123
    
    #Auth 
    mongod.mydb.databse=mydb
    mongod.mydb.username=user
    mongod.mydb.password=pass
    
    mondod.mydb.host=localhost
    mondod.mydb.port=27017
    

    But when I try to do the following

    $ curl http://localhost:9123/docs/mydb 
     {
       "success": false,
       "error": {
        "code": 401,
        "name": "unauthorized"
       }
      }
    

    And the Mora instance logs:

    2015/06/20 05:32:00 [mora] connecting to [db=mongodb://user:[email protected]:27014/mydb] with timeout [0 seconds]
    2015/06/20 05:32:00 [mora][error] unauthorized
    

    Needless to say - I have checked my user authentication info several times - it is correct. The thing is - I have tried the exact same URI for mongoDB in another app (node.js mongodb-rest api) and it works, but for some reason Mora fails to authenticate.

    Any ideas how to go around that?

    opened by atanasbozhkov 3
  • Use Mongo Extended JSON spec for outputing and querying BSON types

    Use Mongo Extended JSON spec for outputing and querying BSON types

    This standardizes how complex BSON data types (like ObjectId, date fields, binary data, etc) get translated to JSON. It uses MongoDB's extended JSON specification: The heavy lifting is done by the mejson go library.

    By treating the input queries as extended JSON, this allows for several types of queries that weren't previously possible. For example, it's now possible to query a date field like:

    ?query={"updated_at":{"$gt":{"$date":1426411642889}}}
    

    Other special BSON types should be queryable in a similar fashion.

    However, the output is backwards incompatible, since these various BSON field types now have a different output style. For example, what might have previously been returned as:

    {
      "_id": "5505507a1c57d0372be0076c",
      "updated_at": "2015-03-15T09:27:22.889Z",
    }
    

    Is now returned as:

    {
      "_id": {
        "$oid": "5505507a1c57d0372be0076c"
      },
      "updated_at": {
        "$date": 1426411642889
      }
    }
    

    String and numeric fields are not affected by this and will still be returned the same.

    The change in package paths for the mgo library as part of this commit are for compatibility with the mejson library (which uses the gopkg.in url instead of the github.com address).

    Note, this pull request also includes the commit to fix number handling from #31. That was included in this pull request too, since the number fixes are needed to properly handle the things like date parsing (since those are all treated as integers).

    opened by GUI 3
  • Add support for setting read consistency mode on sessions

    Add support for setting read consistency mode on sessions

    This adds support for calling SetMode on the session, which allows for choosing different read preferences (secondary, nearest, etc).

    This also makes a small adjustment to point to the latest version of mgo's Dial documentation in the README's uri reference.

    opened by GUI 2
  • Every new post request create a new TCP connection.

    Every new post request create a new TCP connection.

    Is it by design or how can I reuse the TCP connection.

    Looking at netstat this appears to be resulting in a new connection for every post resulting in a large number of concurrent connections being open.

    What is the correct way to reuse connections in this case?

    opened by sapamja 1
  • bind: cannot assign requested address

    bind: cannot assign requested address

    Each time I used another ip for http.server.host setting throws a binding exeception, here is the full log:

    2016/10/24 18:24:55 [mora][info] loading configuration from [mora.properties]
    [restful] 2016/10/24 18:24:55 log.go:30: [restful/swagger] listing is available at http://192.168.56.103:8181/apidocs.json
    [restful] 2016/10/24 18:24:55 log.go:30: [restful/swagger] http://192.168.56.103:8181/apidocs/ is mapped to folder ./swagger-ui/dist
    2016/10/24 18:24:55 [mora][info] ready to serve on http://192.168.56.103:8181
    2016/10/24 18:24:55 listen tcp 192.168.56.103:8181: bind: cannot assign requested address
    

    it works with localhost or empty string, whatever else fails.

    opened by decebal 1
  • XML serialization error

    XML serialization error

    When I'm creating a request to mora using Google Chrome I'm getting error:

    xml: unsupported type: bson.M
    

    Request:

    GET /docs/lab/appg_test_env1/clients/5334dce9b734f91194000001 HTTP/1.1
    Host: 127.0.0.1:4000
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
    
    opened by crackcomm 1
  • Mora reusing

    Mora reusing

    Hey! Today I was writing a small HTTP API using MongoDB as a storage and I just thought, what if I could reuse mora? Tell me what do You think about it and check code which is my first attempt, I have decoupled and a little bit documented the code. It makes mora a little bit more reusable but we can go further.

    opened by crackcomm 1
  • The correct location of the document for every route.

    The correct location of the document for every route.

    It works with both /docs/alias/db/col (when creating) and /docs/alias/db/col/id (when updating or creating) now.

    It fixed location of modified object in Content-Location header when document was updated with putDocument operation.

    Request Url: http://127.0.0.1:8181/docs/local/use1/webservices/52519954b734f9036c000001 Request Method: PUT

    Status Code: 201 ... Content-Location: /docs/local/use1/webservices/52519954b734f9036c000001/52519954b734f9036c000001

    opened by crackcomm 1
  • len(data)=0  when I set skip=1 and limit=1

    len(data)=0 when I set skip=1 and limit=1

    curl 'http://****?skip=1&limit=1'

    response : { "success": true, "data": [] }

    change limit =2 , the resoponse is ok .

    looks like : limit=1, skip >= 1 , empty limit = 1, skip = 0 , ok limit > 1, ok

    I find len(iter.docData)=0 and iter.err = "not found"

    opened by yuqitao 2
  • OPTIONS response prior to performing PUT does not contain Access-Control-Allow-Origin or Access-Control-Expose-Headers headers. Rejected by Chrome.

    OPTIONS response prior to performing PUT does not contain Access-Control-Allow-Origin or Access-Control-Expose-Headers headers. Rejected by Chrome.

    I was attempting to perform a PUT operation from Angular and kept getting rejected by CORS in Chrome. I checked the GET request I did prior to the PUT and noticed it went through fine (though no OPTIONS request was done for the GET). Upon attempting a PUT, the OPTIONS request goes through and returns a 200 as you'd expect with the Allow response header, but not the Access-Control-Allow-Origin or Access-Control-Expose-Headers headers. This caused Chrome to deny the request.

    I've done some digging in mora and go-restful and it looks like the CrossOriginResourceSharing Filter method isn't being called on the OPTIONS request, though I'm not sure if that's by design and the OPTIONSFilter container func should be handling this instead. The CrossOriginResourceSharing filter method does get called successfully for the GET. I haven't checked to see if it gets called for DELETE or POST yet.

    Also, for completeness sake, I haven't touched the cors setting in mora.properties. It was left in its default state of true.

    Thanks for the great project!

    opened by jamesmillerio 1
  • Is there any plan to add api input validation.

    Is there any plan to add api input validation.

    I think its a good idea to validate the documents before we insert into the database, Using some kind of configuration includes a schema definition for every resource managed by the API.

    opened by sapamja 2
  • Is there any api to run mongodb shell?

    Is there any api to run mongodb shell?

    Just like this link https://github.com/mongodb-labs/sleepy.mongoose/wiki/Database-Commands

    When I tried to insert integer to mongodb, I found that mongodb shell is needed for this situation. Finding documents api of mora seems like support query operation only.

    opened by nifflin 2
  • Interger becomes double when updating document with mora.

    Interger becomes double when updating document with mora.

    I wrote a json like {"a":10} into mongodb with mora, but the integer 10 becomes a double number 10.00 after writing. How to wrote a integer?

    opened by nifflin 1
  • Builds failing due to missing goproperties repo

    Builds failing due to missing goproperties repo

    I'm trying to build mora in a new environment, but it's failing because it looks like the emicklei/goproperties repository has been deleted from github: https://github.com/emicklei/goproperties Was this intentional? I think I was building successfully last month when I thinks this repo still existed.

    $ go get
    # cd .; git clone https://github.com/emicklei/goproperties /tmp/gocode/src/github.com/emicklei/goproperties
    Cloning into '/tmp/gocode/src/github.com/emicklei/goproperties'...
    ERROR: Repository not found.
    fatal: Could not read from remote repository.
    

    I found the original upstream repo at dmotylev/goproperties, so I'm not sure if mora could be switched over to use that directly, or if there was relevant changes in your forked repo that are needed.

    Thanks!

    opened by GUI 3
Owner
Ernest Micklei
GCP Architect, Software Engineer, Google Developer Expert, Gopher, Dartian, Javanian, Smalltalker
Ernest Micklei
Go-Mongodb API - A sample REST API ( CRUD operations ) created using Golang

Go-Mongodb_API This is a sample REST API ( CRUD operations ) created using the G

Aman Sharma 1 May 31, 2022
Generic inquiry tool to OPA server for CI process, such as GitHub Actions

opaq opaq is a generic inquiry tool to OPA server. A major purpose of this tool is for inquiry in GitHub Actions. Features Data formatting: OPA server

Masayoshi Mizutani 3 Jan 20, 2022
Repo CRUD - write e read in mongoDB

Meli User - userwrite Este repo é responsável por um cadastro onde teremos somen

Jefferson Otoni Lima 3 Jan 3, 2022
MongoBackup - This is container that takes backup of MongoDB

MongoBackup This is container that takes backup of MongoDB. It is ment to be ran

Michal Maděra 1 Feb 15, 2022
The k8s-generic-webhook is a library to simplify the implementation of webhooks for arbitrary customer resources (CR) in the operator-sdk or controller-runtime.

k8s-generic-webhook The k8s-generic-webhook is a library to simplify the implementation of webhooks for arbitrary customer resources (CR) in the opera

Norwin Schnyder 6 Jul 12, 2022
Package create provides a generic option pattern for creating new values of any type

create Package create provides a generic option pattern for creating new values

null 1 Dec 30, 2021
A Go package providing a generic data type to track maximum and minimum peak values.

go-peak Overview go-peak is a Go package providing a generic data type that tracks the maximum and minimum peak values within a specific period of tim

tunabay 0 Mar 26, 2022
Generic sharded thread safe LRU cache in Go.

cache Cache is a thread safe, generic, and sharded in memory LRU cache object. This is achieved by partitioning values across many smaller LRU (least

Christopher Cooper 10 May 6, 2022
Kubernetes OS Server - Kubernetes Extension API server exposing OS configuration like sysctl via Kubernetes API

KOSS is a Extension API Server which exposes OS properties and functionality using Kubernetes API, so it can be accessed using e.g. kubectl. At the moment this is highly experimental and only managing sysctl is supported. To make things actually usable, you must run KOSS binary as root on the machine you will be managing.

Mateusz Gozdek 3 May 19, 2021
Gemini server running on the dailybuild server

#dailybuild notice This is the titan2 repo for the dailybuild server. It the static binary is built in a GitHub runner and sent over to the dailybuild

null 0 Nov 26, 2021
Go-http-server-docker - Simple sample server using docker and go

go-http-server-docker Simple sample webserver using docker and go.

null 0 Jan 8, 2022
Webhook-server - Webhook Server for KubeDB resources

webhook-server Webhook Server for KubeDB resources Installation To install KubeD

Kubernetes Database 1 Feb 22, 2022
EdgeDB-Golang-Docker-Sample - The sample of connection between EdgeDB Server and Go Echo API Server

EdgeDB Golang Docker Sample 『Go + Docker Composeを使ってEdgeDBを動かしてみた』のサンプルコードです。 使い

null 1 Jul 9, 2022
Rest API for todoapp written in Golang, using clean architecture, CI/CD

todoapp-backend Rest API for todoapp written in Golang, using Clean Architecture and CI/CD (includes unit tests and integration tests). Using: Web fra

Đạo Phạm 5 Jan 14, 2022
Golang Rest Api Sample

Golang Rest Api Sample Şimdi localhost’umuzda HTTP requestleri ile çalışan basit

Furkan Türkyılmaz 4 Jan 9, 2022
A seed repository that contains a Go project that accepts input via a REST API and saves data to an Oracle database.

rest-oracle-go-seed A seed repository that contains a Go project that accepts input via a REST API and saves data to an Oracle database. Why Oracle? T

Stephen 2 Apr 18, 2022
Koios-rest-go-client - Go Client library for Koios API

Koios API Client Library for Go Koios API is Elastic Cardano Query Layer! A cons

howijd.network 6 Mar 18, 2022
Apibuildr - Tool to generate REST apis in golang

ApiBuildr apibuildr is a commandline tool for creating rest apis in golang langu

Chandan Kumar Singha 5 May 10, 2022
Go WhatsApp Multi-Device Implementation in REST API with Multi-Session/Account Support

Go WhatsApp Multi-Device Implementation in REST API This repository contains example of implementation go.mau.fi/whatsmeow package with Multi-Session/

Dimas Restu H 35 Jul 23, 2022