The MongoDB driver for Go

Overview

Build Status GoDoc

The MongoDB driver for Go

This fork has had a few improvements by ourselves as well as several PR's merged from the original mgo repo that are currently awaiting review. Changes are mostly geared towards performance improvements and bug fixes, though a few new features have been added.

Further PR's (with tests) are welcome, but please maintain backwards compatibility.

Detailed documentation of the API is available at GoDoc.

A sub-package that implements the BSON specification is also included, and may be used independently of the driver.

Supported Versions

mgo is known to work well on (and has integration tests against) MongoDB v3.0, 3.2, 3.4 and 3.6.

MongoDB 4.0 is currently experimental - we would happily accept PRs to help improve support!

Changes

  • Fixes attempting to authenticate before every query (details)
  • Removes bulk update / delete batch size limitations (details)
  • Adds native support for time.Duration marshalling (details)
  • Reduce memory footprint / garbage collection pressure by reusing buffers (details, more)
  • Support majority read concerns (details)
  • Improved connection handling (details)
  • Hides SASL warnings (details)
  • Support for partial indexes (details)
  • Fixes timezone handling (details)
  • Integration tests run against MongoDB 3.2 & 3.4 releases (details, more, more)
  • Improved multi-document transaction performance (details, more, more)
  • Fixes cursor timeouts (details)
  • Support index hints and timeouts for count queries (details)
  • Don't panic when handling indexed int64 fields (details)
  • Supports dropping all indexes on a collection (details)
  • Annotates log entries/profiler output with optional appName on 3.4+ (details)
  • Support for read-only views in 3.4+ (details)
  • Support for collations in 3.4+ (details, more)
  • Provide BSON constants for convenience/sanity (details)
  • Consistently unmarshal time.Time values as UTC (details)
  • Enforces best practise coding guidelines (details)
  • GetBSON correctly handles structs with both fields and pointers (details)
  • Improved bson.Raw unmarshalling performance (details)
  • Minimise socket connection timeouts due to excessive locking (details)
  • Natively support X509 client authentication (details)
  • Gracefully recover from a temporarily unreachable server (details)
  • Use JSON tags when no explicit BSON are tags set (details)
  • Support $changeStream tailing on 3.6+ (details)
  • Fix deadlock in cluster synchronisation (details)
  • Implement maxIdleTimeout for pooled connections (details)
  • Connection pool waiting improvements (details)
  • Fixes BSON encoding for $in and friends (details)
  • Add BSON stream encoders (details)
  • Add integer map key support in the BSON encoder (details)
  • Support aggregation collations (details)
  • Support encoding of inline struct references (details)
  • Improved windows test harness (details)
  • Improved type and nil handling in the BSON codec (details, more)
  • Separated network read/write timeouts (details)
  • Expanded dial string configuration options (details)
  • Implement MongoTimestamp (details)
  • Support setting writeConcern for findAndModify operations (details)
  • Add ssl to the dial string options (details)

Thanks to

  • @aksentyev
  • @bachue
  • @bozaro
  • @BenLubar
  • @carldunham
  • @carter2000
  • @cedric-cordenier
  • @cezarsa
  • @DaytonG
  • @ddspog
  • @drichelson
  • @dvic
  • @eaglerayp
  • @feliixx
  • @fmpwizard
  • @gazoon
  • @gedge
  • @gnawux
  • @idy
  • @jameinel
  • @jefferickson
  • @johnlawsharrison
  • @KJTsanaktsidis
  • @larrycinnabar
  • @mapete94
  • @maxnoel
  • @mcspring
  • @Mei-Zhao
  • @peterdeka
  • @Reenjii
  • @roobre
  • @smoya
  • @steve-gray
  • @tbruyelle
  • @wgallagher
Comments
  • Add switch for `omitempty` tag

    Add switch for `omitempty` tag

    Currently we have bson.SetJSONTagFallback to fallback to json tag when no bson tag available, which is pretty useful when using protobuf with mgo, leave generated *.pb.go untouched and get a consistent field name between json & bson.

    With the default omitempty tag generated by protobuf, updated struct fields with empty values will not be encoded, leave the database untouched. Adding a switch for ignoring omitempty will gain another benefit that we can keep protobuf performant and make updating operations easier.

    enhancement needs info 
    opened by kassiansun 20
  • Add signaling support for connection pool waiting

    Add signaling support for connection pool waiting

    The current behaviour when the poolLimit is reached and a new connection is required is to poll every 100ms to see if there is now headroom to make a new connection. This adds tremendous latency to the limit-hit-path.

    This commit changes the checkout behaviour to watch on a condition variable for connections to become available, and the checkin behaviour to signal this variable. This should allow waiters to use connections immediately after they become available.

    A new parameter is also added to DialInfo, PoolTimeout, which is the maximum time that clients will wait for connection headroom to become available. By default this is unlimited.

    Full disclosure - I haven't yet actually tried this in our production environment yet. The only testing this has so far is that it passes the regression tests. I'm pull-requesting this early to get some feedback on the design of this change and to see if this would be useful for other people :). I intend to try and get this into production sometime in the next couple of weeks.

    enhancement 
    opened by KJTsanaktsidis 17
  • read tcp i/o timeout on Windows 10

    read tcp i/o timeout on Windows 10

    After some db calls, we have to restart our Go backend:

    Message: read tcp 127.0.0.1:50743->127.0.0.1:27017: i/o timeout

    MongoDB 3.4.10 Go 1.9.2 windows/amd64 github.com/globalsign/mgo 5be15cc6a44937b6937a9487e1633f8a5fb25e4a

    I think it was ok with: MongoDB 3.2.16 github.com/globalsign/mgo c4a7121bd9f6f7246d9f54b78bdd2eb4d18b4c76 Go 1.9.1 Alpine 3.6

    needs info 
    opened by hexadecy 17
  • MongoDB 3.6: implement the new wire protocol

    MongoDB 3.6: implement the new wire protocol

    Mongodb 3.6: implement the new wire protocol

    WIP - Do not merge

    Goal

    This PR is an attempt to implement the new wire protocol introduced in Mongodb 3.6. This protocol add a new message format: OP_MSG

    OP_MSG is a bi-directional wire protocol opcode introduced in MongoDB 3.6 with the goal of replacing most existing opcodes, merging their use into one extendable opcode.

    Tests currently run against MongoDB 3.6-rc3 (first 3.6 stable version should be released next month) I would like to have some feedbacks on this before going any further !

    What's in this PR

    this PR contains several commits, first 6 are small update to make test pass against last version of each major serie, and the last one is the new wire protocol implementation

    • commit c1fc70d3d748fb4dd0ebb851b1a8d9a908e053fa test against 3.6-rc3

    • commit a594e559227a1301f8aeecb5895cd00127cff502 update mongodb version use last minor version for each major serie

    • commit f6a8010534991d83d812e235a6fc91f5bb801699 test fixes on 3.2.17

    • commit 8242b9c0ab4762bbbb3e6865c91c7ed888a9ae7f fix X509 test fix TestAuthX509CredRDNConstruction test: need to create an user with {"username": subject}
      before trying to login

    • commit 3365de912f1ba8ebbe2372b8d8646c097d927105 Fix auth test on 3.6-rc3

      Make sure that "rs3/127.0.0.1/40031" is elected at primary. Create user before running 'addShard' command as it requires root access Also add a retry mechanism to make sure that shard are correctly added cf https://docs.mongodb.com/manual/tutorial/deploy-shard-cluster/

    • commit 17833465551a225e567eb2250c2759354260c96d re-enable TestViewWithCollation SERVER-31049 was fixed in 3.4.10, so re-enable it in tests

    • commit 6a815bbddf6a20ab1e1ddc981cebeb5045e59946 start implementing new wire protocol

    TODO

    • [x] fix last 9 failing Test:
      • [x] TestDirect
      • [x] TestBulkUpdateErrorUnordered
      • [x] TestAuthLoginLogout
      • [x] TestAuthLoginLogoutAll
      • [x] TestAuthLoginChangePassword
      • [x] TestAuthUpsertUser
      • [x] TestAuthAddUser
      • [x] TestAuthLoginCachingAcrossPoolWithLogout
      • [x] TestTopologySyncWithSlaveSeed
    • [x] add possibility to set writeConcern
    • [x] make sure that bulk op doesn't contain more than 100000 docs
    • [x] enable/disable OP_MSG with an param in connection URI
    enhancement 3.6 feature 
    opened by feliixx 16
  • test killed after 10min on travis with docker mongo

    test killed after 10min on travis with docker mongo

    Hi,

    Every once in a while our mongo suite gets killed on TravicCI. We run go 1.10 and use docker for our test suites. Our Postgres and Neo4j test suites run just fine with this setup but with mgo and Mongo we're having these issues.

    Stacktrace information can be found below. Any idea why this is happening?

    +go test -v -race -coverprofile=coverage.out -covermode=atomic ./...
    === RUN   TestMongoSuiteWithoutCredentials
    2018/03/04 13:50:01 CREATING NEW POOL
    2018/03/04 13:50:01 POOL CREATED <nil>
    2018/03/04 13:50:01 RUNNING MONGO CONTAINER
    2018/03/04 13:50:11 MONGO CONTAINER CREATED <nil>
    2018/03/04 13:50:11 BEFORE testConnect
    2018/03/04 13:50:11 START DialWithTimeout
    2018/03/04 13:50:11 MONGO URL = mongodb://localhost:32768
    SIGQUIT: quit
    PC=0x474643 m=0 sigcode=0
    
    goroutine 31 [syscall]:
    runtime.notetsleepg(0x12be9e0, 0x37e09133e, 0x16)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/lock_futex.go:227 +0x42 fp=0xc420052760 sp=0xc420052730 pc=0x422022
    runtime.timerproc(0x12be9c0)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/time.go:261 +0x2f9 fp=0xc4200527d8 sp=0xc420052760 pc=0x461889
    runtime.goexit()
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc4200527e0 sp=0xc4200527d8 pc=0x472bd1
    created by runtime.(*timersBucket).addtimerLocked
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/time.go:160 +0x107
    
    goroutine 1 [chan receive]:
    testing.(*T).Run(0xc42021c000, 0xd2d7a8, 0x20, 0xd41b80, 0xc4201e5c00)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:825 +0x597
    testing.runTests.func1(0xc42021c000)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:1063 +0xa5
    testing.tRunner(0xc42021c000, 0xc4201e5d48)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:777 +0x16e
    testing.runTests(0xc4201378e0, 0x127b3e0, 0x1, 0x1, 0xc420160800)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:1061 +0x4e2
    testing.(*M).Run(0xc420160800, 0x0)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:978 +0x2ce
    main.main()
    	_testmain.go:90 +0x325
    
    goroutine 19 [syscall]:
    os/signal.signal_recv(0x472bd1)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/sigqueue.go:139 +0xa6
    os/signal.loop()
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/os/signal/signal_unix.go:22 +0x30
    created by os/signal.init.0
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/os/signal/signal_unix.go:28 +0x4f
    
    goroutine 20 [semacquire]:
    sync.runtime_notifyListWait(0xc42023a6e8, 0xc400000000)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/sema.go:510 +0x11a
    sync.(*Cond).Wait(0xc42023a6d8)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/sync/cond.go:56 +0x8e
    github.com/globalsign/mgo.(*mongoCluster).AcquireSocket(0xc42023a6c0, 0x0, 0xc420240a01, 0x6fc23ac00, 0x6fc23ac00, 0x0, 0x0, 0x0, 0x1000, 0x1c5b320, ...)
    	/home/travis/gopath/src/github.com/globalsign/mgo/cluster.go:644 +0xff
    github.com/globalsign/mgo.(*Session).acquireSocket(0xc4202409c0, 0xb9e201, 0x0, 0x0, 0x0)
    	/home/travis/gopath/src/github.com/globalsign/mgo/session.go:4853 +0x271
    github.com/globalsign/mgo.(*Database).Run(0xc42017bc20, 0xc2ee40, 0xda60b0, 0x0, 0x0, 0x0, 0x0)
    	/home/travis/gopath/src/github.com/globalsign/mgo/session.go:799 +0x5e
    github.com/globalsign/mgo.(*Session).Run(0xc4202409c0, 0xc2ee40, 0xda60b0, 0x0, 0x0, 0xcf84e0, 0xc42023a6c0)
    	/home/travis/gopath/src/github.com/globalsign/mgo/session.go:2270 +0xba
    github.com/globalsign/mgo.(*Session).Ping(0xc4202409c0, 0xc42023a6c0, 0x6fc23ac00)
    	/home/travis/gopath/src/github.com/globalsign/mgo/session.go:2299 +0x5d
    github.com/globalsign/mgo.DialWithInfo(0xc4202c0000, 0x17, 0xc4202c0000, 0x0)
    	/home/travis/gopath/src/github.com/globalsign/mgo/session.go:563 +0x566
    github.com/globalsign/mgo.DialWithTimeout(0xc420026d20, 0x17, 0x6fc23ac00, 0x0, 0xc420167780, 0xc4200b0120)
    	/home/travis/gopath/src/github.com/globalsign/mgo/session.go:304 +0xc3
    mongo_test.(*suite).testConnect(0xc42017bf48, 0xc42021c0f0)
    	/home/travis/build/qdentity/qdentity/go/src/mongo/mongo_test.go:36 +0xc8
    mongo_test.TestMongoSuiteWithoutCredentials(0xc42021c0f0)
    	/home/travis/build/qdentity/qdentity/go/src/mongo/mongo_test.go:22 +0x187
    testing.tRunner(0xc42021c0f0, 0xd41b80)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:777 +0x16e
    created by testing.(*T).Run
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/testing/testing.go:824 +0x565
    
    goroutine 30 [semacquire]:
    sync.runtime_notifyListWait(0xc42023a6e8, 0xc400000001)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/sema.go:510 +0x11a
    sync.(*Cond).Wait(0xc42023a6d8)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/sync/cond.go:56 +0x8e
    github.com/globalsign/mgo.(*mongoCluster).AcquireSocket(0xc42023a6c0, 0x1, 0xc420240b01, 0x2540be400, 0x2540be400, 0x0, 0x0, 0x0, 0x1000, 0xc420082700, ...)
    	/home/travis/gopath/src/github.com/globalsign/mgo/cluster.go:644 +0xff
    github.com/globalsign/mgo.(*Session).acquireSocket(0xc420240b60, 0xc5f001, 0x0, 0x0, 0x0)
    	/home/travis/gopath/src/github.com/globalsign/mgo/session.go:4853 +0x271
    github.com/globalsign/mgo.(*Database).Run(0xc4200779b8, 0xc5f0c0, 0xc42000d200, 0xc10ec0, 0xc420232630, 0x0, 0x0)
    	/home/travis/gopath/src/github.com/globalsign/mgo/session.go:799 +0x5e
    github.com/globalsign/mgo.(*Session).Run(0xc420240b60, 0xc5f0c0, 0xc42000d200, 0xc10ec0, 0xc420232630, 0x0, 0x1)
    	/home/travis/gopath/src/github.com/globalsign/mgo/session.go:2270 +0xba
    github.com/globalsign/mgo.(*mongoCluster).isMaster(0xc42023a6c0, 0xc4202c20f0, 0xc420232630, 0xc4202c20f0, 0x0)
    	/home/travis/gopath/src/github.com/globalsign/mgo/cluster.go:182 +0x258
    github.com/globalsign/mgo.(*mongoCluster).syncServer(0xc42023a6c0, 0xc4202c00e0, 0xd, 0xc42001ed20, 0xc4202c00e0, 0xc42023a6c0, 0xc440000000, 0x0)
    	/home/travis/gopath/src/github.com/globalsign/mgo/cluster.go:231 +0x434
    github.com/globalsign/mgo.(*mongoCluster).syncServersIteration.func1.1(0xc420292060, 0xc420026d2a, 0xd, 0xc420292070, 0xc420026d00, 0xc4202867b0, 0xc42023a6c0, 0xc4202867e0, 0xc420286810, 0x0, ...)
    	/home/travis/gopath/src/github.com/globalsign/mgo/cluster.go:553 +0x1fb
    created by github.com/globalsign/mgo.(*mongoCluster).syncServersIteration.func1
    	/home/travis/gopath/src/github.com/globalsign/mgo/cluster.go:525 +0x175
    
    goroutine 11 [semacquire]:
    sync.runtime_Semacquire(0xc42029206c)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/sema.go:56 +0x39
    sync.(*WaitGroup).Wait(0xc420292060)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/sync/waitgroup.go:129 +0xb3
    github.com/globalsign/mgo.(*mongoCluster).syncServersIteration(0xc42023a6c0, 0x0)
    	/home/travis/gopath/src/github.com/globalsign/mgo/cluster.go:582 +0x4c5
    github.com/globalsign/mgo.(*mongoCluster).syncServersLoop(0xc42023a6c0)
    	/home/travis/gopath/src/github.com/globalsign/mgo/cluster.go:390 +0x17c
    created by github.com/globalsign/mgo.newCluster
    	/home/travis/gopath/src/github.com/globalsign/mgo/cluster.go:81 +0x2e3
    
    goroutine 12 [sleep]:
    time.Sleep(0x37e11d600)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/time.go:102 +0x146
    github.com/globalsign/mgo.(*mongoServer).pinger(0xc4202c00e0, 0x479801)
    	/home/travis/gopath/src/github.com/globalsign/mgo/server.go:314 +0x7ad
    created by github.com/globalsign/mgo.newServer
    	/home/travis/gopath/src/github.com/globalsign/mgo/server.go:89 +0x24b
    
    goroutine 34 [IO wait]:
    internal/poll.runtime_pollWait(0x7f50f3494f00, 0x72, 0x128aff0)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/runtime/netpoll.go:173 +0x5e
    internal/poll.(*pollDesc).wait(0xc420234e18, 0x72, 0xda9f00, 0x128aff0, 0xffffffffffffffff)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/internal/poll/fd_poll_runtime.go:85 +0xe5
    internal/poll.(*pollDesc).waitRead(0xc420234e18, 0xc420028800, 0x24, 0x24)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/internal/poll/fd_poll_runtime.go:90 +0x4b
    internal/poll.(*FD).Read(0xc420234e00, 0xc420028840, 0x24, 0x24, 0x0, 0x0, 0x0)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/internal/poll/fd_unix.go:157 +0x22a
    net.(*netFD).Read(0xc420234e00, 0xc420028840, 0x24, 0x24, 0x4ab9ed, 0xc420234e00, 0x0)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/net/fd_unix.go:202 +0x66
    net.(*conn).Read(0xc42000e0c8, 0xc420028840, 0x24, 0x24, 0x0, 0xc4202c24b0, 0xc420062dc0)
    	/home/travis/.gimme/versions/go1.10.linux.amd64/src/net/net.go:176 +0x85
    github.com/globalsign/mgo.fill(0xdb3660, 0xc42000e0c8, 0xc420028840, 0x24, 0x24, 0x0, 0x11)
    	/home/travis/gopath/src/github.com/globalsign/mgo/socket.go:567 +0x64
    github.com/globalsign/mgo.(*mongoSocket).readLoop(0xc4202c24b0)
    	/home/travis/gopath/src/github.com/globalsign/mgo/socket.go:583 +0x15b
    created by github.com/globalsign/mgo.newSocket
    	/home/travis/gopath/src/github.com/globalsign/mgo/socket.go:197 +0x341
    
    rax    0xfffffffffffffffc
    rbx    0x12bb3a0
    rcx    0x474643
    rdx    0x0
    rdi    0x12be9e0
    rsi    0x0
    rbp    0xc4200526e8
    rsp    0xc420052698
    r8     0x0
    r9     0x0
    r10    0xc4200526d8
    r11    0x202
    r12    0xc420079c80
    r13    0x12bb3a0
    r14    0xc420001500
    r15    0x1a354620
    rip    0x474643
    rflags 0x202
    cs     0x33
    fs     0x0
    gs     0x0
    *** Test killed with quit: ran too long (10m0s).
    FAIL	mongo	600.006s
    
    bug 
    opened by dvic 14
  • Data race for socket

    Data race for socket

    Hello! I got race condition randomly. Here is stack:

    WARNING: DATA RACE
    Read at 0x00c4201381b8 by goroutine 94:
      github.com/globalsign/mgo.(*Session).acquireSocket()
          /.../github.com/globalsign/mgo/session.go:4822 +0x768
      github.com/globalsign/mgo.(*Database).Run()
          /.../github.com/globalsign/mgo/session.go:799 +0x6d
      github.com/globalsign/mgo.(*Session).Run()
          /.../github.com/globalsign/mgo/session.go:2270 +0xc3
      github.com/globalsign/mgo.(*mongoCluster).isMaster()
          /.../github.com/globalsign/mgo/cluster.go:182 +0x27c
      github.com/globalsign/mgo.(*mongoCluster).syncServer()
          /.../github.com/globalsign/mgo/cluster.go:231 +0x457
      github.com/globalsign/mgo.(*mongoCluster).syncServersIteration.func1.1()
          /.../github.com/globalsign/mgo/cluster.go:553 +0x200
    
    Previous write at 0x00c4201381b8 by goroutine 33:
      github.com/globalsign/mgo.(*mongoSocket).kill()
          /.../github.com/globalsign/mgo/socket.go:351 +0x251
      github.com/globalsign/mgo.(*mongoSocket).resetNonce.func1()
          /.../github.com/globalsign/mgo/auth.go:123 +0x135
      github.com/globalsign/mgo.(*mongoSocket).readLoop()
          /.../github.com/globalsign/mgo/socket.go:628 +0x1101
    
    Goroutine 94 (running) created at:
      github.com/globalsign/mgo.(*mongoCluster).syncServersIteration.func1()
          /.../github.com/globalsign/mgo/cluster.go:525 +0x174
      github.com/globalsign/mgo.(*mongoCluster).syncServersIteration()
          /.../github.com/globalsign/mgo/cluster.go:580 +0x54b
      github.com/globalsign/mgo.(*mongoCluster).syncServersLoop()
          /.../github.com/globalsign/mgo/cluster.go:390 +0x18e
    
    Goroutine 33 (running) created at:
      github.com/globalsign/mgo.newSocket()
          /.../github.com/globalsign/mgo/socket.go:197 +0x392
      github.com/globalsign/mgo.(*mongoServer).Connect()
          /.../github.com/globalsign/mgo/server.go:183 +0x60c
      github.com/globalsign/mgo.(*mongoServer).AcquireSocket()
          /.../github.com/globalsign/mgo/server.go:129 +0x349
      github.com/globalsign/mgo.(*mongoCluster).syncServer()
          /.../github.com/globalsign/mgo/cluster.go:225 +0x2a5
      github.com/globalsign/mgo.(*mongoCluster).syncServersIteration.func1.1()
          /.../github.com/globalsign/mgo/cluster.go:553 +0x200
    ==================
    

    I can't reproduce it. But I believe, that we need to Lock mutex for s.masterSocket on /.../globalsign/mgo/session.go:4822 before read dead property. Am I right?

    Му OS is Ubuntu 16.04 and GO version is 1.9.2 linux/amd64.

    Thank you in advance!

    bug 
    opened by kirik 12
  • Partial 4.0 Transaction support

    Partial 4.0 Transaction support

    Third time's a charm! I think we did it right this time.

    This time, I pulled the latest version of development and ported my changes into that. This contains the following changes:

    1. Partial 4.0 transaction support. This supports the basic Update, Upsert, Remove, and Insert support. Batch support is not included (but could probably easily be added - it's not a use case we needed at the moment).
    2. dbtest supports creating a replica set now. You have to explicitly enable that.
    3. A few other minor bugs and issues fixed.

    Documentation is included in the repo.

    All changes are, to the best of my knowledge, backwards compatible. All tests pass for me with the exception of some clustering tests that may be related to 4.0 but are not related to my changes to the best of my knowledge.

    We at The Home Depot are excited to be sharing these changes with the community, please let me know if there are further issues that need to be addressed.

    4.0 feature 
    opened by duskglow 11
  • Changestreams db and cluster level #251

    Changestreams db and cluster level #251

    Enhancing change streams in order to support DB level and cluster level changes notifications introduced in mongoDB 4.0 as per issue #251. I strongly ask review of the way i addressed the question as i'm not really sure it is not a workaround. If we go this way, i think it's better to make unexported all session and database methods that were added (Pipe(), NewIter()), except for the Watch() method in order to not pollute with meaningless methods. Tests are still missing and will be added once we agree on the way i implemented the functionality and Mongo 4.0 is added to the test suite.

    4.0 feature 
    opened by peterdeka 11
  • cross operation on multi bulks result in additional records inserts

    cross operation on multi bulks result in additional records inserts

    say we have two bulk variables defined from different collections named "Bulk1" and "Bulk2". After running following code, collection "Bulk1" get 2 document inserted, but collection "Bulk2" also get 2 documents inserted! the reason for this is because actionPool defined in bulk.go is not safe in this situation. i just made a pull request to resolve this issue, hope it will get accepted, thanks

    var blk = DB().C("Bulk1").Bulk()
    var blk2 = DB().C("Bulk2").Bulk()
    for i := 0; i < 2; i++ {
    	blk.Insert(bson.M{})
    	if i==0{
    		blk.Run()
    	}
    }
    blk2.Insert(bson.M{})
    blk.Run()
    blk2.Run()`
    
    bug 
    opened by qfrank 11
  • Change session.DatabaseNames to use the `nameOnly` option (if available)

    Change session.DatabaseNames to use the `nameOnly` option (if available)

    listDatabases by default returns the list of database names as well as some stats about the database (size, etc.). To gather the stats mongod acquires a db level lock, which is very slow (and depending on other locks etc. may cause this to unnecessarily fail).

    Perf comparison:

    basic took 2.772775182s
    nameonly took 93.680166ms
    

    I have also tested that this additional option falls back to the previous behavior on mongo <3.6

    Mongo docs: https://docs.mongodb.com/manual/reference/command/listDatabases/

    enhancement 
    opened by jacksontj 10
  • support MongoDB 4.0 startAtOperationTime option

    support MongoDB 4.0 startAtOperationTime option

    I wasn't sure how to write new tests for this. But this is an attempt to support the new option in MongoDB 4.0 to start the change stream from a specific Timestamp in the oplog.

    needs tests 4.0 feature 
    opened by rwynn 10
  • BSON unmarshal unable to parse $date

    BSON unmarshal unable to parse $date

    We use the issue tracker to track bugs with mgo - if you have a usage question, it's best to try Stack Overflow :)

    Replace this text with your description, and please answer the questions below before submitting your issue to help us out. Thanks!


    What version of MongoDB are you using (mongod --version)?

    db version v6.0.1
    Build Info: {
        "version": "6.0.1",
        "gitVersion": "32f0f9c88dc44a2c8073a5bd47cf779d4bfdee6b",
        "modules": [],
        "allocator": "system",
        "environment": {
            "distarch": "aarch64",
            "target_arch": "aarch64"
        }
    }
    

    What version of Go are you using (go version)?

    go1.19
    

    What did you do?

    If possible, provide a recipe for reproducing the error. A runnable program is great and really helps!

    var tsMap map[string]interface{}
    	bsonData := []byte(`
    {"_id": {"$numberLong": "10004"},"order_date": {"$date": 1456012800000},"purchaser_id": {"$numberLong": "1003"},"quantity": 1,"product_id": {"$numberLong": "107"}}
    
    `)
    err := bson.UnmarshalJSON(bsonData, &tsMap)
    
    assert.NoError(p.T(), err)
    

    Instead of it working, it returned an error, see: err cannot parse date: "{\"$date\": 1456012800000}"

    Can you reproduce the issue on the latest development branch?

    opened by Tang8330 0
  • current

    current

    We use the issue tracker to track bugs with mgo - if you have a usage question, it's best to try Stack Overflow :)

    Replace this text with your description, and please answer the questions below before submitting your issue to help us out. Thanks!


    What version of MongoDB are you using (mongod --version)?

    <mongodb version here>
    

    What version of Go are you using (go version)?

    <go version here>
    

    What operating system and processor architecture are you using (go env)?

    <go env here>
    

    What did you do?

    If possible, provide a recipe for reproducing the error. A runnable program is great and really helps!

    Can you reproduce the issue on the latest development branch?

    opened by chenyiping111 0
  • Panic With ObjectIdHex(s string)

    Panic With ObjectIdHex(s string)

    It is very danger with mgo/bson/bson.go.212,see code; if anyone else who wanted to change request args to mongoDB objectID, the code maybe like this:

    objID := bson.ID
    if IsObjectIdHex(req.ObjectID) {
        objID = ObjectIdHex(req.ObjectID)
    }else {
        // TODO do else logic
    }
    

    And Why not add like this:

    if objID,err := MaybeObjectIdHex(req.ObjectID);err != nil {
        // TODO add return logic
        return
    }
    
        // TODO do correct logic
    

    Any way, panic in runtime is very unfriendly for every programmer.

    opened by zjzjzjzj1874 0
  • Fully automate dev setup with Gitpod

    Fully automate dev setup with Gitpod

    This commit implements a fully-automated development setup using Gitpod.io, an online IDE for GitLab, GitHub, and Bitbucket that enables Dev-Environments-As-Code. This makes it easy for anyone to get a ready-to-code workspace for any branch, issue or pull request almost instantly with a single click.

    opened by taherv 0
Releases(r2018.06.15)
  • r2018.06.15(Jun 15, 2018)

    Includes:

    • Collation support for Count() queries (detials)
    • Support encoding of inline struct references (details)
    • Improved windows test harness (details)
    • Improved type and nil handling in the BSON codec (details, more)
    • Separated network read/write timeouts (details)
    • Expanded dial string configuration options (details)
    • Implement MongoTimestamp (details)
    • Support setting writeConcern for findAndModify operations (details)
    • Add ssl to the dial string options (details)

    Thanks to:

    • @cedric-cordenier
    • @DaytonG
    • @ddspog
    • @gedge
    • @jefferickson
    • @larrycinnabar
    • @Mei-Zhao
    • @roobre
    • @tbruyelle

    Merge PR #191

    Source code(tar.gz)
    Source code(zip)
  • r2018.04.23(Apr 24, 2018)

  • r2018.04.03(Apr 3, 2018)

  • r2018.02.20(Feb 20, 2018)

    Includes:

    • Use JSON struct tags if there are no explicit BSON tags (#91)
    • Experimental $changeStream support (#97)
    • Tests recovery of abnormal connection closures (#100)
    • Returns the txn error to the caller (#81)

    Thanks to:

    • @steve-gray
    • @peterdeka
    • @jameinel

    Note: The $changeStream feature has not had the extra testing typical for a release due to the current lack of a 3.6+ testing environment - please run your own acceptance tests.

    Merge PR #111.

    Source code(tar.gz)
    Source code(zip)
  • r2018.02.13(Feb 13, 2018)

    This is a hotfix release for https://github.com/globalsign/mgo/pull/105.

    Affected users should upgrade as soon as possible though there's no risk of data loss/corruption.

    Thanks to:

    • @changwoo-nam
    • @qhenkart
    • @canthefason
    • @jyoon17
    Source code(tar.gz)
    Source code(zip)
  • r2018.01.15(Jan 15, 2018)

    Includes:

    • Reduced memory usage in bulk operations (https://github.com/globalsign/mgo/pull/56)
    • Native x509 authentication (https://github.com/globalsign/mgo/pull/55)
    • Better connection recovery (https://github.com/globalsign/mgo/pull/69)
    • Example usage (#75 and #78)

    Thanks to:

    • @bachue
    • @csucu
    • @feliixx

    Merge PR #79

    Source code(tar.gz)
    Source code(zip)
  • r2017.11.06(Nov 6, 2017)

    Includes:

    • Improved bson.Raw unmarshalling performance (details)
    • Minimise socket connection timeouts due to excessive locking (details)

    Of note is #52 which should help reduce (but not eliminate) socket errors which seem fairly common (e.g. #21, https://github.com/go-mgo/mgo/issues/473, https://github.com/go-mgo/mgo/issues/498, etc).

    Thanks for the contributions from:

    • @bozaro
    • @idy
    Source code(tar.gz)
    Source code(zip)
  • r2017.10.11(Oct 11, 2017)

  • r2017.09.18(Sep 18, 2017)

  • r2017.07.26(Jul 26, 2017)

    Changes:

    • Cache TXN ID (https://github.com/globalsign/mgo/pull/10)
    • TXN preloading (https://github.com/globalsign/mgo/pull/11)
    • Timezone handling bugfix (https://github.com/globalsign/mgo/pull/13)
    • Cursor timeout bugfix (https://github.com/globalsign/mgo/pull/15)
    • Bounded TXN queues (https://github.com/globalsign/mgo/pull/16)
    • Count() timeout + index hints (https://github.com/globalsign/mgo/pull/17)

    Merge PR: https://github.com/globalsign/mgo/pull/18

    Source code(tar.gz)
    Source code(zip)
  • r2017.06.15(Jul 26, 2017)

  • r2017.05.15(Jul 26, 2017)

  • r2017.04.18(Jul 26, 2017)

Owner
GlobalSign
GlobalSign
Mongo Go Models (mgm) is a fast and simple MongoDB ODM for Go (based on official Mongo Go Driver)

Mongo Go Models Important Note: We changed package name from github.com/Kamva/mgm/v3(uppercase Kamva) to github.com/kamva/mgm/v3(lowercase kamva) in v

kamva 588 Nov 20, 2022
The MongoDB driver for Go

The MongoDB driver for Go This fork has had a few improvements by ourselves as well as several PR's merged from the original mgo repo that are current

GlobalSign 2k Nov 30, 2022
The Go driver for MongoDB

MongoDB Go Driver The MongoDB supported driver for Go. Requirements Installation Usage Bugs / Feature Reporting Testing / Development Continuous Integ

mongodb 7.1k Nov 30, 2022
Go MySQL Driver is a MySQL driver for Go's (golang) database/sql package

Go-MySQL-Driver A MySQL-Driver for Go's database/sql package Features Requirements Installation Usage DSN (Data Source Name) Password Protocol Address

Go SQL Drivers 12.9k Dec 4, 2022
Go driver for PostgreSQL over SSH. This driver can connect to postgres on a server via SSH using the local ssh-agent, password, or private-key.

pqssh Go driver for PostgreSQL over SSH. This driver can connect to postgres on a server via SSH using the local ssh-agent, password, or private-key.

mattn 52 Nov 6, 2022
Simple key-value store abstraction and implementations for Go (Redis, Consul, etcd, bbolt, BadgerDB, LevelDB, Memcached, DynamoDB, S3, PostgreSQL, MongoDB, CockroachDB and many more)

gokv Simple key-value store abstraction and implementations for Go Contents Features Simple interface Implementations Value types Marshal formats Road

Philipp Gillé 493 Nov 28, 2022
Data access layer for PostgreSQL, CockroachDB, MySQL, SQLite and MongoDB with ORM-like features.

upper/db is a productive data access layer (DAL) for Go that provides agnostic tools to work with different data sources

upper.io 3.2k Nov 29, 2022
💲 Golang, Go Fiber, RabbitMQ, MongoDB, Docker, Kubernetes, GitHub Actions

Bank Projeto para simular empréstimos financeiros em um banco para clientes Tecnologias Utilizadas Golang MongoDB RabbitMQ Github Actions Docker Hub D

Jailton Junior 6 Oct 13, 2022
Examples and code to assign a name to your MongoDB, MySQL, PostgreSQL, RabbitMQ, and redis connection.

your connection deserves a name ?? When your app interacts with an external system, assign a name to the connection. An external system in this contex

Andy Grunwald 27 Nov 2, 2022
A MongoDB compatible embeddable database and toolkit for Go.

lungo A MongoDB compatible embeddable database and toolkit for Go. Installation Example Motivation Architecture Features License Installation To get s

Joël Gähwiler 397 Nov 19, 2022
Go-odm, a Golang Object Document Mapping for MongoDB.

A project of SENROK Open Source Go ODM Go-odm, a Golang Object Document Mapping for MongoDB. Table of contents Features Installation Get started Docum

SENROK 4 Nov 4, 2022
Golang MongoDB Integration Examples

Get Program Get a copy of the program: git clone https://github.com/hmdhszd/Go

Hamid Hosseinzadeh 1 Feb 1, 2022
Mirror of Apache Calcite - Avatica Go SQL Driver

Apache Avatica/Phoenix SQL Driver Apache Calcite's Avatica Go is a Go database/sql driver for the Avatica server. Avatica is a sub-project of Apache C

The Apache Software Foundation 103 Nov 3, 2022
Firebird RDBMS sql driver for Go (golang)

firebirdsql (Go firebird sql driver) Firebird RDBMS http://firebirdsql.org SQL driver for Go Requirements Firebird 2.5 or higher Golang 1.13 or higher

Hajime Nakagami 184 Nov 25, 2022
Microsoft ActiveX Object DataBase driver for go that using exp/sql

go-adodb Microsoft ADODB driver conforming to the built-in database/sql interface Installation This package can be installed with the go get command:

mattn 131 Nov 2, 2022
Microsoft SQL server driver written in go language

A pure Go MSSQL driver for Go's database/sql package Install Requires Go 1.8 or above. Install with go get github.com/denisenkom/go-mssqldb . Connecti

null 1.7k Nov 24, 2022
Oracle driver for Go using database/sql

go-oci8 Description Golang Oracle database driver conforming to the Go database/sql interface Installation Install Oracle full client or Instant Clien

mattn 596 Nov 13, 2022
sqlite3 driver for go using database/sql

go-sqlite3 Latest stable version is v1.14 or later not v2. NOTE: The increase to v2 was an accident. There were no major changes or features. Descript

mattn 6.2k Dec 6, 2022
GO DRiver for ORacle DB

Go DRiver for ORacle godror is a package which is a database/sql/driver.Driver for connecting to Oracle DB, using Anthony Tuininga's excellent OCI wra

null 403 Nov 25, 2022