GODRIVER-3587 Use raw bytes in valueReader#2120
Conversation
API Change ReportNo changes found! |
|
@matthewdale , @qingyang-hu This is a high priority ticket. It could be helpful to start with the description and determine if we should accept / reject the design. |
There was a problem hiding this comment.
What are your thoughts about returning the remaining buf as well as the EOF for ReadByte(), peek() and discard() when there are fewer bytes than required. That's also how bufio acts.
matthewdale
left a comment
There was a problem hiding this comment.
The concept makes sense and the API for wrapping the data slice improves the logic. There are a few improvements and rough edges that need to be worked out.
There was a problem hiding this comment.
This appears to only be called with byte slices that are wrapped by a bytes.Reader. Can we make this accept the byte slice directly instead?
| func newBufferedDocumentReader(r io.Reader) *valueReader { | |
| buf, err := io.ReadAll(r) | |
| if err != nil { | |
| panic("bson: could not read document: " + err.Error()) | |
| } | |
| func newBufferedDocumentReader(b []byte) *valueReader { | |
| buf := b |
There was a problem hiding this comment.
We need a way to return this error, not panic. We should defer reading the data from the reader until someone calls one of the Read* methods.
27e063b to
620eb5a
Compare
| if !errors.Is(err, io.EOF) { | ||
| t.Errorf("Expected io.ErrUnexpectedEOF with document length too small. got %v; want %v", err, io.EOF) |
There was a problem hiding this comment.
Read* methods should return io.ErrUnexpectedEOF if the data ends before the requested value can be read. From the io.EOF docs:
... Functions should return EOF only to signal a graceful end of input. If the EOF occurs unexpectedly in a structured data stream, the appropriate error is either [ErrUnexpectedEOF] or some other error giving more detail.
There was a problem hiding this comment.
In the buffered path, EOF is returned from peek, not from attempting a read. Additionally, using bufio's discard would just return io.EOF if a user tries to discard more data than is on the buffer. Since there is no precedence for buffered peek, we just adopted the discard solution from bufio. This is also inline with the v1 implementation.
func TestDiscard_EOF(t *testing.T) {
testData := bytes.Repeat([]byte("a"), 1024)
reader := bufio.NewReader(bytes.NewReader(testData))
n, err := reader.Discard(10000)
assert.Error(t, err)
assert.Equal(t, err, io.EOF)
assert.NotEqual(t, err, io.ErrUnexpectedEOF)
assert.Equal(t, 1024-100, n, "Expected to discard all bytes before EOF")
}TL;DR we don't read anything in the buffered path, so it's unclear why we would use io. ErrUnexpectedEOF.
There was a problem hiding this comment.
Sorry, I was talking about the structured Read* methods, like ReadDocument or ReadBinary. I see now that many of the existing tests expect io.EOF from those methods (example), but it's inconsistent between the different methods. We should either preserve the existing error behavior or standardize on returning io.UnexpectedEOF. The unstructured stream readers (streamingValueReader and bufferedValueReader) should return io.EOF.
There was a problem hiding this comment.
@matthewdale I've created GODRIVER-3613 to investigate this further, per our conversation.
Co-authored-by: Matt Dale <9760375+matthewdale@users.noreply.github.com>
matthewdale
left a comment
There was a problem hiding this comment.
Some optional suggestions, but over all looks good! 👍
| { | ||
| "Array/invalid length", | ||
| TypeArray, | ||
| []byte{0x01, 0x02, 0x03}, | ||
| io.EOF, 0, 0, | ||
| }, |
There was a problem hiding this comment.
Optional: Keyed struct literals are way easier to read than unkeyed because you don't need to remember the struct format. I recommend adding the field names for these test cases.
| t.Errorf("Offset not set at correct position; got %d; want %d", vr.src.pos(), tc.offset) | ||
| } | ||
| }) | ||
| t.Run("ReadBytes", func(t *testing.T) { |
There was a problem hiding this comment.
Optional: Lots of levels of subtests makes the code difficult to read because of the extra indentation and very long functions. Consider making these subtests separate test functions instead.
| }) | ||
|
|
||
| // This test is too complicated to try to abstract using subtests. | ||
| t.Run("ReadBytes & Skip (streaming)", func(t *testing.T) { |
There was a problem hiding this comment.
Optional: Consider making this a separate test function to make the test code easier to read and run.
| (*valueReader).ReadString, | ||
| "", | ||
| io.ErrUnexpectedEOF, | ||
| io.EOF, | ||
| TypeString, |
There was a problem hiding this comment.
Is this the only case where the error differs between the streaming and buffered solutions?
Optional: Consider updating the logic to always return io.EOF. That should allow removing streamingErr and bufferedError from the test cases.
There was a problem hiding this comment.
Suggest keeping these separate to avoid changing errors in production code.
| // bufferedValueReader implements the low-level byteSrc interface by reading | ||
| // directly from an in-memory byte slice. It provides efficient, zero-copy | ||
| // access for parsing BSON when the entire document is buffered in memory. | ||
| type bufferedValueReader struct { |
There was a problem hiding this comment.
Optional: This isn't a ValueReader, so the name is a bit confusing. Consider a name like bufferedByteSrc.
| // | ||
| // Note: this approach trades memory usage for extra buffering and reader calls, | ||
| // so it is less performanted than the in-memory bufferedValueReader. | ||
| type streamingValueReader struct { |
There was a problem hiding this comment.
Optional: This isn't a ValueReader, so the name is a bit confusing. Consider a name like streamingByteSrc.
| "sync" | ||
| ) | ||
|
|
||
| type valueReaderByteSrc interface { |
There was a problem hiding this comment.
Optional: Consider a more concise name like byteSrc.
| vr.src = &bufferedValueReader{} | ||
| vr.src.(*bufferedValueReader).buf = b | ||
| vr.src.(*bufferedValueReader).offset = 0 |
There was a problem hiding this comment.
Optional: Suggested simplification.
| vr.src = &bufferedValueReader{} | |
| vr.src.(*bufferedValueReader).buf = b | |
| vr.src.(*bufferedValueReader).offset = 0 | |
| vr.src = &bufferedValueReader{ | |
| buf: b, | |
| offset: 0, | |
| } |
6c659fe
| type subtest struct { | ||
| name string | ||
| src valueReaderByteSrc | ||
| func TestValueReader_ReadBinary(t *testing.T) { |
There was a problem hiding this comment.
@qingyang-hu , @matthewdale There were no logical changes to the tests. This is the update requested on the PR to avoid nesting subtests and keying test case parameters.
There was a problem hiding this comment.
Yeah, I didn't consider how much my suggestion would balloon the diff size. I think the refactored tests are way easier to read, but it made the PR difficult to review.
* master: (65 commits)
Replace all uses of 'interface{}' with 'any' in the bson package. (#2138)
GODRIVER-3473 Short-cicruit cursor.next() on invalid timeouts (#2135)
GODRIVER-3622 Automatically retry some test tasks. (#2147)
Replace all uses of 'interface{}' with 'any' in the repo docs. (#2142)
GODRIVER-3102: Perf comparison (#2134)
GODRIVER-3587 Use raw bytes in valueReader (#2120)
Replace all uses of 'interface{}' with 'any' in the internal/ packages. (#2140)
Replace all uses of 'interface{}' with 'any' in the x/ packages. (#2137)
GODRIVER-2016 Unskip all Transactions unified spec tests. (#2132)
GODRIVER-2540 Run govulncheck in CI builds. (#2136)
GODRIVER-3549 Update Client BulkWrite prose tests. (#2131)
Add guidelines for contributing features to the Go Driver (#2125)
Bump alcaeus/automatic-merge-up-action from 1.0.0 to 1.0.1 in the actions group (#2126)
Add wrappedMsgOnly to mongo.MarshalError and mongo.MongocryptError.
Bump testdata/specifications from `db69351` to `6689929`
fix wiremessage oob in case of intmin (#2076)
GODRIVER-3399: PoolClearedError should have TransientTransactionError label appended to it (#2114)
PR feedback.
Prevent integration tests from running when testing with -short
Skip AWS Test if no URI (#2102)
...
This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver) | `v1.17.7` → `v2.4.2` |  |  | --- ### Release Notes <details> <summary>mongodb/mongo-go-driver (go.mongodb.org/mongo-driver)</summary> ### [`v2.4.2`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.4.2): MongoDB Go Driver 2.4.2 [Compare Source](mongodb/mongo-go-driver@v2.4.1...v2.4.2) The MongoDB Go Driver Team is pleased to release version 2.4.2 of the official MongoDB Go Driver. #### Release Highlights This release fixes buffer handling in GSSAPI error description and username functions. <!-- Release notes generated using configuration in .github/release.yml at v2.4.2 --> #### What's Changed ##### 🐛 Fixed - GODRIVER-3770 Fix buffer handling in GSSAPI error description and username functions by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2297](mongodb/mongo-go-driver#2297) **Full Changelog**: <mongodb/mongo-go-driver@v2.4.1...v2.4.2> For a full list of tickets included in this release, please see the [list of fixed issues](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20fixVersion%3D2.4.2). Documentation for the Go Driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo?tab=doc) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson?tab=doc). For issues with, questions about, or feedback for the Go Driver, please look into our [support channels](https://www.mongodb.com/docs/manual/support/), including [StackOverflow](https://stackoverflow.com/questions/tagged/mongodb%20go?sort=Newest). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the Go Driver is greatly appreciated! ### [`v2.4.1`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.4.1): MongoDB Go Driver 2.4.1 [Compare Source](mongodb/mongo-go-driver@v2.4.0...v2.4.1) The MongoDB Go Driver Team is pleased to release version 2.4.1 of the official MongoDB Go Driver. <!-- Release notes generated using configuration in .github/release.yml at v2.4.1 --> #### What's Changed ##### 🐛 Fixed - GODRIVER-3704 Fix search index failure on empty "Options". by [@​qingyang-hu](https://github.com/qingyang-hu) in [#​2247](mongodb/mongo-go-driver#2247) ##### 📝 Other Changes - GODRIVER-3696: Add CI/CD label to label checker by [@​alcaeus](https://github.com/alcaeus) in [#​2243](mongodb/mongo-go-driver#2243) **Full Changelog**: <mongodb/mongo-go-driver@v2.4.0...v2.4.1> For a full list of tickets included in this release, please see the [list of fixed issues](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20fixVersion%3D2.4.1). Documentation for the Go Driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo?tab=doc) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson?tab=doc). For issues with, questions about, or feedback for the Go Driver, please look into our [support channels](https://www.mongodb.com/docs/manual/support/), including [StackOverflow](https://stackoverflow.com/questions/tagged/mongodb%20go?sort=Newest). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the Go Driver is greatly appreciated! ### [`v2.4.0`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.4.0): MongoDB Go Driver 2.4.0 [Compare Source](mongodb/mongo-go-driver@v2.3.1...v2.4.0) The MongoDB Go Driver Team is pleased to release version 2.4.0 of the official MongoDB Go Driver. #### Release Highlights > \[!IMPORTANT] > Go Driver v2.4 requires MongoDB 4.2 or newer. This release adds a new method [Client.AppendDriverInfo](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2@​v2.4.0/mongo#Client.AppendDriverInfo) that adds information to the driver metadata sent on subsequent connection handshakes. <!-- Release notes generated using configuration in .github/release.yml at v2.4.0 --> #### What's Changed ##### ✨ New Features - GODRIVER-3544, GODRIVER-3653 Allow Client to Send Client Metadata On-Demand by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2197](mongodb/mongo-go-driver#2197) ##### 📝 Other Changes - GODRIVER-3523 Drop support for MongoDB 4.0. by [@​qingyang-hu](https://github.com/qingyang-hu) in [#​2221](mongodb/mongo-go-driver#2221) - GODRIVER-3288 Stop gossiping $clusterTime on SDAM commands. by [@​qingyang-hu](https://github.com/qingyang-hu) in [#​2150](mongodb/mongo-go-driver#2150) - GODRIVER-3632: Reroute to use perfcomp from DET by [@​zhouselena](https://github.com/zhouselena) in [#​2163](mongodb/mongo-go-driver#2163) - Update README.md by [@​NiteshSingh17](https://github.com/NiteshSingh17) in [#​2176](mongodb/mongo-go-driver#2176) - Rewrite mongo.Connect documentation. by [@​matthewdale](https://github.com/matthewdale) in [#​2185](mongodb/mongo-go-driver#2185) - Allow ignore-for-release label to satisfy label checker \[v2] by [@​matthewdale](https://github.com/matthewdale) in [#​2209](mongodb/mongo-go-driver#2209) - Merge release/2.3 into master by [@​mongodb-drivers-pr-bot](https://github.com/mongodb-drivers-pr-bot)\[bot] in [#​2210](mongodb/mongo-go-driver#2210) - GODRIVER-3667 Update support links. by [@​matthewdale](https://github.com/matthewdale) in [#​2215](mongodb/mongo-go-driver#2215) - GODRIVER-3675 Rename `internal/decimal/decinal128.go` to `decimal128.go` by [@​ggyuchive](https://github.com/ggyuchive) in [#​2220](mongodb/mongo-go-driver#2220) #### New Contributors - [@​NiteshSingh17](https://github.com/NiteshSingh17) made their first contribution in [#​2176](mongodb/mongo-go-driver#2176) - [@​ggyuchive](https://github.com/ggyuchive) made their first contribution in [#​2220](mongodb/mongo-go-driver#2220) **Full Changelog**: <mongodb/mongo-go-driver@v2.3.1...v2.4.0> For a full list of tickets included in this release, please see the [list of fixed issues](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20fixVersion%3D2.4.0). Documentation for the Go Driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo?tab=doc) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson?tab=doc). For issues with, questions about, or feedback for the Go Driver, please look into our [support channels](https://www.mongodb.com/docs/manual/support/), including [StackOverflow](https://stackoverflow.com/questions/tagged/mongodb%20go?sort=Newest). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the Go Driver is greatly appreciated! ### [`v2.3.1`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.3.1): MongoDB Go Driver 2.3.1 [Compare Source](mongodb/mongo-go-driver@v2.3.0...v2.3.1) The MongoDB Go Driver Team is pleased to release version 2.3.1 of the official MongoDB Go Driver. #### Release Highlights This release applies client-level timeouts for tailable/awaitData cursors, and fixes a bug that causes a tight loop when there are no selectable servers. <!-- Release notes generated using configuration in .github/release.yml at v2.3.1 --> #### What's Changed ##### 🐛 Fixed - GODRIVER-3616 Apply client-level timeout to tailable cursors by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2174](mongodb/mongo-go-driver#2174) - GODRIVER-3648 Make server selection block on topology updates. by [@​matthewdale](https://github.com/matthewdale) in [#​2211](mongodb/mongo-go-driver#2211) ##### 📝 Other Changes - Merge release/1.17 into release/2.2 by [@​mongodb-drivers-pr-bot](https://github.com/mongodb-drivers-pr-bot)\[bot] in [#​2193](mongodb/mongo-go-driver#2193) - Merge release/2.2 into release/2.3 by [@​mongodb-drivers-pr-bot](https://github.com/mongodb-drivers-pr-bot)\[bot] in [#​2204](mongodb/mongo-go-driver#2204) **Full Changelog**: <mongodb/mongo-go-driver@v2.3.0...v2.3.1> For a full list of tickets included in this release, please see the [list of fixed issues](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20fixVersion%3D2.3.1). Documentation for the Go Driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo?tab=doc) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson?tab=doc). For issues with, questions about, or feedback for the Go Driver, please look into our [support channels](https://www.mongodb.com/docs/manual/support/), including [StackOverflow](https://stackoverflow.com/questions/tagged/mongodb%20go?sort=Newest). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the Go Driver is greatly appreciated! ### [`v2.3.0`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.3.0): MongoDB Go Driver 2.3.0 [Compare Source](mongodb/mongo-go-driver@v2.2.3...v2.3.0) The MongoDB Go Driver Team is pleased to release version 2.3.0 of the official MongoDB Go Driver. #### Release Highlights > \[!IMPORTANT]\ > Go Driver v2.3 will be the last release to support MongoDB 4.0. Go Driver v2.4 will require MongoDB 4.2 or newer. > \[!IMPORTANT]\ > The minimum Go version for Go Driver v2.3 is Go 1.19. This release improves BSON unmarshal performance to fix the regression in v2.0 and fixes bugs in logging truncation and cursor timeouts. <!-- Release notes generated using configuration in .github/release.yml at v2.3.0 --> #### What's Changed ##### 🐛 Fixed - GODRIVER-3473 Short-cicruit cursor.next() on invalid timeouts by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2135](mongodb/mongo-go-driver#2135) - GODRIVER-3605 Refactor StringN by [@​qingyang-hu](https://github.com/qingyang-hu) in [#​2128](mongodb/mongo-go-driver#2128) - fix wiremessage oob in case of intmin by [@​kobrineli](https://github.com/kobrineli) in [#​2076](mongodb/mongo-go-driver#2076) ##### 📦 Dependency Updates - GODRIVER-3515 Bump the minimum Go Version to 1.19 by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2054](mongodb/mongo-go-driver#2054) ##### 📝 Other Changes - GODRIVER-3587 Use raw bytes in valueReader by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2120](mongodb/mongo-go-driver#2120) - optimize allocations in redactStartedInformationCmd by [@​isopov](https://github.com/isopov) in [#​2129](mongodb/mongo-go-driver#2129) - GODRIVER-3102: Perf comparison by [@​zhouselena](https://github.com/zhouselena) in [#​2134](mongodb/mongo-go-driver#2134) - GODRIVER-3102: Perf comp PR comment pipeline by [@​zhouselena](https://github.com/zhouselena) in [#​2149](mongodb/mongo-go-driver#2149) - Add CODEOWNERS file by [@​alcaeus](https://github.com/alcaeus) in [#​2018](mongodb/mongo-go-driver#2018) - GODRIVER-3444 Adjust getMore maxTimeMS Calculation for tailable awaitData Cursors by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​1925](mongodb/mongo-go-driver#1925) - DEVPROD-17319 - update perf.send command to new results end point by [@​MAhmadShah](https://github.com/MAhmadShah) in [#​2029](mongodb/mongo-go-driver#2029) - GODRIVER-3550 Update Documentation for Go Driver Branching and Merge … by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2045](mongodb/mongo-go-driver#2045) - GODRIVER-3361 Improve connection error message. by [@​qingyang-hu](https://github.com/qingyang-hu) in [#​2027](mongodb/mongo-go-driver#2027) - Add configuration for automated release notes by [@​alcaeus](https://github.com/alcaeus) in [#​2047](mongodb/mongo-go-driver#2047) - Update README.md by [@​rishitb-mongodb](https://github.com/rishitb-mongodb) in [#​2055](mongodb/mongo-go-driver#2055) - GODRIVER-3494 Deprecate hedged read preference methods. by [@​matthewdale](https://github.com/matthewdale) in [#​2100](mongodb/mongo-go-driver#2100) - GODRIVER-3457: Pilot using OpenSSF Scorecard by [@​zhouselena](https://github.com/zhouselena) in [#​2104](mongodb/mongo-go-driver#2104) - GODRIVER-3457: Add OpenSSF Scorecard to README by [@​zhouselena](https://github.com/zhouselena) in [#​2105](mongodb/mongo-go-driver#2105) - GODRIVER-3518: Test flexible numeric comparisons with $$lte by [@​zhouselena](https://github.com/zhouselena) in [#​2106](mongodb/mongo-go-driver#2106) - drivertest: name an anonymous error by [@​mmcclimon](https://github.com/mmcclimon) in [#​2115](mongodb/mongo-go-driver#2115) - GODRIVER-3397 Remove the MONGODB-CR auth mechanism. by [@​matthewdale](https://github.com/matthewdale) in [#​2103](mongodb/mongo-go-driver#2103) - GODRIVER-3399: PoolClearedError should have TransientTransactionError label appended to it by [@​zhouselena](https://github.com/zhouselena) in [#​2114](mongodb/mongo-go-driver#2114) - Add guidelines for contributing features to the Go Driver by [@​alcaeus](https://github.com/alcaeus) in [#​2125](mongodb/mongo-go-driver#2125) #### New Contributors - [@​MAhmadShah](https://github.com/MAhmadShah) made their first contribution in [#​2029](mongodb/mongo-go-driver#2029) - [@​rishitb-mongodb](https://github.com/rishitb-mongodb) made their first contribution in [#​2055](mongodb/mongo-go-driver#2055) - [@​mmcclimon](https://github.com/mmcclimon) made their first contribution in [#​2115](mongodb/mongo-go-driver#2115) **Full Changelog**: <mongodb/mongo-go-driver@v2.2.3...v2.3.0> For a full list of tickets included in this release, please see the [list of fixed issues](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20fixVersion%3D2.3.0). Documentation for the Go Driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo?tab=doc) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson?tab=doc). Questions and inquiries can be asked on the [MongoDB Developer Community](https://www.mongodb.com/community/forums/tag/golang). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the Go Driver is greatly appreciated! ### [`v2.2.3`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.2.3): MongoDB Go Driver 2.2.3 [Compare Source](mongodb/mongo-go-driver@v2.2.2...v2.2.3) The MongoDB Go Driver Team is pleased to release version 2.2.3 of the official MongoDB Go Driver. #### Release Notes This release updates the `DefaultClient` to use a non-default global `DefaultTransport`. This way, the program will not panic if the user changes the global `http.DefaultTransport` variable. The client is created as needed. *** For a full list of tickets included in this release, please see the [list of fixed issues](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20fixVersion%3D2.2.3). **Full Changelog**: [v2.2.2...v2.2.3](mongodb/mongo-go-driver@v2.2.2...v2.2.3) Documentation for the Go Driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo?tab=doc) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson?tab=doc). Questions and inquiries can be asked on the [MongoDB Developer Community](https://www.mongodb.com/community/forums/tag/golang). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the Go Driver is greatly appreciated! ### [`v2.2.2`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.2.2): MongoDB Go Driver 2.2.2 [Compare Source](mongodb/mongo-go-driver@v2.2.1...v2.2.2) The MongoDB Go Driver Team is pleased to release version 2.2.2 of the official MongoDB Go Driver. #### Release Notes This release resolves three bugs in the Go Driver: it removes a buggy and unnecessary connection liveness check that could run unexpectedly or fail intermittently when maxIdleTimeMS was set, and it fixes an issue in decoding to overwrite prepopulated slice. Also, a unmarshaler for gridfs.File is added. *** For a full list of tickets included in this release, please see the [list of fixed issues](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20fixVersion%3D2.2.2). **Full Changelog**: [v2.2.1...v2.2.2](mongodb/mongo-go-driver@v2.2.1...v2.2.2) Documentation for the Go Driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo?tab=doc) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson?tab=doc). Questions and inquiries can be asked on the [MongoDB Developer Community](https://www.mongodb.com/community/forums/tag/golang). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the Go Driver is greatly appreciated! #### What's Changed - Ignore unmaintained branches when merging up by [@​alcaeus](https://github.com/alcaeus) in [#​2062](mongodb/mongo-go-driver#2062) - GODRIVER-3549 Fix timeouts in CSE custom endpoint test ([#​2028](mongodb/mongo-go-driver#2028)) ([#​2031](mongodb/mongo-go-driver#2031)) by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2061](mongodb/mongo-go-driver#2061) - GODRIVER-3516 Remove isAlive by [@​linfeip](https://github.com/linfeip) in [#​2060](mongodb/mongo-go-driver#2060) - GODRIVER-3560 Assume ec2 role explicitly in CI by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2080](mongodb/mongo-go-driver#2080) - GODRIVER-3524 Sync updates to reflect showExpandedEvents omissions by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2084](mongodb/mongo-go-driver#2084) - GODRIVER-3565 Add UnmarshalBSON to GridFSFile by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2077](mongodb/mongo-go-driver#2077) - GODRIVER-3574 Align BSON interface slice decoding with json package. by [@​qingyang-hu](https://github.com/qingyang-hu) in [#​2075](mongodb/mongo-go-driver#2075) #### New Contributors - [@​linfeip](https://github.com/linfeip) made their first contribution in [#​2060](mongodb/mongo-go-driver#2060) **Full Changelog**: <mongodb/mongo-go-driver@v2.2.1...v2.2.2> ### [`v2.2.1`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.2.1): MongoDB Go Driver 2.2.1 [Compare Source](mongodb/mongo-go-driver@v2.2.0...v2.2.1) The MongoDB Go Driver Team is pleased to release version 2.2.1 of the official MongoDB Go Driver. #### Release Notes This release enhances BSON encoding performance by using `sync.Pool` for value writers and readers during encoding and decoding, leading to better memory allocation and improved efficiency. Note that further regressions from version 1 may be addressed in GODRIVER-3450, with this update focusing on reinstating optimizations removed in the version 2 implementation. Performance may not be 1-1 with v1 since v2 suggests using the `bufio` package for byte management rather than a raw byte slice. #### What's Changed - GODRIVER-3546 Fix timeouts in CSE custom endpoint test ([#​2028](mongodb/mongo-go-driver#2028)) by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2031](mongodb/mongo-go-driver#2031) - GODRIVER-3533 Optimize value reader and writer ([#​2022](mongodb/mongo-go-driver#2022)) by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2030](mongodb/mongo-go-driver#2030) - DEVPROD-17319 Update perf.send command to new results end point by [@​prestonvasquez](https://github.com/prestonvasquez) in [#​2032](mongodb/mongo-go-driver#2032) **Full Changelog**: <mongodb/mongo-go-driver@v2.2.0...v2.2.1> **JIRA Release Notes**: <https://jira.mongodb.org/projects/GODRIVER/versions/43096> Documentation for the Go Driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo?tab=doc) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson?tab=doc). Questions and inquiries can be asked on the [MongoDB Developer Community](https://www.mongodb.com/community/forums/tag/golang). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the Go Driver is greatly appreciated! ### [`v2.2.0`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.2.0): MongoDB Go Driver 2.2.0 [Compare Source](mongodb/mongo-go-driver@v2.1.0...v2.2.0) The MongoDB Go Driver team is pleased to release version 2.2.0 of the official MongoDB Go Driver. #### Release Notes This release includes support for a global `omitempty` setting to omit empty values when encoding BSON, instead of requiring tagging on individual struct fields. We have implemented support for [`errors.Is`](https://pkg.go.dev/errors#Is) and [`errors.As`](https://pkg.go.dev/errors#As) in all stable public APIs for improved error management. Also included is support to configure the lifetime of the [Data Encryption Key (DEK)](https://www.mongodb.com/docs/manual/core/csfle/fundamentals/manage-keys/) cache as well as [Kubernetes Support for OIDC](https://www.mongodb.com/docs/drivers/go/current/fundamentals/enterprise-auth/#kubernetes). Additionally **MongoDB Server Version 3.6** has been marked as end-of-life (EOL) and is no longer supported by the driver. Users are advised to upgrade to a supported version. *** For a full list of tickets included in this release, please see the links below: - [New Features](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20type%3D%22new%20feature%22%20and%20status%20%3D%20Closed%20and%20fixVersion%3D2.2.0) - [Improvements](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20type%3Dimprovement%20and%20status%20%3D%20Closed%20and%20fixVersion%3D2.2.0) - [Bugs](https://jira.mongodb.org/issues/?jql=project%3DGODRIVER%20and%20type%3DBug%20and%20status%3Dclosed%20and%20fixVersion%3D2.2.0) - [Tasks](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20type%3D%22task%22%20and%20status%20%3D%20Closed%20and%20fixVersion%3D2.2.0) **Full Changelog**: [v2.1.0...v2.2.0](mongodb/mongo-go-driver@v2.1.0...v2.2.0) **Note**: The Go Driver team has implemented a workflow action to automatically merge up PRs between v1 and master. This has resulted in an over-extended changelog for this version, including v1 commits for PRs that have already been cherry-picked to master. Please see the associated tickets for commits directly associated with this release. Documentation for the MongoDB Go Driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson). Questions and inquiries can be asked on the [MongoDB Developer Community](https://www.mongodb.com/community/forums/tag/golang). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the MongoDB Go Driver is greatly appreciated! ### [`v2.1.0`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.1.0): MongoDB Go Driver 2.1.0 [Compare Source](mongodb/mongo-go-driver@v2.0.1...v2.1.0) The MongoDB Go Driver Team is pleased to release version 2.1.0 of the official MongoDB Go Driver. #### Release Notes This release adds support for the new bulk write API added in [MongoDB 8.0](https://www.mongodb.com/docs/manual/release-notes/8.0/#new-bulk-write-command) and the new BSON vector datatype used in [Atlas Vector Search](https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-overview/). **Additionally, support for MongoDB 3.6 and below was deprecated in v2.0.0 and will be dropped in the next minor version release (v2.2.0).** ##### New BulkWrite API The new [Client.BulkWrite](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo#Client.BulkWrite) method can perform many insert, update, and delete operations on multiple databases and collections in one request. In contrast, the existing [Collection.BulkWrite](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo#Collection.BulkWrite) method can only modify a single collection. ##### BSON Vector Datatype The new [bson.Vector](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson#Vector) type makes inserting and querying vector data using [Atlas Vector Search](https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-overview/) easier and more efficient. *** For a full list of tickets included in this release, please see the links below: - [New Features](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20type%3D%22new%20feature%22%20and%20status%20%3D%20Closed%20and%20fixVersion%3D2.1.0) - [Improvements](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20type%3Dimprovement%20and%20status%20%3D%20Closed%20and%20fixVersion%3D2.1.0) - [Bugs](https://jira.mongodb.org/issues/?jql=project%3DGODRIVER%20and%20type%3DBug%20and%20status%3Dclosed%20and%20fixVersion%3D2.1.0) - [Tasks](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20type%3D%22task%22%20and%20status%20%3D%20Closed%20and%20fixVersion%3D2.1.0) **Full Changelog**: [v2.0.1...v2.1.0](mongodb/mongo-go-driver@v2.0.1...v2.1.0) Documentation for the MongoDB Go Driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson). Questions and inquiries can be asked on the [MongoDB Developer Community](https://www.mongodb.com/community/forums/tag/golang). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the MongoDB Go Driver is greatly appreciated! ### [`v2.0.1`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.0.1): MongoDB Go Driver 2.0.1 [Compare Source](mongodb/mongo-go-driver@v2.0.0...v2.0.1) The MongoDB Go Driver team is pleased to release version 2.0.1 of the official MongoDB Go Driver. #### Release Notes This release includes various bug fixes and improvements: - Allows SRV hostnames with only a domain name and TLD (e.g. "service-name.tld"). - Detects joined errors correctly in [IsNetworkError](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo#IsNetworkError) and [WithTransaction](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo#Session.WithTransaction). - Fixes a memory buffer reuse bug in [MarshalValue](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson#MarshalValue) that can cause marshaled bytes to become corrupted. - No longer returns nil from [MergeClientOptions](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo/options#MergeClientOptions) when only one nil argument is passed. - Skips calling [UnmarshalBSONValue](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson#ValueUnmarshaler.UnmarshalBSONValue) only if the associated BSON field value is null and the Go value is a pointer. *** For a full list of tickets included in this release, please see the links below: - [Bugs](https://jira.mongodb.org/issues/?jql=project%3DGODRIVER%20and%20type%3DBug%20and%20status%3Dclosed%20and%20fixVersion%3D2.0.1) - [Tasks](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20type%3D%22task%22%20and%20status%20%3D%20Closed%20and%20fixVersion%3D2.0.1) **Full Changelog**: [v2.0.0...v2.0.1](mongodb/mongo-go-driver@v2.0.0...v2.0.1) Documentation for the MongoDB Go Driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/mongo) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/v2/bson). Questions and inquiries can be asked on the [MongoDB Developer Community](https://www.mongodb.com/community/forums/tag/golang). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the MongoDB Go Driver is greatly appreciated! ### [`v2.0.0`](https://github.com/mongodb/mongo-go-driver/releases/tag/v2.0.0): MongoDB Go Driver 2.0.0 [Compare Source](mongodb/mongo-go-driver@v1.17.7...v2.0.0) The MongoDB Go Driver Team is pleased to release version 2.0.0 of the official MongoDB Go driver. #### Release Notes This release includes a host of new features and several breaking changes. Below, we provide an overview of the key highlights, changes, and guidance for migrating from version 1.x to 2.0. See the [MongoDB docs](https://www.mongodb.com/docs/drivers/go/upcoming/whats-new/#what-s-new-in-2.0) for a complete list of breaking changes and additional information on what's new. This release includes several breaking changes that users must account for when upgrading, including: - **Options Package**: The approach to managing options has been restructured to use a builder pattern, changing how options are constructed and applied. This alteration allows for more flexible and dynamic option configurations, but may require changes to existing code that directly manipulates options objects. - **Mongo Package**: The `Client.Connect()` method has been removed, favoring `mongo.Connect()` to streamline connection handling. - **Event Package**: References to the `description` package have been transitioned to `event.ServerDescription` and `event.TopologyDescription`, impacting how server and topology details are accessed. - **BSON Package Consolidation**: All previously separate BSON-related packages, including `bsoncodec`, `bsonoptions`, `bsonrw`, and `mgocompat`, have been merged into the single `bson` package. Many functionalities from these packages have been reorganized or renamed. For instance, `bson.NewRegistryBuilder` has been replaced with new APIs tailored for registry configuration, and interfaces like `ValueReader` and `ValueWriter` are now part of the consolidated `bson` package. This consolidation simplifies the package structure but requires updates to import paths and related logic. To ease the transition to 2.0, **a comprehensive [migration guide](https://github.com/mongodb/mongo-go-driver/blob/master/docs/migration-2.0.md) has been provided**. This guide includes detailed instructions to navigate the changes, including the updated options pattern, altered imports, and revised method signatures. Additionally, this release deprecates support for MongoDB server versions below 3.6. Support for MongoDB 3.6 will be dropped in a future update. New features will no longer be added to the 1.x versions. However, critical bug fixes and CVE resolutions will continue to be back-ported to 1.x versions for one year until January 15, 2026. After that date, support for 1.x versions will be discontinued. We express our gratitude to our contributors and the community for all the support in developing this release. *** For a full list of tickets included in this release, please see the links below: - [Projects](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20type%3Depic%20and%20status%20%3D%20Closed%20and%20fixVersion%3D2.0.0) - [New Features](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20type%3D%22new%20feature%22%20and%20status%20%3D%20Closed%20and%20fixVersion%3D2.0.0) - [Improvements](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20type%3Dimprovement%20and%20status%20%3D%20Closed%20and%20fixVersion%3D2.0.0) - [Bugs](https://jira.mongodb.org/issues/?jql=project%3DGODRIVER%20and%20type%3DBug%20and%20status%3Dclosed%20and%20fixVersion%3D2.0.0) - [Tasks](https://jira.mongodb.org/issues/?jql=project%3Dgodriver%20and%20type%3D%22task%22%20and%20status%20%3D%20Closed%20and%20fixVersion%3D2.0.0) **Full Changelog**: [v1.17.1...v2.0.0](mongodb/mongo-go-driver@v1.17.1...v2.0.0) Documentation for the Go driver can be found on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/mongo?tab=doc) and the [MongoDB documentation site](https://docs.mongodb.com/ecosystem/drivers/go/). BSON library documentation is also available on [pkg.go.dev](https://pkg.go.dev/go.mongodb.org/mongo-driver/bson?tab=doc). Questions and inquiries can be asked on the [MongoDB Developer Community](https://community.mongodb.com/). Bugs can be reported in the [Go Driver project in the MongoDB JIRA](https://jira.mongodb.org/secure/CreateIssue!default.jspa?pid=14289) where a list of [current issues](https://jira.mongodb.org/browse/GODRIVER) can be found. Your feedback on the Go driver is greatly appreciated! </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0Mi45Mi4yIiwidXBkYXRlZEluVmVyIjoiNDIuOTIuMiIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIl19--> Co-authored-by: Renovate Bot <renovate@yourdomain.com> Co-authored-by: riverajo <2325467+riverajo@users.noreply.github.com> Reviewed-on: https://git.rivera.ninja/joe/fitness-app/pulls/29 Co-authored-by: renovate-bot <rivera.d.joseph@gmaill.com> Co-committed-by: renovate-bot <rivera.d.joseph@gmaill.com>
GODRIVER-3587
Summary
In v2 we provided users with the ability to stream data into the
ValueReader(see the temporarily removed "stream" test for more information). Because of this we can't simply revert to the v1 solution, i.e. using raw bytes instead of bufio. This PR suggests that we addbufferedValueReaderandstreamingValueReaderto optimize workflow and avoid backwards breaking changes, respectively. Both must implement the minimal set of functions required byvalueReader:And then
valueReaderwill be updated to maintain avalueReaderByteSrcimplementation object:NewDocumentReaderwill usestreamingValueReader, since that is the current expectation. AndnewDocumentReaderwill usebufferedValueReaderto optimize in the non-default case.If accepted,
streamingValueReaderwill be added in a subsequent PR.Benchmarking
Benchmark: https://gist.github.com/prestonvasquez/77eecc18699736b3d7d62a7a30c5769e
streamingValueReaderindicates the same regression noted in the analysis:bufferedValueReaderis still worse than V1 but largely improved:Background & Motivation
The v2 solution is less optimized because of the allocation requirements for streaming.