Skip to content

READY feat(bigtable) : add dynamic channel support#2

Merged
sushanb merged 23 commits intomainfrom
go2
Dec 4, 2025
Merged

READY feat(bigtable) : add dynamic channel support#2
sushanb merged 23 commits intomainfrom
go2

Conversation

@sushanb
Copy link
Copy Markdown
Owner

@sushanb sushanb commented Nov 3, 2025

No description provided.

Comment on lines +644 to +648
newConns := make([]*connEntry, numCurrent+n)
copy(newConns, currentConns)
copy(newConns[numCurrent:], newEntries[:n])
p.conns.Store(newConns)
btopt.Debugf(p.logger, "bigtable_connpool: Added %d connections, new size: %d\n", n, len(newConns))
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems to still have the same problem as we discussed ... you are exposing unprimed channels to the pool.

How about adding a wait group for all of the goroutines and waiting on all of the to finish?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

index int
}

// removeConnections returns true if the pool size changed.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please expand the comment explaining which connections will be targeted for removal

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expanded.


dynamicConfig btopt.DynamicChannelPoolConfig // Keep the config for options
// background monitors
monitors []Monitor
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The relationship between a pool and a monitor is very murky. It seems like a Monitor should know about a pool and control the pool. But for some reason the pool owns the references to the monitor? Why not have the client own struct own it and have a cleaner separation?

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--> Why not have the client own struct own it and have a cleaner separation?

yeah it is murky.

what do you prefer?

  1. client owning the reference to the monitor and controlling the pool.

  2. or passing the functions instead of the entire pool reference to the monitor? similar to the MetricsExporter

NewDynamicScaleMonitor(
pool.dynamicConfig,
pool.ConnectionLoadStatsSuppolier,
pool.addConnections,
pool.removeConnections,
)

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

made it part of client.

@sushanb sushanb changed the title feat(bigtable) : add dynamic channel support NOT_READY feat(bigtable) : add dynamic channel support Nov 12, 2025
@sushanb sushanb changed the base branch from go1 to main November 15, 2025 20:32
@sushanb sushanb force-pushed the go2 branch 2 times, most recently from 9cc5825 to 57713c3 Compare November 16, 2025 23:14
@sushanb sushanb changed the base branch from main to go1 November 16, 2025 23:16
@sushanb sushanb changed the base branch from go1 to main November 16, 2025 23:18
@sushanb sushanb changed the base branch from main to go1 November 16, 2025 23:19
@sushanb sushanb changed the base branch from go1 to main November 16, 2025 23:28
@sushanb sushanb changed the title NOT_READY feat(bigtable) : add dynamic channel support READY feat(bigtable) : add dynamic channel support Nov 17, 2025
@sushanb sushanb merged commit 8579aed into main Dec 4, 2025
sushanb added a commit that referenced this pull request Dec 21, 2025
* WIP

* WIO

* Fix

* Address feedback

* remove alts

* FIX

* WIO

* WIP

* WIP

* Make conn draining timeout 30mins

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Review comments

* WIP

* WIP

* go2 complete
sushanb added a commit that referenced this pull request Dec 21, 2025
* WIP

* WIO

* Fix

* Address feedback

* remove alts

* FIX

* WIO

* WIP

* WIP

* Make conn draining timeout 30mins

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* Review comments

* WIP

* WIP

* go2 complete
sushanb added a commit to googleapis/google-cloud-go that referenced this pull request Dec 22, 2025
sushanb pushed a commit that referenced this pull request Feb 6, 2026
…3782)

PR created by the Librarian CLI to initialize a release. Merging this PR
will auto trigger a release.

Librarian Version: v0.8.0
Language Image:
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/librarian-go@sha256:01189c9771ac4150742aed38eb52e19a008018889066002742034b7f82db070f
<details><summary>bigtable: 1.42.0</summary>

##
[1.42.0](googleapis/google-cloud-go@bigtable/v1.41.0...bigtable/v1.42.0)
(2026-02-04)

### Features

* add TieredStorageConfig to table admin api (PiperOrigin-RevId:
863493708)
([611f239](googleapis@611f2392))

* add dynamic channel support (#2) (googleapis#13504)
([65a3c4e](googleapis@65a3c4ec))

* add ip protocol for bigtable connection (googleapis#13520)
([6f86983](googleapis@6f86983e))

* add metrics exporter for outstandingrpcs and perconnectionerror count
(googleapis#13510)
([7f1b36a](googleapis@7f1b36ac))

* add client startup time metrics. (googleapis#13521)
([8f90da3](googleapis@8f90da38))

* introduce prime() method and load counting based on unary,streaming
and ability to remove connection (googleapis#13419)
([c6773ce](googleapis@c6773ceb))

### Bug Fixes

* handle client region and client host name based on ote… (googleapis#13752)
([0429170](googleapis@0429170f))

* fix a nil pointer deference in remove connections when… (googleapis#13535)
([9d2ba82](googleapis@9d2ba82e))

### Performance Improvements

* use passthrough with emulator endpoint (googleapis#13739)
([dafbca2](googleapis@dafbca22))

</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants