Skip to content

chore: loadShards changes to more cleanly support 2.x feature#25556

Merged
gwossum merged 1 commit into2.7from
gw/25555/gw_loadShards_cleanup_2.x
Nov 15, 2024
Merged

chore: loadShards changes to more cleanly support 2.x feature#25556
gwossum merged 1 commit into2.7from
gw/25555/gw_loadShards_cleanup_2.x

Conversation

@gwossum
Copy link
Copy Markdown
Member

@gwossum gwossum commented Nov 15, 2024

  • chore: loadShards changes to more cleanly support 2.x feature (chore: loadShards changes to more cleanly support 2.x feature #25513)

  • chore: move shardID parsing and shard filtering into walkShardsAndProcess

  • chore: make it impossible to miss sending shardResponse or marking shard as complete

  • chore: always count number of shards (preparation for 2.x related feature)

  • chore: explicitly load series files and create indices serially

Explicitly load series files and create indices serially. Also avoid passing them to work functions that don't need them.

  • chore: rework loadShards for changes necessary to cancel loading process

  • chore: comment improvements

  • fix: fix race conditions in TestStore_StartupShardProgress and TestStore_BadShardLoading

  • chore: avoid logging nil error

  • chore: refactor shard loading and shard walking

Refactor loadShards and CreateShard to use a common shardLoader class that makes thread-safety easier. Refactor walkShardsAndProcess into findShards.

  • chore: improve comment

  • chore: rename OpenShard to ReopenShard and implement with shardLoader

Rename Store.OpenShard to Store.ReopenShard and implement using a shardLoader object. Changes to tests as necessary.

  • chore: avoid resetting shard options and locking on Reopen

Avoid resetting shard options when reopening a shard. Proper mutex locker in Shard.ReopenShard.

  • chore: fix formatting issue

  • chore: warn on mixed index types in Store.CreateShard

  • chore: change from info to warn when invalid shard IDs found in path

  • chore: use coarser locking in Store.ReopenShard

  • chore: fix typo in comment

  • chore: code simplification

(cherry picked from commit 0bc167b)

  • chore: fix logging issues in Store.loadShards

Fix reporting shards not opening correctly when they actually did. Fix race condition with logging in loadShards.

(cherry picked from commit 65683bf)

  • chore: remove unnecessary fmt.Sprintf calls

Remove unnecessary fmt.Sprintf calls for static code checks in main-2.x.

(cherry picked from commit 8497fbf)

  • chore: remove unnecessary blank identifier

  • chore: remove unnecessary blank identifier

(cherry picked from commit 5c7479e)

Closes: #25555

Closes #

Describe your proposed changes here.

  • I've read the contributing section of the project README.
  • Signed CLA (if not already signed).

* chore: loadShards changes to more cleanly support 2.x feature (#25513)

* chore: move shardID parsing and shard filtering into walkShardsAndProcess

* chore: make it impossible to miss sending shardResponse or marking shard as complete

* chore: always count number of shards (preparation for 2.x related feature)

* chore: explicitly load series files and create indices serially

Explicitly load series files and create indices serially. Also
avoid passing them to work functions that don't need them.

* chore: rework loadShards for changes necessary to cancel loading process

* chore: comment improvements

* fix: fix race conditions in TestStore_StartupShardProgress and TestStore_BadShardLoading

* chore: avoid logging nil error

* chore: refactor shard loading and shard walking

Refactor loadShards and CreateShard to use a common shardLoader class that
makes thread-safety easier. Refactor walkShardsAndProcess into findShards.

* chore: improve comment

* chore: rename OpenShard to ReopenShard and implement with shardLoader

Rename Store.OpenShard to Store.ReopenShard and implement using a
shardLoader object. Changes to tests as necessary.

* chore: avoid resetting shard options and locking on Reopen

Avoid resetting shard options when reopening a shard.
Proper mutex locker in Shard.ReopenShard.

* chore: fix formatting issue

* chore: warn on mixed index types in Store.CreateShard

* chore: change from info to warn when invalid shard IDs found in path

* chore: use coarser locking in Store.ReopenShard

* chore: fix typo in comment

* chore: code simplification

(cherry picked from commit 0bc167b)

* chore: fix logging issues in Store.loadShards

Fix reporting shards not opening correctly when they actually did.
Fix race condition with logging in loadShards.

(cherry picked from commit 65683bf)

* chore: remove unnecessary fmt.Sprintf calls

Remove unnecessary fmt.Sprintf calls for static code checks in main-2.x.

(cherry picked from commit 8497fbf)

* chore: remove unnecessary blank identifier

* chore: remove unnecessary blank identifier

(cherry picked from commit 5c7479e)

Closes: #25555
@gwossum gwossum added area/tsm area/2.x OSS 2.0 related issues and PRs labels Nov 15, 2024
@gwossum gwossum self-assigned this Nov 15, 2024
@gwossum gwossum marked this pull request as ready for review November 15, 2024 21:33
Copy link
Copy Markdown
Contributor

@davidby-influx davidby-influx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - clean cherry-pick

@gwossum gwossum merged commit 941a41b into 2.7 Nov 15, 2024
@gwossum gwossum deleted the gw/25555/gw_loadShards_cleanup_2.x branch November 15, 2024 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/tsm area/2.x OSS 2.0 related issues and PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants