Split ranges along accounting and zone configuration prefix boundaries.#179
Merged
spencerkimball merged 7 commits intomasterfrom Nov 21, 2014
Merged
Conversation
Each store registers a gossip callback for accounting and zone config maps. On callback, the zone does a binary search using the rangesByKey sorted slice to find possible ranges which might be intersected by config map prefixes. On possible hits, the range is dissected along prefix boundary lines and if there's a necessary split, the range is split and the process retried. This allows new zones or accounting prefixes to be specified and ranges will automatically be split to fit.
storage/store.go
Outdated
Contributor
There was a problem hiding this comment.
Either the comment or the behavior seems wrong. It looks like this returns true if there is nothing to split.
Member
Author
There was a problem hiding this comment.
Changed this to be more understandable. This method no longer returns a bool.
added 2 commits
November 20, 2014 20:43
In the process found an error in the prefix config code exercisable when there are successive prefixes in a config map, such as "db1" and "db2". Added a unittest for that case. Also, discovered we were always re-gossiping config data on every write because we weren't paying attention to a "real" dirty bit-- we were always copying the struct in the ranged for loop and so never setting the dirty bit to false after gossiping.
Member
Author
|
PTAL...made some rather significant changes to fix some stuff up including config prefix code & unittests as well as mechanism for ranges to gossip configs. |
added 3 commits
November 20, 2014 21:45
This informs the callback callee whether or not the actual contents of the info were updated. This can save a lot of work in the event that the new info just has a shorter number of hops or an updated timestamp but the same contents--we expect this to be a common occurence.
storage/range.go
Outdated
Contributor
There was a problem hiding this comment.
Is this correct? Wouldn't this happen if a configuration group spanned multiple complete ranges?
Contributor
There was a problem hiding this comment.
Nevermind, this isn't what I thought it was.
Member
Author
There was a problem hiding this comment.
Configuration groups cannot span ranges. The illegality of splitting mid-config group is enforced in storage/engine/mvcc.go:803 at IsValidSplitKey(). I'm adding a comment here to that effect.
Contributor
|
LGTM |
gossip/group.go
Outdated
Member
There was a problem hiding this comment.
this comment makes it sound like contentsChanged=false if the key was previously not set. Below it looks like you do the opposite.
Member
Author
|
Addressed feedback. |
5bc7c5f to
3441b68
Compare
Member
|
LGTM |
spencerkimball
added a commit
that referenced
this pull request
Nov 21, 2014
…ig-changes Split ranges along accounting and zone configuration prefix boundaries.
tbg
referenced
this pull request
in tbg/cockroach
Aug 31, 2016
We have been seeing long startup times which disappear spontaneously. During a
restart of the beta cluster, the following goroutine was observed, which suggests
that we were spending a lot of time GCing replicas on startup.
engine ??:0 _Cfunc_DBIterNext(cockroachdb#324, cockroachdb#323, 0, 0, 0, 0, 0, 0, 0)
engine rocksdb.go:1135 (*rocksDBIterator).Next(cockroachdb#235)
storage replica_data_iter.go:104 (*ReplicaDataIterator).Next(cockroachdb#316)
storage store.go:1748 (*Store).destroyReplicaData(#109, cockroachdb#317, 0, 0)
storage store.go:841 (*Store).Start.func2(0x101b, cockroachdb#300, 0x36, 0x40, cockroachdb#301, 0x36, 0x40, cockroachdb#315, 0x3, 0x4, ...)
storage store.go:734 IterateRangeDescriptors.func1(cockroachdb#306, 0x40, 0x41, cockroachdb#307, 0x92, 0x92, cockroachdb#341, 0, 0x186c, 0x4000, ...)
engine mvcc.go:1593 MVCCIterate(cockroachdb#329, #68, #47, #81, cockroachdb#232, 0x9, 0x10, cockroachdb#233, 0xb, 0x10, ...)
storage store.go:738 IterateRangeDescriptors(cockroachdb#330, cockroachdb#196, #47, #81, cockroachdb#195, #179, #110)
storage store.go:867 (*Store).Start(#109, cockroachdb#330, cockroachdb#196, #179, #185, 0x1)
server node.go:405 (*Node).initStores(#78, cockroachdb#330, cockroachdb#196, #98, 0x1, 0x1, #179, 0, #55)
server node.go:330 (*Node).start(#78, cockroachdb#330, cockroachdb#196, #42, #129, #98, 0x1, 0x1, 0, 0, ...)
server server.go:431 (*Server).Start(#5, cockroachdb#330, cockroachdb#196, #95, 0x1)
cli start.go:368 runStart(#34, #178, 0, 0x9, 0, 0)
cobra command.go:599 (*Command).execute(#34, #177, 0x9, 0x9, #34, #177)
cobra command.go:689 (*Command).ExecuteC(#33, #70, cockroachdb#343, #72)
cobra command.go:648 (*Command).Execute(#33, #71, cockroachdb#343)
cli cli.go:96 Run(#64, 0xa, 0xa, 0, 0)
main main.go:37 main()
tbg
referenced
this pull request
in tbg/cockroach
Aug 31, 2016
We have been seeing long startup times which disappear spontaneously. During a
restart of the beta cluster, the following goroutine was observed, which suggests
that we were spending a lot of time GCing replicas on startup.
engine ??:0 _Cfunc_DBIterNext(cockroachdb#324, cockroachdb#323, 0, 0, 0, 0, 0, 0, 0)
engine rocksdb.go:1135 (*rocksDBIterator).Next(cockroachdb#235)
storage replica_data_iter.go:104 (*ReplicaDataIterator).Next(cockroachdb#316)
storage store.go:1748 (*Store).destroyReplicaData(#109, cockroachdb#317, 0, 0)
storage store.go:841 (*Store).Start.func2(0x101b, cockroachdb#300, 0x36, 0x40, cockroachdb#301, 0x36, 0x40, cockroachdb#315, 0x3, 0x4, ...)
storage store.go:734 IterateRangeDescriptors.func1(cockroachdb#306, 0x40, 0x41, cockroachdb#307, 0x92, 0x92, cockroachdb#341, 0, 0x186c, 0x4000, ...)
engine mvcc.go:1593 MVCCIterate(cockroachdb#329, #68, #47, #81, cockroachdb#232, 0x9, 0x10, cockroachdb#233, 0xb, 0x10, ...)
storage store.go:738 IterateRangeDescriptors(cockroachdb#330, cockroachdb#196, #47, #81, cockroachdb#195, #179, #110)
storage store.go:867 (*Store).Start(#109, cockroachdb#330, cockroachdb#196, #179, #185, 0x1)
server node.go:405 (*Node).initStores(#78, cockroachdb#330, cockroachdb#196, #98, 0x1, 0x1, #179, 0, #55)
server node.go:330 (*Node).start(#78, cockroachdb#330, cockroachdb#196, #42, #129, #98, 0x1, 0x1, 0, 0, ...)
server server.go:431 (*Server).Start(#5, cockroachdb#330, cockroachdb#196, #95, 0x1)
cli start.go:368 runStart(#34, #178, 0, 0x9, 0, 0)
cobra command.go:599 (*Command).execute(#34, #177, 0x9, 0x9, #34, #177)
cobra command.go:689 (*Command).ExecuteC(#33, #70, cockroachdb#343, #72)
cobra command.go:648 (*Command).Execute(#33, #71, cockroachdb#343)
cli cli.go:96 Run(#64, 0xa, 0xa, 0, 0)
main main.go:37 main()
koorosh
pushed a commit
to koorosh/cockroach
that referenced
this pull request
May 13, 2021
* CRDB-2271: Truncating long queries with CSS Applying a tried and true hack using `-webkit-line-clamp` to truncate queries in the statements table to a couple of lines followed by an ellipsis. To allow a longer preview, I am applying the same fix to the tooltip, but clamping the lines to 15 (there were around 30 characters per line and design asked for ~425 characters. It's close enough). * Adding truncation to Transactions table Since statements and transactions have enough variation to not be able to make this change at the common table level, I'm adding truncation to the transactions table styles to mirror the change in the statements page table. * Adding trucation to Statement col in Sessions Adding trucation to the Statements column on the Sessions page to be consistent with how statements appear on Statements and Tranactions pages. * Truncating long app names in Statements app filter If a very long name is displayed in the Statements application filter, the name may cause the dropdown to extend outside of the view or at least beyond the table and cause excessive horizontal scrolling. To correct this I have added the `nowrap` option to the `<Text>` component inside of the Dropdown trigger. The "nowrap" prop will constrain the the text to a single line but does not offer a constraint for the width. To constrain the width I have added a local style set to a max-width of `75ch` (75 characters wide). Just a note, that I have used this opportunity to bring in the `<Text>` from `ui-components` to replace the local one here, which was the inspiration for the common component. The width and display of the menu are controlled by the Dropdown component and I didn't want to alter the display of all dropdowns that use that component. So instead I extended the dropdown component to take an additional `itemsClassname` prop to apply a class name to the element rendered for the menu items. This way I can supply another local style to constrain the width of the menu (to `80ch`) and apply text truncation. * Replacing repetitive styles with Sass mixins DRYing up the code a little with two Sass mixins: text-overflow and line-clamp.
koorosh
pushed a commit
to koorosh/cockroach
that referenced
this pull request
May 18, 2021
* CRDB-2271: Truncating long queries with CSS Applying a tried and true hack using `-webkit-line-clamp` to truncate queries in the statements table to a couple of lines followed by an ellipsis. To allow a longer preview, I am applying the same fix to the tooltip, but clamping the lines to 15 (there were around 30 characters per line and design asked for ~425 characters. It's close enough). * Adding truncation to Transactions table Since statements and transactions have enough variation to not be able to make this change at the common table level, I'm adding truncation to the transactions table styles to mirror the change in the statements page table. * Adding trucation to Statement col in Sessions Adding trucation to the Statements column on the Sessions page to be consistent with how statements appear on Statements and Tranactions pages. * Truncating long app names in Statements app filter If a very long name is displayed in the Statements application filter, the name may cause the dropdown to extend outside of the view or at least beyond the table and cause excessive horizontal scrolling. To correct this I have added the `nowrap` option to the `<Text>` component inside of the Dropdown trigger. The "nowrap" prop will constrain the the text to a single line but does not offer a constraint for the width. To constrain the width I have added a local style set to a max-width of `75ch` (75 characters wide). Just a note, that I have used this opportunity to bring in the `<Text>` from `ui-components` to replace the local one here, which was the inspiration for the common component. The width and display of the menu are controlled by the Dropdown component and I didn't want to alter the display of all dropdowns that use that component. So instead I extended the dropdown component to take an additional `itemsClassname` prop to apply a class name to the element rendered for the menu items. This way I can supply another local style to constrain the width of the menu (to `80ch`) and apply text truncation. * Replacing repetitive styles with Sass mixins DRYing up the code a little with two Sass mixins: text-overflow and line-clamp.
koorosh
pushed a commit
to koorosh/cockroach
that referenced
this pull request
May 31, 2021
* CRDB-2271: Truncating long queries with CSS Applying a tried and true hack using `-webkit-line-clamp` to truncate queries in the statements table to a couple of lines followed by an ellipsis. To allow a longer preview, I am applying the same fix to the tooltip, but clamping the lines to 15 (there were around 30 characters per line and design asked for ~425 characters. It's close enough). * Adding truncation to Transactions table Since statements and transactions have enough variation to not be able to make this change at the common table level, I'm adding truncation to the transactions table styles to mirror the change in the statements page table. * Adding trucation to Statement col in Sessions Adding trucation to the Statements column on the Sessions page to be consistent with how statements appear on Statements and Tranactions pages. * Truncating long app names in Statements app filter If a very long name is displayed in the Statements application filter, the name may cause the dropdown to extend outside of the view or at least beyond the table and cause excessive horizontal scrolling. To correct this I have added the `nowrap` option to the `<Text>` component inside of the Dropdown trigger. The "nowrap" prop will constrain the the text to a single line but does not offer a constraint for the width. To constrain the width I have added a local style set to a max-width of `75ch` (75 characters wide). Just a note, that I have used this opportunity to bring in the `<Text>` from `ui-components` to replace the local one here, which was the inspiration for the common component. The width and display of the menu are controlled by the Dropdown component and I didn't want to alter the display of all dropdowns that use that component. So instead I extended the dropdown component to take an additional `itemsClassname` prop to apply a class name to the element rendered for the menu items. This way I can supply another local style to constrain the width of the menu (to `80ch`) and apply text truncation. * Replacing repetitive styles with Sass mixins DRYing up the code a little with two Sass mixins: text-overflow and line-clamp.
koorosh
pushed a commit
to koorosh/cockroach
that referenced
this pull request
May 31, 2021
* CRDB-2271: Truncating long queries with CSS Applying a tried and true hack using `-webkit-line-clamp` to truncate queries in the statements table to a couple of lines followed by an ellipsis. To allow a longer preview, I am applying the same fix to the tooltip, but clamping the lines to 15 (there were around 30 characters per line and design asked for ~425 characters. It's close enough). * Adding truncation to Transactions table Since statements and transactions have enough variation to not be able to make this change at the common table level, I'm adding truncation to the transactions table styles to mirror the change in the statements page table. * Adding trucation to Statement col in Sessions Adding trucation to the Statements column on the Sessions page to be consistent with how statements appear on Statements and Tranactions pages. * Truncating long app names in Statements app filter If a very long name is displayed in the Statements application filter, the name may cause the dropdown to extend outside of the view or at least beyond the table and cause excessive horizontal scrolling. To correct this I have added the `nowrap` option to the `<Text>` component inside of the Dropdown trigger. The "nowrap" prop will constrain the the text to a single line but does not offer a constraint for the width. To constrain the width I have added a local style set to a max-width of `75ch` (75 characters wide). Just a note, that I have used this opportunity to bring in the `<Text>` from `ui-components` to replace the local one here, which was the inspiration for the common component. The width and display of the menu are controlled by the Dropdown component and I didn't want to alter the display of all dropdowns that use that component. So instead I extended the dropdown component to take an additional `itemsClassname` prop to apply a class name to the element rendered for the menu items. This way I can supply another local style to constrain the width of the menu (to `80ch`) and apply text truncation. * Replacing repetitive styles with Sass mixins DRYing up the code a little with two Sass mixins: text-overflow and line-clamp.
Closed
Closed
pav-kv
pushed a commit
to pav-kv/cockroach
that referenced
this pull request
Mar 12, 2024
…h-assert-(node_test.go) Test: Replace t.error/fatal with assert/request in [node_test.go]
ajstorm
pushed a commit
to ajstorm/cockroach
that referenced
this pull request
Jul 23, 2025
* CRDB-2271: Truncating long queries with CSS Applying a tried and true hack using `-webkit-line-clamp` to truncate queries in the statements table to a couple of lines followed by an ellipsis. To allow a longer preview, I am applying the same fix to the tooltip, but clamping the lines to 15 (there were around 30 characters per line and design asked for ~425 characters. It's close enough). * Adding truncation to Transactions table Since statements and transactions have enough variation to not be able to make this change at the common table level, I'm adding truncation to the transactions table styles to mirror the change in the statements page table. * Adding trucation to Statement col in Sessions Adding trucation to the Statements column on the Sessions page to be consistent with how statements appear on Statements and Tranactions pages. * Truncating long app names in Statements app filter If a very long name is displayed in the Statements application filter, the name may cause the dropdown to extend outside of the view or at least beyond the table and cause excessive horizontal scrolling. To correct this I have added the `nowrap` option to the `<Text>` component inside of the Dropdown trigger. The "nowrap" prop will constrain the the text to a single line but does not offer a constraint for the width. To constrain the width I have added a local style set to a max-width of `75ch` (75 characters wide). Just a note, that I have used this opportunity to bring in the `<Text>` from `ui-components` to replace the local one here, which was the inspiration for the common component. The width and display of the menu are controlled by the Dropdown component and I didn't want to alter the display of all dropdowns that use that component. So instead I extended the dropdown component to take an additional `itemsClassname` prop to apply a class name to the element rendered for the menu items. This way I can supply another local style to constrain the width of the menu (to `80ch`) and apply text truncation. * Replacing repetitive styles with Sass mixins DRYing up the code a little with two Sass mixins: text-overflow and line-clamp.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Each store registers a gossip callback for accounting and zone config
maps. On callback, the zone does a binary search using the rangesByKey
sorted slice to find possible ranges which might be intersected by
config map prefixes. On possible hits, the range is dissected along
prefix boundary lines and if there's a necessary split, the range is
split and the process retried.
This allows new zones or accounting prefixes to be specified and ranges
will automatically be split to fit.