Skip to content

ui/cluster-ui: add wait time insights to active executions#85081

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
xinhaoz:txn-contention-ui
Aug 11, 2022
Merged

ui/cluster-ui: add wait time insights to active executions#85081
craig[bot] merged 1 commit intocockroachdb:masterfrom
xinhaoz:txn-contention-ui

Conversation

@xinhaoz
Copy link
Copy Markdown
Contributor

@xinhaoz xinhaoz commented Jul 26, 2022

Closes: #79127
Closes: #79131

This commit introduces a new section, 'Wait Time Insights' to the
active stmts/txns details pages. The section is included if the txn
being viewed is experiencing contention and includes info on the
blocked schema, table, index name, time spent blocking, and the
executions blocking or waiting for the viewed execution.

The section is powered by querying the crdb_internal.cluster_locks
table via the SQL api in /api/v2/. The table informatioin is
refreshed at an interval of 10s while on active execution pages,
and is requested along with session information (to give info on
active txns/stmts).

Only users having VIEWACTIVITY or higher permissions can view this
feature.

Refactor note: to remove duplication across shared selector logic
in the active txn components, activeExecutionsCommon.selectors.ts
has been created. This file exports combiner functions for active
txn selectors. The combiner func step contains the bulk of the logic
to transform data from the redux state to component props, thus future
changes to this logic will not need to be duplicated across packages.

Release note (ui change): A new section, 'Wait Time Insights' has
been added to the active statement and transaction details page.
The section is included if the txn being viewed is experiencing
contention and includes info on the blocked schema, table, index
name, time spent blocking, and the txns blocking or waiting for the
viewed txn. Only users having VIEWACTIVITY or higher can view
this feature. The column 'Time Spent Waiting' has been added to
the active executions tables that shows the total amount of time
an execution has been waiting for a lock.


Note to reviewers: only the 2nd commit is relevant, see first commit here: #85080

https://www.loom.com/share/53d8a74f9c9041a9b967e32dc370a153

@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@xinhaoz xinhaoz force-pushed the txn-contention-ui branch 5 times, most recently from 59c059a to 8cfb2c3 Compare July 27, 2022 21:23
@xinhaoz xinhaoz requested a review from a team July 27, 2022 21:24
@xinhaoz
Copy link
Copy Markdown
Contributor Author

xinhaoz commented Jul 27, 2022

@cockroachdb/sql-observability I'd like to do the same refactor shown here for the selector logic for active statements pages and possibly other pages. I notice when we make changes to how we transform data from the redux store to our pages we literally copy paste the code across db-console and cluster-ui workspaces. This is pretty tedious to maintain but we can avoid the bulk of that if we just export the combiner functions for the selectors, as shown here. LMK your thoughts and I can file issues, maybe smth to do for a flex friday =)

@xinhaoz
Copy link
Copy Markdown
Contributor Author

xinhaoz commented Jul 27, 2022

The CC changes here are WIP. Couldn't test those changes yet and might just revert and leave them for another PR.

@xinhaoz xinhaoz force-pushed the txn-contention-ui branch from 8cfb2c3 to 20102df Compare August 3, 2022 19:02
@xinhaoz xinhaoz marked this pull request as ready for review August 3, 2022 19:05
@xinhaoz xinhaoz force-pushed the txn-contention-ui branch from 20102df to a47cf01 Compare August 3, 2022 19:08
@xinhaoz xinhaoz changed the title ui/cluster-ui: add wait time insights to active transaction details ui/cluster-ui: add wait time insights to active executions Aug 3, 2022
Copy link
Copy Markdown
Contributor

@maryliag maryliag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 4 files at r1, 33 of 33 files at r3, 23 of 40 files at r4.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @maryliag and @xinhaoz)


pkg/ui/workspaces/cluster-ui/src/activeExecutions/activeStatementUtils.ts line 99 at r4 (raw file):

            : "Preparing",
        start: TimestampToMoment(query.start),
        elapsedTimeMillis: time.diff(TimestampToMoment(query.start), "ms"),

might be worth just calling elapsedTime and leaving the unit to be generated by the format Duration function, which adds the relevant unit


pkg/ui/workspaces/cluster-ui/src/activeExecutions/activeStatementUtils.ts line 228 at r4 (raw file):

        status: "Executing" as ExecutionStatus,
        start: TimestampToMoment(activeTxn.start),
        elapsedTimeMillis: time.diff(TimestampToMoment(activeTxn.start), "ms"),

same comment as above


pkg/ui/workspaces/cluster-ui/src/activeExecutions/types.ts line 28 at r4 (raw file):

export interface ActiveExecution {
  statementID?: string; // This may not be present for a tranaction.

nit: transaction


pkg/ui/workspaces/cluster-ui/src/detailsPanels/waitTimeInsightsPanel.tsx line 34 at r4 (raw file):

  BLOCKED_INDEX: "Blocked Index",
  BLOCKED_ROW: "Blocked Row",
  WAIT_TIME: "Time Spenet Waiting",

nit: spent


pkg/ui/workspaces/cluster-ui/src/detailsPanels/waitTimeInsightsPanel.tsx line 38 at r4 (raw file):

    `${capitalize(execType)} ID: ${id} waiting on`,
  WAITING_TXNS_TABLE_TITLE: (id: string, execType: ExecutionType): string =>
    `${capitalize(execType)} waiting for ID: ${id}`,

nit: add an s, so it looks like statements waiting for...
you don't need the : on those 2 messages


pkg/ui/workspaces/cluster-ui/src/selectors/activeExecutions.selectors.ts line 30 at r4 (raw file):

// This file contains selector functions used across active execution
// pages that are specific to cluster-ui.
// They should NOT be exported with the cluster-ui package.

I'm a little confused by this comment, this is used for cluster-ui, but should not be exported to cluster-ui, so how cluster-ui is using these functions?


pkg/ui/workspaces/cluster-ui/src/store/clusterLocks/clusterLocks.reducer.ts line 43 at r4 (raw file):

      state.valid = false;
    },
    // Define actions that don't change state

nit: period


pkg/ui/workspaces/cluster-ui/src/store/sessions/sessions.sagas.ts line 46 at r4 (raw file):

    takeLatest(actions.request, requestSessionsSaga),
    takeLatest(
      actions.received,

what was this being used for?


pkg/ui/workspaces/cluster-ui/src/transactionDetails/activeTransactionDetails.tsx line 53 at r4 (raw file):

const TXN_EXECUTION_ID_LABEL = "Transaction Execution ID";

export const GeneralActiveTxnInsightsLabels = {

this is becoming a wordy name, I don't think the general is adding anything here and can be removed


pkg/ui/workspaces/cluster-ui/src/transactionsPage/activeTransactionsView.tsx line 87 at r4 (raw file):

  useEffect(() => {
    // Refresh  every 10 seconds.
    // if (!transactions || transactions.length === 0) {

I think you forgot to put the comment back here


pkg/ui/workspaces/db-console/src/redux/apiReducers.ts line 296 at r4 (raw file):

  api.getSessions,
  "sessions",
  // The sessions page is a real time view, so need a fairly quick update pace.

is this comment still valid since you changed to null?

@xinhaoz xinhaoz force-pushed the txn-contention-ui branch from a47cf01 to ae28201 Compare August 4, 2022 22:18
Copy link
Copy Markdown
Contributor Author

@xinhaoz xinhaoz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @maryliag)


pkg/ui/workspaces/cluster-ui/src/activeExecutions/activeStatementUtils.ts line 99 at r4 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

might be worth just calling elapsedTime and leaving the unit to be generated by the format Duration function, which adds the relevant unit

The unit displayed will still be generated by the duration function, I actually made this change so that this would happen (was previously just displaying seconds). I just need to approximate the elapsed time here based on the lastUpdated value since it's not actually returned by the server. We can file that as an issue so as to get a more accurate elapsed time.


pkg/ui/workspaces/cluster-ui/src/activeExecutions/activeStatementUtils.ts line 228 at r4 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

same comment as above

(responded to other comment)


pkg/ui/workspaces/cluster-ui/src/activeExecutions/types.ts line 28 at r4 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

nit: transaction

Done.


pkg/ui/workspaces/cluster-ui/src/detailsPanels/waitTimeInsightsPanel.tsx line 34 at r4 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

nit: spent

Done.


pkg/ui/workspaces/cluster-ui/src/detailsPanels/waitTimeInsightsPanel.tsx line 38 at r4 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

nit: add an s, so it looks like statements waiting for...
you don't need the : on those 2 messages

Done.
The : was part of the figma but I can remove it if it makes more sense!


pkg/ui/workspaces/cluster-ui/src/selectors/activeExecutions.selectors.ts line 30 at r4 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

I'm a little confused by this comment, this is used for cluster-ui, but should not be exported to cluster-ui, so how cluster-ui is using these functions?

I noted they shouldn't be exported with the package since they're not to be used in db-console. Since they're local to cluster-ui we don't need to export them since they'll just be used by the components connecting to redux within the pkg. The file activeExecutionsCommon.selectors and the active stmt utils contains the funcs that can be reused across the two pkgs for the selectors and contains all the shared logic in transforming data for the UI pages. This file just uses those functions to create selectors based off the specific state in cluster-ui.


pkg/ui/workspaces/cluster-ui/src/store/clusterLocks/clusterLocks.reducer.ts line 43 at r4 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

nit: period

Done.


pkg/ui/workspaces/cluster-ui/src/store/sessions/sessions.sagas.ts line 46 at r4 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

what was this being used for?

This was used to perform polling previously (invalidation causes refresh). But since we're manually polling now on the page there's no need for this and I thought the added logic here might muck up the polling being done by the pages so I removed it.


pkg/ui/workspaces/cluster-ui/src/transactionDetails/activeTransactionDetails.tsx line 53 at r4 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

this is becoming a wordy name, I don't think the general is adding anything here and can be removed

Done.


pkg/ui/workspaces/cluster-ui/src/transactionsPage/activeTransactionsView.tsx line 87 at r4 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

I think you forgot to put the comment back here

Ah yes, fixed!


pkg/ui/workspaces/db-console/src/redux/apiReducers.ts line 296 at r4 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

is this comment still valid since you changed to null?

Updated comment to better reflect how this api is being used.

Copy link
Copy Markdown
Contributor

@maryliag maryliag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I notice on the loom that on the details page, the values for blocked schema, blocked database and blocked index are empty, only the blocked table is populated, is that expected?

Reviewed 1 of 37 files at r5, 36 of 36 files at r6.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @xinhaoz)


pkg/ui/workspaces/cluster-ui/src/detailsPanels/waitTimeInsightsPanel.tsx line 76 at r6 (raw file):

              <Col className="gutter-row" span={12}>
                <SummaryCard className={cx("summary-card")}>
                  <SummaryCardItem

if viewactivityredacted won't have that value, better hide this info on that case.
Similar to showing the column itself, otherwise they will see 0/empty and think there is no contention

@xinhaoz xinhaoz force-pushed the txn-contention-ui branch from ae28201 to e64da35 Compare August 8, 2022 14:51
Copy link
Copy Markdown
Contributor Author

@xinhaoz xinhaoz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maryliag The DB and schema wasn't being populated in the component at the time I made the loom, that's since been fixed. For all the other values, they are just what is being returned in the query, so the index returned just so happened to be the empty string. Perhaps this is a bug within the virtual table -- I'll ask.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @maryliag)


pkg/ui/workspaces/cluster-ui/src/detailsPanels/waitTimeInsightsPanel.tsx line 76 at r6 (raw file):

Previously, maryliag (Marylia Gutierrez) wrote…

if viewactivityredacted won't have that value, better hide this info on that case.
Similar to showing the column itself, otherwise they will see 0/empty and think there is no contention

VIEWACTIVITYREDACTED can show the wait time values and blocked table/schema/db/index, they just won't be able to see which txns are waiting/blocking it. I left it in since I thought it might be useful info -- do you think it's better to just remove everything for REDACTED users?

Copy link
Copy Markdown
Contributor

@maryliag maryliag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the explanation! :lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @maryliag and @xinhaoz)


pkg/ui/workspaces/cluster-ui/src/detailsPanels/waitTimeInsightsPanel.tsx line 76 at r6 (raw file):

Previously, xinhaoz (Xin Hao Zhang) wrote…

VIEWACTIVITYREDACTED can show the wait time values and blocked table/schema/db/index, they just won't be able to see which txns are waiting/blocking it. I left it in since I thought it might be useful info -- do you think it's better to just remove everything for REDACTED users?

ah okay, I thought they couldn't see anything. In that case, makes sense to keep it in

@xinhaoz xinhaoz force-pushed the txn-contention-ui branch 8 times, most recently from e525744 to ebf72a5 Compare August 10, 2022 19:16
Closes: cockroachdb#79127
Closes: cockroachdb#79131

This commit introduces a new section, 'Wait Time Insights' to the
active stmts/txns details pages. The section is included if the txn
being viewed is experiencing contention and includes info on the
blocked schema, table, index name, time spent blocking, and the
executions blocking or waiting for the viewed execution.

The section is powered by querying the `crdb_internal.cluster_locks`
table via the SQL api in `/api/v2/`. The table informatioin is
refreshed at an interval of 10s while on active execution pages,
and is requested along with session information (to give info on
active txns/stmts).

Only users having VIEWACTIVITY or higher permissions can view this
feature.

Refactor note: to remove  duplication across shared selector logic
in the active txn components, `activeExecutionsCommon.selectors.ts`
has been created. This file exports  combiner functions for active
txn selectors. The combiner func step contains the bulk of the logic
to transform data from the redux state to component props, thus future
changes to this logic will not need to be duplicated across packages.

Release note (ui change): A new section, 'Wait Time Insights' has
been added to the active statement and transaction details page.
The section is included if the txn being viewed is experiencing
contention and includes info on the blocked schema, table, index
name, time spent blocking, and the txns blocking or waiting for the
viewed txn.  Only users having `VIEWACTIVITY` or higher can view
this feature. The column 'Time Spent Waiting' has been added to
the active executions tables that shows the total amount of time
an execution has been waiting for a lock.
@xinhaoz xinhaoz force-pushed the txn-contention-ui branch from ebf72a5 to 3556435 Compare August 11, 2022 02:24
@xinhaoz
Copy link
Copy Markdown
Contributor Author

xinhaoz commented Aug 11, 2022

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 11, 2022

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 11, 2022

Build succeeded:

@craig craig bot merged commit e011dd4 into cockroachdb:master Aug 11, 2022
@xinhaoz xinhaoz deleted the txn-contention-ui branch August 11, 2022 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ui: add blocking information for active queries/txns in WAITING state cluster-ui: surface blocked transaction information for active executions

3 participants