Skip to content

Conversation

@zuston
Copy link
Member

@zuston zuston commented Dec 6, 2023

What changes were proposed in this pull request?

Currently, the tags label is generated by self when registering metrics for shuffle-server. This is wrong, which will lack the GRPC or NETTY_GRPC tags.

Why are the changes needed?

Fix: #1348

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Neen't


// the value is used for client/server compatible, eg, online upgrade
public static final String SHUFFLE_SERVER_VERSION = "ss_v4";
public static final String METRICS_TAG_LABEL_NAME = "label";
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the name of label is not proper in proemetheus.

Copy link
Contributor

@jerqi jerqi Dec 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @smallzhongfeng @xianjingfeng
Is it ok for you? Do you have influence for your production environment?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you just change the name, the impact is not bad, but I remember that GRPC or NETTY_GRPC is available. I have tested this before. 🤔️

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested in this morning, but I haven't seen this. Like this,
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it may be that a certain PR has rolled back this function. If you can ensure that the tag parameter configuration is correct and the error will still be reported, I agree with this change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @leixm The tag function of the online cluster should be normal now, right?

@zuston zuston requested a review from xianjingfeng December 6, 2023 03:50
private Constants() {}

// the value is used for client/server compatible, eg, online upgrade
public static final String SHUFFLE_SERVER_VERSION = "ss_v4";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this default label also need to be deleted?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. This is necessary for upgrade when having in-compatible changes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok


// the value is used for client/server compatible, eg, online upgrade
public static final String SHUFFLE_SERVER_VERSION = "ss_v4";
public static final String METRICS_TAG_LABEL_NAME = "label";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it may be that a certain PR has rolled back this function. If you can ensure that the tag parameter configuration is correct and the error will still be reported, I agree with this change.


// the value is used for client/server compatible, eg, online upgrade
public static final String SHUFFLE_SERVER_VERSION = "ss_v4";
public static final String METRICS_TAG_LABEL_NAME = "label";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI @leixm The tag function of the online cluster should be normal now, right?

@zuston
Copy link
Member Author

zuston commented Dec 8, 2023

Could you help review again @smallzhongfeng @leixm

Copy link
Contributor

@smallzhongfeng smallzhongfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LTGM, thanks @zuston

@zuston zuston closed this Dec 13, 2023
@zuston zuston reopened this Dec 13, 2023
@codecov-commenter
Copy link

codecov-commenter commented Dec 13, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (f57360f) 0.00% compared to head (778ebbd) 54.12%.
Report is 10 commits behind head on master.

Additional details and impacted files
@@              Coverage Diff              @@
##             master    #1349       +/-   ##
=============================================
+ Coverage          0   54.12%   +54.12%     
- Complexity        0     2713     +2713     
=============================================
  Files             0      398      +398     
  Lines             0    21560    +21560     
  Branches          0     2041     +2041     
=============================================
+ Hits              0    11670    +11670     
- Misses            0     9176     +9176     
- Partials          0      714      +714     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@zuston zuston merged commit fc64c7c into apache:master Dec 13, 2023
lifeSo added a commit to lifeSo/incubator-uniffle that referenced this pull request Dec 22, 2023
* master: (23 commits)
  format
  use synchronized
  format
  modify to singleton
  format
  format
  format
  format
  format
  change Coordinator to singleton like ShuffleServerClientFactory.java
  [apache#1189] feat(server): Add netty used direct memory size metric (apache#1363)
  format
  format
  format
  rename
  file rename
  [apache#960] fix(dashboard): simplify dependency and correct the startup script (apache#1347)
  [apache#1348] improvement(metrics): Unify tags generation for shuffle-server metrics reporter (apache#1349)
  [MINOR] chore: fix kubernetes ci pipeline (apache#1368)
  [MINOR] fix(spark): Fix NPE for ShuffleWriteClientImpl.unregisterShuffle (apache#1367)
  ...
lifeSo added a commit to lifeSo/incubator-uniffle that referenced this pull request Dec 26, 2023
…re_1119_new_2023_12_23

* feature_1119_BUFFER_LIMIT_OF_HUGE_PARTITION:
  add file for commit
  [apache#1189] feat(server): Add netty used direct memory size metric (apache#1363)
  format
  format
  format
  rename
  file rename
  [apache#960] fix(dashboard): simplify dependency and correct the startup script (apache#1347)
  [apache#1348] improvement(metrics): Unify tags generation for shuffle-server metrics reporter (apache#1349)
  [MINOR] chore: fix kubernetes ci pipeline (apache#1368)
  [MINOR] fix(spark): Fix NPE for ShuffleWriteClientImpl.unregisterShuffle (apache#1367)
  format code
  format code
  添加堆外内存监控的代码
zuston added a commit to zuston/incubator-uniffle that referenced this pull request Jan 18, 2024
…-server metrics reporter (apache#1349)

### What changes were proposed in this pull request?

Currently, the tags label is generated by self when registering metrics for shuffle-server. This is wrong, which will lack the GRPC or NETTY_GRPC tags.

### Why are the changes needed?

Fix: apache#1348

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Neen't
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement] Unify tags generation for shuffle-server metrics reporter

4 participants