-
Notifications
You must be signed in to change notification settings - Fork 168
Closed
Description
Code of Conduct
- I agree to follow this project's Code of Conduct
Search before asking
- I have searched in the issues and found no similar issues.
Describe the bug
The connection between the client and shufflerserver is not closed. As a result, the maptask container does not exit.
1.map container does not exit when reduce is running
2.When cluster resources are used up and some maps are not allocated, am waits for a one-minute timeout, kills the completed map container, and allocates resources to other maps
error log like:
2023-03-11 02:42:16,826 INFO [Ping Checker] org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: Expired:attempt_1676901654399_1531374_m_000190_0 Timed out after 60 secs
In the close method of the RssMapOutputCollector, closing the shuffle client solves this problem
Affects Version(s)
master
Uniffle Server Log Output
No response
Uniffle Engine Log Output
No response
Uniffle Server Configurations
No response
Uniffle Engine Configurations
No response
Additional context
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Metadata
Metadata
Assignees
Labels
No labels