Skip to content

Update testNodeSelectorRouting so that it does not depend on iteration order#89879

Merged
idegtiarenko merged 3 commits intoelastic:mainfrom
idegtiarenko:fix_testNodeSelectorRouting
Sep 9, 2022
Merged

Update testNodeSelectorRouting so that it does not depend on iteration order#89879
idegtiarenko merged 3 commits intoelastic:mainfrom
idegtiarenko:fix_testNodeSelectorRouting

Conversation

@idegtiarenko
Copy link
Copy Markdown
Contributor

@idegtiarenko idegtiarenko commented Sep 7, 2022

New allocator might allocate shards in different order that will affect order of elements in the iteration.
Fixes RoutingIteratorTests » testNodeSelectorRouting (different order of iterating).

Related to: #86429

@idegtiarenko idegtiarenko added >test Issues or PRs that are addressing/adding tests :Distributed/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) Team:Distributed Meta label for distributed team. v8.5.0 labels Sep 7, 2022
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

elasticsearchmachine commented Sep 7, 2022

Pinging @elastic/es-distributed (Team:Distributed)

@idegtiarenko
Copy link
Copy Markdown
Contributor Author

idegtiarenko commented Sep 9, 2022

This is caused by 2 different allocation possibilities:

routing_nodes:
-----node_id[node1][V]
--------[test][0], node[node1], [P], s[STARTED], a[id=k8m_YHiLTiawZiXHAKILlw]
-----node_id[node2][V]
--------[test][0], node[node2], [R], recovery_source[peer recovery], s[INITIALIZING], a[id=v0Q3RGPkR7ighebIonQEmQ], unassigned_info[[reason=INDEX_CREATED], at[2022-09-07T12:51:06.802Z], delayed=false, allocation_status[no_attempt]]
---- unassigned


routing_nodes:
-----node_id[node1][V]
--------[test][0], node[node1], [R], recovery_source[peer recovery], s[INITIALIZING], a[id=F_H852zOTZGG144Tn0wPYg], unassigned_info[[reason=INDEX_CREATED], at[2022-09-07T12:51:38.210Z], delayed=false, allocation_status[no_attempt]]
-----node_id[node2][V]
--------[test][0], node[node2], [P], s[STARTED], a[id=v_QeYR8GScC5U6w3ZJ8_Yw]
---- unassigned

that are caused by different iteration order in Set12. See:

    /**
     * For set and map iteration, we will iterate in "reverse" stochastically,
     * decided at bootstrap time.
     */
    private static final boolean REVERSE;

DaveCTurner
DaveCTurner approved these changes Sep 9, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@idegtiarenko idegtiarenko merged commit 7154765 into elastic:main Sep 9, 2022
@idegtiarenko idegtiarenko deleted the fix_testNodeSelectorRouting branch September 9, 2022 11:10
weizijun added a commit to weizijun/elasticsearch that referenced this pull request Sep 9, 2022
@weizijun
Merge branch 'main' into fixed-parse-tsid
1b2d52b 
* main: (176 commits)
  Fix RandomSamplerAggregatorTests testAggregationSamplingNestedAggsScaled test failure (elastic#89958)
  [Downsampling] Replace document map with SMILE encoded doc (elastic#89495)
  Remove full cluster state from error logging in MasterService (elastic#89960)
  [ML] Truncate categorization fields (elastic#89827)
  [TSDB] Removed `summary` and `histogram` metric types (elastic#89937)
  Update testNodeSelectorRouting so that it does not depend on iteration order (elastic#89879)
  Make sure listener is resolved when file queue is cleared (elastic#89929)
  [Stable plugin api] Extensible annotation (elastic#89903)
  Fix double sending of response in TransportOpenIdConnectPrepareAuthenticationAction (elastic#89930)
  Make sure ivy repo directory exists before downloading artifacts
  Use 'file://' scheme for local repository URL
  Use DRA artifacts for release build CI jobs
  Log unsuccessful attempts to get credentials from web identity tokens (elastic#88241)
  Script: Write Field API path manipulation (elastic#89889)
  Fetch health info action (elastic#89820)
  Fix memory leak in TransportDeleteExpiredDataAction (elastic#89935)
  [ML] Performance improvements for categorization jobs (elastic#89824)
  [DOCS] Revert changes for ES_JAVA_OPTS (elastic#89931)
  Fix deadlock bug exposed by a test (elastic#89934)
  [Downsampling] Remove `FieldValueFetcher` validator (elastic#89497)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@DaveCTurner DaveCTurner DaveCTurner approved these changes

Assignees

No one assigned

Labels

:Distributed/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) Team:Distributed Meta label for distributed team. >test Issues or PRs that are addressing/adding tests v8.5.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@idegtiarenko @elasticsearchmachine @DaveCTurner