Skip to content

[BUG] Errors/Broken operations during rolling upgrade of clusters from 1.3 to 2.0 #2228

@saikaranam-amazon

Description

@saikaranam-amazon

What is the bug?
Errors/Broken search results during rolling upgrade of clusters from 1.3 to 2.0

How can one reproduce the bug?

  1. Create 1.3 cluster with atleast 2 nodes
  2. Create index with 2 primaries to allocate atleast one primary per node.
  3. Upgrade one of the node to 2.0 OS version.
  4. Invoke search query to invoke search on all the shards from 1.3 node.
  5. See that there are failures to execute the search on this request
"took" : 40,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 1,
    "failures" : [
      {
        "shard" : 1,
        "index" : "test-index",
        "node" : "O7kxX-lMTAKvXBj91-LQ8Q",
        "reason" : {
          "type" : "exception",
          "reason" : "java.lang.ClassNotFoundException: com.amazon.opendistroforelasticsearch.security.user.User",
          "caused_by" : {
            "type" : "class_not_found_exception",
            "reason" : "class_not_found_exception: com.amazon.opendistroforelasticsearch.security.user.User"
          }
        }
      }
    ]
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        ".....
          }
        }
      }
    ]

Notice that the failed shard count is 1

What is the expected behavior?
Rolling upgrade of clusters should complete without any issues.

What is your host/environment?

  • OS: 1.3 to OS: 2.0 upgrade
  • Plugins: Security

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingsprint backlogtriagedIssues labeled as 'Triaged' have been reviewed and are deemed actionable.v1.3.7

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions