Skip to content

Inconsistent treatment of invalid characters in index and bulk requests #61763

@cjcenizal

Description

@cjcenizal

A user has gotten into a situation where the Index Management UI fails to load indices named like %{[@metadata][beat]}-%{[@metadata][version]}-2020.08.23 (elastic/kibana#75869). This occurs because we're fetching indices using the get index API (https://github.com/elastic/kibana/pull/66422/files#diff-784d69eb6ea217af4edd2559d5ca2445R44), which complains about these indices' names with an error like this one:

{
  "statusCode":405,
  "error":"Method Not Allowed",
  "message":"invalid escape sequence `%{[' at index 5 of: test,%{[@metadata][beat]}-%{[@metadata][version]}-2020.08.23, allowed: [PUT, DELETE, HEAD, GET]"
}

This looks similar to other problems users have reported regarding Logstash creating indices with these names (https://discuss.elastic.co/t/logstash-index-name-as-metadata-beat-metadata-version/184225). I suspect that Logstash is using the bulk API, which will allow creation of an index with this name (disallowed by the index API):

POST _bulk
{ "index" : { "_index" : "%{[@metadata][beat]}-%{[@metadata][version]}-2020.08.23", "_id" : "1" } }

It seems reasonable for the index API to reject invalid characters like %, [, and {, but inconsistent for the bulk API to allow them. It also seems trappy because then users can get into the situation as in the first link I shared, in which parts of the stack break for no clear reason, due to unexpected interactions between other parts of the stack.

Possibly related: #48308, #9379

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions