Skip to content

[BUG] Index field does not show up in the unmapped_index_fields array of mappings view API response #620

@amsiglan

Description

@amsiglan

What is the bug?
Log index field not returned as part of the unmapped_index_fields array in the response for get mappings view API.

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Create an index for google workspace log type. For that we can simply ingest below document
POST /gworkspace/_doc
{
  "google_workspace.drive.visibility": "shared_externally",
  "google_workspace.drive.target": "conference",
  "google_workspace.event": "conference"
}
  1. Create below detection rule for log type gworkspace:
id: 25b9c01c-350d-4b95-bed1-836d04a4f324
logsource:
  product: gworkspace
title: my rule
description: Drive visibility checker
tags: []
falsepositives: []
level: critical
status: experimental
references: []
author: unknown
detection:
  condition: Selection_1
  Selection_1:
    google_workspace.drive.visibility|contains:
      - shared
  1. Go to Create detector page and select data source gworkspace and log type gworkspace. The response from the view mappings API is
{
    "ok": true,
    "response": {
        "properties": {},
        "unmapped_index_fields": [
            "google_workspace.event",
            "google_workspace.drive.target"
        ],
        "unmapped_field_aliases": [
            "google_workspace.event.name",
            "google_workspace.admin.service.name",
            "google_workspace.admin.new_value"
        ]
    }
}

which means the field google_workspace.drive.visibility has been automatically mapped and is correct.
4. Abandon the create detector flow
5. Edit the detection rule from step 2 above and change it to

id: 25b9c01c-350d-4b95-bed1-836d04a4f324
logsource:
  product: gworkspace
title: my rule
description: Drive visibility checker
tags: []
falsepositives: []
level: critical
status: experimental
references: []
author: unknown
detection:
  condition: Selection_1
  Selection_1:
    google_workspace.drive-visibility|contains:
      - shared
  1. Repeat step 3 and check the response of the view mappings API.
{
    "ok": true,
    "response": {
        "properties": {},
        "unmapped_index_fields": [
            "google_workspace.event",
            "google_workspace.drive.target"
        ],
        "unmapped_field_aliases": [
            "google_workspace.event.name",
            "google_workspace.admin.service.name",
            "google_workspace.admin.new_value",
            "google_workspace.drive-visibility"
        ]
    }
}

We see that now we have google_workspace.drive-visibility under unmapped_field_aliases but the log index field google_workspace.drive.visibility is not present under the unmapped_index_fields array.

The expected response is

{
    "ok": true,
    "response": {
        "properties": {},
        "unmapped_index_fields": [
            "google_workspace.event",
            "google_workspace.drive.target",
            "google_workspace.drive.visibility"
        ],
        "unmapped_field_aliases": [
            "google_workspace.event.name",
            "google_workspace.admin.service.name",
            "google_workspace.admin.new_value",
            "google_workspace.drive-visibility"
        ]
    }
}

What is your host/environment?

  • Opensearch 2.10

Do you have any screenshots?
N/A

Do you have any additional context?
N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions