Skip to content

Exposed again the pipelines queue.data and queue.capacity subdocuments for _node/stats#11923

Merged
andsel merged 1 commit intoelastic:masterfrom
andsel:fix/re_expose_capacity_and_data_for_queue_metrics
Jun 4, 2020
Merged

Exposed again the pipelines queue.data and queue.capacity subdocuments for _node/stats#11923
andsel merged 1 commit intoelastic:masterfrom
andsel:fix/re_expose_capacity_and_data_for_queue_metrics

Conversation

@andsel
Copy link
Copy Markdown
Contributor

@andsel andsel commented May 25, 2020

Avoid to reassign the subdocument for queue metrics preferring a merge
With PR #10576 the PluginsStats.report(stats) overwrites the subsection related to queue instead of merge with newly created entries.

With the overwriting of queue subdocument the HTTP with extended_stats return something like

"queue": {
  "type": "persisted",
  "events_count": 0,
  "queue_size_in_bytes": 1,
  "max_queue_size_in_bytes": 1073741824 
}

instead of the original form:

"queue": {
  "type": "persisted",
  "capacity": {
    "max_queue_size_in_bytes": 1073741824,
    "page_capacity_in_bytes": 67108864,
    "queue_size_in_bytes": 1, 
    "max_unread_events": 0
  },
  "events": 0,
  "data": {
    "path": "/home/andrea/workspace/logstash_andsel/data/queue/main",
    "storage_type": "ext4",
    "free_space_in_bytes": 57728352256
  }
}

To not break anything published, this PR does the merge of both, duplicating some data

Copy link
Copy Markdown
Member

@jsvd jsvd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To not break anything published, this PR does the merge of both, duplicating some data

Can you elaborate here on what gets duplicated with this change?

@andsel
Copy link
Copy Markdown
Contributor Author

andsel commented May 27, 2020

After this change for a persisted queue the _node/stats output becomes something like:

"queue": {
    "data": {
      "storage_type": "ext4",
      "path": "/home/andrea/workspace/logstash_andsel/data/queue/main",
      "free_space_in_bytes": 58074423296
    },
    "capacity": {
      "page_capacity_in_bytes": 67108864,
      "queue_size_in_bytes": 335, *
      "max_unread_events": 0,
      "max_queue_size_in_bytes": 1073741824 **
    },
    "events": 0, ***
    "type": "persisted",
    "events_count": 0, ***
    "queue_size_in_bytes": 335, *
    "max_queue_size_in_bytes": 1073741824 **
  },
  "hash": "3ff22011c5f904fd46f676f1392cbb1914b9f2e463af0fee296a74ed0df9a147",
  "ephemeral_id": "3dd64d86-2373-4f53-afbd-7e1a7c1f23e2"
}

where the * marked are duplications

Copy link
Copy Markdown
Member

@jsvd jsvd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andsel like we discussed, let's merge this for now but create an issue labelled breaking change to remove the duplication and reorganize this json document response for 8.0

LGTM

…s for _node/stats

Avoid to reassing the subdocument for queue metrics preferring a merge
With PR elastic#10576 the PluginsStats.report(stats) overwrites the subsection related to queue instead of merge with newly created entries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants