docs: add 429 semantic_text troubleshooting#2507
Conversation
🔍 Preview links for changed docs |
| These errors are often related to [backlogged tasks](task-queue-backlog.md), [bulk index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) sizing, or the ingest target's [`refresh_interval` setting](elasticsearch://reference/elasticsearch/index-settings/index-modules.md). | ||
| These errors are often related to [backlogged tasks](task-queue-backlog.md), [bulk index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) sizing, or the ingest target's [`refresh_interval` setting](elasticsearch://reference/elasticsearch/index-settings/index-modules.md).¹ | ||
|
|
||
| ¹ : Starting from versions 8.19+ and 9.1+, indexing to a semantic_text field can also cause 429 errors if the batch may otherwise incur an Out Of Memory (OOM) error. |
There was a problem hiding this comment.
I don't think we typically use footnotes in our docs? It would probably be better to put this in the doc and use our applies to syntax to tag the appropriate version number. Note that we'll want a follow up PR for 8.19 as this uses asciidoc format, so we'll only need applies to 9.1 in markdown.
There was a problem hiding this comment.
Makes perfect sense, (I just like footnotes 🙈). Will adjust, thanks!
|
|
||
| 1. First, try reducing the batch size of documents in your indexing requests. | ||
| 2. If reducing batch size doesn't resolve the issue, then consider scaling up your machine resources. | ||
| 3. A last resort option is to adjust the indexing_pressure.memory.coordinating.limit cluster setting. The default value is 10% of the heap. Increasing this limit allows more memory to be used for coordinating operations before rejections occur. This adjustment should only be considered after exhausting other options, as setting this value too high may risk Out of Memory (OOM) errors in your cluster. A cluster restart is required for this change to take effect. No newline at end of file |
There was a problem hiding this comment.
This may warrant a warning in the docs, not just information in the list? Also, we should note that cluster settings can't be changed in serverless - that should be taken care of with applies to notation.
There was a problem hiding this comment.
Great pointers, will have a look. Thanks!
Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co>
…-content into add_429_semantic_text_note
Mikep86
left a comment
There was a problem hiding this comment.
Content looks good! But the ongoing battle with inline applies_to continues :)
|
|
||
| These errors are often related to [backlogged tasks](task-queue-backlog.md), [bulk index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) sizing, or the ingest target's [`refresh_interval` setting](elasticsearch://reference/elasticsearch/index-settings/index-modules.md). | ||
| These errors are often related to [backlogged tasks](task-queue-backlog.md), [bulk index](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-bulk) sizing, or the ingest target's [`refresh_interval` setting](elasticsearch://reference/elasticsearch/index-settings/index-modules.md). | ||
| {applies_to}`stack: ga 9.1`{applies_to}`serverless: ga`Another cause of indexing pressure rejections might be the use of the [`semantic_text`](https://www.elastic.co/docs/reference/elasticsearch/mapping-reference/semantic-text) field type, which can cause rejections when indexing large batches of documents if the batch may otherwise incur an Out Of Memory (OOM) error. |
There was a problem hiding this comment.
This line renders oddly due to the the inline applies_to:
Perhaps we should make a new sub-section for semantic text here and use a section applies_to instead? Or put this content in a note block?
There was a problem hiding this comment.
Tried out a note block.
Any better? 😅
|
|
||
| 1. Reduce the batch size of documents in your indexing requests. | ||
| 2. If reducing batch size doesn't resolve the issue, then consider scaling up your machine resources. | ||
| {applies_to}`serverless: unavailable`3. A last resort option is to adjust the `indexing_pressure.memory.coordinating.limit` cluster setting. The default value is 10% of the heap. Increasing this limit allows more memory to be used for coordinating operations before rejections occur. |
There was a problem hiding this comment.
Similar rendering issue here:
Moving the applies_to to after the "3." could help
There was a problem hiding this comment.
1. Reduce the batch size of documents in your indexing requests.
2. If reducing batch size doesn't resolve the issue, then consider scaling up your machine resources.
3. {applies_to}`serverless: unavailable` A last resort option is to adjust the `indexing_pressure.memory.coordinating.limit` cluster setting. The default value is 10% of the heap. Increasing this limit allows more memory to be used for coordinating operations before rejections occur.
Will render as:
There was a problem hiding this comment.
Oh definitely! Thanks both, apologies! Was playing around with the markdown yesterday and left it in an inconsistent state.
Thanks for the suggestion!
❓ Is there a way to render this locally so that I can try a few things out?
There was a problem hiding this comment.
@mromaios Certainly! The instructions for running a local docs build are here: https://elastic.github.io/docs-builder/contribute/locally/
You'd need to install the docs-builder tool and then clone the docs-content repo.
Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com>
…-content into add_429_semantic_text_note
Mikep86
left a comment
There was a problem hiding this comment.
Formatting looks much better now, thanks!
|
Thanks for the reviews everyone 🙏 ! Will follow up with an 8.19 PR as well. |
This PR adds some troubleshooting information for addressing 429 responses due to
semantic_textingestion.