Skip to content

admin: create new json streaming api and switch admin stats json format to use it#28988

Merged
jmarantz merged 61 commits intoenvoyproxy:mainfrom
jmarantz:admin-json-streaming
Aug 31, 2023
Merged

admin: create new json streaming api and switch admin stats json format to use it#28988
jmarantz merged 61 commits intoenvoyproxy:mainfrom
jmarantz:admin-json-streaming

Conversation

@jmarantz
Copy link
Copy Markdown
Contributor

@jmarantz jmarantz commented Aug 11, 2023

Commit Message: When admin endpoint /stats?format=json is requested, all histograms are copied into a protobuf structure for subsequent serialization via the protobuf json serializer.

This approach scales poorly with large numbers of clusters or listeners, as every byte of histogram information must be copied into memory twice: Once when copying into the protobuf structure, and once when serializing into bytes. With the new 'detailed' histograms needed for graphically rendering them into /stats?format=html, this can represent a significant burst of memory and CPU time, and might risk an OOM.

This change removes the protobuf as an intermediate representation, and incorporates a streaming JSON serializer, which meshes well with the http streaming model used by stats. This methodology decreases the CPU time spent in a new histograms benchmark by almost 15x (249ms per hundred histograms to 17ms per hundred histograms).

The new serialization class is fairly simple -- its value-add is to simply keep track of punctuation across nested arrays and classes, and otherwise transfers API calls like adding a number or string into the response buffer.

Additional Description:

--------------------------------------------------------------------------------
Benchmark                      Time            CPU         Iterations
--------------------------------------------------------------------------------
BM_HistogramsJson  (before)    249 ms          249 ms       3
BM_HistogramsJson  (after)     16.6 ms         16.6 ms     41

Risk Level: low
Testing: The existing stats_render_test, a new json serializer test, plus test/integration/admin_html/web_test.sh
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a

jmarantz added 20 commits July 2, 2023 10:42
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
…needed.

Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
@repokitteh-read-only
Copy link
Copy Markdown

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #28988 was opened by jmarantz.

see: more, trace.

Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
…h paging, disabling format for that block.

Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
…r having emitted the key.

Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
Signed-off-by: Joshua Marantz <jmarantz@google.com>
…bug-only field.

Signed-off-by: Joshua Marantz <jmarantz@google.com>
…ext.

Signed-off-by: Joshua Marantz <jmarantz@google.com>
@jmarantz
Copy link
Copy Markdown
Contributor Author

I decided to simplify the streamer implementation by keeping the stack only for debug, and removing Json::Streamer::clear() -- to clear you need to destruct all objects in the proper order. This simplifies the logic and only adds a small complexity to the call-site in StatsJsonRender, to keep the json-related data in its own structure which can be cleared during StatsJsonRender::finalize(). This removed a few blocks of code and some special-cases.

ravenblackx
ravenblackx previously approved these changes Aug 28, 2023
Copy link
Copy Markdown
Contributor

@ravenblackx ravenblackx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much nicer, thanks! Glad to see the back of the double-close support. :)

@jmarantz
Copy link
Copy Markdown
Contributor Author

Thanks! Greg can you do a senior pass?

@jmarantz
Copy link
Copy Markdown
Contributor Author

/retest

1 similar comment
@ravenblackx
Copy link
Copy Markdown
Contributor

/retest

Copy link
Copy Markdown
Member

@ggreenway ggreenway left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this approach!

/wait

delim_ = ",";
ASSERT(!histograms_initialized_);
json_->stats_array_->addMap()->addEntries(
{{"name", name}, {"value", static_cast<double>(value)}});
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there could be loss of precision by converting to double. I think it would be better to add native int64/uint64 support to the json streamer.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch! If this is consumed by JavaScript this doesn't matter, I scanned JSON docs and it does not specify what the data type is in any language; just the syntax. So if we have another language reading json it could read high-precision integers and we should write them.

This also I think will be slightly faster.

Signed-off-by: Joshua Marantz <jmarantz@google.com>
…sion during serialization.

Signed-off-by: Joshua Marantz <jmarantz@google.com>
ggreenway
ggreenway previously approved these changes Aug 30, 2023
Signed-off-by: Joshua Marantz <jmarantz@google.com>
@jmarantz
Copy link
Copy Markdown
Contributor Author

@ravenblackx @ggreenway need another stamp -- I had to use hex to specify very big integers in the test to get gcc to compile.

@jmarantz
Copy link
Copy Markdown
Contributor Author

Thanks for the great feedback, Greg & Raven!

@jmarantz jmarantz merged commit f43f0e2 into envoyproxy:main Aug 31, 2023
@jmarantz jmarantz deleted the admin-json-streaming branch August 31, 2023 00:18
jmarantz added a commit that referenced this pull request Sep 8, 2023
A performance regression was found in a late commit as part of #28988 where we switched from a low-precision fast serialization of doubles (absl::StrCat) to a high-precision slow serialization of doubles (absl::StrFormat("%.15g", ...). This resolves that issue using std::to_chars which is faster than absl::StrCat and is also high precision. The downside is it doesn't work on all platforms, so an intermediate high-precision medium-speed solution was found for Apple and GCC. The serialize-double-to-buffer functionality is moved out of Json::Streamer and into a new Buffer::Util namespace.

Latest Json histogram serialization performance:
```
------------------------------------------------------------------------
Benchmark                              Time             CPU   Iterations
------------------------------------------------------------------------
BM_HistogramsJson                   17.3 ms         17.3 ms           41

Signed-off-by: Joshua Marantz <jmarantz@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants