Skip to content

HTTP2: Add DumpState support for HTTP2#14923

Merged
snowp merged 8 commits intoenvoyproxy:mainfrom
KBaichoo:h2-rebase
Feb 17, 2021
Merged

HTTP2: Add DumpState support for HTTP2#14923
snowp merged 8 commits intoenvoyproxy:mainfrom
KBaichoo:h2-rebase

Conversation

@KBaichoo
Copy link
Copy Markdown
Contributor

@KBaichoo KBaichoo commented Feb 3, 2021

Signed-off-by: Kevin Baichoo kbaichoo@google.com

For an explanation of how to fill out the fields, please see the relevant section
in PULL_REQUESTS.md

Commit Message: HTTP2: Add DumpState support for HTTP2
Additional Description: While dispatching, the HTTP2 Connection object registers itself as a scopetrackedobject. If we crash during parsing pipeline, we'll dump information such as the current streams, current buffer, etc.
Risk Level: medium
Testing:unit tests
Docs Changes: N/A
Release Notes: Included
Platform Specific Features: N/A
Related Issue #14249

This is a follow up to PR #14812.

Originally got some feedback on this in KBaichoo#2.

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
@KBaichoo
Copy link
Copy Markdown
Contributor Author

KBaichoo commented Feb 3, 2021

/assign @antoniovicente

Copy link
Copy Markdown
Contributor

@antoniovicente antoniovicente left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it looks good otherwise.

case '\t':
os << "\\t";
break;
case '\v':
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also escape '\0'

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
@KBaichoo
Copy link
Copy Markdown
Contributor Author

KBaichoo commented Feb 4, 2021

/retest

@repokitteh-read-only
Copy link
Copy Markdown

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #14923 (comment) was created by @KBaichoo.

see: more, trace.

Copy link
Copy Markdown
Contributor Author

@KBaichoo KBaichoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @antoniovicente !

case '\t':
os << "\\t";
break;
case '\v':
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
antoniovicente
antoniovicente previously approved these changes Feb 6, 2021

void ConnectionImpl::StreamImpl::dumpState(std::ostream& os, int indent_level) const {
const char* spaces = spacesForLevel(indent_level);
os << "ConnectionImpl::StreamImpl " << this << DUMP_MEMBER(stream_id_)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think you're missing spaces in this line

os << spaces << "ConnectionImpl::StreamImpl "

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

OutputBufferStream ostream{buffer.data(), buffer.size()};

Stats::TestUtil::MemoryTest memory_test;
server_->dumpState(ostream, 0);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: passing in a non-zero number of spaces here may make testing easier/more comprehensive. suggestion: set to 2 instead of 0.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. It now generates 2 spaces for all of these tests. Note: the spacesForLevel generates min(12, 2*i) spaces for i in [0, inf).

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
@KBaichoo
Copy link
Copy Markdown
Contributor Author

KBaichoo commented Feb 8, 2021

@antoniovicente since you're the maintainer on-call can you assign someone from @envoyproxy/senior-maintainers who is a non-googler? Thanks!

@KBaichoo
Copy link
Copy Markdown
Contributor Author

KBaichoo commented Feb 9, 2021

/retest

@repokitteh-read-only
Copy link
Copy Markdown

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #14923 (comment) was created by @KBaichoo.

see: more, trace.

Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
@KBaichoo
Copy link
Copy Markdown
Contributor Author

KBaichoo commented Feb 9, 2021

/retest

@repokitteh-read-only
Copy link
Copy Markdown

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #14923 (comment) was created by @KBaichoo.

see: more, trace.

@KBaichoo
Copy link
Copy Markdown
Contributor Author

KBaichoo commented Feb 9, 2021

PTAL @snowp . @antoniovicente suggested you'd be a good senior-maintainer to look at this.

Thanks!

/assign @snowp

antoniovicente
antoniovicente previously approved these changes Feb 10, 2021
Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
Signed-off-by: Kevin Baichoo <kbaichoo@google.com>
@KBaichoo
Copy link
Copy Markdown
Contributor Author

/retest

Windows seems like a flake

@repokitteh-read-only
Copy link
Copy Markdown

Retrying Azure Pipelines:
Check envoy-presubmit isn't fully completed, but will still attempt retrying.
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #14923 (comment) was created by @KBaichoo.

see: more, trace.

@KBaichoo
Copy link
Copy Markdown
Contributor Author

/retest

@repokitteh-read-only
Copy link
Copy Markdown

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #14923 (comment) was created by @KBaichoo.

see: more, trace.

@snowp snowp merged commit 3f740bc into envoyproxy:main Feb 17, 2021
@moderation
Copy link
Copy Markdown
Contributor

moderation commented Feb 17, 2021

I'm getting a compilation error on RHEL 8.3 with Clang 10

image

no member named 'for_each_n' in namespace 'std'

Is this a Clang feature issue or something else @KBaichoo ?

Edit: this line is the only one in the codebase that uses for_each_n. May be a C++17 or greater feature

@KBaichoo
Copy link
Copy Markdown
Contributor Author

KBaichoo commented Feb 17, 2021

It is a C++17 feature: https://en.cppreference.com/w/cpp/algorithm/for_each_n

Edit: What's the procedure then? I'm fine with just replacing it.

@moderation
Copy link
Copy Markdown
Contributor

This appears to be the first introduction of a C++17 feature and it breaks Clang 10. The current build requirements are outlined at https://github.com/envoyproxy/envoy/blob/main/docs/root/start/building.rst. Will likely need the @envoyproxy/senior-maintainers to chime in

@jrajahalme
Copy link
Copy Markdown
Contributor

for_each_n also breaks build on GCC 7. Apparently it is not implemented in libstdc++, maybe libc++ works. Build docs state that both are supported, though. Maybe code the loop without using for_each_n?

@snowp
Copy link
Copy Markdown
Contributor

snowp commented Feb 17, 2021

Yeah I think the simplest thing to do here is to just avoid the use of std::for_each_n since it seems to be poorly supported. @lizan do you have any thoughts here? Is this just due to std lib implementations poorly supporting C++17 features?

@antoniovicente
Copy link
Copy Markdown
Contributor

IIRC there are other places where the Envoy codebase depends on C++ 17. Is there some compiler flag that can be used to disallow for_each_n in c++ 17 if we're not ready to support it yet?

@KBaichoo
Copy link
Copy Markdown
Contributor Author

We use structured bindings in

auto [iterator, did_insert] =
which is c++17

@snowp
Copy link
Copy Markdown
Contributor

snowp commented Feb 18, 2021

Language/library support in compilers tend to be on a per feature basis, so I can totally believe that some of the older standard library versions don't support this feature while still supporting structured bindings. I think if we want to maintain support for GCC-7 we would probably want to avoid using this, similar to how we avoid using std::filesystem due to spotty support on some platforms (e.g. mobile).

Is there some compiler flag that can be used to disallow for_each_n in c++ 17 if we're not ready to support it yet?
I doubt it, probably our best bet would be a lint check

@KBaichoo
Copy link
Copy Markdown
Contributor Author

See: #15093 for a lint of this as you suggested.

@lizan
Copy link
Copy Markdown
Member

lizan commented Feb 18, 2021

It must be due to older libstdc++, libstdc++-9 has support for that. @moderation can you try with gcc-toolset-9?

@moderation
Copy link
Copy Markdown
Contributor

@lizan I have gcc-toolset-10 in my RHEL 8 environment and will try that but I recall moving to the clang 10 for some reason. Will try and report back

@moderation
Copy link
Copy Markdown
Contributor

@lizan @KBaichoo With no config flag and therefore gcc 10 + libstdc++ and a very limited set of extensions enabled the first failure was in the quic code (I forgot to comment that out).
image

I commented that extension out and the next problem I hit was
image

So with the addition of the single C++17 feature it doesn't look like there is a way to build on RHEL 8.3 with either GCC 10 or Clang 10

@lizan
Copy link
Copy Markdown
Member

lizan commented Feb 18, 2021

You can use libstdc++ >= 9 with clang, no need use gcc, but that gcc 10 issue is a one issue that we should address.

What's your output of clang -x c++ -v -c /dev/null? If you see it picks libstdc++-8, you might need symlink ln -s /opt/rh/gcc-toolset-9/ /usr/lib/gcc/x86_64-redhat-linux/9 to let clang to pick a newer libstc++.

@rgs1
Copy link
Copy Markdown
Member

rgs1 commented Feb 22, 2021

This broke for me too in ubuntu 18 fwiw. Are we planning to revert or document/fix fwd?

@rgs1
Copy link
Copy Markdown
Member

rgs1 commented Feb 22, 2021

Ah nvm #15093 is the fix, i'll wait for that (or add it to our repo in the meanwhile).

@moderation
Copy link
Copy Markdown
Contributor

Looking forward to #15093 too as we are still blocked with Clang 10 on RHEL8.3 (and GCC is broken too but we'd rather stick with Clang)

@antoniovicente
Copy link
Copy Markdown
Contributor

PR #15093 seems stuck in review. It would be good to unblock folks using ubuntu 18 and RHEL8.3. What is the recommended path forward?

rgs1 pushed a commit to rgs1/envoy that referenced this pull request Feb 23, 2021
This reverts commit 3f740bc.

Because it breaks builds on RHEL 8 and Ubuntu 18. See:

envoyproxy#15093

Signed-off-by: Raul Gutierrez Segales <rgs@pinterest.com>
@rgs1
Copy link
Copy Markdown
Member

rgs1 commented Feb 23, 2021

I'd kindly suggest we revert (PR here #15153) for now, so we can unblock people who need to track master closely.

@antoniovicente
Copy link
Copy Markdown
Contributor

PR #15093 was merged, ubuntu 18 and RHEL8.3 builds should work again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants