Skip to content

Use envoy-wasm as upstream#2252

Merged
istio-testing merged 6 commits intoistio:masterfrom
bianpengyuan:test-wasm
Jun 1, 2019
Merged

Use envoy-wasm as upstream#2252
istio-testing merged 6 commits intoistio:masterfrom
bianpengyuan:test-wasm

Conversation

@bianpengyuan
Copy link
Copy Markdown
Contributor

No description provided.

@bianpengyuan bianpengyuan requested a review from kyessenov May 31, 2019 20:54
@googlebot googlebot added the cla: yes Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA. label May 31, 2019
@istio-testing istio-testing requested review from lambdai and rshriram May 31, 2019 20:54
@kyessenov
Copy link
Copy Markdown
Contributor

kyessenov commented May 31, 2019 via email

@bianpengyuan
Copy link
Copy Markdown
Contributor Author

/retest proxy-presubmit-tsan

@bianpengyuan
Copy link
Copy Markdown
Contributor Author

/test proxy-presubmit-tsan

@kyessenov
Copy link
Copy Markdown
Contributor

Per envoyproxy/envoy-wasm#19 (comment) sanitizer builds might be broken due to WAVM.

@bianpengyuan
Copy link
Copy Markdown
Contributor Author

@kyessenov the failure seems to be caused by flakiness in an existing test (lru_cache), and there is a typo in lgtm so could you do it again?

@PiotrSikora
Copy link
Copy Markdown
Contributor

As @kyessenov said, WAVM is leaking memory, but we don't exercise WASM tests in istio/proxy, and the local tests are passing...

Perhaps we should be running upstream tests (all of them, not only WASM) as part of Q&A as well? Otherwise, we might be shipping broken build (e.g. due to different build options that upstream).

cc @duderino

@kyessenov
Copy link
Copy Markdown
Contributor

kyessenov commented May 31, 2019 via email

@istio-testing
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bianpengyuan, kyessenov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@bianpengyuan
Copy link
Copy Markdown
Contributor Author

@PiotrSikora This is on master branch, we are not trying to ship anything from this code yet, but just want to include the null sandbox API to unblock development. Are you worrying about daily build? Ideally envoy-wasm should be running all envoy test soon, right?

@istio-testing istio-testing merged commit c77759c into istio:master Jun 1, 2019
@PiotrSikora
Copy link
Copy Markdown
Contributor

@bianpengyuan no, what I'm saying that we have false sense of correctness, because we never run upstream tests (and I'm not talking only about envoy-wasm, but envoy in general) using the binaries that we build in istio/proxy.

@bianpengyuan
Copy link
Copy Markdown
Contributor Author

@PiotrSikora So it is a general question always existing with this repo. My understanding is that upstream should be responsible for testing its core functionality and filter interface, so that istio-proxy or any out of tree filter could focus on testing its own functionality, which is self-contained. Is this wrong and we should always run more upstream tests? Seems like quite a burden to me :)

@kyessenov
Copy link
Copy Markdown
Contributor

@bianpengyuan @PiotrSikora filed https://github.com/istio/proxy/issues/2256. We can break envoy tests if we change dependency versions, so we have to test it again here.

@pitlv2109
Copy link
Copy Markdown
Member

Hey is there anything else now that I need to do to build Istio proxy? Running bazel build src/envoy:envoy would give errors such as An error occurred during the fetch of repository 'envoy_deps' and External dependency build failed...

Revert this PR would allow bazel build src/envoy:envoy to work.

@bianpengyuan
Copy link
Copy Markdown
Contributor Author

@pitlv2109 what's the error output?

@pitlv2109
Copy link
Copy Markdown
Member

It seems like it has something to do with me not having a -ASM version of clang. I'm not familiar with WebAssembly so not sure. Here is the whole log https://gist.github.com/pitlv2109/7b60048ae7939bb7788e29622baf9838 Thanks!

@bianpengyuan
Copy link
Copy Markdown
Contributor Author

Yeah looks like you need to install clang-7 and clang++-7.

@pitlv2109
Copy link
Copy Markdown
Member

Thanks! I'll give it a try

@pitlv2109
Copy link
Copy Markdown
Member

Looks like there is now some linking issue.

Linking of rule '//src/envoy:envoy' failed (Exit 1) envoy_cc_wrapper failed: error executing command .../7e01d71ebfad769394e7addc8f5b132f/external/local_config_cc/extra_tools/envoy_cc_wrapper -o bazel-out/k8-fastbuild/bin/src/envoy/envoy ... (remaining 69 argument(s) skipped)

Just curious, why did we switch to use envoy-wasm?

@bianpengyuan
Copy link
Copy Markdown
Contributor Author

It is to unblock development of mixer v2 because webassembly filter API is only available in envoy-wasm repo and has not been upstreamed. I assume your change should have nothing to do with webassembly filter, so feel free to change workspace back to use envoy upstream directly in your local environment.

@PiotrSikora
Copy link
Copy Markdown
Contributor

@bianpengyuan do you need WASM runtimes right now or only the null sandbox?

@bianpengyuan
Copy link
Copy Markdown
Contributor Author

@PiotrSikora We'd like a way to build only null sandbox for presubmit to avoid excessive build time, and build envoy with full WASM runtime at post-submit and run extension as a module in order to make sure the development is not breaking WASM integration.

@duderino
Copy link
Copy Markdown

@bianpengyuan @kyessenov @mandarjog @PiotrSikora I'm a bit worried about this PR.

It's making it harder for us to maintain the release-1.0, release-1.1, and release-1.2 branches because we are supposed to merge into master first, then cherry pick into one of those branches, but master is now very different from our release branches.

Can you move your work to a different branch?

CC: @crazyxy who is trying to get #2270 merged to unblock the 1.2 release and is running into problems on master

@PiotrSikora
Copy link
Copy Markdown
Contributor

Envoy in master is at exactly the same version as in release-1.2 (+ WASM changes, but they are self-contained), so I'm not sure how it affects #2270.

@duderino
Copy link
Copy Markdown

Well for starters, I can't even build that PR which I was trying to do to debug some of the automated test failures on Yan's PR

________ running 'download_from_google_storage --no_resume --no_auth -u --bucket v8-wasm-spec-tests -s v8/test/wasm-spec-tests/tests.tar.gz.sha1' in '/home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy_deps_cache_0b3e9e759c817528ff346c390d0b536b/v8.dep.build/wasm-c-api-dc8cc29340e0c374dd350e0de8ff4b22d429af1e/v8'
0> Downloading v8/test/wasm-spec-tests/tests.tar.gz...
0> Removed v8/test/wasm-spec-tests/tests...
0> Extracting 72 entries from v8/test/wasm-spec-tests/tests.tar.gz to v8/test/wasm-spec-tests/tests
Downloading 1 files took 3.595652 second(s)
________ running '/usr/bin/python v8/build/linux/sysroot_scripts/install-sysroot.py --arch=x86' in '/home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy_deps_cache_0b3e9e759c817528ff346c390d0b536b/v8.dep.build/wasm-c-api-dc8cc29340e0c374dd350e0de8ff4b22d429af1e/v8'
________ running '/usr/bin/python v8/build/linux/sysroot_scripts/install-sysroot.py --arch=x64' in '/home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy_deps_cache_0b3e9e759c817528ff346c390d0b536b/v8.dep.build/wasm-c-api-dc8cc29340e0c374dd350e0de8ff4b22d429af1e/v8'
________ running '/usr/bin/python v8/third_party/binutils/download.py' in '/home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy_deps_cache_0b3e9e759c817528ff346c390d0b536b/v8.dep.build/wasm-c-api-dc8cc29340e0c374dd350e0de8ff4b22d429af1e/v8'
________ running '/usr/bin/python v8/tools/clang/scripts/update.py' in '/home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy_deps_cache_0b3e9e759c817528ff346c390d0b536b/v8.dep.build/wasm-c-api-dc8cc29340e0c374dd350e0de8ff4b22d429af1e/v8'
Downloading https://commondatastorage.googleapis.com/chromium-browser-clang/Linux_x64/clang-354873-1.tgz .......... Done.
________ running '/usr/bin/python v8/build/util/lastchange.py -o v8/build/util/LASTCHANGE' in '/home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy_deps_cache_0b3e9e759c817528ff346c390d0b536b/v8.dep.build/wasm-c-api-dc8cc29340e0c374dd350e0de8ff4b22d429af1e/v8'
+===================================================+
|   METRICS COLLECTION WILL START IN 7 EXECUTIONS   |
|                                                   |
| To suppress this message opt in or out using:     |
| $ gclient metrics [--opt-in] [--opt-out]          |
| For more information please see metrics.README.md |
| in your depot_tools checkout or visit             |
| https://goo.gl/yNpRDV.                            |
+===================================================+
++ 1560453830.365143200 patch -p1
patching file BUILD.gn
Hunk #1 succeeded at 1555 with fuzz 2 (offset 431 lines).
Hunk #2 succeeded at 1606 with fuzz 2 (offset 455 lines).
Hunk #3 succeeded at 1622 with fuzz 2 (offset 456 lines).
++ 1560453830.367716252 cp -p ../../src/wasm-v8-lowlevel.cc src/wasm-v8-lowlevel.cc
++ 1560453830.369808358 cp -p ../../src/wasm-v8-lowlevel.hh include/wasm-v8-lowlevel.hh
++ 1560453830.371798046 tools/dev/v8gen.py x64.release -- v8_monolithic=true v8_use_external_startup_data=false v8_enable_i18n_support=false v8_enable_gdbjit=false use_custom_libcxx=false

Hint: You can raise verbosity (-vv) to see the output of failed commands.

Traceback (most recent call last):
  File "tools/dev/v8gen.py", line 307, in <module>
    sys.exit(gen.main())
  File "tools/dev/v8gen.py", line 301, in main
    return self._options.func()
  File "tools/dev/v8gen.py", line 169, in cmd_gen
    gn_outdir,
  File "tools/dev/v8gen.py", line 211, in _call_cmd
    stderr=subprocess.STDOUT,
  File "/usr/lib/python2.7/subprocess.py", line 223, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['/usr/bin/python', '-u', 'tools/mb/mb.py', 'gen', '-f', 'infra/mb/mb_config.pyl', '-m', 'developer_default', '-b', 'x64.release', 'out.gn/x64.release']' returned non-zero exit status 1

real	4m3.305s
user	4m23.063s
sys	0m26.887s
Makefile:64: recipe for target '/home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy_deps_cache_0b3e9e759c817528ff346c390d0b536b/v8.dep' failed
make[1]: *** [/home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy_deps_cache_0b3e9e759c817528ff346c390d0b536b/v8.dep] Error 1
make[1]: Leaving directory '/home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy_deps'

real	4m3.488s
user	7m56.072s
sys	0m42.903s
DEBUG: /home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy/bazel/repositories.bzl:60:5: External dep build exited with return code: 2
DEBUG: /home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy/bazel/repositories.bzl:62:9:  External dependency build failed, check above log for errors and ensure all prerequisites at https://github.com/envoyproxy/envoy/blob/master/bazel/README.md#quick-start-bazel-build-for-developers are met.
ERROR: /home/jblatt/.cache/bazel/_bazel_jblatt/b6258e8f3eabbcfebfab41d31f03811a/external/envoy/bazel/repositories.bzl:189:13: no such package '@envoy_deps//': External dep build failed and referenced by '//external:wavm_with_llvm'
ERROR: Analysis of target '//src/envoy/http/jwt_auth:http_filter_factory' failed; build aborted: Analysis failed
INFO: Elapsed time: 260.303s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (441 packages loaded, 16885 targets configured)
FAILED: Build did NOT complete successfully (441 packages loaded, 16885 targets configured)
    Fetching @envoy_deps; fetching 243s
Makefile:48: recipe for target 'test' failed
make: *** [test] Error 1

@PiotrSikora
Copy link
Copy Markdown
Contributor

Fair enough, #2273 should fix it (I was postponing update until @jplevyak's PRs are merged, but I guess we can do another update then).

PiotrSikora added a commit to PiotrSikora/proxy that referenced this pull request Jun 20, 2019
Backported from istio#2252.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
PiotrSikora added a commit to PiotrSikora/proxy that referenced this pull request Jun 20, 2019
Backported from istio#2252.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
PiotrSikora added a commit to PiotrSikora/proxy that referenced this pull request Jun 20, 2019
Backported from istio#2252.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
PiotrSikora added a commit to PiotrSikora/proxy that referenced this pull request Jun 20, 2019
Backported from istio#2252.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
duderino pushed a commit that referenced this pull request Jul 23, 2019
Backported from #2252.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
fpesce pushed a commit that referenced this pull request Jul 23, 2019
* Always use Bazel-0.25 for macOS on CircleCI (release-1.2).

Backported from #2252.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>

* review: update Bazel for Linux on CircleCI.

Signed-off-by: Piotr Sikora <piotrsikora@google.com>
@duderino
Copy link
Copy Markdown

duderino commented Aug 9, 2019

@bianpengyuan @kyessenov in retrospect, I'm more convinced that this should not have been merged to master and instead should have been done on a branch

@bianpengyuan
Copy link
Copy Markdown
Contributor Author

@duderino What is the downside of shipping with envoy-wasm? Sorry if this is an obvious question but I found it is hard for me to think of any... The extra code are all filter based and any feature shipped with envoy wasm filter will be considered experimental or very early alpha. They will not be enabled by default. Also we are not building istio proxy with v8 or wavm, but just a wrapper of envoy's filter API, so it seems to be not risky at all.

@kyessenov
Copy link
Copy Markdown
Contributor

Master is just another branch. We fully intend to ship the branch with envoy-wasm as the base, so it's not clear to me why the designation makes a difference.

@PiotrSikora
Copy link
Copy Markdown
Contributor

@duderino why?

@duderino
Copy link
Copy Markdown

duderino commented Aug 9, 2019

The immediate trigger for most people, I think, is that Istio has never carried such a large fork from upstream Envoy before. Yes, envoy-wasm will be upstreamed, but it will undergo changes during the code reviews which may break backwards compatibility.

So the argument should be reversed - why do we want to ship envoy-wasm with Istio 1.3? Are there features we want to expose early? Or is it just easier for us to do all our development on master?

@kyessenov
Copy link
Copy Markdown
Contributor

envoy-wasm is a requirement for prometheus and stackdriver adapters in mixer v2 architecture.
We should coordinate with @mandarjog .

@bianpengyuan
Copy link
Copy Markdown
Contributor Author

bianpengyuan commented Aug 9, 2019

Regarding backward compatibility, do you mean wasm API's compatibility? I think we are not going to advertise it anywhere or publish it as feature on istio.io. That said it is not even an alpha feature, so I guess it is OK to make breaking changes when upstream? If people want to try out WASM based filter with 1.3, they should expect changes.

@PiotrSikora
Copy link
Copy Markdown
Contributor

@duderino this is not a fork, it's an up-to-date mirror of envoyproxy/envoy with extensions hosted there instead of here (i.e. istio/proxy) to ease the upstreaming process.

There is zero difference between extensions hosted in envoyproxy/envoy-wasm and those hosted here, so people making an issue out of it just don't know what they're talking about.

@PiotrSikora
Copy link
Copy Markdown
Contributor

In any case, we're going to branch release-1.3 from master as-is (i.e. based on envoyproxy/envoy-wasm).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: yes Set by the Google CLA bot to indicate the author of a PR has signed the Google CLA.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants