Skip to content

go/proxyd: Make endpoints match Geth, better logging#1730

Merged
tynes merged 1 commit intoethereum-optimism:developfrom
mslipper:feat/alt-ws-server
Nov 11, 2021
Merged

go/proxyd: Make endpoints match Geth, better logging#1730
tynes merged 1 commit intoethereum-optimism:developfrom
mslipper:feat/alt-ws-server

Conversation

@mslipper
Copy link
Copy Markdown
Contributor

@mslipper mslipper commented Nov 9, 2021

This PR makes the following changes:

Makes the RPC/WS endpoints match Geth

Rather than use custom paths for RPC/WS endpoints, Geth has the RPC and WS servers listen on different ports. This PR makes proxyd match that behavior. To configure Proxyd in this mode, use the following configuration:

[server]
# Host for the proxyd RPC server to listen on.
rpc_host = "0.0.0.0"
# Port for the above.
rpc_port = 8080
# Host for the proxyd WS server to listen on.
ws_host = "0.0.0.0"
# Port for the above
ws_port = 8085
# Maximum client body size, in bytes, that the server will accept.
max_body_size_bytes = 10485760

Better logging

This PR adds logging to all incoming requests, attempted WS connections, and exchanged WS messages. Additionally, it adds a randomly-generated req_id field to log output that will allow us to trace requests as they go from the web server, to the proxied backend, and back.

Example WS logs:

INFO [11-09|15:31:09.261] received WS connection                   req_id=63cfe34c38d6faec0989
INFO [11-09|15:31:09.774] accepted WS connection                   auth=foobar req_id=63cfe34c38d6faec0989
INFO [11-09|15:31:27.599] error preparing client message           auth=foobar req_id=63cfe34c38d6faec0989 err="parse error"
INFO [11-09|15:31:33.844] forwarded WS message to backend          method=eth_subscribe auth=foobar req_id=63cfe34c38d6faec0989
INFO [11-09|15:31:33.924] forwarded WS message to client           auth=foobar req_id=63cfe34c38d6faec0989

RPC:

INFO [11-09|15:34:07.287] received RPC request                     req_id=9293a41182a3bbb840fd auth=foobar
INFO [11-09|15:34:07.817] forwarded RPC request                    method=eth_blockNumber auth=foobar req_id=9293a41182a3bbb840fd

Change to how websocket backend groups are defined

Only one backend group can be websocket-enabled. Previously, this was done by defining a ws_enabled property on the backend group to proxy websockets to. To make this simpler, you can now define a single ws_backend_group config property at the top of the file. Example:

# List of WS methods to whitelist.
ws_method_whitelist = [
  "eth_subscribe",
  "eth_call",
  "eth_chainId"
]
# Enable WS on this backend group. There can only be one WS-enabled backend group.
ws_backend_group = "main"

[backends]
# A map of backends by name.
[backends.infura]
# The URL to contact the backend at.
rpc_url = "foo"
# The WS URL to contact the backend at.
ws_url = "bar"
max_rps = 3
max_ws_conns = 1

[backend_groups]
[backend_groups.main]
backends = ["infura"]

Note that all backends in the websocket backend group need a websocket URL.

The WS backend group is explicitly defined in the config rather than populated from the backends that have ws_urls because we may want the flexibility to create a custom group just for websocket-enabled endpoints in the future.

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Nov 9, 2021

🦋 Changeset detected

Latest commit: abe231b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@eth-optimism/proxyd Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@mslipper mslipper force-pushed the feat/alt-ws-server branch 5 times, most recently from 1cb5b1c to d36e6be Compare November 9, 2021 22:33
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Nov 9, 2021

Codecov Report

Merging #1730 (e61de1e) into develop (b3363ac) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop    #1730   +/-   ##
========================================
  Coverage    76.52%   76.52%           
========================================
  Files           82       82           
  Lines         3041     3041           
  Branches       466      466           
========================================
  Hits          2327     2327           
  Misses         714      714           
Flag Coverage Δ
batch-submitter 61.74% <ø> (ø)
contracts 86.05% <ø> (ø)
core-utils 63.27% <ø> (ø)
data-transport-layer 37.86% <ø> (ø)
message-relayer 83.48% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b3363ac...e61de1e. Read the comment docs.

Copy link
Copy Markdown
Contributor

@tynes tynes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a tabs/spaces thing?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea - I ran make fmt on it.

@mslipper mslipper changed the title Make endpoints match Geth, better logging go/proxyd: Make endpoints match Geth, better logging Nov 10, 2021
Copy link
Copy Markdown
Contributor

@optimisticben optimisticben left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing, fantastic work! 🚀 🚀

@tynes tynes merged commit 4096cd3 into ethereum-optimism:develop Nov 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants