doc: add interop-test description for orca#29632
Conversation
|
There's a related discussion to this in grpc/grpc-java#9079 . I think we need to make it a bit more generic. The approach right now is poorly suited to cross-language testing. |
| listener (configure load report interval to be 1s). The interop-testing client will run with a | ||
| service config to select the load balancing config (using argument `--service_config_json`), so that | ||
| it effectively uses this newly registered custom LB policy. A load report reference can be passed | ||
| from the call to the LB policy through, e.g. CallOptions, to receive metric reports. |
There was a problem hiding this comment.
How will the test framework interact with this load report reference to verify the resulting load reports?
There was a problem hiding this comment.
The test framework will fill in the reference with the load report callback API. The test case can then verify the reference has the result. I updated the doc to make it more explicit.
There was a problem hiding this comment.
Don't we have a common test framework for all languages? If so, how will the test framework fill in the reference for each language-specific client, given that each language will have a different API for this?
There was a problem hiding this comment.
The test framework doesn't do any assertions (other than assert return code is 0). This is non-xDS interop where we have assertions in each language. The test framework is involved to enable test_backend_metrics_load_balancer, but then it is per-language APIs for the test case to interact with that LB policy.
There was a problem hiding this comment.
Ah, okay. In that case, this makes sense. Thanks!
There was a problem hiding this comment.
yea each language should
- implement the
test_backend_metrics_load_balancerin interop test client - implement interoptest server
- implement test case described in test procedure.
Use
./run-test-client.sh --use_tls=false --test_case=orca_per_rpc --service_config_json='{"loadBalancingConfig":[{"test_backend_metrics_load_balancer":{}}]}'
to enable the loadbalancer. The client should pass with cross-language server: e.g. a java client and Go server runs:
./run-test-server.sh --use_tls=false
@markdroth @dfawley If things look good to you, i'll merge this PR and proto change today to unblock java impl.
| listener (configure load report interval to be 1s). The interop-testing client will run with a | ||
| service config to select the load balancing config (using argument `--service_config_json`), so that | ||
| it effectively uses this newly registered custom LB policy. A load report reference can be passed | ||
| from the call to the LB policy through, e.g. CallOptions, to receive metric reports. |
There was a problem hiding this comment.
The test framework doesn't do any assertions (other than assert return code is 0). This is non-xDS interop where we have assertions in each language. The test framework is involved to enable test_backend_metrics_load_balancer, but then it is per-language APIs for the test case to interact with that LB policy.
add cross language readme doc for orca integration test.
Next step: add internal CI for the tests