Skip to content

[PSM Interop] Gamma baseline test PoC#33504

Closed
sergiitk wants to merge 18 commits intogrpc:masterfrom
sergiitk:psm-interop-gamma-baseline
Closed

[PSM Interop] Gamma baseline test PoC#33504
sergiitk wants to merge 18 commits intogrpc:masterfrom
sergiitk:psm-interop-gamma-baseline

Conversation

@sergiitk
Copy link
Copy Markdown
Member

@sergiitk sergiitk commented Jun 21, 2023 •
edited
Loading

It's alive.

Running tests under Python 3.9.17: /Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/venv/bin/python
I0718 17:15:28.888420 140704716400192 xds_k8s_testcase.py:130] ----- Testing GammaBaselineTest -----
I0718 17:15:28.888513 140704716400192 xds_k8s_testcase.py:131] Logs timezone: PDT
I0718 17:15:28.888693 140704716400192 skips.py:124] Detected language and version: TestConfig(client_lang='java', server_lang='java', version=None)
I0718 17:15:29.962364 140704716400192 k8s.py:173] Using kubernetes context "sergii-gamma", active host: https://34.168.7.137
I0718 17:15:30.030090 140704716400192 k8s.py:173] Using kubernetes context "None", active host: https://34.168.7.137
[ RUN      ] GammaBaselineTest.test_ping_pong
I0718 17:15:30.039905 140704716400192 xds_k8s_testcase.py:626] Test run resource prefix: sergiitk, suffix: 20230719-0015-49qfm
I0718 17:15:32.883258 140704716400192 xds_k8s_testcase.py:240] --- Starting subTest __main__.GammaBaselineTest.test_ping_pong.1_run_test_server ---
I0718 17:15:32.883581 140704716400192 gamma_server_runner.py:107] Deploying GAMMA xDS test server "psm-grpc-server" to k8s namespace sergiitk-server-20230719-0015-49qfm: test_port=8080 maintenance_port=8080 secure_mode=False replica_count=1
I0718 17:15:32.883937 140704716400192 k8s_base_runner.py:261] Loading k8s manifest template: /Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/kubernetes-manifests/namespace.yaml
I0718 17:15:32.891043 140704716400192 k8s_base_runner.py:264] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
  name: sergiitk-server-20230719-0015-49qfm
  labels:
    name: sergiitk-server-20230719-0015-49qfm
    owner: xds-k8s-interop-test
...

I0718 17:15:33.011967 140704716400192 k8s_base_runner.py:284] Namespace sergiitk-server-20230719-0015-49qfm created
I0718 17:15:33.012229 140704716400192 k8s_base_runner.py:261] Loading k8s manifest template: /Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/kubernetes-manifests/gamma/tdmesh.yaml
I0718 17:15:33.015131 140704716400192 k8s_base_runner.py:264] Rendered template kubernetes-manifests/gamma/tdmesh.yaml:
---
kind: TDMesh
apiVersion: net.gke.io/v1alpha1
metadata:
  name: sergiitk-mesh-20230719-0015-49qfm
  namespace: sergiitk-server-20230719-0015-49qfm
  labels:
    owner: xds-k8s-interop-test
spec:
  gatewayClassName: gke-td
  allowedRoutes:
    namespaces:
      from: All
    kinds:
    - group: net.gke.io
      kind: TDGRPCRoute
...

I0718 17:15:33.074143 140704716400192 k8s_base_runner.py:284] TDMesh sergiitk-mesh-20230719-0015-49qfm created
I0718 17:15:33.074419 140704716400192 k8s_base_runner.py:261] Loading k8s manifest template: /Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/kubernetes-manifests/gamma/service.yaml
I0718 17:15:33.077228 140704716400192 k8s_base_runner.py:264] Rendered template kubernetes-manifests/gamma/service.yaml:
---
apiVersion: v1
kind: Service
metadata:
  name: psm-grpc-server
  namespace: sergiitk-server-20230719-0015-49qfm
  labels:
    owner: xds-k8s-interop-test
spec:
  type: ClusterIP
  selector:
    app: psm-grpc-server
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
...

I0718 17:15:33.142441 140704716400192 k8s_base_runner.py:284] Service psm-grpc-server created
I0718 17:15:33.142723 140704716400192 k8s_base_runner.py:261] Loading k8s manifest template: /Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/kubernetes-manifests/gamma/route_grpc.yaml
I0718 17:15:33.145455 140704716400192 k8s_base_runner.py:264] Rendered template kubernetes-manifests/gamma/route_grpc.yaml:
---
kind: GRPCRoute
apiVersion: gateway.networking.k8s.io/v1alpha2
metadata:
  name: route-psm-grpc-server
  namespace: sergiitk-server-20230719-0015-49qfm
  labels:
    owner: xds-k8s-interop-test
spec:
  parentRefs:
  - name: sergiitk-mesh-20230719-0015-49qfm
    namespace: sergiitk-server-20230719-0015-49qfm
    group: net.gke.io
    kind: TDMesh
  hostnames:
  - psm-grpc-server-20230719-0015-49qfm
  rules:
  - backendRefs:
    - name: psm-grpc-server
      port: 8080
      namespace: sergiitk-server-20230719-0015-49qfm
...

I0718 17:15:33.203818 140704716400192 k8s_base_runner.py:284] GRPCRoute route-psm-grpc-server created
I0718 17:15:33.203936 140704716400192 k8s_base_runner.py:673] Waiting for NEG for service psm-grpc-server
I0718 17:15:51.731146 140704716400192 k8s_base_runner.py:678] Service psm-grpc-server: detected NEG=k8s1-56c34c5f-sergiitk-server-20230719--psm-grpc-se-80-3a18cb21 in zones=['us-west1-a']
I0718 17:15:51.731341 140704716400192 k8s_base_runner.py:341] Granting roles/iam.workloadIdentityUser to serviceAccount:sergiitk-grpc-gke.svc.id.goog[sergiitk-server-20230719-0015-49qfm/psm-grpc-server] for GCP Service Account xds-k8s-interop-tests@sergiitk-grpc-gke.iam.gserviceaccount.com
I0718 17:15:52.512472 140704716400192 k8s_base_runner.py:261] Loading k8s manifest template: /Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/kubernetes-manifests/service-account.yaml
I0718 17:15:52.514675 140704716400192 k8s_base_runner.py:264] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: psm-grpc-server
  namespace: sergiitk-server-20230719-0015-49qfm
  labels:
    owner: xds-k8s-interop-test
  annotations:
    iam.gke.io/gcp-service-account: xds-k8s-interop-tests@sergiitk-grpc-gke.iam.gserviceaccount.com
...

I0718 17:15:52.563453 140704716400192 k8s_base_runner.py:284] ServiceAccount psm-grpc-server created
I0718 17:15:52.563828 140704716400192 k8s_base_runner.py:261] Loading k8s manifest template: /Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/kubernetes-manifests/server.deployment.yaml
I0718 17:15:52.574136 140704716400192 k8s_base_runner.py:264] Rendered template kubernetes-manifests/server.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: psm-grpc-server
  namespace: sergiitk-server-20230719-0015-49qfm
  labels:
    app: psm-grpc-server
    deployment_id: psm-grpc-server-z9qkgbpk
    owner: xds-k8s-interop-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: psm-grpc-server
      deployment_id: psm-grpc-server-z9qkgbpk
  template:
    metadata:
      labels:
        app: psm-grpc-server
        deployment_id: psm-grpc-server-z9qkgbpk
        owner: xds-k8s-interop-test
    spec:
      serviceAccountName: psm-grpc-server
      containers:
      - name: psm-grpc-server
        image: gcr.io/grpc-testing/xds-interop/java-server:v1.55.x
        imagePullPolicy: Always
        startupProbe:
          tcpSocket:
            port: 8080
          periodSeconds: 3
          failureThreshold: 1000
        args:
          - "--port=8080"
        ports:
          - containerPort: 8080
        env:
          - name: GRPC_XDS_BOOTSTRAP
            value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
          - name: GRPC_XDS_EXPERIMENTAL_V3_SUPPORT
            value: "true"
        volumeMounts:
          - mountPath: /tmp/grpc-xds/
            name: grpc-td-conf
            readOnly: true
        resources:
          limits:
            cpu: 800m
            memory: 512Mi
          requests:
            cpu: 100m
            memory: 512Mi
      initContainers:
        - name: grpc-td-init
          image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
          imagePullPolicy: Always
          args:
            - "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
            - "--vpc-network-name=default"
            - "--node-metadata=app=sergiitk-server-20230719-0015-49qfm-psm-grpc-server"
          resources:
            limits:
              cpu: 100m
              memory: 100Mi
            requests:
              cpu: 10m
              memory: 100Mi
          volumeMounts:
            - mountPath: /tmp/bootstrap/
              name: grpc-td-conf
      volumes:
        - name: grpc-td-conf
          emptyDir:
            medium: Memory
...

I0718 17:15:52.647361 140704716400192 k8s_base_runner.py:284] Deployment psm-grpc-server created
I0718 17:15:52.647485 140704716400192 k8s_base_runner.py:608] Waiting for deployment psm-grpc-server to initialize 1 pod(s)
I0718 17:15:52.727461 140704716400192 k8s_base_runner.py:618] Deployment psm-grpc-server initialized 1 pod(s): ['psm-grpc-server-dbfb45546-zgh4f']
I0718 17:15:52.727596 140704716400192 k8s_base_runner.py:628] Waiting for pod psm-grpc-server-dbfb45546-zgh4f to start
I0718 17:15:56.934978 140704716400192 k8s_base_runner.py:631] Pod psm-grpc-server-dbfb45546-zgh4f ready, IP: 10.118.1.72
I0718 17:15:56.935140 140704716400192 k8s_base_runner.py:590] Waiting for deployment psm-grpc-server to report 1 available replica(s)
I0718 17:16:03.209808 140704716400192 k8s_base_runner.py:599] Deployment psm-grpc-server has 1 replicas available
I0718 17:16:03.209944 140704716400192 k8s_base_runner.py:639] LOCAL DEV MODE: Enabling port forwarding to 10.118.1.72:8080
I0718 17:16:03.834635 140704716400192 k8s_port_forwarder.py:118] Forwarding from 127.0.0.1:49687 -> 8080
I0718 17:16:03.836701 140704716400192 xds_k8s_testcase.py:245] --- Finished subTest __main__.GammaBaselineTest.test_ping_pong.1_run_test_server ---
I0718 17:16:03.836954 140704716400192 xds_k8s_testcase.py:240] --- Starting subTest __main__.GammaBaselineTest.test_ping_pong.2_start_test_client ---
I0718 17:16:03.837235 140704716400192 k8s_xds_client_runner.py:107] Deploying xDS test client "psm-grpc-client" to k8s namespace sergiitk-client-20230719-0015-49qfm: server_target=xds:///psm-grpc-server-20230719-0015-49qfm rpc=UnaryCall qps=25 metadata='' secure_mode=False print_response=False
I0718 17:16:03.837713 140704716400192 k8s_base_runner.py:749] GCP Logs Explorer link to psm-grpc-client:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22sergiitk-grpc-gke%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22sergiitk-client-20230719-0015-49qfm%22;timeRange=2023-07-19T00:16:03.837452Z%2F2023-07-19T00:46:03.837452Z?project=sergiitk-grpc-gke
I0718 17:16:03.838325 140704716400192 k8s_base_runner.py:261] Loading k8s manifest template: /Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/kubernetes-manifests/namespace.yaml
I0718 17:16:03.841230 140704716400192 k8s_base_runner.py:264] Rendered template kubernetes-manifests/namespace.yaml:
---
apiVersion: v1
kind: Namespace
metadata:
  name: sergiitk-client-20230719-0015-49qfm
  labels:
    name: sergiitk-client-20230719-0015-49qfm
    owner: xds-k8s-interop-test
...

I0718 17:16:03.870648 140704716400192 k8s_base_runner.py:284] Namespace sergiitk-client-20230719-0015-49qfm created
I0718 17:16:03.870846 140704716400192 k8s_base_runner.py:341] Granting roles/iam.workloadIdentityUser to serviceAccount:sergiitk-grpc-gke.svc.id.goog[sergiitk-client-20230719-0015-49qfm/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@sergiitk-grpc-gke.iam.gserviceaccount.com
I0718 17:16:04.323599 140704716400192 k8s_base_runner.py:261] Loading k8s manifest template: /Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/kubernetes-manifests/service-account.yaml
I0718 17:16:04.326192 140704716400192 k8s_base_runner.py:264] Rendered template kubernetes-manifests/service-account.yaml:
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: psm-grpc-client
  namespace: sergiitk-client-20230719-0015-49qfm
  labels:
    owner: xds-k8s-interop-test
  annotations:
    iam.gke.io/gcp-service-account: xds-k8s-interop-tests@sergiitk-grpc-gke.iam.gserviceaccount.com
...

I0718 17:16:04.375728 140704716400192 k8s_base_runner.py:284] ServiceAccount psm-grpc-client created
I0718 17:16:04.376194 140704716400192 k8s_base_runner.py:261] Loading k8s manifest template: /Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/kubernetes-manifests/client.deployment.yaml
I0718 17:16:04.385794 140704716400192 k8s_base_runner.py:264] Rendered template kubernetes-manifests/client.deployment.yaml:
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: psm-grpc-client
  namespace: sergiitk-client-20230719-0015-49qfm
  labels:
    app: psm-grpc-client
    deployment_id: psm-grpc-client-1u4pmuss
    owner: xds-k8s-interop-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: psm-grpc-client
      deployment_id: psm-grpc-client-1u4pmuss
  template:
    metadata:
      labels:
        app: psm-grpc-client
        deployment_id: psm-grpc-client-1u4pmuss
        owner: xds-k8s-interop-test
    spec:
      serviceAccountName: psm-grpc-client
      containers:
      - name: psm-grpc-client
        image: gcr.io/grpc-testing/xds-interop/java-client:v1.56.x
        imagePullPolicy: Always
        startupProbe:
          tcpSocket:
            port: 8079
          periodSeconds: 3
          failureThreshold: 1000
        args:
          - "--server=xds:///psm-grpc-server-20230719-0015-49qfm"
          - "--stats_port=8079"
          - "--qps=25"
          - "--rpc=UnaryCall"
          - "--metadata="
          - "--print_response=False"
        ports:
          - containerPort: 8079
        env:
          - name: GRPC_XDS_BOOTSTRAP
            value: "/tmp/grpc-xds/td-grpc-bootstrap.json"
          - name: GRPC_XDS_EXPERIMENTAL_ENABLE_RING_HASH
            value: "true"
          - name: GRPC_XDS_EXPERIMENTAL_ENABLE_RETRY
            value: "true"
          - name: GRPC_EXPERIMENTAL_XDS_CUSTOM_LB_CONFIG
            value: "true"
          - name: GRPC_EXPERIMENTAL_ENABLE_OUTLIER_DETECTION
            value: "true"
        volumeMounts:
          - mountPath: /tmp/grpc-xds/
            name: grpc-td-conf
            readOnly: true
        resources:
          limits:
            cpu: 800m
            memory: 512Mi
          requests:
            cpu: 100m
            memory: 512Mi
      initContainers:
        - name: grpc-td-init
          image: gcr.io/grpc-testing/td-grpc-bootstrap:d6baaf7b0e0c63054ac4d9bedc09021ff261d599
          imagePullPolicy: Always
          args:
            - "--output=/tmp/bootstrap/td-grpc-bootstrap.json"
            - "--vpc-network-name=default"
            - "--config-mesh-experimental=gketd-sergiitk-mesh-20230719-0015-49qfm"
          resources:
            limits:
              cpu: 100m
              memory: 100Mi
            requests:
              cpu: 10m
              memory: 100Mi
          volumeMounts:
            - mountPath: /tmp/bootstrap/
              name: grpc-td-conf
      volumes:
        - name: grpc-td-conf
          emptyDir:
            medium: Memory
...

I0718 17:16:04.453361 140704716400192 k8s_base_runner.py:284] Deployment psm-grpc-client created
I0718 17:16:04.453492 140704716400192 k8s_base_runner.py:608] Waiting for deployment psm-grpc-client to initialize 1 pod(s)
I0718 17:16:04.536607 140704716400192 k8s_base_runner.py:618] Deployment psm-grpc-client initialized 1 pod(s): ['psm-grpc-client-5c6769fb54-nj86w']
I0718 17:16:04.536742 140704716400192 k8s_base_runner.py:628] Waiting for pod psm-grpc-client-5c6769fb54-nj86w to start
I0718 17:16:08.741082 140704716400192 k8s_base_runner.py:631] Pod psm-grpc-client-5c6769fb54-nj86w ready, IP: 10.118.1.73
I0718 17:16:08.741197 140704716400192 k8s_base_runner.py:590] Waiting for deployment psm-grpc-client to report 1 available replica(s)
I0718 17:16:11.912405 140704716400192 k8s_base_runner.py:599] Deployment psm-grpc-client has 1 replicas available
I0718 17:16:11.912571 140704716400192 k8s_base_runner.py:639] LOCAL DEV MODE: Enabling port forwarding to 10.118.1.73:8079
I0718 17:16:12.382785 140704716400192 k8s_port_forwarder.py:118] Forwarding from 127.0.0.1:49696 -> 8079
I0718 17:16:12.389114 140704716400192 client_app.py:188] [psm-grpc-client-5c6769fb54-nj86w] Waiting to report a READY channel to xds:///psm-grpc-server-20230719-0015-49qfm
I0718 17:16:16.445281 140704716400192 client_app.py:221] [psm-grpc-client-5c6769fb54-nj86w] Server channel: <Channel channel_id=5 target=xds:///psm-grpc-server-20230719-0015-49qfm state=IDLE>
I0718 17:16:26.479290 140704716400192 client_app.py:221] [psm-grpc-client-5c6769fb54-nj86w] Server channel: <Channel channel_id=5 target=xds:///psm-grpc-server-20230719-0015-49qfm state=TRANSIENT_FAILURE>
I0718 17:16:36.511304 140704716400192 client_app.py:221] [psm-grpc-client-5c6769fb54-nj86w] Server channel: <Channel channel_id=5 target=xds:///psm-grpc-server-20230719-0015-49qfm state=TRANSIENT_FAILURE>
I0718 17:16:46.538242 140704716400192 client_app.py:221] [psm-grpc-client-5c6769fb54-nj86w] Server channel: <Channel channel_id=5 target=xds:///psm-grpc-server-20230719-0015-49qfm state=TRANSIENT_FAILURE>
I0718 17:16:56.564682 140704716400192 client_app.py:221] [psm-grpc-client-5c6769fb54-nj86w] Server channel: <Channel channel_id=5 target=xds:///psm-grpc-server-20230719-0015-49qfm state=TRANSIENT_FAILURE>
I0718 17:17:12.595000 140704716400192 client_app.py:221] [psm-grpc-client-5c6769fb54-nj86w] Server channel: <Channel channel_id=5 target=xds:///psm-grpc-server-20230719-0015-49qfm state=TRANSIENT_FAILURE>
I0718 17:17:37.623841 140704716400192 client_app.py:221] [psm-grpc-client-5c6769fb54-nj86w] Server channel: <Channel channel_id=5 target=xds:///psm-grpc-server-20230719-0015-49qfm state=CONNECTING>
I0718 17:18:02.655199 140704716400192 client_app.py:221] [psm-grpc-client-5c6769fb54-nj86w] Server channel: <Channel channel_id=5 target=xds:///psm-grpc-server-20230719-0015-49qfm state=READY>
I0718 17:18:02.698497 140704716400192 client_app.py:234] [psm-grpc-client-5c6769fb54-nj86w] Found subchannel in state READY: <Subchannel subchannel_id=25 target=[[[/10.118.1.72:8080]/{io.grpc.xds.InternalXdsAttributes.localityWeight=1000000, io.grpc.xds.AddressFilter.PATH_CHAIN_KEY=null, io.grpc.xds.InternalXdsAttributes.clusterName=cloud-internal-istio:cloud_mp_1040920224690_4043754803713185215, io.grpc.xds.InternalXdsAttributes.locality=Locality{region=, zone=, subZone=pv:us-west1-a_4628828790806189527_neg}, io.grpc.xds.InternalXdsAttributes.serverWeight=1000000}]] state=READY>
I0718 17:18:02.698754 140704716400192 client_app.py:199] [psm-grpc-client-5c6769fb54-nj86w] Channel to xds:///psm-grpc-server-20230719-0015-49qfm transitioned to state READY: <Channel channel_id=5 target=xds:///psm-grpc-server-20230719-0015-49qfm state=READY>
I0718 17:18:02.698961 140704716400192 xds_k8s_testcase.py:245] --- Finished subTest __main__.GammaBaselineTest.test_ping_pong.2_start_test_client ---
I0718 17:18:02.699416 140704716400192 xds_k8s_testcase.py:240] --- Starting subTest __main__.GammaBaselineTest.test_ping_pong.3_test_server_received_rpcs_from_test_client ---
I0718 17:18:02.699965 140704716400192 grpc.py:72] [psm-grpc-client-5c6769fb54-nj86w:49696] >> RPC LoadBalancerStatsService.GetClientStats(request=LoadBalancerStatsRequest({'numRpcs': 100, 'timeoutSec': 1200}), wait_for_ready=True, timeout=1200)
I0718 17:18:06.707231 140704716400192 xds_k8s_testcase.py:559] [psm-grpc-client-5c6769fb54-nj86w] << Received LoadBalancerStatsResponse:
num_failures: 0
rpcs_by_peer:
  psm-grpc-server-dbfb45546-zgh4f: 100
rpcs_by_method:
  UnaryCall:
    psm-grpc-server-dbfb45546-zgh4f: 100

I0718 17:18:06.707433 140704716400192 xds_k8s_testcase.py:245] --- Finished subTest __main__.GammaBaselineTest.test_ping_pong.3_test_server_received_rpcs_from_test_client ---
I0718 17:18:06.707602 140704716400192 xds_k8s_testcase.py:645] ----- TestMethod __main__.GammaBaselineTest.test_ping_pong teardown -----
I0718 17:18:06.894940 140704716400192 k8s_base_runner.py:538] Deleting deployment psm-grpc-client
I0718 17:18:06.895068 140704716400192 k8s_port_forwarder.py:126] Shutting down port forwarding, pid 17162
I0718 17:18:06.898217 140704716400192 k8s_port_forwarder.py:131] Port forwarding stopped
I0718 17:18:17.029206 140704716400192 k8s_base_runner.py:360] Revoking roles/iam.workloadIdentityUser from serviceAccount:sergiitk-grpc-gke.svc.id.goog[sergiitk-client-20230719-0015-49qfm/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@sergiitk-grpc-gke.iam.gserviceaccount.com
I0718 17:18:17.613954 140704716400192 k8s_base_runner.py:564] Deleting service account psm-grpc-client
I0718 17:18:17.714504 140704716400192 k8s_base_runner.py:576] Deleting namespace sergiitk-client-20230719-0015-49qfm
I0718 17:18:47.843647 140704716400192 k8s_base_runner.py:505] Deleting GRPCRoute route-psm-grpc-server
I0718 17:18:47.947566 140704716400192 k8s_base_runner.py:493] Deleting GAMMA mesh sergiitk-mesh-20230719-0015-49qfm
I0718 17:19:48.539755 140704716400192 retryers.py:206] Retrying framework.xds_k8s_testcase.IsolatedXdsKubernetesTestCase.cleanup in 10.0 seconds as it raised RetryError: Retry error calling framework.infrastructure.k8s.KubernetesNamespace.get_gamma_mesh: timeout 0:01:00 (h:mm:ss) exceeded. Check result callback returned False..
I0718 17:19:58.540241 140704716400192 k8s_base_runner.py:538] Deleting deployment psm-grpc-client
I0718 17:19:58.561030 140704716400192 k8s_base_runner.py:543] Deployment psm-grpc-client deletion failed: Kubernetes API returned 404 Not Found: deployments.apps "psm-grpc-client" not found
I0718 17:19:58.561197 140704716400192 k8s_base_runner.py:360] Revoking roles/iam.workloadIdentityUser from serviceAccount:sergiitk-grpc-gke.svc.id.goog[sergiitk-client-20230719-0015-49qfm/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@sergiitk-grpc-gke.iam.gserviceaccount.com
I0718 17:19:58.731001 140704716400192 k8s_base_runner.py:564] Deleting service account psm-grpc-client
I0718 17:19:58.751422 140704716400192 k8s_base_runner.py:568] Service account psm-grpc-client deletion failed: Kubernetes API returned 404 Not Found: serviceaccounts "psm-grpc-client" not found
I0718 17:19:58.751564 140704716400192 k8s_base_runner.py:505] Deleting GRPCRoute route-psm-grpc-server
I0718 17:19:58.772300 140704716400192 k8s_base_runner.py:509] GRPCRoute route-psm-grpc-server deletion failed: Kubernetes API returned 404 Not Found: grpcroutes.gateway.networking.k8s.io "route-psm-grpc-server" not found
I0718 17:19:58.772423 140704716400192 k8s_base_runner.py:493] Deleting GAMMA mesh sergiitk-mesh-20230719-0015-49qfm
I0718 17:20:59.533383 140704716400192 retryers.py:206] Retrying framework.xds_k8s_testcase.IsolatedXdsKubernetesTestCase.cleanup in 10.0 seconds as it raised RetryError: Retry error calling framework.infrastructure.k8s.KubernetesNamespace.get_gamma_mesh: timeout 0:01:00 (h:mm:ss) exceeded. Check result callback returned False..
I0718 17:21:09.537684 140704716400192 k8s_base_runner.py:538] Deleting deployment psm-grpc-client
I0718 17:21:09.592813 140704716400192 k8s_base_runner.py:543] Deployment psm-grpc-client deletion failed: Kubernetes API returned 404 Not Found: deployments.apps "psm-grpc-client" not found
I0718 17:21:09.592936 140704716400192 k8s_base_runner.py:360] Revoking roles/iam.workloadIdentityUser from serviceAccount:sergiitk-grpc-gke.svc.id.goog[sergiitk-client-20230719-0015-49qfm/psm-grpc-client] for GCP Service Account xds-k8s-interop-tests@sergiitk-grpc-gke.iam.gserviceaccount.com
I0718 17:21:09.781543 140704716400192 k8s_base_runner.py:564] Deleting service account psm-grpc-client
I0718 17:21:09.832563 140704716400192 k8s_base_runner.py:568] Service account psm-grpc-client deletion failed: Kubernetes API returned 404 Not Found: serviceaccounts "psm-grpc-client" not found
I0718 17:21:09.832705 140704716400192 k8s_base_runner.py:505] Deleting GRPCRoute route-psm-grpc-server
I0718 17:21:09.884527 140704716400192 k8s_base_runner.py:509] GRPCRoute route-psm-grpc-server deletion failed: Kubernetes API returned 404 Not Found: grpcroutes.gateway.networking.k8s.io "route-psm-grpc-server" not found
I0718 17:21:09.884692 140704716400192 k8s_base_runner.py:493] Deleting GAMMA mesh sergiitk-mesh-20230719-0015-49qfm
E0718 17:22:10.469890 140704716400192 xds_k8s_testcase.py:667] Got error during teardown
Traceback (most recent call last):
  File "/Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/framework/xds_k8s_testcase.py", line 665, in tearDown
    retryer(self.cleanup)
  File "/Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/venv/lib/python3.9/site-packages/tenacity/__init__.py", line 423, in __call__
    do = self.iter(retry_state=retry_state)
  File "/Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/venv/lib/python3.9/site-packages/tenacity/__init__.py", line 369, in iter
    return self.retry_error_callback(retry_state=retry_state)
  File "/Users/sergiitk/Development/grpc/tools/run_tests/xds_k8s_test_driver/framework/helpers/retryers.py", line 141, in error_handler
    raise RetryError(
framework.helpers.retryers.RetryError: Retry error calling framework.xds_k8s_testcase.IsolatedXdsKubernetesTestCase.cleanup: 3 attempts exhausted. Last exception: RetryError: Retry error calling framework.infrastructure.k8s.KubernetesNamespace.get_gamma_mesh: timeout 0:01:00 (h:mm:ss) exceeded. Check result callback returned False.
I0718 17:22:10.473983 140704716400192 xds_k8s_testcase.py:669] ----- Test client/server logs -----
I0718 17:22:10.474282 140704716400192 k8s_base_runner.py:749] GCP Logs Explorer link to psm-grpc-client-1u4pmuss:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22sergiitk-grpc-gke%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22sergiitk-client-20230719-0015-49qfm%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-1u4pmuss%22;timeRange=2023-07-19T00:16:03.837408Z%2F2023-07-19T00:18:47.843590Z?project=sergiitk-grpc-gke
I0718 17:22:10.474583 140704716400192 k8s_base_runner.py:749] GCP Logs Explorer link to psm-grpc-client-1u4pmuss:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22sergiitk-grpc-gke%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22sergiitk-client-20230719-0015-49qfm%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-1u4pmuss%22;timeRange=2023-07-19T00:16:03.837408Z%2F2023-07-19T00:19:58.751541Z?project=sergiitk-grpc-gke
I0718 17:22:10.474887 140704716400192 k8s_base_runner.py:749] GCP Logs Explorer link to psm-grpc-client-1u4pmuss:
https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22sergiitk-grpc-gke%22%0Aresource.labels.container_name%3D%22psm-grpc-client%22%0Aresource.labels.namespace_name%3D%22sergiitk-client-20230719-0015-49qfm%22%0Alabels.%22k8s-pod%2Fdeployment_id%22%3D%22psm-grpc-client-1u4pmuss%22;timeRange=2023-07-19T00:16:03.837408Z%2F2023-07-19T00:21:09.832684Z?project=sergiitk-grpc-gke
I0718 17:22:10.475039 140704716400192 k8s_base_runner.py:698] No completed deployments of psm-grpc-server
[       OK ] GammaBaselineTest.test_ping_pong
----------------------------------------------------------------------
Ran 1 test in 401.440s

OK

There's some cleanup errors at the end, but everything else works great!

The log with ansi colors: gamma.log

@sergiitk sergiitk added disposition/DO NOT MERGE release notes: no Indicates if PR should not be in release notes area/psm interop labels Jun 21, 2023
@sergiitk sergiitk force-pushed the psm-interop-gamma-baseline branch from a265ad1 to 86f8915 Compare June 26, 2023 21:35
@sergiitk sergiitk changed the title [Psm Interop] Gamma baseline test PoC [PSM Interop] Gamma baseline test PoC Jun 27, 2023
@sergiitk sergiitk force-pushed the psm-interop-gamma-baseline branch 2 times, most recently from 9e25881 to ca19484 Compare July 10, 2023 21:26
@sergiitk sergiitk force-pushed the psm-interop-gamma-baseline branch from e4ec041 to cd9dd61 Compare July 17, 2023 17:53
@sergiitk sergiitk force-pushed the psm-interop-gamma-baseline branch from 0a035f3 to 81e112c Compare July 18, 2023 01:42
@sergiitk sergiitk requested a review from gnossen July 19, 2023 00:32
@sergiitk sergiitk marked this pull request as ready for review July 19, 2023 00:34
@sergiitk sergiitk mentioned this pull request Aug 25, 2023
Merged
sergiitk added a commit that referenced this pull request Aug 25, 2023
@sergiitk
[PSM Interop] Initial support for GAMMA tests (#34151)
2c5abd3 
Adds initial support for K8s
[GAMMA](https://gateway-api.sigs.k8s.io/concepts/gamma/) (Gateway API
for Service Mesh) initiative.

- Add framework support for loading CRD-based APIs using k8s python
dynamic client
- Add basic mesh baseline test (aka ping-pong) using GAMMA setup
- Implement initial framework changes needed to run PSM tests on
GAMMA-enabled cluster using
[TDMesh](https://cloud.google.com/traffic-director/docs/gke-gateway-overview#gateway-api)
and GRPCRoute.

Based on #33504.
@sergiitk
Copy link
Copy Markdown
Member Author

sergiitk commented Aug 25, 2023

Superceed by #34151.

@sergiitk sergiitk closed this Aug 25, 2023
sergiitk added a commit to grpc/psm-interop that referenced this pull request Nov 8, 2023
@sergiitk
[PSM Interop] Initial support for GAMMA tests (#34151)
40724df 
Adds initial support for K8s
[GAMMA](https://gateway-api.sigs.k8s.io/concepts/gamma/) (Gateway API
for Service Mesh) initiative.

- Add framework support for loading CRD-based APIs using k8s python
dynamic client
- Add basic mesh baseline test (aka ping-pong) using GAMMA setup
- Implement initial framework changes needed to run PSM tests on
GAMMA-enabled cluster using
[TDMesh](https://cloud.google.com/traffic-director/docs/gke-gateway-overview#gateway-api)
and GRPCRoute.

Based on grpc/grpc#33504.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gnossen gnossen Awaiting requested review from gnossen

Assignees

No one assigned

Labels

area/psm interop disposition/DO NOT MERGE release notes: no Indicates if PR should not be in release notes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@sergiitk