Skip to content

Flaky test: ErrorFromParentLB_ResourceNotFound #8803

@arjan-bal

Description

@arjan-bal

Example failure: https://github.com/grpc/grpc-go/actions/runs/20744847668/job/59559610610?pr=8802

Logs
--- FAIL: Test (14.06s)
    --- FAIL: Test/ErrorFromParentLB_ResourceNotFound (5.01s)
        balancer_test.go:179: Created new snapshot cache...
        tlogger.go:133: INFO server.go:717 [core] [Server #405] Server created  (t=+283.071µs)
        balancer_test.go:179: Registered Aggregated Discovery Service (ADS)...
        balancer_test.go:179: xDS management server serving at: 127.0.0.1:44629...
        tlogger.go:133: INFO server.go:913 [core] [Server #405 ListenSocket #406] ListenSocket created  (t=+785.634µs)
        tlogger.go:133: INFO server.go:717 [core] [Server #407] Server created  (t=+1.615151ms)
        tlogger.go:133: INFO server.go:913 [core] [Server #407 ListenSocket #408] ListenSocket created  (t=+1.720709ms)
        stubserver.go:300: Started test service backend at "127.0.0.1:36899"
        server.go:229: Created new resource snapshot...
        logging.go:30: setting snapshot for node 1de034c9-4b1e-4d8a-a5a0-6acc79b3c7dc
        server.go:235: Updated snapshot cache with resource snapshot...
        tlogger.go:133: INFO clientconn.go:1799 [core] original dial target is: "xds:///listener-my-service-client-side-xds"  (t=+1.905145ms)
        tlogger.go:133: INFO clientconn.go:514 [core] [Channel #409] Channel created for target "xds:///listener-my-service-client-side-xds"  (t=+1.959637ms)
        tlogger.go:133: INFO clientconn.go:245 [core] [Channel #409] parsed dial target is: resolver.Target{URL:url.URL{Scheme:"xds", Opaque:"", User:(*url.Userinfo)(nil), Host:"", Path:"/listener-my-service-client-side-xds", RawPath:"", OmitHost:false, ForceQuery:false, RawQuery:"", Fragment:"", RawFragment:""}}  (t=+1.98777ms)
        tlogger.go:133: INFO clientconn.go:246 [core] [Channel #409] Channel authority set to "listener-my-service-client-side-xds"  (t=+2.025441ms)
        tlogger.go:133: INFO clientconn.go:618 [core] [Channel #409] Channel Connectivity change to CONNECTING  (t=+2.069333ms)
        tlogger.go:133: INFO pool.go:309 [xds] xDS node ID: 1de034c9-4b1e-4d8a-a5a0-6acc79b3c7dc  (t=+2.362623ms)
        tlogger.go:133: INFO xds_resolver.go:152 [xds] [xds-resolver 0xc000a10700] Creating resolver for target: xds:///listener-my-service-client-side-xds  (t=+2.425111ms)
        tlogger.go:133: INFO clientconn.go:1799 [core] original dial target is: "passthrough:///127.0.0.1:44629"  (t=+2.505041ms)
        tlogger.go:133: INFO clientconn.go:514 [core] [Channel #410] Channel created for target "passthrough:///127.0.0.1:44629"  (t=+2.528835ms)
        tlogger.go:133: INFO clientconn.go:245 [core] [Channel #410] parsed dial target is: resolver.Target{URL:url.URL{Scheme:"passthrough", Opaque:"", User:(*url.Userinfo)(nil), Host:"", Path:"/127.0.0.1:44629", RawPath:"", OmitHost:false, ForceQuery:false, RawQuery:"", Fragment:"", RawFragment:""}}  (t=+2.551478ms)
        tlogger.go:133: INFO clientconn.go:246 [core] [Channel #410] Channel authority set to "127.0.0.1:44629"  (t=+2.582836ms)
        tlogger.go:133: INFO clientconn.go:618 [core] [Channel #410] Channel Connectivity change to CONNECTING  (t=+2.718621ms)
        tlogger.go:133: INFO resolver_wrapper.go:211 [core] [Channel #410] Resolver state updated: {
              "Addresses": [
                {
                  "Addr": "127.0.0.1:44629",
                  "ServerName": "",
                  "Attributes": null,
                  "BalancerAttributes": null,
                  "Metadata": null
                }
              ],
              "Endpoints": [
                {
                  "Addresses": [
                    {
                      "Addr": "127.0.0.1:44629",
                      "ServerName": "",
                      "Attributes": null,
                      "BalancerAttributes": null,
                      "Metadata": null
                    }
                  ],
                  "Attributes": null
                }
              ],
              "ServiceConfig": null,
              "Attributes": null
            } (resolver returned new addresses)  (t=+2.836192ms)
        tlogger.go:133: INFO balancer_wrapper.go:121 [core] [Channel #410] Channel switches to new LB policy "pick_first"  (t=+2.875255ms)
        tlogger.go:133: INFO clientconn.go:921 [core] [Channel #410 SubChannel #411] Subchannel created  (t=+2.926702ms)
        tlogger.go:133: INFO clientconn.go:419 [core] [Channel #410] Channel exiting idle mode  (t=+2.950797ms)
        tlogger.go:133: INFO clientconn.go:1296 [core] [Channel #410 SubChannel #411] Subchannel Connectivity change to CONNECTING  (t=+2.999418ms)
        logging.go:30: open watch 13 for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment map[endpoints-listener-my-service-client-side-xds:{}] from nodeID "1de034c9-4b1e-4d8a-a5a0-6acc79b3c7dc", version "3"
        tlogger.go:133: INFO clientconn.go:1296 [core] [Channel #409 SubChannel #417] Subchannel Connectivity change to READY  (t=+12.705241ms)
        tlogger.go:133: INFO balancergroup.go:510 [xds] [weighted-target-lb 0xc00086eee0] Balancer state update from child {region="region-1", zone="zone-1", sub_zone="subzone-1"}, new state: {ConnectivityState:READY Picker:0xc0009cee40}  (t=+12.757449ms)
        tlogger.go:133: INFO aggregator.go:253 [xds] [weighted-target-lb 0xc00086eee0] Child pickers with config: map[{region="region-1", zone="zone-1", sub_zone="subzone-1"}:weight:1,picker:0xc0009cee40,state:READY,stateToAggregate:READY]  (t=+12.780002ms)
        tlogger.go:133: INFO balancergroup.go:510 [xds] [priority-lb 0xc0005a6e00] Balancer state update from child priority-0-0, new state: {ConnectivityState:READY Picker:0xc0006a2d68}  (t=+12.801402ms)
        tlogger.go:133: INFO balancergroup.go:510 [xds] [xds-cluster-manager-lb 0xc000074c60] Balancer state update from child cluster:cluster-listener-my-service-client-side-xds, new state: {ConnectivityState:READY Picker:0xc0006a2d68}  (t=+12.817752ms)
        tlogger.go:133: INFO balancerstateaggregator.go:147 [xds] [xds-cluster-manager-lb 0xc000074c60] State update from sub-balancer "cluster:cluster-listener-my-service-client-side-xds": {ConnectivityState:READY Picker:0xc0006a2d68}  (t=+12.83261ms)
        tlogger.go:133: INFO balancerstateaggregator.go:191 [xds] [xds-cluster-manager-lb 0xc000074c60] Child pickers: map[cluster:cluster-listener-my-service-client-side-xds:picker:0xc0006a2d68,state:READY,stateToAggregate:READY]  (t=+12.847689ms)
        tlogger.go:133: INFO clientconn.go:618 [core] [Channel #409] Channel Connectivity change to READY  (t=+12.866894ms)
        balancer_test.go:288: Timeout when waiting for EDS resource to be requested
        tlogger.go:133: INFO clientconn.go:618 [core] [Channel #409] Channel Connectivity change to SHUTDOWN  (t=+5.005803112s)
        tlogger.go:133: INFO resolver_wrapper.go:112 [core] [Channel #409] Closing the name resolver  (t=+5.005831325s)
        tlogger.go:133: INFO balancer_wrapper.go:159 [core] [Channel #409] ccBalancerWrapper: closing  (t=+5.005868154s)
        tlogger.go:129: WARNING ads_stream.go:402 [xds] [xds-client 0xc000676c60] [xds-channel 0xc0008107d0] [ads-stream 0xc0006445a0] Sending ADS request for type "type.googleapis.com/envoy.config.listener.v3.Listener", resources: [], version: "3", nonce: "10" failed: EOF  (t=+5.005955649s)
        tlogger.go:133: INFO clusterimpl.go:360 [xds] [xds-cluster-impl-lb 0xc0000c7100] Shutdown  (t=+5.00597248s)
        tlogger.go:133: INFO balancergroup.go:341 [xds] [priority-lb 0xc0005a6e00] Removing child policy for child "priority-0-0"  (t=+5.005994762s)
        tlogger.go:133: INFO clusterresolver.go:354 [xds] [xds-cluster-resolver-lb 0xc0006aa000] Shutdown  (t=+5.006007376s)
        tlogger.go:133: INFO cdsbalancer.go:411 [xds] [cds-lb 0xc0004f7680] Shutdown  (t=+5.006019428s)
        tlogger.go:133: INFO clustermanager.go:196 [xds] [xds-cluster-manager-lb 0xc000074c60] Shutdown  (t=+5.006033194s)
        tlogger.go:129: WARNING ads_stream.go:441 [xds] [xds-client 0xc000676c60] [xds-channel 0xc0008107d0] [ads-stream 0xc0006445a0] ADS stream closed: rpc error: code = Canceled desc = context canceled  (t=+5.00604678s)
        tlogger.go:133: INFO ads_stream.go:160 [xds] [xds-client 0xc000676c60] [xds-channel 0xc0008107d0] [ads-stream 0xc0006445a0] Shutdown ADS stream  (t=+5.006070354s)
        tlogger.go:133: INFO clientconn.go:618 [core] [Channel #410] Channel Connectivity change to SHUTDOWN  (t=+5.006103666s)
        tlogger.go:133: INFO resolver_wrapper.go:112 [core] [Channel #410] Closing the name resolver  (t=+5.006119717s)
        tlogger.go:133: INFO balancer_wrapper.go:159 [core] [Channel #410] ccBalancerWrapper: closing  (t=+5.006139944s)
        tlogger.go:133: INFO clientconn.go:1296 [core] [Channel #410 SubChannel #411] Subchannel Connectivity change to SHUTDOWN  (t=+5.006198244s)
        tlogger.go:133: INFO clientconn.go:1672 [core] [Channel #410 SubChannel #411] Subchannel deleted  (t=+5.006216438s)
        tlogger.go:133: INFO clientconn.go:1231 [core] [Channel #410] Channel deleted  (t=+5.006344619s)
        tlogger.go:133: INFO channel.go:142 [xds] [xds-client 0xc000676c60] [xds-channel 0xc0008107d0] Shutdown  (t=+5.006370126s)
        tlogger.go:133: INFO xdsclient.go:200 [xds] [xds-client 0xc000676c60] Shutdown  (t=+5.006389873s)
        tlogger.go:133: INFO xds_resolver.go:254 [xds] [xds-resolver 0xc000a10700] Shutdown  (t=+5.006406194s)
        tlogger.go:133: INFO clientconn.go:1296 [core] [Channel #409 SubChannel #417] Subchannel Connectivity change to SHUTDOWN  (t=+5.006450357s)
        tlogger.go:133: INFO clientconn.go:1672 [core] [Channel #409 SubChannel #417] Subchannel deleted  (t=+5.006466487s)
        tlogger.go:133: INFO clientconn.go:1231 [core] [Channel #409] Channel deleted  (t=+5.006585781s)
        tlogger.go:133: INFO server.go:849 [core] [Server #407 ListenSocket #408] ListenSocket deleted  (t=+5.006644181s)
        tlogger.go:133: INFO server.go:849 [core] [Server #405 ListenSocket #406] ListenSocket deleted  (t=+5.006725573s)

Metadata

Metadata

Labels

Area: xDSIncludes everything xDS related, including LB policies used with xDS.Type: Bug

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions