nsc to connect nsmgr via tcp

[zolug]

Hi,

I noticed that nsmgr normally has a tcp listener specified as well besides the unix socket:
NSM_LISTEN_ON: unix:///var/lib/networkservicemesh/nsm.io.sock,tcp://:5001

I wonder if an NSC could leverage TCP to ask for a specific network service in the cluster?

I gave it a try basically relying on example basic, cmd-nsc and cmd-nse-icmp-responder.
The nsc managed to connect with the collocated nsmgr, but the request failed at the local vpp-forwarder:

Nov 14 16:21:09.637ESC[37m [TRAC] [id:8a75672a-f41b-42cd-9cdd-ea5eef310595] [type:networkService] ESC[0m(1.1)   request={"connection":{"id":"8a75672a-f41b-42cd-9cdd-ea5eef310595","network_service":"my-kernel-svc","context":{"ip_context":{"excluded_prefixes":["10.96.0.0/16","10.244.0.0/16"]}},"labels":{"nodeName":"kind-worker"},"path":{"index":1,"path_segments":[{"name":"nsc","id":"nsc-0","token":"eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJzcGlmZmU6Ly9leGFtcGxlLm9yZy9ucy9kZWZhdWx0L3NhL2RlZmF1bHQiLCJhdWQiOlsic3BpZmZlOi8vZXhhbXBsZS5vcmcvbnMvbnNtL3NhL25zbWdyLXNhIl0sImV4cCI6MTY2ODQ0MzQ2OX0.gCsW1Tm17HISOgUY0mdUHiEhaYPCYVp-Rrf6TASrlUY8voTSAPGj-w4_575tROZItytpExUj5k6mz2H1_lXRyg","expires":{"seconds":1668443469,"nanos":622149106}},{"name":"nsmgr-n4mzr","id":"8a75672a-f41b-42cd-9cdd-ea5eef310595","token":"eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJzcGlmZmU6Ly9leGFtcGxlLm9yZy9ucy9uc20vc2EvbnNtZ3Itc2EiLCJhdWQiOlsic3BpZmZlOi8vZXhhbXBsZS5vcmcvbnMvZGVmYXVsdC9zYS9kZWZhdWx0Il0sImV4cCI6MTY2ODQ0MzQ2OX0.rpT5iD1QMv86LAcg0ySuFYGhpFReMXb3LMgT0Psh4t4vQd6diOdcsq5pLtliIE8CQ-8JmsNe0CUFBBJE2LDKjg","expires":{"seconds":1668443469,"nanos":623643475}}]}},"mechanism_preferences":[{"cls":"LOCAL","type":"KERNEL","parameters":{"inodeURL":"inode://4/4026535708","name":"nsm-1"}}]}
...
Nov 14 16:21:09.642ESC[37m [TRAC] [id:7c5abadb-2f68-46c5-bcac-fd2efb386da6] [type:networkService] ESC[0m(9)         ⎆ sdk/pkg/networkservice/common/mechanisms/recvfd/recvFDServer.Request()
Nov 14 16:21:24.619ESC[31m [ERRO] [id:7c5abadb-2f68-46c5-bcac-fd2efb386da6] [type:networkService] ESC[0m(9.1)           timeout in recvfd waiting for inode://4/4026535708: context deadline exceeded
Nov 14 16:21:24.619ESC[37m [TRAC] [id:7c5abadb-2f68-46c5-bcac-fd2efb386da6] [type:networkService] ESC[0m(8.1)          request-response=null

NSM version: 1.6.1
Spire: 1.2.2
cmd-nsc: ghcr.io/networkservicemesh/ci/cmd-nsc:5de2e87
cmd-nse-icmp-responder: ghcr.io/networkservicemesh/ci/cmd-nse-icmp-responder:9e479a6

Reproduction:

• start Kind with 2 workers and a controller
• deployed nsm (with vpp-forwarder) in k8s namespace nsm.
• add a service to nsm k8s namespace with internalTrafficPolicy: Local for nsmgr port 5001 to force NSC to connect the collocated nsmgr:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: nsm-nsmgr-svc
  namespace: nsm
spec:
  selector:
    app: nsmgr
  ports:
    - name: nsm-nsmgr-svc
      protocol: TCP
      port: 5001
      targetPort: 5001
  internalTrafficPolicy: Local
EOF

• deploy cmd-nse-icmp-responder on node kind-worker:

cat <<EOF | kubectl apply -f -
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nse-kernel
  labels:
    app: nse-kernel
spec:
  selector:
    matchLabels:
      app: nse-kernel
  template:
    metadata:
      labels:
        app: nse-kernel
        "spiffe.io/spiffe-id": "true"
    spec:
      containers:
        - name: nse
          image: ghcr.io/networkservicemesh/ci/cmd-nse-icmp-responder:9e479a6
          imagePullPolicy: IfNotPresent
          env:
            - name: SPIFFE_ENDPOINT_SOCKET
              value: unix:///run/spire/sockets/agent.sock
            - name: NSM_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: NSM_LOG_LEVEL
              value: TRACE
            - name: NSM_CONNECT_TO
              value: unix:///var/lib/networkservicemesh/nsm.io.sock
            - name: NSM_CIDR_PREFIX
              value: 172.16.1.0/24
            - name: NSM_SERVICE_NAMES
              value: my-kernel-svc
          volumeMounts:
            - name: spire-agent-socket
              mountPath: /run/spire/sockets
              readOnly: true
            - name: nsm-socket
              mountPath: /var/lib/networkservicemesh
              readOnly: true
          resources:
            requests:
              cpu: 100m
              memory: 40Mi
            limits:
              memory: 80Mi
              cpu: 200m
      nodeSelector:
        kubernetes.io/hostname: kind-worker
      volumes:
        - name: spire-agent-socket
          hostPath:
            path: /run/spire/sockets
            type: Directory
        - name: nsm-socket
          hostPath:
            path: /var/lib/networkservicemesh
            type: DirectoryOrCreate
EOF

• deploy cmd-nsc on the same node as the NSE requesting service via kernel mechanism:

cat <<EOF | kubectl apply -f -
---
apiVersion: v1
kind: Pod
metadata:
  name: nsc
  labels:
    app: nsc  
    "spiffe.io/spiffe-id": "true"
spec:
  containers:
  - name: nsc
    image: ghcr.io/networkservicemesh/ci/cmd-nsc:5de2e87
    imagePullPolicy: Always
    env:
      - name: SPIFFE_ENDPOINT_SOCKET
        value: unix:///run/spire/sockets/agent.sock
      - name: NSM_LOG_LEVEL
        value: TRACE
      - name: NSM_NETWORK_SERVICES
        value: kernel://my-kernel-svc/nsm-1
      - name: NSM_NAME
        valueFrom:
          fieldRef:
            fieldPath: metadata.name
      - name: NSM_LIVENESSCHECKENABLED
        value: "false"
      - name: NSM_CONNECT_TO
        value: nsm-nsmgr-svc.nsm:5001
      - name: NSM_LOCALDNSSERVERENABLED
        value: "false"
    volumeMounts:
      - name: spire-agent-socket
        mountPath: /run/spire/sockets
        readOnly: true
      - name: nsm-socket
        mountPath: /var/lib/networkservicemesh
        readOnly: true
    resources:
      requests:
        cpu: 100m
        memory: 40Mi
      limits:
        memory: 80Mi
        cpu: 200m
  volumes:
    - name: spire-agent-socket
      hostPath:
        path: /run/spire/sockets
        type: Directory
    - name: nsm-socket
      hostPath:
        path: /var/lib/networkservicemesh
        type: DirectoryOrCreate
  nodeSelector:
    kubernetes.io/hostname: kind-worker
EOF