deny-all policy not working

[pkaramol]

What you expected to happen?

Block all traffic to a service of a specific namespace (i.e. staging)

What happened?

Traffic allowed from bothdefault and staging namespace

How to reproduce it?

Create the following namespace

kind: Namespace
apiVersion: v1
metadata:
  name: staging
  annotations:
    net.beta.kubernetes.io/network-policy: |
      {
        "ingress": {
          "isolation": "DefaultDeny"
        }
      }

Spin up a busybox for testing purposes in the default namespace:

kubectl run busybox --rm -ti --image=busybox /bin/sh  --namespace=default

...and my ui service (supposed to be listening on port 80) in staging namespace is reachable!

/ # wget --spider ui.staging.svc.cluster.local
Connecting to ui.staging.svc.cluster.local (100.68.222.37:80)

I have also tried to just create via command line a namespace (without) yml file

and apply the following presumably deny-all policy (didn't work either)

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: default-deny-all
  namespace: staging
spec:
  podSelector: {}
  ingress: []

Anything else we need to know?

Using aws.

Cluster deployed with

$ kops version
Version 1.8.0 (git-5099bc5)

... in private topology

Versions:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.1", GitCommit:"f38e43b221d08850172a9a4ea785a86a3ffa3b3a", GitTreeState:"clean", BuildDate:"2017-10-11T23:27:35Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.4", GitCommit:"9befc2b8928a9426501d3bf62f72849d5cbcd5a3", GitTreeState:"clean", BuildDate:"2017-11-20T05:17:43Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}

weaveworks/weave-npc                               2.0.5               55ed7c451d70        3 months ago        54.7 MB
weaveworks/weave-kube                              2.0.5               b73b5c64c5d3        3 months ago        101 MB

Logs:

$ kubectl logs -n kube-system <weave-net-pod> weave

weave logs are just alternations of

Discovered remote MAC
Expired MAC

I applied the network policy (or updated the existing namespace) after my application was deployed, just for the record.

[brb]

@pkaramol Can you please include logs of weave-npc (kubectl logs -n kube-system <weave-net-pod> weave-npc) and iptables-save output?

Also, keep in mind, that Weave Net 2.0.5 implements only legacy NetworkPolicy (extensions/v1beta1.NetworkPolicy), while since Weave Net 2.1.1 we support new NetworkPolicy (networkingv1.NetworkPolicy). So, in your case the default-deny-all netpol won't work. See https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.7.md#network for more details. The upcoming kops 1.8.1 version will include the latest Weave Net (2.1.3), so you will be able to use the latest NetworkPolicy.

[pkaramol]

@brb
Unfortunately I have brought my cluster down.
Nonetheless, as stated in my opening post I am quite sure I did not any specific output when fetching the wave pod logs, other than these two messages

Discovered remote MAC <some_mac_address>
Expired MAC <some_mac_address>

I believe however this should be an issue easy to reproduce should someone find 10-20 mins.

[brb]

Discovered remote MAC <some_mac_address>
Expired MAC <some_mac_address>

These are the logs of the weave Pod, but not the weave-npc Pod which implements Network Policy.

[brb]

I couldn't reproduce it. Please re-open if the issue persists.