Multinet Static IP Allocation

[infinitydon]

Is your feature request related to a problem? Please describe.
There are some Telco CNFs that require persistent IP allocation, an example will be the gateway functionality of 5G UPF, whereby UEs require a default gateway to route the IP traffic to the internet/corporate network.

Describe the solution you'd like
Have the following network config:

apiVersion: projectcalico.org/v3
kind: Network
metadata:
  name: green
spec:
  vni: 89
  range: "172.22.0.0/16"
---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: network-green-conf
spec:
  config: |-
    {
      "name": "network-green",
      "cniVersion": "0.3.0",
      "plugins": [
        {
          "type": "calico",
          "log_level": "debug",
          "datastore_type": "kubernetes",
          "mtu": 0,
          "ipam": {
              "type": "static"
          },
          "policy": {
              "type": "k8s"
          },
          "kubernetes": {
              "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"
          },
          "dataplane_options": {
            "type": "grpc",
            "socket": "unix:///var/run/calico/cni-server.sock",
            "network_name": "green"
          }
        }
      ]
    } 

Sample pod template:

apiVersion: v1
kind: Pod
metadata:
  name: memifpod-2
  annotations:
    k8s.v1.cni.cncf.io/networks: |
      [
        {
          "name": "network-green-conf",
          "interface": "memif1",
          "ips": ["172.22.0.1"]
        }
      ]    
spec:
  containers:
  - name: mvpp
    image: calicovpp/vpp:latest
    command: ["tail", "-f", "/dev/null"]
    imagePullPolicy: IfNotPresent
    securityContext:
      privileged: true

Describe alternatives you've considered
I have tried the above context but it's not working, keeps giving errors:

  Warning  FailedCreatePodSandBox  0s    kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "4fcdac839a327b8d2e455131323df039fbe9051fc5239ff65d6963497c609941" network for pod "memifpod-2": networkPlugin cni failed to set up pod "memifpod-2_default" network: CmdAdd (shim): CNI request failed with status 400: '&{ContainerID:4fcdac839a327b8d2e455131323df039fbe9051fc5239ff65d6963497c609941 Netns:/proc/19355/ns/net IfName:eth0 Args:IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=memifpod-2;K8S_POD_INFRA_CONTAINER_ID=4fcdac839a327b8d2e455131323df039fbe9051fc5239ff65d6963497c609941 Path: StdinData:[123 34 99 97 112 97 98 105 108 105 116 105 101 115 34 58 123 34 98 97 110 100 119 105 100 116 104 34 58 116 114 117 101 44 34 112 111 114 116 77 97 112 112 105 110 103 115 34 58 116 114 117 101 125 44 34 99 108 117 115 116 101 114 78 101 116 119 111 114 107 34 58 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 47 49 48 45 99 97 108 105 99 111 46 99 111 110 102 108 105 115 116 34 44 34 99 110 105 67 111 110 102 105 103 68 105 114 34 58 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 34 44 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 51 46 49 34 44 34 108 111 103 76 101 118 101 108 34 58 34 118 101 114 98 111 115 101 34 44 34 108 111 103 84 111 83 116 100 101 114 114 34 58 116 114 117 101 44 34 110 97 109 101 34 58 34 109 117 108 116 117 115 45 99 110 105 45 110 101 116 119 111 114 107 34 44 34 114 117 110 116 105 109 101 67 111 110 102 105 103 34 58 123 34 112 111 114 116 77 97 112 112 105 110 103 115 34 58 91 93 125 44 34 115 111 99 107 101 116 68 105 114 34 58 34 47 114 117 110 47 109 117 108 116 117 115 47 34 44 34 116 121 112 101 34 58 34 109 117 108 116 117 115 45 115 104 105 109 34 44 10 32 32 32 32 34 99 104 114 111 111 116 68 105 114 34 58 32 34 47 104 111 115 116 114 111 111 116 34 44 10 32 32 32 32 34 99 111 110 102 68 105 114 34 58 32 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 34 44 10 32 32 32 32 34 108 111 103 84 111 83 116 100 101 114 114 34 58 32 116 114 117 101 44 10 32 32 32 32 34 108 111 103 76 101 118 101 108 34 58 32 34 100 101 98 117 103 34 44 10 32 32 32 32 34 108 111 103 70 105 108 101 34 58 32 34 47 116 109 112 47 109 117 108 116 117 115 46 108 111 103 34 44 10 32 32 32 32 34 98 105 110 68 105 114 34 58 32 34 47 111 112 116 47 99 110 105 47 98 105 110 34 44 10 32 32 32 32 34 99 110 105 68 105 114 34 58 32 34 47 118 97 114 47 108 105 98 47 99 110 105 47 109 117 108 116 117 115 34 44 10 32 32 32 32 34 115 111 99 107 101 116 68 105 114 34 58 32 34 47 104 111 115 116 47 114 117 110 47 109 117 108 116 117 115 47 34 10 125 10]} {ContainerID:4fcdac839a327b8d2e455131323df039fbe9051fc5239ff65d6963497c609941 Netns:/proc/19355/ns/net IfName:eth0 Args:IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=memifpod-2;K8S_POD_INFRA_CONTAINER_ID=4fcdac839a327b8d2e455131323df039fbe9051fc5239ff65d6963497c609941 Path: StdinData:[123 34 99 97 112 97 98 105 108 105 116 105 101 115 34 58 123 34 98 97 110 100 119 105 100 116 104 34 58 116 114 117 101 44 34 112 111 114 116 77 97 112 112 105 110 103 115 34 58 116 114 117 101 125 44 34 99 108 117 115 116 101 114 78 101 116 119 111 114 107 34 58 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 47 49 48 45 99 97 108 105 99 111 46 99 111 110 102 108 105 115 116 34 44 34 99 110 105 67 111 110 102 105 103 68 105 114 34 58 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 34 44 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 51 46 49 34 44 34 108 111 103 76 101 118 101 108 34 58 34 118 101 114 98 111 115 101 34 44 34 108 111 103 84 111 83 116 100 101 114 114 34 58 116 114 117 101 44 34 110 97 109 101 34 58 34 109 117 108 116 117 115 45 99 110 105 45 110 101 116 119 111 114 107 34 44 34 114 117 110 116 105 109 101 67 111 110 102 105 103 34 58 123 34 112 111 114 116 77 97 112 112 105 110 103 115 34 58 91 93 125 44 34 115 111 99 107 101 116 68 105 114 34 58 34 47 114 117 110 47 109 117 108 116 117 115 47 34 44 34 116 121 112 101 34 58 34 109 117 108 116 117 115 45 115 104 105 109 34 44 10 32 32 32 32 34 99 104 114 111 111 116 68 105 114 34 58 32 34 47 104 111 115 116 114 111 111 116 34 44 10 32 32 32 32 34 99 111 110 102 68 105 114 34 58 32 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 34 44 10 32 32 32 32 34 108 111 103 84 111 83 116 100 101 114 114 34 58 32 116 114 117 101 44 10 32 32 32 32 34 108 111 103 76 101 118 101 108 34 58 32 34 100 101 98 117 103 34 44 10 32 32 32 32 34 108 111 103 70 105 108 101 34 58 32 34 47 116 109 112 47 109 117 108 116 117 115 46 108 111 103 34 44 10 32 32 32 32 34 98 105 110 68 105 114 34 58 32 34 47 111 112 116 47 99 110 105 47 98 105 110 34 44 10 32 32 32 32 34 99 110 105 68 105 114 34 58 32 34 47 118 97 114 47 108 105 98 47 99 110 105 47 109 117 108 116 117 115 34 44 10 32 32 32 32 34 115 111 99 107 101 116 68 105 114 34 58 32 34 47 104 111 115 116 47 114 117 110 47 109 117 108 116 117 115 47 34 10 125 10]} ERRORED: error configuring pod [default/memifpod-2] networking: [default/memifpod-2/:network-green]: error adding container to network "network-green": plugin type="calico" failed (add): invalid CIDR 172.22.0.1: invalid CIDR address: 172.22.0.1

Additional context
None.

[infinitydon]

if I make the IP with /32, it gives the following error:

" network: CmdAdd (shim): CNI request failed with status 400: '&{ContainerID:ed252910bd9e6bbaad3d96a48e9865010784c42e96ede56ed94d90e801dcac4e Netns:/proc/9231/ns/net IfName:eth0 Args:IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=memifpod-2;K8S_POD_INFRA_CONTAINER_ID=ed252910bd9e6bbaad3d96a48e9865010784c42e96ede56ed94d90e801dcac4e Path: StdinData:[123 34 99 97 112 97 98 105 108 105 116 105 101 115 34 58 123 34 98 97 110 100 119 105 100 116 104 34 58 116 114 117 101 44 34 112 111 114 116 77 97 112 112 105 110 103 115 34 58 116 114 117 101 125 44 34 99 108 117 115 116 101 114 78 101 116 119 111 114 107 34 58 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 47 49 48 45 99 97 108 105 99 111 46 99 111 110 102 108 105 115 116 34 44 34 99 110 105 67 111 110 102 105 103 68 105 114 34 58 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 34 44 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 51 46 49 34 44 34 108 111 103 76 101 118 101 108 34 58 34 118 101 114 98 111 115 101 34 44 34 108 111 103 84 111 83 116 100 101 114 114 34 58 116 114 117 101 44 34 110 97 109 101 34 58 34 109 117 108 116 117 115 45 99 110 105 45 110 101 116 119 111 114 107 34 44 34 114 117 110 116 105 109 101 67 111 110 102 105 103 34 58 123 34 112 111 114 116 77 97 112 112 105 110 103 115 34 58 91 93 125 44 34 115 111 99 107 101 116 68 105 114 34 58 34 47 114 117 110 47 109 117 108 116 117 115 47 34 44 34 116 121 112 101 34 58 34 109 117 108 116 117 115 45 115 104 105 109 34 44 10 32 32 32 32 34 99 104 114 111 111 116 68 105 114 34 58 32 34 47 104 111 115 116 114 111 111 116 34 44 10 32 32 32 32 34 99 111 110 102 68 105 114 34 58 32 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 34 44 10 32 32 32 32 34 108 111 103 84 111 83 116 100 101 114 114 34 58 32 116 114 117 101 44 10 32 32 32 32 34 108 111 103 76 101 118 101 108 34 58 32 34 100 101 98 117 103 34 44 10 32 32 32 32 34 108 111 103 70 105 108 101 34 58 32 34 47 116 109 112 47 109 117 108 116 117 115 46 108 111 103 34 44 10 32 32 32 32 34 98 105 110 68 105 114 34 58 32 34 47 111 112 116 47 99 110 105 47 98 105 110 34 44 10 32 32 32 32 34 99 110 105 68 105 114 34 58 32 34 47 118 97 114 47 108 105 98 47 99 110 105 47 109 117 108 116 117 115 34 44 10 32 32 32 32 34 115 111 99 107 101 116 68 105 114 34 58 32 34 47 104 111 115 116 47 114 117 110 47 109 117 108 116 117 115 47 34 10 125 10]} {ContainerID:ed252910bd9e6bbaad3d96a48e9865010784c42e96ede56ed94d90e801dcac4e Netns:/proc/9231/ns/net IfName:eth0 Args:IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=memifpod-2;K8S_POD_INFRA_CONTAINER_ID=ed252910bd9e6bbaad3d96a48e9865010784c42e96ede56ed94d90e801dcac4e Path: StdinData:[123 34 99 97 112 97 98 105 108 105 116 105 101 115 34 58 123 34 98 97 110 100 119 105 100 116 104 34 58 116 114 117 101 44 34 112 111 114 116 77 97 112 112 105 110 103 115 34 58 116 114 117 101 125 44 34 99 108 117 115 116 101 114 78 101 116 119 111 114 107 34 58 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 47 49 48 45 99 97 108 105 99 111 46 99 111 110 102 108 105 115 116 34 44 34 99 110 105 67 111 110 102 105 103 68 105 114 34 58 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 34 44 34 99 110 105 86 101 114 115 105 111 110 34 58 34 48 46 51 46 49 34 44 34 108 111 103 76 101 118 101 108 34 58 34 118 101 114 98 111 115 101 34 44 34 108 111 103 84 111 83 116 100 101 114 114 34 58 116 114 117 101 44 34 110 97 109 101 34 58 34 109 117 108 116 117 115 45 99 110 105 45 110 101 116 119 111 114 107 34 44 34 114 117 110 116 105 109 101 67 111 110 102 105 103 34 58 123 34 112 111 114 116 77 97 112 112 105 110 103 115 34 58 91 93 125 44 34 115 111 99 107 101 116 68 105 114 34 58 34 47 114 117 110 47 109 117 108 116 117 115 47 34 44 34 116 121 112 101 34 58 34 109 117 108 116 117 115 45 115 104 105 109 34 44 10 32 32 32 32 34 99 104 114 111 111 116 68 105 114 34 58 32 34 47 104 111 115 116 114 111 111 116 34 44 10 32 32 32 32 34 99 111 110 102 68 105 114 34 58 32 34 47 104 111 115 116 47 101 116 99 47 99 110 105 47 110 101 116 46 100 34 44 10 32 32 32 32 34 108 111 103 84 111 83 116 100 101 114 114 34 58 32 116 114 117 101 44 10 32 32 32 32 34 108 111 103 76 101 118 101 108 34 58 32 34 100 101 98 117 103 34 44 10 32 32 32 32 34 108 111 103 70 105 108 101 34 58 32 34 47 116 109 112 47 109 117 108 116 117 115 46 108 111 103 34 44 10 32 32 32 32 34 98 105 110 68 105 114 34 58 32 34 47 111 112 116 47 99 110 105 47 98 105 110 34 44 10 32 32 32 32 34 99 110 105 68 105 114 34 58 32 34 47 118 97 114 47 108 105 98 47 99 110 105 47 109 117 108 116 117 115 34 44 10 32 32 32 32 34 115 111 99 107 101 116 68 105 114 34 58 32 34 47 104 111 115 116 47 114 117 110 47 109 117 108 116 117 115 47 34 10 125 10]} ERRORED: error configuring pod [default/memifpod-2] networking: [default/memifpod-2/:network-red]: error adding container to network "network-red": plugin type="calico" failed (add): ARGS: error parsing value of pair "IP=172.22.0.1/32": invalid IP address: 172.22.0.1/32

[hedibouattour]

I tried to reproduce your deployment but I am getting this error instead:
ERRORED: error configuring pod [default/memifpod-2] networking: [default/memifpod-2/c678973a-11f5-42d1-8e39-279ea6ba2775:network-green]: error adding container to network "network-green": plugin type="calico" failed (add): failed to find plugin "static" in path [/opt/cni/bin]
Actually network attachment definitions looks for the ipam type in /opt/cni/bin. Do you have "static" as binary in /opt/cni/bin?

[infinitydon]

I have static in my CNI bin:

[ec2-user@ip-10-0-1-45 bin]$ ls -ltrh /opt/cni/bin/
total 313M
-rwxr-xr-x 1 root root 5.7M May 13  2020 firewall
-rwxr-xr-x 1 root root 3.3M May 13  2020 sbr
-rwxr-xr-x 1 root root 4.5M May 13  2020 bridge
-rwxr-xr-x 1 root root 4.0M May 13  2020 host-device
-rwxr-xr-x 1 root root 4.2M May 13  2020 ipvlan
-rwxr-xr-x 1 root root 4.2M May 13  2020 macvlan
-rwxr-xr-x 1 root root 4.4M May 13  2020 ptp
-rwxr-xr-x 1 root root 4.2M May 13  2020 vlan
-rwxr-xr-x 1 root root  12M May 13  2020 dhcp
-rwxr-xr-x 1 root root 2.8M May 13  2020 static
-rwxr-xr-x 1 root root  38M Mar  7 17:13 multus-shim
-rwxr-xr-x 1 root root 3.7M Mar  7 17:14 bandwidth
-rwsr-xr-x 1 root root  56M Mar  7 17:14 calico
-rwsr-xr-x 1 root root  56M Mar  7 17:14 calico-ipam
-rwxr-xr-x 1 root root 2.3M Mar  7 17:14 flannel
-rwxr-xr-x 1 root root 3.2M Mar  7 17:14 host-local
-rwsr-xr-x 1 root root  56M Mar  7 17:14 install
-rwxr-xr-x 1 root root 3.2M Mar  7 17:14 loopback
-rwxr-xr-x 1 root root 3.6M Mar  7 17:14 portmap
-rwxr-xr-x 1 root root 3.4M Mar  7 17:14 tuning
-rwxr-xr-x 1 root root  42M Mar  7 17:26 whereabouts
[ec2-user@ip-10-0-1-45 bin]$
[ec2-user@ip-10-0-1-45 bin]$

[hedibouattour]

Hello @infinitydon, we've been investigating this. Indeed there is a bug when calico tries to use static as a plugin. There is a difference between the spec awaited from calico side [0] and the spec used in the cni args. [1]
[0] https://github.com/projectcalico/calico/blob/757bc074b06e683e30fd400ff9eb3a521367d83c/cni-plugin/pkg/types/types.go#L160
[1] https://www.cni.dev/docs/conventions/#cni_args
We pinpointed this issue and we'll try to resolve it and keep you updated.

[infinitydon]

Thanks @hedibouattour for the update.