From 84738b608c9d4d4fadbd0fda97f33bc6aeeacd8b Mon Sep 17 00:00:00 2001 From: Christopher Tauchen Date: Fri, 13 Mar 2026 18:32:02 +0000 Subject: [PATCH] Restore CC Felix config pages to use FelixConfig component The Vale PR #2554 inadvertently reverted the CC Felix config migration (PR #2560) back to static HTML tables. This restores the 4 affected MDX pages to use the auto-generated FelixConfig React component. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../node/felix/configuration.mdx | 547 +++++++----------- .../reference/resources/felixconfig.mdx | 411 ++++++------- .../node/felix/configuration.mdx | 547 +++++++----------- .../reference/resources/felixconfig.mdx | 411 ++++++------- 4 files changed, 826 insertions(+), 1090 deletions(-) diff --git a/calico-cloud/reference/component-resources/node/felix/configuration.mdx b/calico-cloud/reference/component-resources/node/felix/configuration.mdx index 9d2df35fca..72b8cce4ad 100644 --- a/calico-cloud/reference/component-resources/node/felix/configuration.mdx +++ b/calico-cloud/reference/component-resources/node/felix/configuration.mdx @@ -4,16 +4,7 @@ description: Configure Felix, the daemon that runs on every machine that provide # Configuring Felix -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - - - - -If you have installed Calico using the operator, you cannot modify the environment provided to felix directly. To configure felix, see the [FelixConfiguration](../../../resources/felixconfig.mdx) resource instead. - - - +import FelixConfig from '../../../../_includes/components/FelixConfig'; :::note @@ -39,338 +30,210 @@ defaults, and it should be rare to have to explicitly set them. The full list of parameters which can be set is as follows. -### General configuration - -| Configuration file parameter | Environment variable | Description | Schema | -| ----------------------------------- | ---------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | -| `DataplaneWatchdogTimeout` | `FELIX_DATAPLANEWATCHDOGTIMEOUT` | Deprecated: superseded by `HealthTimeoutOverrides`. Timeout before the main data plane goroutine is determined to have hung and Felix will report non-live and non-ready. Can be increased if the liveness check incorrectly fails (for example if Felix is running slowly on a heavily loaded system). [Default: `90`] | int | -| `AwsSrcDstCheck` | `FELIX_AWSSRCDSTCHECK` | Set the [source-destination-check](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCheck) when using AWS EC2 instances. Check [IAM role and profile configuration](../../../resources/felixconfig.mdx#aws-iam-rolepolicy-for-source-destination-check-configuration) for setting the necessary permission for this setting to work. [Default: `DoNothing`] | `DoNothing`, `Disable`, `Enable` | -| `AWSSecondaryIPSupport` | `FELIX_AWSSECONDARYIPSUPPORT` | Controls whether Felix will create secondary AWS ENIs for AWS-backed IP pools. This feature is documented in the [egress gateways on AWS guide](../../../../networking/egress/egress-gateway-aws.mdx). Should only be enabled on AWS. [Default: "Disabled"] | `Enabled`, `EnabledENIPerWorkload`, `Disabled` | -| `AWSSecondaryIPRoutingRulePriority` | `FELIX_AWSSECONDARYIPROUTINGRULEPRIORITY` | Controls the priority of the policy-based routing rules used to implement AWS-backed IP addresses. Should only be changed to avoid conflicts if your nodes have additional policy based routing rules. [Default: 101] | int | -| `AWSRequestTimeout` | `FELIX_AWSREQUESTTIMEOUT` Timeout used for communicating with the AWS API (seconds). [Default: "30"] | int | -| `DatastoreType` | `FELIX_DATASTORETYPE` | The datastore that Felix should read endpoints and policy information from. [Default: `etcdv3`] | `etcdv3`, `kubernetes` | -| `DeviceRouteSourceAddress` | `FELIX_DEVICEROUTESOURCEADDRESS` | IPv4 address to use as the source hint on device routes programmed by Felix [Default: No source hint is set on programmed routes and for local traffic from host to workload the source address will be chosen by the kernel.] | `` | -| `DeviceRouteSourceAddressIPv6` | `FELIX_DEVICEROUTESOURCEADDRESSIPV6` | IPv6 address to use as the source hint on device routes programmed by Felix [Default: No source hint is set on programmed routes and for local traffic from host to workload the source address will be chosen by the kernel.] | `` | -| `DeviceRouteProtocol` | `FELIX_DEVICEROUTEPROTOCOL` | This defines the route protocol added to programmed device routes. [Default: `RTPROT_BOOT`] | int | -| `DisableConntrackInvalidCheck` | `FELIX_DISABLECONNTRACKINVALIDCHECK` | Disable the dropping of packets that aren't either a valid handshake or part of an established connection. [Default: `false`] | boolean | -| `EndpointReportingDelaySecs` | `FELIX_ENDPOINTREPORTINGDELAYSECS` | Set the endpoint reporting delay between status check intervals, in seconds. Only used if endpoint reporting is enabled. [Default: `1`] | int | -| `EndpointReportingEnabled` | `FELIX_ENDPOINTREPORTINGENABLED` | Enable the endpoint status reporter. [Default: `false`] | boolean | -| `ExternalNodesCIDRList` | `FELIX_EXTERNALNODESCIDRLIST` | Comma-delimited list of IPv4 or CIDR of external-non-calico-nodes from which IPIP traffic is accepted by calico-nodes. [Default: ""] | string | -| `FailsafeInboundHostPorts` | `FELIX_FAILSAFEINBOUNDHOSTPORTS` | List of PortProto struct objects including UDP/TCP/SCTP ports and CIDRs that Felix will allow incoming traffic to host endpoints on irrespective of the security policy. This is useful to avoid accidentally cutting off a host with incorrect configuration. For backwards compatibility, if the protocol is not specified, it defaults to `"tcp"`. If a CIDR is not specified, it will allow traffic from all addresses. To disable all inbound host ports, use the value `[]`. The default value allows ssh access, DHCP, BGP, etcd and the Kubernetes API. [Default: `[{"port":22,"protocol":"tcp"},{"port":68,"protocol":"udp"},{"port":179,"protocol":"tcp"},{"port":2379,"protocol":"tcp"}, {"port":2380,"protocol":"tcp"}, {"port":5473,"protocol":"tcp"}, {"port":6443,"protocol":"tcp"}, {"port":6666,"protocol":"tcp"}, {"port":6667,"protocol":"tcp"}]`] | list | -| `FailsafeOutboundHostPorts` | `FELIX_FAILSAFEOUTBOUNDHOSTPORTS` | List of PortProto struct objects including UDP/TCP/SCTP ports and CIDRs that Felix will allow outgoing traffic from host endpoints to irrespective of the security policy. This is useful to avoid accidentally cutting off a host with incorrect configuration. For backwards compatibility, if the protocol is not specified, it defaults to `"tcp"`. If a CIDR is not specified, it will allow traffic from all addresses. To disable all outbound host ports, use the value `[]`. The default value opens etcd's standard ports to ensure that Felix does not get cut off from etcd as well as allowing DHCP, DNS, BGP and the Kubernetes API. [Default: `[{"port":53,"protocol":"udp"},{"port":67,"protocol":"udp"}, {"port":179,"protocol":"tcp"}, {"port":2379,"protocol":"tcp"}, {"port":2380,"protocol":"tcp"}, {"port":5473,"protocol":"tcp"}, {"port":6443,"protocol":"tcp"}, {"port": 6666,"protocol":"tcp"}, {"port":6667,"protocol":"tcp"}]`] | list | | -| `FelixHostname` | `FELIX_FELIXHOSTNAME` | The hostname Felix reports to the plugin. Should be used if the hostname Felix autodetects is incorrect or does not match what the plugin will expect. [Default: `socket.gethostname()`] | string | -| `HealthEnabled` | `FELIX_HEALTHENABLED` | When enabled, exposes felix health information via an http endpoint. | boolean | -| `HealthHost` | `FELIX_HEALTHHOST` | The address on which Felix will respond to health requests. [Default: `localhost`] | string | -| `HealthPort` | `FELIX_HEALTHPORT` | The port on which Felix will respond to health requests. [Default: `9099`] | int | -| `HealthTimeoutOverrides` | `FELIX_HEALTHTIMEOUTOVERRIDES` | Allows the internal watchdog timeouts of individual subcomponents to be overridden; example: "InternalDataplaneMainLoop=30s,CalculationGraph=2m". This is useful for working around "false positive" liveness timeouts that can occur in particularly stressful workloads or if CPU is constrained. For a list of active subcomponents, see Felix's logs. [Default: ``] | Comma-delimited list of key/value pairs where the values are durations: `1s`, `10s`, `5m`, etc. | -| `IpInIpEnabled` | `FELIX_IPINIPENABLED` | Optional, you shouldn't need to change this setting as Felix calculates if IPIP should be enabled based on the existing IP Pools. When set, this overrides whether Felix should configure an IPinIP interface on the host. When explicitly disabled in FelixConfiguration, Felix will not clean up addresses from the `tunl0` interface (use this if you need to add addresses to that interface and don't want to have them removed). [Default: unset] | optional boolean | -| `IpInIpMtu` | `FELIX_IPINIPMTU` | The MTU to set on the IPIP tunnel device. Zero value means auto-detect. See [Configuring MTU](../../../../networking/configuring/mtu.mdx) [Default: `0`] | int | -| `IPv4VXLANTunnelAddr` | | IP address of the IPv4 VXLAN tunnel. This is system configured and should not be updated manually. | string | -| `LogFilePath` | `FELIX_LOGFILEPATH` | The full path to the Felix log. Set to `none` to disable file logging. [Default: `/var/log/calico/felix.log`] | string | -| `LogSeverityFile` | `FELIX_LOGSEVERITYFILE` | The log severity above which logs are sent to the log file. [Default: `Info`] | `Debug`, `Info`, `Warning`, `Error`, `Fatal` | -| `LogSeverityScreen` | `FELIX_LOGSEVERITYSCREEN` | The log severity above which logs are sent to the stdout. [Default: `Info`] | `Debug`, `Info`, `Warning`, `Error`, `Fatal` | -| `LogSeveritySys` | `FELIX_LOGSEVERITYSYS` | The log severity above which logs are sent to the syslog. Set to `none` for no logging to syslog. [Default: `Info`] | `Debug`, `Info`, `Warning`, `Error`, `Fatal` | -| `LogDebugFilenameRegex` | `FELIX_LOGDEBUGFILENAMEREGEX` | Controls which source code files have their Debug log output included in the logs. Only logs from files with names that match the given regular expression are included. The filter only applies to Debug level logs. [Default: `""`] | regex | -| `PolicySyncPathPrefix` | `FELIX_POLICYSYNCPATHPREFIX` | File system path where Felix notifies services of policy changes over Unix domain sockets. This is required only if you're configuring [L7 logs](../../../../observability/elastic/l7/configure.mdx), or [egress gateways](../../../../networking/egress/index.mdx). Set to `""` to disable. [Default: `""`] | string | -| `PrometheusGoMetricsEnabled` | `FELIX_PROMETHEUSGOMETRICSENABLED` | Set to `false` to disable Go runtime metrics collection, which the Prometheus client does by default. This reduces the number of metrics reported, reducing Prometheus load. [Default: `true`] | boolean | -| `PrometheusMetricsEnabled` | `FELIX_PROMETHEUSMETRICSENABLED` | Set to `true` to enable the Prometheus metrics server in Felix. [Default: `false`] | boolean | -| `PrometheusMetricsHost` | `FELIX_PROMETHEUSMETRICSHOST` | TCP network address that the Prometheus metrics server should bind to. [Default: `""`] | string | -| `PrometheusMetricsPort` | `FELIX_PROMETHEUSMETRICSPORT` | TCP port that the Prometheus metrics server should bind to. [Default: `9091`] | int | -| `PrometheusProcessMetricsEnabled` | `FELIX_PROMETHEUSPROCESSMETRICSENABLED` | Set to `false` to disable process metrics collection, which the Prometheus client does by default. This reduces the number of metrics reported, reducing Prometheus load. [Default: `true`] | boolean | -| `PrometheusWireguardMetricsEnabled` | `FELIX_PROMETHEUSWIREGUARDMETRICSENABLED` | Set to `false` to disable WireGuard device metrics collection, which Felix does by default. [Default: `true`] | boolean | -| `RemoveExternalRoutes` | `FELIX_REMOVEEXTERNALROUTES` | Whether or not to remove device routes that have not been programmed by Felix. Disabling this will allow external applications to also add device routes. [Default: `true`] | bool | -| `ReportingIntervalSecs` | `FELIX_REPORTINGINTERVALSECS` | Interval at which Felix reports its status into the datastore. 0 means disabled and is correct for Kubernetes-only clusters. Must be non-zero in OpenStack deployments. [Default: `30`] | int | -| `ReportingTTLSecs` | `FELIX_REPORTINGTTLSECS` | Time-to-live setting for process-wide status reports. [Default: `90`] | int | -| `RouteTableRange` | `FELIX_ROUTETABLERANGE` | _deprecated in favor of `RouteTableRanges`_ Calico programs additional Linux route tables for various purposes. `RouteTableRange` specifies the indices of the route tables that Calico should use. [Default: `""`] | `-` | -| `RouteTableRanges` | `FELIX_ROUTETABLERANGES` | Calico programs additional Linux route tables for various purposes. `RouteTableRanges` specifies a set of table index ranges that Calico should use. Deprecates `RouteTableRange`, overrides `RouteTableRange`. [Default: `"1-250"`] | `-,-,...` | -| `RouteSyncDisabled` | `FELIX_ROUTESYNCDISABLED` | Set to `true` to disable Calico programming routes to local workloads. [Default: `false`] | boolean | -| `VXLANEnabled` | `FELIX_VXLANENABLED` | Optional, you shouldn't need to change this setting as Felix calculates if VXLAN should be enabled based on the existing IP Pools. When set, this overrides whether Felix should create the VXLAN tunnel device for VXLAN networking. [Default: unset] | optional boolean | -| `VXLANMTU` | `FELIX_VXLANMTU` | The MTU to set on the IPv4 VXLAN tunnel device. Zero value means auto-detect. Also controls NodePort MTU when eBPF enabled. See [Configuring MTU](../../../../networking/configuring/mtu.mdx) [Default: `0`] | int | -| `VXLANMTUV6` | `FELIX_VXLANMTUV6` | The MTU to set on the IPv6 VXLAN tunnel device. Zero value means auto-detect. Also controls NodePort MTU when eBPF enabled. See [Configuring MTU](../../../../networking/configuring/mtu.mdx) [Default: `0`] | int | -| `VXLANPort` | `FELIX_VXLANPORT` | The UDP port to use for VXLAN. [Default: `4789`] | int | -| `VXLANTunnelMACAddr` | | MAC address of the IPv4 VXLAN tunnel. This is system configured and should not be updated manually. | string | -| `VXLANVNI` | `FELIX_VXLANVNI` | The virtual network ID to use for VXLAN. [Default: `4096`] | int | -| `AllowVXLANPacketsFromWorkloads` | `FELIX_ALLOWVXLANPACKETSFROMWORKLOADS` | Set to `true` to allow VXLAN encapsulated traffic from workloads. [Default: `false`] | boolean | -| `AllowIPIPPacketsFromWorkloads` | `FELIX_ALLOWIPIPPACKETSFROMWORKLOADS` | Set to `true` to allow IPIP encapsulated traffic from workloads. [Default: `false`] | boolean | -| `TyphaAddr` | `FELIX_TYPHAADDR` | IPv4 address at which Felix should connect to Typha. [Default: none] | string | -| `TyphaK8sServiceName` | `FELIX_TYPHAK8SSERVICENAME` | Name of the Typha Kubernetes service | string | -| `Ipv6Support` | `FELIX_IPV6SUPPORT` | Enable $[prodname] networking and security for IPv6 traffic as well as for IPv4. | boolean | -| `RouteSource` | `FELIX_ROUTESOURCE` | Where Felix gets is routing information from for VXLAN and the BPF data plane. The CalicoIPAM setting is more efficient because it supports route aggregation, but it only works when Calico's IPAM or host-local IPAM is in use. Use the WorkloadIPs setting if you are using Calico's VXLAN or BPF data plane and not using Calico IPAM or host-local IPAM. [Default: "CalicoIPAM"] | 'CalicoIPAM', or 'WorkloadIPs' | -| `mtuIfacePattern` | `FELIX_MTUIFACEPATTERN` | Pattern used to discover the host's interface for MTU auto-detection. [Default: `^((en\|wl\|ww\|sl\|ib)[Pcopsvx].*\|(eth\|wlan\|wwan).*)`] | regex | -| `TPROXYMode` | `FELIX_TPROXYMODE` | Sets transparent proxying mode. [Default: "Disabled"] | 'Disabled', 'Enabled' | -| `TPROXYPort` | `FELIX_TPROXYPORT` | What local ports is the proxied traffic sent to. [Default: `16001`] | int | -| `FeatureDetectOverride` | `FELIX_FEATUREDETECTOVERRIDE` | Is used to override the feature detection. Values are specified in a comma separated list with no spaces, example; "SNATFullyRandom=true,MASQFullyRandom=false,RestoreSupportsLock=true,IPIPDeviceIsL3=true. "true" or "false" will force the feature, empty or omitted values are auto-detected. [Default: `""`] | string | - -### etcd datastore configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ----------------------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- | -| `EtcdCaFile` | `FELIX_ETCDCAFILE` | Path to the file containing the root certificate of the certificate authority (CA) that issued the etcd server certificate. Configures Felix to trust the CA that signed the root certificate. The file may contain multiple root certificates, causing Felix to trust each of the CAs included. To disable authentication of the server by Felix, set the value to `none`. [Default: `/etc/ssl/certs/ca-certificates.crt`] | string | -| `EtcdCertFile` | `FELIX_ETCDCERTFILE` | Path to the file containing the client certificate issued to Felix. Enables Felix to participate in mutual TLS authentication and identify itself to the etcd server. Example: `/etc/felix/cert.pem` (optional) | string | -| `EtcdEndpoints` | `FELIX_ETCDENDPOINTS` | Comma-delimited list of etcd endpoints to connect to. Example: `http://127.0.0.1:2379,http://127.0.0.2:2379`. | `://:` | -| `EtcdKeyFile` | `FELIX_ETCDKEYFILE` | Path to the file containing the private key matching Felix's client certificate. Enables Felix to participate in mutual TLS authentication and identify itself to the etcd server. Example: `/etc/felix/key.pem` (optional) | string | - -### Kubernetes API datastore configuration - -The Kubernetes API datastore driver reads its configuration from Kubernetes-provided environment variables. - -### iptables data plane configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ------------------------------------ | ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- | -| `ChainInsertMode` | `FELIX_CHAININSERTMODE` | Controls whether Felix hooks the kernel's top-level iptables chains by inserting a rule at the top of the chain or by appending a rule at the bottom. `Insert` is the safe default since it prevents $[prodname]'s rules from being bypassed. If you switch to `Append` mode, be sure that the other rules in the chains signal acceptance by falling through to the $[prodname] rules, otherwise the $[prodname] policy will be bypassed. [Default: `Insert`] | `Insert`, `Append` | -| `DefaultEndpointToHostAction` | `FELIX_DEFAULTENDPOINTTOHOSTACTION` | This parameter controls what happens to traffic that goes from a workload endpoint to the host itself (after the traffic hits the endpoint egress policy). By default $[prodname] blocks traffic from workload endpoints to the host itself with an iptables `Drop` action. If you want to allow some or all traffic from endpoint to host, set this parameter to `Return` or `Accept`. Use `Return` if you have your own rules in the iptables "INPUT" chain; $[prodname] will insert its rules at the top of that chain, then `Return` packets to the "INPUT" chain once it has completed processing workload endpoint egress policy. Use `Accept` to unconditionally accept packets from workloads after processing workload endpoint egress policy. [Default: `Drop`] | `Drop`, `Return`, `Accept` | -| `GenericXDPEnabled` | `FELIX_GENERICXDPENABLED` | When enabled, Felix can fallback to the non-optimized `generic` XDP mode. This should only be used for testing since it doesn't improve performance over the non-XDP mode. [Default: `false`] | boolean | -| `InterfaceExclude` | `FELIX_INTERFACEEXCLUDE` | A comma-separated list of interface names that should be excluded when Felix is resolving host endpoints. The default value ensures that Felix ignores Kubernetes' internal `kube-ipvs0` device. If you want to exclude multiple interface names using a single value, the list supports regular expressions. For regular expressions you must wrap the value with `/`. For example having values `/^kube/,veth1` will exclude all interfaces that begin with `kube` and also the interface `veth1`. [Default: `kube-ipvs0`] | string | -| `IpsetsRefreshInterval` | `FELIX_IPSETSREFRESHINTERVAL` | Period, in seconds, at which Felix re-checks the IP sets in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable IP sets refresh. [Default: `10`] | int | -| `IptablesBackend` | `FELIX_IPTABLESBACKEND` | This parameter controls which variant of iptables Felix uses. Set this to `Auto` for auto detection of the backend. If a specific backend is needed then use `nft` for hosts using a netfilter backend or `Legacy` for others. [Default: `Auto`] | `Legacy`, `NFT`, `Auto` | -| `IptablesFilterAllowAction` | `FELIX_IPTABLESFILTERALLOWACTION` | This parameter controls what happens to traffic that is allowed by a Felix policy chain in the iptables filter table (i.e., a normal policy chain). The default will immediately `Accept` the traffic. Use `Return` to send the traffic back up to the system chains for further processing. [Default: `Accept`] | `Accept`, `Return` | -| `IptablesLockFilePath` | `FELIX_IPTABLESLOCKFILEPATH` | _Deprecated:_ For iptables versions prior to v1.6.2, location of the iptables lock file (later versions of iptables always use value "/run/xtables.lock"). You may need to change this if the lock file is not in its standard location (for example if you have mapped it into Felix's container at a different path). [Default: `/run/xtables.lock`] | string | -| `IptablesLockProbeIntervalMillis` | `FELIX_IPTABLESLOCKPROBEINTERVALMILLIS` | Time, in milliseconds, that Felix will wait between attempts to acquire the iptables lock if it is not available. Lower values make Felix more responsive when the lock is contended, but use more CPU. [Default: `50`] | int | -| `IptablesLockTimeoutSecs` | `FELIX_IPTABLESLOCKTIMEOUTSECS` | Time, in seconds, that Felix will wait for the iptables lock. Versions of iptables prior to v1.6.2 support disabling the iptables lock by setting this value to 0; v1.6.2 and above do not so Felix will default to 10s if a non-positive number is used. To use this feature, Felix must share the iptables lock file with all other processes that also take the lock. When running Felix inside a container, this typically requires the file /run/xtables.lock on the host to be mounted into the `$[nodecontainer]` or `calico/felix` container. [Default: `0` disabled for iptables <v1.6.2 or 10s for later versions] | int | -| `IptablesMangleAllowAction` | `FELIX_IPTABLESMANGLEALLOWACTION` | This parameter controls what happens to traffic that is allowed by a Felix policy chain in the iptables mangle table (i.e., a pre-DNAT policy chain). The default will immediately `Accept` the traffic. Use `Return` to send the traffic back up to the system chains for further processing. [Default: `Accept`] | `Accept`, `Return` | -| `IptablesMarkMask` | `FELIX_IPTABLESMARKMASK` | Mask that Felix selects its IPTables Mark bits from. Should be a 32 bit hexadecimal number with at least 8 bits set, none of which clash with any other mark bits in use on the system. When using $[prodname] with Kubernetes' `kube-proxy` in IPVS mode, [we recommend allowing at least 16 bits](#ipvs-bits). [Default: `0xffff0000`] | netmask | -| `IptablesNATOutgoingInterfaceFilter` | `FELIX_IPTABLESNATOUTGOINGINTERFACEFILTER` | This parameter can be used to limit the host interfaces on which Calico will apply SNAT to traffic leaving a Calico IPAM pool with "NAT outgoing" enabled. This can be useful if you have a main data interface, where traffic should be SNATted and a secondary device (such as the docker bridge) which is local to the host and doesn't require SNAT. This parameter uses the iptables interface matching syntax, which allows `+` as a wildcard. Most users will not need to set this. Example: if your data interfaces are eth0 and eth1 and you want to exclude the docker bridge, you could set this to `eth+` | string | -| `IptablesPostWriteCheckIntervalSecs` | `FELIX_IPTABLESPOSTWRITECHECKINTERVALSECS` | Period, in seconds, after Felix has done a write to the data plane that it schedules an extra read back to check the write was not clobbered by another process. This should only occur if another application on the system doesn't respect the iptables lock. [Default: `1`] | int | -| `IptablesRefreshInterval` | `FELIX_IPTABLESREFRESHINTERVAL` | Period, in seconds, at which Felix re-checks all iptables state to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable iptables refresh. [Default: `90`] | int | -| `LogPrefix` | `FELIX_LOGPREFIX` | The log prefix that Felix uses when rendering LOG rules. [Default: `calico-packet`] | string | -| `MaxIpsetSize` | `FELIX_MAXIPSETSIZE` | Maximum size for the ipsets used by Felix. Should be set to a number that is greater than the maximum number of IP addresses that are ever expected in a selector. [Default: `1048576`] | int | -| `NATPortRange` | `FELIX_NATPORTRANGE` | Port range used by iptables for port mapping when doing outgoing NAT. (Example: `32768:65000`). [Default: iptables maps source ports below 512 to other ports below 512: those between 512 and 1023 inclusive will be mapped to ports below 1024, and other ports will be mapped to 1024 or above. Where possible, no port alteration will occur.] | string | -| `NATOutgoingAddress` | `FELIX_NATOUTGOINGADDRESS` | Source address used by iptables for an SNAT rule when doing outgoing NAT. [Default: an iptables `MASQUERADE` rule is used for outgoing NAT which will use the address on the interface traffic is leaving on.] | `` | -| `NetlinkTimeoutSecs` | `FELIX_NETLINKTIMEOUTSECS` | Time, in seconds, that Felix will wait for netlink (i.e. routing table list/update) operations to complete before giving up and retrying. [Default: `10`] | float | -| `RouteRefreshInterval` | `FELIX_ROUTEREFRESHINTERVAL` | Period, in seconds, at which Felix re-checks the routes in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable route refresh. [Default: `90`] | int | -| `ServiceLoopPrevention` | `FELIX_SERVICELOOPPREVENTION` | When [service IP advertisement is enabled](../../../../networking/configuring/advertise-service-ips.mdx), prevent routing loops to service IPs that are not in use, by dropping or rejecting packets that do not get DNAT'd by kube-proxy. Unless set to "Disabled", in which case such routing loops continue to be allowed. [Default: `Drop`] | `Drop`, `Reject`, `Disabled` | -| `WorkloadSourceSpoofing` | `FELIX_WORKLOADSOURCESPOOFING` | Controls whether pods can enable source IP address spoofing with the `cni.projectcalico.org/allowedSourcePrefixes` annotation. When set to `Any`, pods can use this annotation to send packets from any IP address. [Default: `Disabled`] | `Any`, `Disabled` | -| `XDPRefreshInterval` | `FELIX_XDPREFRESHINTERVAL` | Period, in seconds, at which Felix re-checks the XDP state in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable XDP refresh. [Default: `90`] | int | -| `XDPEnabled` | `FELIX_XDPENABLED` | Enable XDP acceleration for host endpoint policies. [Default: `true`] | boolean | - -### eBPF data plane configuration - -eBPF data plane mode uses the Linux Kernel's eBPF virtual machine to implement networking and policy instead of iptables. When BPFEnabled is set to `true`, Felix will: - -- Require a v5.3 Linux kernel. -- Implement policy with eBPF programs instead of iptables. -- Activate its embedded implementation of `kube-proxy` to implement Kubernetes service load balancing. -- Disable support for IPv6. - -See [Enable the eBPF data plane](../../../../operations/ebpf/enabling-ebpf.mdx) for step-by step instructions to enable this feature. - -| Configuration parameter / Environment variable | Description | Schema | Default | -| ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | -------------- | ------ | ------ | ----------------- | -| BPFEnabled /
FELIX_BPFENABLED | Enable eBPF data plane mode. eBPF mode has a number of limitations, see the [HOWTO guide](../../../../operations/ebpf/enabling-ebpf.mdx). | true, false | false | -| BPFDisableUnprivileged /
FELIX_BPFDISABLEUNPRIVILEGED | If true, Felix sets the kernel.unprivileged_bpf_disabled sysctl to disable unprivileged use of BPF. This ensures that unprivileged users cannot access Calico's BPF maps and cannot insert their own BPF programs to interfere with the ones that $[prodname] installs. | true, false | true | -| BPFLogLevel /
FELIX_BPFLOGLEVEL | The log level used by the BPF programs. The logs are emitted to the BPF trace pipe, accessible with the command `tc exec BPF debug`. | Off,Info,Debug | Off | -| BPFDataIfacePattern /
FELIX_BPFDATAIFACEPATTERN | Controls which interfaces Felix should attach BPF programs to catch traffic to/from the external network. This needs to match the interfaces that Calico workload traffic flows over as well as any interfaces that handle incoming traffic to NodePorts and services from outside the cluster. It should not match the workload interfaces (usually named cali...).. | regular expression | `^(en[opsvx].\* | eth.\* | tunl0$ | wireguard.cali$)` | -| BPFConnectTimeLoadBalancingEnabled /
FELIX_BPFCONNECTTIMELOADBALANCINGENABLED | Controls whether Felix installs the connect-time load balancer. In the current release, the connect-time load balancer is required for the host to reach kubernetes services. | true,false | true | -| BPFExternalServiceMode /
FELIX_BPFEXTERNALSERVICEMODE | Controls how traffic from outside the cluster to NodePorts and ClusterIPs is handled. In Tunnel mode, packet is tunneled from the ingress host to the host with the backing pod and back again. In DSR mode, traffic is tunneled to the host with the backing pod and then returned directly; this requires a network that allows direct return. | Tunnel,DSR | Tunnel | -| BPFExtToServiceConnmark /
FELIX_BPFEXTTOSERVICECONNMARK | Controls a 32bit mark that is set on connections from an external client to a local service. This mark allows us to control how packets of that connection are routed within the host and how is routing interpreted by RPF check. | int | 0 | -| BPFKubeProxyIptablesCleanupEnabled /
FELIX_BPFKUBEPROXYIPTABLESCLEANUPENABLED | Controls whether Felix will clean up the iptables rules created by the Kubernetes `kube-proxy`; should only be enabled if `kube-proxy` is not running. | true,false | true | -| BPFKubeProxyMinSyncPeriod /
FELIX_BPFKUBEPROXYMINSYNCPERIOD | Controls the minimum time between data plane updates for Felix's embedded `kube-proxy` implementation. | seconds | `1` | -| BPFKubeProxyEndpointSlicesEnabled /
FELIX_BPFKUBEPROXYENDPOINTSLICESENABLED | Controls whether Felix's embedded kube-proxy derives its services from Kubernetes' EndpointSlices resources. Using EndpointSlices is more efficient but it requires EndpointSlices support to be enabled at the Kubernetes API server. | true,false | false | -| BPFMapSizeConntrack /
FELIX_BPFMapSizeConntrack | Controls the size of the conntrack map. This map must be large enough to hold an entry for each active connection. Warning: changing the size of the conntrack map can cause disruption. | int | 512000 | -| BPFMapSizeNATFrontend /
FELIX_BPFMapSizeNATFrontend | Controls the size of the NAT frontend map. FrontendMap should be large enough to hold an entry for each nodeport, external IP and each port in each service. | int | 65536 | -| BPFMapSizeNATBackend /
FELIX_BPFMapSizeNATBackend | Controls the size of the NAT backend map. This is the total number of endpoints. This is mostly more than the size of the number of services. | int | 262144 | -| BPFMapSizeNATAffinity /
FELIX_BPFMapSizeNATAffinity | Controls the size of the NAT affinity map. | int | 65536 | -| BPFMapSizeIPSets /
FELIX_BPFMapSizeIPSets | Controls the size of the IPSets map. The IP sets map must be large enough to hold an entry for each endpoint matched by every selector in the source/destination matches in network policy. Selectors such as "all()" can result in large numbers of entries (one entry per endpoint in that case). | int | 1048576 | -| BPFMapSizeRoute /
FELIX_BPFMapSizeRoute | Controls the size of the route map. The routes map should be large enough to hold one entry per workload and a handful of entries per host (enough to cover its own IPs and tunnel IPs). | int | 262144 | -| BPFHostConntrackBypass /
FELIX_BPFHostConntrackBypass | Controls whether to bypass Linux conntrack in BPF mode for workloads and services. | true,false | true | -| BPFPolicyDebugEnabled /
FELIX_BPFPOLICYDEBUGENABLED | In eBPF data plane mode, Felix records detailed information about the BPF policy programs, which can be examined with the calico-bpf command-line tool. | true, false | true | - -### Windows-specific configuration - -| Configuration parameter | Environment variable | Description | Schema | Default | -| ------------------------------- | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ------------------------------------------- | -| windowsFlowLogsFileDirectory |
FELIX_WINDOWSFLOWLOGSFILEDIRECTORY | Set the directory where flow logs files are stored on Windows nodes. This parameter only takes effect when `flowLogsFileEnabled` is set to `true`. | string | `c:\\TigeraCalico\\flowlogs` | -| windowsFlowLogsPositionFilePath |
FELIX_WINDOWSFLOWLOGSPOSITIONFILEPATH | Specify the position of the external pipeline that reads flow logs on Windows nodes. This parameter only takes effect when `FlowLogsDynamicAggregationEnabled` is set to `true`. | string | `c:\\TigeraCalico\\flowlogs\\flows.log.pos` | -| windowsStatsDumpFilePath |
FELIX_WINDOWSTATSDUMPFILEPATH | Specify the position of the file used for dumping flow log statistics on Windows nodes. Note this is an internal setting that users shouldn't need to modify. | string | `c:\\TigeraCalico\\stats\\dump` | -| WindowsDNSCacheFile |
FELIX_WINDOWSDNSCACHEFILE | Specify the name of the file that Felix uses to preserve learned DNS information when restarting. | string | `c:\\TigeraCalico\\felix-dns-cache.txt` | -| WindowsDNSExtraTTL |
FELIX_WINDOWSDNSEXTRATTL | Specify extra time in seconds to keep IPs and alias names that are learned from DNS, in addition to each name or IP's advertised TTL. | seconds | `120` | - -### Kubernetes-specific configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ----------------------- | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ | -| `KubeNodePortRanges` | `FELIX_KUBENODEPORTRANGES` | A list of port ranges that Felix should treat as Kubernetes node ports. Only when `kube-proxy` is configured to use IPVS mode: Felix assumes that traffic arriving at the host of one of these ports will ultimately be forwarded instead of being terminated by a host process. [Default: `30000:32767`] | Comma-delimited list of `:` port ranges or single ports. | -| `KubeMasqueradeBit` | `FELIX_KUBEMASQUERADEBIT` | KubeMasqueradeBit should be set to the same value as --iptables-masquerade-bit of kube-proxy when TPROXY is used. This defaults to the corresponding kube-proxy default value so it only needs to change if kube-proxy is using a non-standard setting. Must be within the range of 0-31. OpenShift sets the bit to 0 by default. [Default: 14] | integer | +## Spec -:::note +### Datastore connection - When using $[prodname] with Kubernetes' `kube-proxy` in IPVS mode, $[prodname] uses additional -iptables mark bits to store an ID for each local $[prodname] endpoint. For example, the default `IptablesMarkMask` value, -`0xffff0000` gives $[prodname] 16 bits, up to 6 of which are used for internal purposes, leaving 10 bits for endpoint -IDs. 10 bits is enough for 1024 different values and $[prodname] uses 2 of those for internal purposes, leaving enough -for 1022 endpoints on the host. + -::: +### Process: Feature detection/overrides + + + +### Process: Go runtime + + + +### Process: Health port and timeouts + + + +### Process: Logging + + + +### Process: Prometheus metrics + + + +### Dataplane: Common + + + +### Dataplane: iptables + + + +### Dataplane: nftables + + + +### Dataplane: eBPF + + + +### Dataplane: Windows + + + +### Dataplane: OpenStack support + + + +### Dataplane: XDP acceleration for iptables dataplane + + + +### Overlay: VXLAN overlay + + + +### Overlay: IP-in-IP + + -### Bare metal specific configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ----------------------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | -| `InterfacePrefix` | `FELIX_INTERFACEPREFIX` | The interface name prefix that identifies workload endpoints and so distinguishes them from host endpoint interfaces. Accepts more than one interface name prefix in comma-delimited format, e.g., `tap,cali`. Note: in environments other than bare metal, the orchestrators configure this appropriately. For example our Kubernetes and Docker integrations set the `cali` value, and our OpenStack integration sets the `tap` value. [Default: `cali`] | string | - -### $[prodname] specific configuration - -| Setting | Environment variable | Default | Meaning | -| --------------------------------------- | --------------------------------------------- | ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `DropActionOverride` | `FELIX_DROPACTIONOVERRIDE` | `Drop` | How to treat packets that are disallowed by the current $[prodname] policy. For more detail please see below. | -| `LogDropActionOverride` | `FELIX_LOGDROPACTIONOVERRIDE` | `false` | Set to `true` to add the `DropActionOverride` to the syslog entries. For more detail please see below. | -| `PrometheusReporterEnabled` | `FELIX_PROMETHEUSREPORTERENABLED` | `false` | Set to `true` to enable Prometheus reporting of denied packet metrics. For more detail please see below. | -| `PrometheusReporterPort` | `FELIX_PROMETHEUSREPORTERPORT` | `9092` | The TCP port on which to report denied packet metrics. | -| `PrometheusReporterCertFile` | `FELIX_PROMETHEUSREPORTERCERTFILE` | None | Certificate for encrypting Prometheus denied packet metrics. | -| `PrometheusReporterKeyFile` | `FELIX_PROMETHEUSREPORTERKEYFILE` | None | Private key for encrypting Prometheus denied packet metrics. | -| `PrometheusReporterCAFile` | `FELIX_PROMETHEUSREPORTERCAFILE` | None | Trusted CA file for clients attempting to read Prometheus denied packet metrics. | -| `PrometheusMetricsCertFile` | `FELIX_PROMETHEUSMETRICSCERTFILE` | None | Certificate for encrypting general Felix Prometheus metrics. | -| `PrometheusMetricsKeyFile` | `FELIX_PROMETHEUSMETRICSKEYFILE` | None | Private key for encrypting general Felix Prometheus metrics. | -| `PrometheusMetricsCAFile` | `FELIX_PROMETHEUSMETRICSCAFILE` | None | Trusted CA file for clients attempting to read general Felix Prometheus metrics. | -| `IPSecMode` | `FELIX_IPSECMODE` | None | Controls which mode IPsec is operating on. The only supported value is `PSK`. An empty value means IPsec is not enabled. | -| `IPSecAllowUnsecuredTraffic` | `FELIX_IPSECALLOWUNSECUREDTRAFFIC` | `false` | When set to false, only IPsec-protected traffic will be allowed on the packet paths where IPsec is supported. When set to true, IPsec will be used but non-IPsec traffic will be accepted. In general, setting this to `true` is less safe since it allows an attacker to inject packets. However, it is useful when transitioning from non-IPsec to IPsec since it allows traffic to flow while the cluster negotiates the IPsec mesh. | -| `IPSecIKEAlgorithm` | `FELIX_IPSECIKEALGORITHM` | `aes128gcm16-prfsha256-ecp256` | IPsec IKE algorithm. Default is NIST suite B recommendation. | -| `IPSecESPAlgorithm` | `FELIX_IPSECESPALGORITHM` | `aes128gcm16-ecp256` | IPsec ESP algorithm. Default is NIST suite B recommendation. | -| `IPSecLogLevel` | `FELIX_IPSECLOGLEVEL` | `Info` | Controls log level for IPsec components. Set to `None` for no logging. Other valid values are `Notice`, `Info`, `Debug` and `Verbose`. | -| `IPSecPSKFile` | `FELIX_IPSECPSKFILE` | None | The path to the pre shared key file for IPsec. | -| `FlowLogsFileEnabled` | `FELIX_FLOWLOGSFILEENABLED` | `false` | Set to `true`, enables flow logs. If set to `false` no flow logging will occur. Flow logs are written to a file `flows.log` and sent to Elasticsearch. The location of this file can be configured using the `FlowLogsFileDirectory` field. File rotation settings for this `flows.log` file can be configured using the fields `FlowLogsFileMaxFiles` and `FlowLogsFileMaxFileSizeMB`. Note that flow log exports to Elasticsearch are dependent on flow logs getting written to this file. Setting this parameter to `false` will disable flow logs. | -| `FlowLogsFileIncludeLabels` | `FELIX_FLOWLOGSFILEINCLUDELABELS` | `false` | Set to `true` to include endpoint label information in flow logs. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFileIncludePolicies` | `FELIX_FLOWLOGSFILEINCLUDEPOLICIES` | `false` | Set to `true` to include policy match information in flow logs. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFileIncludeService` | `FELIX_FLOWLOGSFILEINCLUDESERVICE` | `false` | Set to `true` to include destination service information in flow logs. The service information is derived from pre-DNAT destination IP and is therefore only available on the node where DNAT occurs. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFileDirectory` | `FELIX_FLOWLOGSFILEDIRECTORY` | `/var/log/calico/flowlogs` | The directory where flow logs files are stored. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFileMaxFiles` | `FELIX_FLOWLOGSFILEMAXFILES` | `5` | The number of files to keep when rotating flow log files. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFileMaxFileSizeMB` | `FELIX_FLOWLOGSFILEMAXFILESIZEMB` | `100` | The max size in MB of flow logs files before rotation. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFlushInterval` | `FELIX_FLOWLOGSFLUSHINTERVAL` | `300` | The period, in seconds, at which Felix exports the flow logs. | -| `FlowLogsEnableNetworkSets` | `FELIX_FLOWLOGSENABLENETWORKSETS` | `false` | Whether to specify the network set a flow log originates from. | -| `FlowLogsFileAggregationKindForAllowed` | `FELIX_FLOWLOGSFILEAGGREGATIONKINDFORALLOWED` | `2` | How much to aggregate the flow logs sent to Elasticsearch for allowed traffic. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. `0` means no aggregation, `1` means aggregate all flows that share a source port on each node, `2` means aggregate all flows that share source ports or are from the same ReplicaSet and `3` means aggregate all flows that share destination and source ports and are from the same ReplicateSet | -| `FlowLogsFileAggregationKindForDenied` | `FELIX_FLOWLOGSFILEAGGREGATIONKINDFORDENIED` | `1` | How much to aggregate the flow logs sent to Elasticsearch for denied traffic. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. `0` means no aggregation, `1` means aggregate all flows that share a source port on each node, and `2` means aggregate all flows that share source ports or are from the same ReplicaSet and `3` means aggregate all flows that share destination and source ports and are from the same ReplicateSet. | -| `FlowLogsDynamicAggregationEnabled` | `FELIX_FLOWLOGSDYNAMICAGGREGATIONENABLED` | `false` | Enable dynamic aggregation for flow logs. This will increase aggregation up to the maximum level allowed (which is 3 and means aggregate all flows that share destination and source ports and are from the same ReplicateSet) when it detects the pipeline for reading flow logs is stalled. It will revert to its initial aggregation level when this condition changes. The initial aggregation level can be specified using `FlowLogsFileAggregationKindForAllowed` and `FlowLogsFileAggregationKindForDenied`. If these values are not specified, default values of `2` and `1` will be used. | -| `FlowLogsPositionFilePath` | `FELIX_FLOWLOGSPOSITIONPATH` | `/var/log/calico/flows.log.pos` | Default path of the position file. It is used to read the current state of pipeline for flow logs. This parameter will be used only when `FlowLogsDynamicAggregationEnabled` is set to `true` | -| `FlowLogsAggregationThresholdBytes` | `FELIX_FLOWLOGSAGGREGATIONTHRESHOLDBYTES` | `8192` | Default threshold to determine how far behind the pipeline for flow logs can get before aggregation starts in. Detecting a difference of 8192 bytes means increase 1 level, while a difference of 16384 means increasing two levels. This parameter will be used only when `FlowLogsDynamicAggregationEnabled` is set to `true`. | -| `FlowLogsCollectProcessInfo` | `FELIX_FLOWLOGSCOLLECTPROCESSINFO` | `true` | If enabled Felix will load the kprobe BPF programs to collect process info. | -| `FlowLogsCollectTcpStats` | `FELIX_FLOWLOGSCOLLECTTCPSTATS` | `true` | If enabled Felix will collect TCP socket stats using BPF and requires a recent kernel that supports BPF | -| `FlowLogsCollectProcessPath` | `FELIX_FLOWLOGSCOLLECTPROCESSPATH` | `true` | If enabled, along with FlowLogsCollectProcessInfo, each flow log will contain the full path of the process executable and the arguments with which the executable was invoked. If path or arguments cannot be determined, Felix will fallback to using task names and arguments will be empty. For full functionality, this feature should be enabled via operator see [Enabling process path](../../../../observability/elastic/flow/processpath.mdx) | -| `FlowLogsFilePerFlowProcessLimit` | `FELIX_FLOWLOGSFILEPERFLOWPROCESSLIMIT` | `2` | Specify the maximum number of flow log entries with distinct process information beyond which process information will be aggregated. | -| `FlowLogsFilePerFlowProcessArgsLimit` | `FELIX_FLOWLOGSFILEPERFLOWPROCESSARGSLIMIT` | `5` | Specify the maximum number of arguments beyond which the process arguments will be aggregated. | -| `FlowLogsPolicyEvaluationMode` | `FELIX_FLOWLOGSPOLICYEVALUATIONMODE` | `Continuous` | Defines how policies are evaluated and reflected in flow logs. `OnNewConnection`: In this mode, staged policies are only evaluated when new connections are made in the data plane. Staged/active policy changes will not be reflected in the `pending_policies` field of flow logs for long lived connections. `Continuous`: Felix evaluates active flows on a regular basis to determine the rule traces in the flow logs. Any policy updates that impact a flow will be reflected in the `pending_policies` field, offering a near-real-time view of policy changes across flows. | -| `DNSCacheFile` | `FELIX_DNSCACHEFILE` | `/var/run/calico/felix-dns-cache.txt` | The name of the file that Felix uses to preserve learned DNS information when restarting. | -| `DNSCacheSaveInterval` | `FELIX_DNSCACHESAVEINTERVAL` | `60` | The periodic interval at which Felix saves learned DNS information to the cache file. | -| `DNSCacheEpoch` | `FELIX_DNSCACHEEPOCH` | `0` | An arbitrary number that can be changed, at runtime, to tell Felix to discard all its learned DNS information. | -| `DNSExtraTTL` | `FELIX_DNSEXTRATTL` | `0` | Extra time, in seconds, to keep IPs and alias names that are learned from DNS, in addition to each name or IP's advertised TTL. | -| `DNSTrustedServers` | `FELIX_DNSTRUSTEDSERVERS` | `k8s-service:kube-dns` | The DNS servers that Felix should trust. Each entry here must be `[:]` - indicating an explicit DNS server IP - or `k8s-service:[/][:port]` - indicating a Kubernetes DNS service. `` defaults to the first service port, or 53 for an IP, and `` to `kube-system`. An IPv6 address with a port must use the square brackets convention, for example `[fd00:83a6::12]:5353`. Note that Felix (calico-node) will need RBAC permission to read the details of each service specified by a `k8s-service:...` form. | -| `DNSLogsFileEnabled` | `FELIX_DNSLOGSFILEENABLED` | `false` | Set to `true`, enables DNS logs. If set to `false` no DNS logging will occur. DNS logs are written to a file `dns.log` and sent to Elasticsearch. The location of this file can be configured using the `DNSLogsFileDirectory` field. File rotation settings for this `dns.log` file can be configured using the fields `DNSLogsFileMaxFiles` and `DNSLogsFileMaxFileSizeMB`. Note that DNS log exports to Elasticsearch are dependent on DNS logs getting written to this file. Setting this parameter to `false` will disable DNS logs. | -| `DNSLogsFileDirectory` | `FELIX_DNSLOGSFILEDIRECTORY` | `/var/log/calico/dnslogs` | The directory where DNS logs files are stored. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | -| `DNSLogsFileMaxFiles` | `FELIX_DNSLOGSFILEMAXFILES` | `5` | The number of files to keep when rotating DNS log files. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | -| `DNSLogsFileMaxFileSizeMB` | `FELIX_DNSLOGSFILEMAXFILESIZEMB` | `100` | The max size in MB of DNS log files before rotation. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | -| `DNSLogsFlushInterval` | `FELIX_DNSLOGSFLUSHINTERVAL` | `300` | The period, in seconds, at which Felix exports DNS logs. | -| `DNSLogsFileAggregationKind` | `FELIX_DNSLOGSFILEAGGREGATIONKIND` | `1` | How much to aggregate DNS logs. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. `0` means no aggregation, `1` means aggregate similar DNS logs from workloads in the same ReplicaSet. | -| `DNSLogsFileIncludeLabels` | `FELIX_DNSLOGSFILEINCLUDELABELS` | `true` | Whether to include client and server workload labels in DNS logs. | -| `DNSLogsFilePerNodeLimit` | `FELIX_DNSLOGSFILEPERNODELIMIT` | `0` (no limit) | Limit on the number of DNS logs that can be emitted within each flush interval. When this limit has been reached, Felix counts the number of unloggable DNS responses within the flush interval, and emits a WARNING log with that count at the same time as it flushes the buffered DNS logs. | -| `DNSLogsLatency` | `FELIX_DNSLOGSLATENCY` | `true` | Indicates to include measurements of DNS request/response latency in each DNS log. | -| `EgressIPSupport` | `FELIX_EGRESSIPSUPPORT` | `Disabled` | Defines three different support modes for egress gateway function. `Disabled` means egress gateways are not supported. `EnabledPerNamespace` means egress gateway function is enabled and can be configured on a per-namespace basis (but per-pod egress annotations are ignored). `EnabledPerNamespaceOrPerPod` means egress gateway function is enabled and can be configured per-namespace or per-pod (with per-pod egress annotations overriding namespace annotations). | -| `EgressIPVXLANPort` | `FELIX_EGRESSIPVXLANPORT` | `4097` | Port to use for egress gateway VXLAN traffic. A value of `0` means "use the kernel default". | -| `EgressIPVXLANVNI` | `FELIX_EGRESSIPVXLANVNI` | `4790` | Virtual network ID to use for egress gateway VXLAN traffic. A value of `0` means "use the kernel default". | -| `EgressIPRoutingRulePriority` | `FELIX_EGRESSIPROUTINGRULEPRIORITY` | `100` | Priority value to use for the egress gateway routing rule. | -| `L7LogsFileEnabled` | `FELIX_L7LOGSFILEENABLED` | `true` | If set to `false` no L7 logging will occur. L7 logs are written to a file `l7.log` and sent to Elasticsearch. The location of this file can be configured using the `L7LogsFileDirectory` field. File rotation settings for this `l7.log` file can be configured using the fields `L7LogsFileMaxFiles` and `L7LogsFileMaxFileSizeMB`. Note that L7 log exports to Elasticsearch are dependent on L7 logs getting written to this file. | -| `L7LogsFileDirectory` | `FELIX_L7LOGSFILEDIRECTORY` | `/var/log/calico/l7logs` | The directory where L7 log files are stored. This parameter only takes effect when `L7LogsFileEnabled` is `true`. | -| `L7LogsFileMaxFiles` | `FELIX_L7LOGSFILEMAXFILES` | `5` | The number of files to keep when rotating L7 log files. This parameter only takes effect when `L7LogsFileEnabled` is `true`. | -| `L7LogsFileMaxFileSizeMB` | `FELIX_L7LOGSFILEMAXFILESIZEMB` | `100` | The max size in MB of L7 log files before rotation. This parameter only takes effect when `L7LogsFileEnabled` is `true`. | -| `L7LogsFlushInterval` | `FELIX_L7LOGSFLUSHINTERVAL` | `300` | The period, in seconds, at which Felix exports L7 logs. | -| `L7LogsFileAggregationHTTPHeaderInfo` | `FELIX_L7LOGSFILEAGGREGATIONHTTPHEADERINFO` | `ExcludeL7HTTPHeaderInfo` | How to handle HTTP header information for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7HTTPHeaderInfo` and `IncludeL7HTTPHeaderInfo`. | -| `L7LogsFileAggregationHTTPMethod` | `FELIX_L7LOGSFILEAGGREGATIONHTTPMETHOD` | `IncludeL7HTTPMethod` | How to handle HTTP method data for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7HTTPMethod` and `IncludeL7HTTPMethod`. | -| `L7LogsFileAggregationServiceInfo` | `FELIX_L7LOGSFILEAGGREGATIONSERVICEINFO` | `IncludeL7ServiceInfo` | How to handle service information for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7ServiceInfo` and `IncludeL7ServiceInfo`. | -| `L7LogsFileAggregationDestinationInfo` | `FELIX_L7LOGSFILEAGGREGATIONDESTINATIONINFO` | `IncludeL7DestinationInfo` | How to handle destination metadata for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7DestinationInfo` and `IncludeL7DestinationInfo`. | -| `L7LogsFileAggregationSourceInfo` | `FELIX_L7LOGSFILEAGGREGATIONSOURCEINFO` | `IncludeL7SourceInfoNoPort` | How to handle source metadata for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7SourceInfo`, `IncludeL7SourceInfoNoPort`, and `IncludeL7SourceInfo`. | -| `L7LogsFileAggregationResponseCode` | `FELIX_L7LOGSFILEAGGREGATIONRESPONSECODE` | `IncludeL7ResponseCode` | How to handle response code data for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7ResponseCode` and `IncludeL7ResponseCode`. | -| `L7LogsFileAggregationTrimURL` | `FELIX_L7LOGSFILEAGGREGATIONTRIMURL` | `IncludeL7FullURL` | How to handle URL data for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7URL`, `TrimURLQuery`, `TrimURLQueryAndPath`, and `IncludeL7FullURL`. | -| `L7LogsFileAggregationNumURLPath` | `FELIX_L7LOGSFILEAGGREGATIONNUMURLPATH` | `5` | How many components in the path to limit the URL by. This parameter only takes effect when `L7LogsFileAggregationTrimURL` is set to `IncludeL7FullURL`. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Negative values set the limit to infinity. | - -DropActionOverride controls what happens to each packet that is denied by -the current $[prodname] policy - i.e. by the ordered combination of all the -configured policies and profiles that apply to that packet. It may be -set to one of the following values: - -- `Drop` -- `Accept` -- `LogAndDrop` -- `LogAndAccept` - -Normally the `Drop` or `LogAndDrop` value should be used, as dropping a -packet is the obvious implication of that packet being denied. However when -experimenting, or debugging a scenario that is not behaving as you expect, the -`Accept` and `LogAndAccept` values can be useful: then the packet will be -still be allowed through. - -When set to `LogAndDrop` or `LogAndAccept`, each denied packet is logged in -syslog, with an entry like this: - -``` -May 18 18:42:44 ubuntu kernel: [ 1156.246182] calico-drop: IN=tunl0 OUT=cali76be879f658 MAC= SRC=192.168.128.30 DST=192.168.157.26 LEN=60 TOS=0x00 PREC=0x00 TTL=62 ID=56743 DF PROTO=TCP SPT=56248 DPT=80 WINDOW=29200 RES=0x00 SYN URGP=0 MARK=0xa000000 -``` - -If the `LogDropActionOverride` flag is set, then the `DropActionOverride` will also appear in the syslog entry: - -``` -May 18 18:42:44 ubuntu kernel: [ 1156.246182] calico-drop LOGandDROP: IN=tunl0 OUT=cali76be879f658 MAC= SRC=192.168.128.30 DST=192.168.157.26 LEN=60 TOS=0x00 PREC=0x00 TTL=62 ID=56743 DF PROTO=TCP SPT=56248 DPT=80 WINDOW=29200 RES=0x00 SYN URGP=0 MARK=0xa000000 -``` - -When the reporting of denied packet metrics is enabled, Felix keeps counts of -recently denied packets and publishes these as Prometheus metrics on the port -configured by the `PrometheusReporterPort` setting. - -Note that denied packet metrics are independent of the DropActionOverride -setting. Specifically, if packets that would normally be denied are being -allowed through by a setting of `Accept` or `LogAndAccept`, those packets -still contribute to the denied packet metrics as just described. - -### Felix-Typha Configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ----------------------- | --------------------------- | ----------------------------------------------------------------------------------------- | ------ | -| `TyphaAddr` | `FELIX_TYPHAADDR` | Address of the Typha Server when running outside a K8S Cluster, in the format IP:PORT | string | -| `TyphaK8sServiceName` | `FELIX_TYPHAK8SSERVICENAME` | Service Name of Typha Deployment when running inside a K8S Cluster | string | -| `TyphaK8sNamespace` | `FELIX_TYPHAK8SNAMESPACE` | Namespace of Typha Deployment when running inside a K8S Cluster. [Default: `kube-system`] | string | -| `TyphaReadTimeout` | `FELIX_TYPHAREADTIMEOUT` | Timeout of Felix when reading information from Typha, in seconds. [Default: 30] | int | -| `TyphaWriteTimeout` | `FELIX_TYPHAWRITETIMEOUT` | Timeout of Felix when writing information to Typha, in seconds. [Default: 30] | int | - -### Felix-Typha TLS configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ----------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | -| `TyphaCAFile` | `FELIX_TYPHACAFILE` | Path to the file containing the root certificate of the CA that issued the Typha server certificate. Configures Felix to trust the CA that signed the root certificate. The file may contain multiple root certificates, causing Felix to trust each of the CAs included. Example: `/etc/felix/ca.pem` | string | -| `TyphaCertFile` | `FELIX_TYPHACERTFILE` | Path to the file containing the client certificate issued to Felix. Enables Felix to participate in mutual TLS authentication and identify itself to the Typha server. Example: `/etc/felix/cert.pem` | string | -| `TyphaCN` | `FELIX_TYPHACN` | If set, the `Common Name` that Typha's certificate must have. If you have enabled TLS on the communications from Felix to Typha, you must set a value here or in `TyphaURISAN`. You can set values in both, as well, such as to facilitate a migration from using one to the other. If either matches, the communication succeeds. [Default: none] | string | -| `TyphaKeyFile` | `FELIX_TYPHAKEYFILE` | Path to the file containing the private key matching the Felix client certificate. Enables Felix to participate in mutual TLS authentication and identify itself to the Typha server. Example: `/etc/felix/key.pem` (optional) | string | -| `TyphaURISAN` | `FELIX_TYPHAURISAN` | If set, a URI SAN that Typha's certificate must have. We recommend populating this with a [SPIFFE](https://github.com/spiffe/spiffe/blob/master/standards/SPIFFE-ID.md#2-spiffe-identity) string that identifies Typha. All Typha instances should use the same SPIFFE ID. If you have enabled TLS on the communications from Felix to Typha, you must set a value here or in `TyphaCN`. You can set values in both, as well, such as to facilitate a migration from using one to the other. If either matches, the communication succeeds. [Default: none] | string | - -For more information on how to use and set these variables, refer to -[Connections from Node to Typha (Kubernetes)](../../../../operations/comms/crypto-auth.mdx#connections-from-node-to-typha-kubernetes). - -### PacketCapture configuration - -The following parameters fine tune packet capture rotation: - -| Configuration parameter | Environment variable | Description | Schema | -| ------------------------ | --------------------------------- | --------------------------------------------------------------------------------------------- | ------ | -| `CaptureDir` | `FELIX_CAPTUREDIR` | Controls the directory where packet capture files are stored. Example: `/var/log/calico/pcap` | string | -| `CaptureMaxSizeBytes` | `FELIX_CAPTUREMAXSIZEBYTES` | Controls the maximum size in bytes for a packet capture file before rotation. | int | -| `CaptureRotationSeconds` | `FELIX_CAPTUREMAXROTATIONSECONDS` | Controls the rotation period in seconds for a packet capture file. | int | -| `CaptureMaxFiles` | `FELIX_CAPTUREMAXFILES` | Controls the maximum number rotated packet capture files. | int | - -### WireGuard configuration - -| Configuration parameter | Environment variable | Description | Schema | | -| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | ------- | -------------- | -| wireguardEnabled | Enable encryption for IPv4 on WireGuard supported nodes in cluster. When enabled, pod to pod traffic will be sent over encrypted tunnels between the nodes. | `true`, `false` | boolean | `false` | -| wireguardEnabledV6 | Enable encryption for IPv6 on WireGuard supported nodes in cluster. When enabled, pod to pod traffic will be sent over encrypted tunnels between the nodes. | `true`, `false` | boolean | `false` | -| wireguardInterfaceName | Name of the IPv4 WireGuard interface created by Felix. If you change the name, and want to clean up the previously-configured interface names on each node, this is a manual process. | string | string | wireguard.cali | -| wireguardInterfaceNameV6 | Name of the IPv6 WireGuard interface created by Felix. If you change the name, and want to clean up the previously-configured interface names on each node, this is a manual process. | string | string | wg-v6.cali | -| wireguardListeningPort | Port used by IPv4 WireGuard tunnels. Felix sets up an IPv4 WireGuard tunnel on each node specified by this port. Available for configuration only in the global FelixConfiguration resource; setting it per host, config-file or environment variable will not work. | 1-65535 | int | 51820 | -| wireguardListeningPortV6 | Port used by IPv6 WireGuard tunnels. Felix sets up an IPv6 WireGuard tunnel on each node specified by this port. Available for configuration only in the global FelixConfiguration resource; setting it per host, config-file or environment variable will not work. | 1-65535 | int | 51821 | -| wireguardMTU | MTU set on the IPv4 WireGuard interface created by Felix. Zero value means auto-detect. See [Configuring MTU](../../../../networking/configuring/mtu.mdx). | int | int | 0 | -| wireguardMTUV6 | MTU set on the IPv6 WireGuard interface created by Felix. Zero value means auto-detect. See [Configuring MTU](../../../../networking/configuring/mtu.mdx). | int | int | 0 | -| wireguardRoutingRulePriority | WireGuard routing rule priority value set up by Felix. If you change the default value, set it to a value most appropriate to routing rules for your nodes. | 1-32765 | int | 99 | -| wireguardHostEncryptionEnabled | **Experimental**: Adds host-namespace workload IP's to WireGuard's list of peers. Should **not** be enabled when WireGuard is enabled on a cluster's control plane node, as networking deadlock can occur. | true, false | boolean | false | -| wireguardKeepAlive | WireguardKeepAlive controls WireGuard PersistentKeepalive option. Set 0 to disable. [Default: 0] | int | int | 25 | - -For more information on encrypting in-cluster traffic with WireGuard, refer to -[Encrypt cluster pod traffic](../../../../compliance/encrypt-cluster-pod-traffic.mdx) +### Overlay: Wireguard + + + +### Overlay: IPSec + + + +### Flow logs: Prometheus reports + + + +### Flow logs: Syslog reports + + + +### Flow logs: file reports + + + +### DNS logs / policy + + + +### L7 logs + + + +### AWS integration + + + +### Egress gateway + + + +### External network support + + + +### Packet capture + + + +### L7 proxy + + + +### Debug/test-only (generally unsupported) + + + +### Usage reporting + + ## Environment variables @@ -383,10 +246,6 @@ include `FELIX_ETCDSCHEME`, `FELIX_ETCDKEYFILE`, `FELIX_ETCDCERTFILE`, `FELIX_ETCDCAFILE`, `FELIX_FELIXHOSTNAME`, `FELIX_LOGFILEPATH` and `FELIX_METADATAADDR`. -:::note -To set a parameter to an empty value using an environment variable, assign an empty string to it (e.g., `FELIX_FAILSAFEINBOUNDHOSTPORTS=""`). -::: - ## Configuration file On startup, Felix reads an ini-style configuration file. The path to @@ -432,7 +291,3 @@ a global setting and a per-host override. ``` For more information, see [Felix Configuration Resource](../../../resources/felixconfig.mdx). - - - - diff --git a/calico-cloud/reference/resources/felixconfig.mdx b/calico-cloud/reference/resources/felixconfig.mdx index f8671ef5f2..fc3aeb5f1d 100644 --- a/calico-cloud/reference/resources/felixconfig.mdx +++ b/calico-cloud/reference/resources/felixconfig.mdx @@ -1,7 +1,10 @@ --- description: API for this Calico Enterprise resource. +toc_max_heading_level: 4 --- +import FelixConfig from '../../_includes/components/FelixConfig'; + # Felix configuration A Felix configuration resource (`FelixConfiguration`) represents Felix configuration options for the cluster. @@ -37,196 +40,213 @@ spec: ### Spec -| Field | Description | Accepted Values | Schema | Default | -| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| awsSrcDstCheck | Controls automatically setting [source-destination-check](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCheck) on an AWS EC2 instance running Felix. Setting the value to `Enable` will set the check value in the instance description to `true`. For `Disable`, the check value will be `false`. Setting must be `Disable` if you want the EC2 instance to process traffic not matching the host interface IP address. For example, EKS cluster using Calico CNI with `VXLANMode=CrossSubnet`. Check [IAM role and profile configuration](#aws-iam-rolepolicy-for-source-destination-check-configuration) for setting the necessary permission for this setting to work. | DoNothing, Enable, Disable | string | `DoNothing` | -| awsSecondaryIPSupport | Controls whether Felix will create secondary AWS ENIs for AWS-backed IP pools. This feature is documented in the [egress gateways on AWS guide](../../networking/egress/egress-gateway-aws.mdx). Should only be enabled on AWS. | `Enabled`, `EnabledENIPerWorkload`, `Disabled` | string | `Disabled` | -| awsSecondaryIPRoutingRulePriority | Controls the priority of the policy-based routing rules used to implement AWS-backed IP addresses. Should only be changed to avoid conflicts if your nodes have additional policy based routing rules. | 0-4294967295 | int | 101 | -| awsRequestTimeout | Timeout used for communicating with the AWS API. | `5s`, `10s`, `1m` etc. | duration | `30s` | -| dropActionOverride | Controls what happens to each packet that is denied by the current $[prodname] policy. Normally the `Drop` or `LogAndDrop` value should be used. However when experimenting or debugging a scenario that is not behaving as you expect, the `Accept` and `LogAndAccept` values can be useful: then the packet will be still be allowed through. When one of the `LogAnd...` values is set, each denied packet is logged in syslog.\* | `Drop`, `Accept`, `LogAndDrop`, `LogAndAccept` | string | `Drop` | -| chainInsertMode | Controls whether Felix hooks the kernel's top-level iptables chains by inserting a rule at the top of the chain or by appending a rule at the bottom. `Insert` is the safe default since it prevents $[prodname]'s rules from being bypassed. If you switch to `Append` mode, be sure that the other rules in the chains signal acceptance by falling through to the $[prodname] rules, otherwise the $[prodname] policy will be bypassed. | `Insert`, `Append` | string | `Insert` | -| healthTimeoutOverrides | A list of overrides for Felix's internal liveness/readiness timeouts. | see [below](#health-timeout-overrides) | List of `HealthTimeoutOverride` objects | `[]` | -| dataplaneWatchdogTimeout | Deprecated, use `healthTimeoutOverrides` instead. Timeout before the main data plane goroutine is determined to have hung and Felix will report non-live and non-ready. Can be increased if the liveness check incorrectly fails (for example if Felix is running slowly on a heavily loaded system). | `90s`, `120s`, `10m` etc. | duration | `90s` | -| defaultEndpointToHostAction | This parameter controls what happens to traffic that goes from a workload endpoint to the host itself (after the traffic hits the endpoint egress policy). By default $[prodname] blocks traffic from workload endpoints to the host itself with an iptables "DROP" action. If you want to allow some or all traffic from endpoint to host, set this parameter to `Return` or `Accept`. Use `Return` if you have your own rules in the iptables "INPUT" chain; $[prodname] will insert its rules at the top of that chain, then `Return` packets to the "INPUT" chain once it has completed processing workload endpoint egress policy. Use `Accept` to unconditionally accept packets from workloads after processing workload endpoint egress policy. | Drop, Return, Accept | string | `Drop` | -| deviceRouteSourceAddress | IPv4 address to set as the source hint for routes programmed by Felix. When not set the source address for local traffic from host to workload will be determined by the kernel. | IPv4 | string | `""` | -| deviceRouteSourceAddressIPv6 | IPv6 address to set as the source hint for routes programmed by Felix. When not set the source address for local traffic from host to workload will be determined by the kernel. | IPv6 | string | `""` | -| deviceRouteProtocol | This defines the route protocol added to programmed device routes. | Protocol | int | RTPROT_BOOT | -| externalNodesCIDRList | A comma-delimited list of CIDRs of external non-calico nodes that can source tunnel traffic for acceptance by calico-nodes. | IPv4 | string | `""` | -| failsafeInboundHostPorts | UDP/TCP/SCTP protocol/cidr/port groupings that Felix will allow incoming traffic to host endpoints on irrespective of the security policy. This is useful to avoid accidentally cutting off a host with incorrect configuration. The default value allows SSH access, etcd, BGP, DHCP and the Kubernetes API. | | List of [ProtoPort](#protoport) |

- protocol: tcp
  port: 22
- protocol: udp
  port: 68
- protocol: tcp
  port: 179
- protocol: tcp
  port: 2379
- protocol: tcp
  port: 2380
- protocol: tcp
  port: 5473
- protocol: tcp
  port: 6443
- protocol: tcp
  port: 6666
- protocol: tcp
  port: 6667

| -| failsafeOutboundHostPorts | UDP/TCP/SCTP protocol/port groupings that Felix will allow outgoing traffic from host endpoints to irrespective of the security policy. This is useful to avoid accidentally cutting off a host with incorrect configuration. The default value opens etcd's standard ports to ensure that Felix does not get cut off from etcd as well as allowing DHCP, DNS, BGP and the Kubernetes API. | | List of [ProtoPort](#protoport) |

- protocol: udp
  port: 53
- protocol: udp
  port: 67
- protocol: tcp
  port: 179
- protocol: tcp
  port: 2379
- protocol: tcp
  port: 2380
- protocol: tcp
  port: 5473
- protocol: tcp
  port: 6443
- protocol: tcp
  port: 6666
- protocol: tcp
  port: 6667

| -| featureDetectOverride | Is used to override the feature detection. Values are specified in a comma separated list with no spaces, example; "SNATFullyRandom=true,MASQFullyRandom=false,RestoreSupportsLock=". "true" or "false" will force the feature, empty or omitted values are auto-detected. | string | string | `""` | -| genericXDPEnabled | When enabled, Felix can fallback to the non-optimized `generic` XDP mode. This should only be used for testing since it doesn't improve performance over the non-XDP mode. | true,false | boolean | `false` | -| interfaceExclude | A comma-separated list of interface names that should be excluded when Felix is resolving host endpoints. The default value ensures that Felix ignores Kubernetes' internal `kube-ipvs0` device. If you want to exclude multiple interface names using a single value, the list supports regular expressions. For regular expressions you must wrap the value with `/`. For example having values `/^kube/,veth1` will exclude all interfaces that begin with `kube` and also the interface `veth1`. | string | string | `kube-ipvs0` | -| interfacePrefix | The interface name prefix that identifies workload endpoints and so distinguishes them from host endpoint interfaces. Note: in environments other than bare metal, the orchestrators configure this appropriately. For example our Kubernetes and Docker integrations set the 'cali' value, and our OpenStack integration sets the 'tap' value. | string | string | `cali` | -| ipipEnabled | Optional, you shouldn't need to change this setting as Felix calculates if IPIP should be enabled based on the existing IP Pools. When set, this overrides whether Felix should configure an IPinIP interface on the host. When explicitly disabled in FelixConfiguration, Felix will not clean up addresses from the `tunl0` interface (use this if you need to add addresses to that interface and don't want to have them removed). | `true`, `false`, unset | optional boolean | unset | -| ipipMTU | The MTU to set on the tunnel device. Zero value means auto-detect. See [Configuring MTU](../../networking/configuring/mtu.mdx) | int | int | `0` | -| ipsetsRefreshInterval | Period at which Felix re-checks the IP sets in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable IP sets refresh. | `5s`, `10s`, `1m` etc. | duration | `10s` | -| iptablesFilterAllowAction | This parameter controls what happens to traffic that is accepted by a Felix policy chain in the iptables filter table (i.e. a normal policy chain). The default will immediately `Accept` the traffic. Use `Return` to send the traffic back up to the system chains for further processing. | Accept, Return | string | `Accept` | -| iptablesBackend | This parameter controls which variant of iptables Felix uses. If using Felix on a system that uses the netfilter-backed iptables binaries, set this to `nft`. | Legacy, nft | string | automatic detection | -| iptablesLockFilePath | Location of the iptables lock file. You may need to change this if the lock file is not in its standard location (for example if you have mapped it into Felix's container at a different path). | string | string | `/run/xtables.lock` | -| iptablesLockProbeInterval | Time that Felix will wait between attempts to acquire the iptables lock if it is not available. Lower values make Felix more responsive when the lock is contended, but use more CPU. | `5s`, `10s`, `1m` etc. | duration | `50ms` | -| iptablesLockTimeout | Time that Felix will wait for the iptables lock, or 0, to disable. To use this feature, Felix must share the iptables lock file with all other processes that also take the lock. When running Felix inside a container, this requires the /run directory of the host to be mounted into the $[nodecontainer] or calico/felix container. | `5s`, `10s`, `1m` etc. | duration | `0` (Disabled) | -| iptablesMangleAllowAction | This parameter controls what happens to traffic that is accepted by a Felix policy chain in the iptables mangle table (i.e. a pre-DNAT policy chain). The default will immediately `Accept` the traffic. Use `Return` to send the traffic back up to the system chains for further processing. | `Accept`, `Return` | string | `Accept` | -| iptablesMarkMask | Mask that Felix selects its IPTables Mark bits from. Should be a 32 bit hexadecimal number with at least 8 bits set, none of which clash with any other mark bits in use on the system. | netmask | netmask | `0xffff0000` | -| iptablesNATOutgoingInterfaceFilter | This parameter can be used to limit the host interfaces on which Calico will apply SNAT to traffic leaving a Calico IPAM pool with "NAT outgoing" enabled. This can be useful if you have a main data interface, where traffic should be SNATted and a secondary device (such as the docker bridge) which is local to the host and doesn't require SNAT. This parameter uses the iptables interface matching syntax, which allows `+` as a wildcard. Most users will not need to set this. Example: if your data interfaces are eth0 and eth1 and you want to exclude the docker bridge, you could set this to `eth+` | string | string | `""` | -| iptablesPostWriteCheckInterval | Period after Felix has done a write to the data plane that it schedules an extra read back to check the write was not clobbered by another process. This should only occur if another application on the system doesn't respect the iptables lock. | `5s`, `10s`, `1m` etc. | duration | `1s` | -| iptablesRefreshInterval | Period at which Felix re-checks all iptables state to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable iptables refresh. | `5s`, `10s`, `1m` etc. | duration | `90s` | -| ipv6Support | IPv6 support for Felix | `true`, `false` | boolean | `true` | -| logFilePath | The full path to the Felix log. Set to `none` to disable file logging. | string | string | `/var/log/calico/felix.log` | -| logPrefix | The log prefix that Felix uses when rendering LOG rules. | string | string | `calico-packet` | -| logSeverityFile | The log severity above which logs are sent to the log file. | Same as logSeveritySys | string | `Info` | -| logSeverityScreen | The log severity above which logs are sent to the stdout. | Same as logSeveritySys | string | `Info` | -| logSeveritySys | The log severity above which logs are sent to the syslog. Set to `none` for no logging to syslog. | Debug, Info, Warning, Error, Fatal | string | `Info` | -| logDebugFilenameRegex | controls which source code files have their Debug log output included in the logs. Only logs from files with names that match the given regular expression are included. The filter only applies to Debug level logs. | regex | string | `""` | -| maxIpsetSize | Maximum size for the ipsets used by Felix. Should be set to a number that is greater than the maximum number of IP addresses that are ever expected in a selector. | int | int | `1048576` | -| metadataAddr | The IP address or domain name of the server that can answer VM queries for cloud-init metadata. In OpenStack, this corresponds to the machine running nova-api (or in Ubuntu, nova-api-metadata). A value of `none` (case insensitive) means that Felix should not set up any NAT rule for the metadata path. | IPv4, hostname, none | string | `127.0.0.1` | -| metadataPort | The port of the metadata server. This, combined with global.MetadataAddr (if not 'None'), is used to set up a NAT rule, from 169.254.169.254:80 to MetadataAddr:MetadataPort. In most cases this should not need to be changed. | int | int | `8775` | -| natOutgoingAddress | The source address to use for outgoing NAT. By default an iptables MASQUERADE rule determines the source address which will use the address on the host interface the traffic leaves on. | IPV4 | string | `""` | -| policySyncPathPrefix | File system path where Felix notifies services of policy changes over Unix domain sockets. This is required only if you're configuring [L7 logs](../../observability/elastic/l7/configure.mdx), or [egress gateways](../../networking/egress/index.mdx). Set to `""` to disable. | string | string | `""` | -| prometheusGoMetricsEnabled | Set to `false` to disable Go runtime metrics collection, which the Prometheus client does by default. This reduces the number of metrics reported, reducing Prometheus load. | boolean | boolean | `true` | -| prometheusMetricsEnabled | Set to `true` to enable the experimental Prometheus metrics server in Felix. | boolean | boolean | `false` | -| prometheusMetricsHost | TCP network address that the Prometheus metrics server should bind to. | IPv4, IPv6, Hostname | string | `""` | -| prometheusMetricsPort | TCP port that the Prometheus metrics server should bind to. | int | int | `9091` | -| prometheusProcessMetricsEnabled | Set to `false` to disable process metrics collection, which the Prometheus client does by default. This reduces the number of metrics reported, reducing Prometheus load. | boolean | boolean | `true` | -| prometheusReporterEnabled | Set to `true` to enable configure Felix to keep count of recently denied packets and publish these as Prometheus metrics. Note that denied packet metrics are independent of the `dropActionOverride` setting. Specifically, if packets that would normally be denied are being allowed through by a setting of `Accept` or `LogAndAccept`, those packets still get counted as denied packets. | `true`, `false` | boolean | `false` | -| prometheusReporterPort | The TCP port on which to report denied packet metrics, if `prometheusReporterEnabled` is set to `true`. | | | `9092` | -| removeExternalRoutes | Whether or not to remove device routes that have not been programmed by Felix. Disabling this will allow external applications to also add device routes. | bool | boolean | `true` | -| reportingInterval | Interval at which Felix reports its status into the datastore. 0 means disabled and is correct for Kubernetes-only clusters. Must be non-zero in OpenStack deployments. | `5s`, `10s`, `1m` etc. | duration | `30s` | -| reportingTTL | Time-to-live setting for process-wide status reports. | `5s`, `10s`, `1m` etc. | duration | `90s` | -| routeRefreshInterval | Period at which Felix re-checks the routes in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable route refresh. | `5s`, `10s`, `1m` etc. | duration | `90s` | -| ipsecMode | Controls which mode IPsec is operating on. The only supported value is `PSK`. An empty value means IPsec is not enabled. | PSK | string | `""` | -| ipsecAllowUnsecuredTraffic | When set to `false`, only IPsec-protected traffic will be allowed on the packet paths where IPsec is supported. When set to `true`, IPsec will be used but non-IPsec traffic will be accepted. In general, setting this to `true` is less safe since it allows an attacker to inject packets. However, it is useful when transitioning from non-IPsec to IPsec since it allows traffic to flow while the cluster negotiates the IPsec mesh. | `true`, `false` | boolean | `false` | -| ipsecIKEAlgorithm | IPsec IKE algorithm. Default is NIST suite B recommendation. | string | string | `aes128gcm16-prfsha256-ecp256` | -| ipsecESPAlgorithm | IPsec ESP algorithm. Default is NIST suite B recommendation. | string | string | `aes128gcm16-ecp256` | -| ipsecLogLevel | Controls log level for IPsec components. Set to `None` for no logging. | `None`, `Notice`, `Info`, `Debug`, `Verbose` | string | `Info` | -| ipsecPSKFile | The path to the pre shared key file for IPsec. | string | string | `""` | -| flowLogsFileEnabled | Set to `true`, enables flow logs. If set to `false` no flow logging will occur. Flow logs are written to a file `flows.log` and sent to Elasticsearch. The location of this file can be configured using the `flowLogsFileDirectory` field. File rotation settings for this `flows.log` file can be configured using the fields `flowLogsFileMaxFiles` and `flowLogsFileMaxFileSizeMB`. Note that flow log exports to Elasticsearch are dependent on flow logs getting written to this file. Setting this parameter to `false` will disable flow logs. | `true`, `false` | boolean | `false` | -| flowLogsFileDirectory | Set the directory where flow logs files are stored on Linux nodes. This parameter only takes effect when `flowLogsFileEnabled` is set to `true`. | string | string | `/var/log/calico/flowlogs` | -| flowLogsPositionFilePath | Specify the position of the external pipeline that reads flow logs on Linux nodes. This parameter only takes effect when `FlowLogsDynamicAggregationEnabled` is set to `true`. | string | string | `/var/log/calico/flows.log.pos` | -| flowLogsFileMaxFiles | Set the number of log files to keep. This parameter only takes effect when `flowLogsFileEnabled` is set to `true`. | int | int | `5` | -| flowLogsFileMaxFileSizeMB | Set the max size in MB of flow logs files before rotation. This parameter only takes effect when `flowLogsFileEnabled` is set to `true`. | int | int | `100` | -| flowLogsFlushInterval | The period, in seconds, at which Felix exports the flow logs. | int | int | `300s` | -| flowLogsFileAggregationKindForAllowed | How much to aggregate the flow logs sent to Elasticsearch for allowed traffic. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. | 0-2 | [AggregationKind](#aggregationkind) | 2 | -| flowLogsFileAggregationKindForDenied | How much to aggregate the flow logs sent to Elasticsearch for denied traffic. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. | 0-2 | [AggregationKind](#aggregationkind) | 1 | -| flowLogsFileIncludeService | When set to `true`, include destination service information in the aggregated flow log. Note that service information will only be included when the flow can be explicitly determined to be bound to a service (e.g. pre-DNAT destination matches a service ClusterIP). | `true`, `false` | boolean | `false` | -| flowLogsFileIncludeLabels | When set to `true`, include source and destination endpoint labels in the aggregated flow log. Note that only Kubernetes endpoints or network sets are included; arbitrary networks do not contain labels. | `true`, `false` | boolean | `false` | -| flowLogsFileIncludePolicies | When set to `true`, include all policies in the aggregated flow logs that acted upon and matches the flow log traffic. | `true`, `false` | boolean | `false` | -| flowLogsDestDomainsByClient | When set to true, top-level domains are strictly associated with the source IP that originally queried the domains. (default: true) -| flowLogsEnableNetworkSets | When set to `true`, include an arbitrary network set in the aggregated flow log that matches the IP address of the flow log endpoint. | `true`, `false` | boolean | `false` | -| flowLogsCollectProcessInfo | When set to `true`, Felix will load the kprobe BPF programs to collect process info. | `true`, `false` | boolean | `false` | -| flowLogsCollectTcpStats | When set to `true`, Felix will collect the TCP socket stats. | `true`, `false` | boolean | `true` | -| flowLogsCollectProcessPath | When set to `true`, along with flowLogsCollectProcessInfo, each flow log will include the full path of the executable and the arguments with which the executable was invoked. | `true`, `false` | boolean | `false` | -| flowLogsFilePerFlowProcessLimit | Specify the maximum number of flow log entries with distinct process information beyond which process information will be aggregated | int | int | `2` | -| flowLogsFileNatOutgoingPortLimit | Specify the maximum number of distinct post SNAT ports that will appear in the flowLogs | int | int | `3` | -| flowLogsFilePerFlowProcessArgsLimit | Specify the maximum number of unique arguments in the flowlogs beyond which process arguments will be aggregated | int | int | `5` | -| flowLogsFileDomainsLimit | Specify the maximum number of top-level domains to include in a flow log. This only applies to source reported flows to destinations external to the cluster. | int | int | `5` | -| flowLogsPolicyEvaluationMode | Defines how policies are evaluated and reflected in flow logs. `OnNewConnection`: In this mode, staged policies are only evaluated when new connections are made in the data plane. Staged/active policy changes will not be reflected in the `pending_policies` field of flow logs for long lived connections. `Continuous`: Felix evaluates active flows on a regular basis to determine the rule traces in the flow logs. Any policy updates that impact a flow will be reflected in the `pending_policies` field, offering a near-real-time view of policy changes across flows. | `Continuous`, `OnNewConnection` | string | `Continuous` | -| statsDumpFilePath | Specify the position of the file used for dumping flow log statistics on Linux nodes. Note this is an internal setting that users shouldn't need to modify. | string | string | `/var/log/calico/stats/dump` | -| routeTableRange | _deprecated in favor of `RouteTableRanges`_ Calico programs additional Linux route tables for various purposes. `RouteTableRange` specifies the indices of the route tables that Calico should use. | | [RouteTableRanges](#routetablerange) | `""` | -| routeTableRanges | Calico programs additional Linux route tables for various purposes. `RouteTableRanges` specifies a set of table index ranges that Calico should use. Deprecates `RouteTableRange`, overrides `RouteTableRange` | | [RouteTableRanges](#routetableranges) | `[{"min": 1, "max": 250}]` | -| routeSyncDisabled | Set to `true` to disable Calico programming routes to local workloads. | boolean | boolean | `false` | -| serviceLoopPrevention | When [service IP advertisement is enabled](../../networking/configuring/advertise-service-ips.mdx), prevent routing loops to service IPs that are not in use, by dropping or rejecting packets that do not get DNAT'd by kube-proxy. Unless set to "Disabled", in which case such routing loops continue to be allowed. | `Drop`, `Reject`, `Disabled` | string | `Drop` | -| workloadSourceSpoofing | Controls whether pods can enable source IP address spoofing with the `cni.projectcalico.org/allowedSourcePrefixes` annotation. When set to `Any`, pods can use this annotation to send packets from any IP address. | `Any`, `Disabled` | string | `Disabled` | -| vxlanEnabled | Optional, you shouldn't need to change this setting as Felix calculates if VXLAN should be enabled based on the existing IP Pools. When set, this overrides whether Felix should create the VXLAN tunnel device for VXLAN networking. | `true`, `false`, unset | optional boolean | unset | -| vxlanMTU | MTU to use for the IPv4 VXLAN tunnel device. Zero value means auto-detect. Also controls NodePort MTU when eBPF enabled. | int | int | `0` | -| vxlanMTUV6 | MTU to use for the IPv6 VXLAN tunnel device. Zero value means auto-detect. Also controls NodePort MTU when eBPF enabled. | int | int | `0` | -| vxlanPort | Port to use for VXLAN traffic. A value of `0` means "use the kernel default". | int | int | `4789` | -| vxlanVNI | Virtual network ID to use for VXLAN traffic. A value of `0` means "use the kernel default". | int | int | `4096` | -| allowVXLANPacketsFromWorkloads | Set to `true` to allow VXLAN encapsulated traffic from workloads. | boolean | boolean | `false` | -| allowIPIPPacketsFromWorkloads | Set to `true` to allow IPIP encapsulated traffic from workloads. | boolean | boolean | `false` | -| windowsFlowLogsFileDirectory | Set the directory where flow logs files are stored on Windows nodes. This parameter only takes effect when `flowLogsFileEnabled` is set to `true`. | string | string | `c:\\TigeraCalico\\flowlogs` | -| windowsFlowLogsPositionFilePath | Specify the position of the external pipeline that reads flow logs on Windows nodes. This parameter only takes effect when `FlowLogsDynamicAggregationEnabled` is set to `true`. | string | string | `c:\\TigeraCalico\\flowlogs\\flows.log.pos` | -| windowsStatsDumpFilePath | Specify the position of the file used for dumping flow log statistics on Windows nodes. Note this is an internal setting that users shouldn't need to modify. | string | string | `c:\\TigeraCalico\\stats\\dump` | -| windowsDNSCacheFile | Specify the name of the file that Calico uses to preserve learnt DNS information when restarting. | string | string | `c:\\TigeraCalico\\felix-dns-cache.txt` | -| windowsDNSExtraTTL | Specify extra time in seconds to keep IPs and alias names that are learnt from DNS, in addition to each name or IP's advertised TTL. | int | int | `120` | -| wireguardEnabled | Enable encryption on WireGuard supported nodes in cluster. When enabled, pod to pod traffic will be sent over encrypted tunnels between the nodes. | `true`, `false` | boolean | `false` | -| wireguardEnabledV6 | Enable encryption for IPv6 on WireGuard supported nodes in cluster. When enabled, pod to pod traffic will be sent over encrypted tunnels between the nodes. | `true`, `false` | boolean | `false` | -| wireguardInterfaceName | Name of the WireGuard interface created by Felix. If you change the name, and want to clean up the previously-configured interface names on each node, this is a manual process. Felix expect the name to end with either .cali or .calico suffix. | string | string | wireguard.cali | -| wireguardInterfaceNameV6 | Name of the IPv6 WireGuard interface created by Felix. If you change the name, and want to clean up the previously-configured interface names on each node, this is a manual process. Felix expect the name to end with either .cali or .calico suffix. | string | string | wg-v6.cali | -| wireguardListeningPort | Port used by WireGuard tunnels. Felix sets up WireGuard tunnel on each node specified by this port. Available for configuration only in the global FelixConfiguration resource; setting it per host, config-file or environment variable will not work. | 1-65535 | int | 51820 | -| wireguardListeningPortV6 | Port used by IPv6 WireGuard tunnels. Felix sets up an IPv6 WireGuard tunnel on each node specified by this port. Available for configuration only in the global FelixConfiguration resource; setting it per host, config-file or environment variable will not work. | 1-65535 | int | 51821 | -| wireguardMTU | MTU set on the WireGuard interface created by Felix. Zero value means auto-detect. See [Configuring MTU](../../networking/configuring/mtu.mdx). | int | int | 0 | -| wireguardMTUV6 | MTU set on the IPv6 WireGuard interface created by Felix. Zero value means auto-detect. See [Configuring MTU](../../networking/configuring/mtu.mdx). | int | int | 0 | -| wireguardRoutingRulePriority | WireGuard routing rule priority value set up by Felix. If you change the default value, set it to a value most appropriate to routing rules for your nodes. | 1-32765 | int | 99 | -| wireguardHostEncryptionEnabled | **Experimental**: Adds host-namespace workload IP's to WireGuard's list of peers. Should **not** be enabled when WireGuard is enabled on a cluster's control plane node, as networking deadlock can occur. | true, false | boolean | false | -| wireguardKeepAlive | WireguardKeepAlive controls WireGuard PersistentKeepalive option. Set 0 to disable. [Default: 0] | `5s`, `10s`, `1m` etc. | duration | `0` | -| xdpRefreshInterval | Period at which Felix re-checks the XDP state in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable XDP refresh. | `5s`, `10s`, `1m` etc. | duration | `90s` | -| xdpEnabled | When `bpfEnabled` is `false`: enable XDP acceleration for host endpoint policies. When `bpfEnabled` is `true`, XDP is automatically used for Calico policy where that makes sense, regardless of this setting. [Default: `true`] | true,false | boolean | `true` | -| dnsCacheFile | The name of the file that Felix uses to preserve learnt DNS information when restarting. | file name | string | `/var/run/calico/felix-dns-cache.txt` | -| dnsCacheSaveInterval | The period, in seconds, at which Felix saves learnt DNS information to the cache file. | `5s`, `10s`, `1m` etc. | duration | `60s` | -| dnsCacheEpoch | An arbitrary number that can be changed, at runtime, to tell Felix to discard all its learnt DNS information. | int | int | `0` | -| dnsExtraTTL | Extra time to keep IPs and alias names that are learnt from DNS, in addition to each name or IP's advertised TTL. | `5s`, `10s`, `1m` etc. | duration | `0s` | -| dnsTrustedServers | The DNS servers that Felix should trust. Each entry here must be `[:]` - indicating an explicit DNS server IP - or `k8s-service:[/][:port]` - indicating a Kubernetes DNS service. `` defaults to the first service port, or 53 for an IP, and `` to `kube-system`. An IPv6 address with a port must use the square brackets convention, for example `[fd00:83a6::12]:5353`. Note that Felix (calico-node) will need RBAC permission to read the details of each service specified by a `k8s-service:...` form. | IPs or service names | comma-separated strings | `k8s-service:kube-dns` | -| dnsLogsFileEnabled | Set to `true`, enables DNS logs. If set to `false` no DNS logging will occur. DNS logs are written to a file `dns.log` and sent to Elasticsearch. The location of this file can be configured using the `DNSLogsFileDirectory` field. File rotation settings for this `dns.log` file can be configured using the fields `DNSLogsFileMaxFiles` and `DNSLogsFileMaxFileSizeMB`. Note that DNS log exports to Elasticsearch are dependent on DNS logs getting written to this file. Setting this parameter to `false` will disable DNS logs. | `true`, `false` | boolean | `false` | -| dnsLogsFileDirectory | The directory where DNS logs files are stored. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | directory | string | `/var/log/calico/dnslogs` | -| dnsLogsFileMaxFiles | The number of files to keep when rotating DNS log files. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | int | int | `5` | -| dnsLogsFileMaxFileSizeMB | The max size in MB of DNS log files before rotation. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | int | int | `100` | -| dnsLogsFlushInterval | The period, in seconds, at which Felix exports DNS logs. | int | int | `300s` | -| dnsLogsFileAggregationKind | How much to aggregate DNS logs. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. `0` means no aggregation, `1` means aggregate similar DNS logs from workloads in the same ReplicaSet. | `0`,`1` | int | `1` | -| dnsLogsFileIncludeLabels | Whether to include client and server workload labels in DNS logs. | `true`, `false` | boolean | `true` | -| dnsLogsFilePerNodeLimit | Limit on the number of DNS logs that can be emitted within each flush interval. When this limit has been reached, Felix counts the number of unloggable DNS responses within the flush interval, and emits a WARNING log with that count at the same time as it flushes the buffered DNS logs. | int | int | `0` (no limit) | -| dnsLogsLatency | Indicates to include measurements of DNS request/response latency in each DNS log. | `true`, `false` | boolean | `true` | -| dnsPolicyMode | DNSPolicyMode specifies how DNS policy programming will be handled. | `Inline`, `NoDelay`, `DelayDNSResponse`, `DelayDeniedPacket` | [DNSPolicyMode](#dnspolicymode) | `Inline` | -| bpfDNSPolicyMode | BPFDNSPolicyMode specifies how DNS policy programming will be handled. | `NoDelay`, `Inline` | [BPFDNSPolicyMode](#bpfdnspolicymode) | `Inline` | -| nftablesDNSPolicyMode | Specifies how DNS policy programming will be handled for NFTables. Options include `NoDelay`, `DelayDNSResponse`, and `DelayDeniedPacket`. See [NFTablesDNSPolicyMode](#nftablesdnspolicymode) for details. | `NoDelay`, `DelayDNSResponse`, `DelayDeniedPacket` | [NFTablesDNSPolicyMode](#nftablesdnspolicymode) | `DelayDeniedPacket` | -| dnsPolicyNfqueueID | DNSPolicyNfqueueID is the NFQUEUE ID to use for DNS Policy re-evaluation when the domains IP hasn't been programmed to ipsets yet. This value can be changed to avoid conflicts with other users of NFQUEUEs. Used when `DNSPolicyMode` is `DelayDeniedPacket`. | 0-65535 | int | `100` | -| dnsPolicyNfqueueSize | DNSPolicyNfqueueID is the size of the NFQUEUE for DNS policy re-evaluation. This is the maximum number of denied packets that may be queued up pending re-evaluation. Used when `DNSPolicyMode` is `DelayDeniedPacket`. | 0-65535 | int | `100` | -| dnsPacketsNfqueueID | DNSPacketsNfqueueID is the NFQUEUE ID to use for capturing DNS packets to ensure programming IPSets occurs before the response is released. Used when `DNSPolicyMode` is `DelayDNSResponse`. | 0-65535 | int | `101` | -| dnsPacketsNfqueueSize | DNSPacketsNfqueueSize is the size of the NFQUEUE for captured DNS packets. This is the maximum number of DNS packets that may be queued awaiting programming in the data plane. Used when `DNSPolicyMode` is `DelayDNSResponse`. | 0-65535 | int | `100` | -| dnsPacketsNfqueueMaxHoldDuration | DNSPacketsNfqueueMaxHoldDuration is the max length of time to hold on to a DNS response while waiting for the data plane to be programmed. Used when `DNSPolicyMode` is `DelayDNSResponse`. | `5s`, `10s`, `1m` etc. | duration | `3s` | -| bpfEnabled | Enable eBPF data plane mode. eBPF mode has some limitations, see the [HOWTO guide](../../operations/ebpf/enabling-ebpf.mdx) for more details. | true, false | boolean | false | -| bpfDisableUnprivileged | If true, Felix sets the kernel.unprivileged_bpf_disabled sysctl to disable unprivileged use of BPF. This ensures that unprivileged users cannot access Calico's BPF maps and cannot insert their own BPF programs to interfere with the ones that $[prodname] installs. | true, false | boolean | true | -| bpfLogLevel | In eBPF data plane mode, the log level used by the BPF programs. The logs are emitted to the BPF trace pipe, accessible with the command `tc exec bpf debug`. This is a tech preview feature and subject to change in future releases. | Off,Info,Debug | string | Off | -| bpfDataIfacePattern | In eBPF data plane mode, controls which interfaces Felix should attach BPF programs to catch traffic to/from the external network. This needs to match the interfaces that Calico workload traffic flows over as well as any interfaces that handle incoming traffic to NodePorts and services from outside the cluster. It should not match the workload interfaces (usually named cali...).. This is a tech preview feature and subject to change in future releases. | regular expression | string | ^(en.*|eth.*|tunl0$) | -| bpfConnectTimeLoadBalancingEnabled | In eBPF data plane mode, controls whether Felix installs the connect-time load balancer. In the current release, the connect-time load balancer is required for the host to reach kubernetes services. This is a tech preview feature and subject to change in future releases. | true,false | boolean | true | -| bpfExternalServiceMode | In eBPF data plane mode, controls how traffic from outside the cluster to NodePorts and ClusterIPs is handled. In Tunnel mode, packet is tunneled from the ingress host to the host with the backing pod and back again. In DSR mode, traffic is tunneled to the host with the backing pod and then returned directly; this requires a network that allows direct return. | Tunnel,DSR | string | Tunnel | -| bpfKubeProxyIptablesCleanupEnabled | In eBPF data plane mode, controls whether Felix will clean up the iptables rules created by the Kubernetes `kube-proxy`; should only be enabled if `kube-proxy` is not running. This is a tech preview feature and subject to change in future releases. | true,false | boolean | true | -| bpfKubeProxyMinSyncPeriod | In eBPF data plane mode, controls the minimum time between data plane updates for Felix's embedded `kube-proxy` implementation. | `5s`, `10s`, `1m` etc. | duration | `1s` | -| BPFKubeProxyEndpointSlicesEnabled | In eBPF data plane mode, controls whether Felix's embedded kube-proxy derives its services from Kubernetes' EndpointSlices resources. Using EndpointSlices is more efficient but it requires EndpointSlices support to be enabled at the Kubernetes API server. | true,false | boolean | false | -| bpfMapSizeConntrack | In eBPF data plane mode, controls the size of the conntrack map. | int | int | 512000 | -| bpfMapSizeIPSets | In eBPF data plane mode, controls the size of the ipsets map. | int | int | 1048576 | -| bpfMapSizeNATAffinity | In eBPF data plane mode, controls the size of the NAT affinity map. | int | int | 65536 | -| bpfMapSizeNATFrontend | In eBPF data plane mode, controls the size of the NAT front end map. | int | int | 65536 | -| bpfMapSizeNATBackend | In eBPF data plane mode, controls the size of the NAT back end map. | int | int | 262144 | -| bpfMapSizeRoute | In eBPF data plane mode, controls the size of the route map. | int | int | 262144 | -| bpfPolicyDebugEnabled | In eBPF data plane mode, controls whether felix will collect policy dump for each interface. | true, false | boolean | true | -| routeSource | Where Felix gets is routing information from for VXLAN and the BPF data plane. The CalicoIPAM setting is more efficient because it supports route aggregation, but it only works when Calico's IPAM or host-local IPAM is in use. Use the WorkloadIPs setting if you are using Calico's VXLAN or BPF data plane and not using Calico IPAM or host-local IPAM. | CalicoIPAM,WorkloadIPs | string | `CalicoIPAM` | -| mtuIfacePattern | Pattern used to discover the host's interface for MTU auto-detection. | regex | string | ^((en|wl|ww|sl|ib)[opsvx].*|(eth|wlan|wwan).*) | -| bpfForceTrackPacketsFromIfaces | Forces traffic from these interfaces in BPF mode to skip Calico's iptables NOTRACK rule, allowing traffic from those interfaces to be tracked by Linux conntrack. Use only for interfaces that are not used for the Calico fabric, for example, a docker bridge device for non-Calico-networked containers. | A list of strings | A list of strings | docker+ | -| bpfDisableGROForIfaces | BPFDisableGROForIfaces is a regular expression that controls which interfaces Felix should disable the Generic Receive Offload [GRO] option. It should not match the workload interfaces (usually named cali...). | regex | string | "" | -| egressIPSupport | Defines three different support modes for egress gateway function. `Disabled` means egress gateways are not supported. `EnabledPerNamespace` means egress gateway function is enabled and can be configured on a per-namespace basis (but per-pod egress annotations are ignored). `EnabledPerNamespaceOrPerPod` means egress gateway function is enabled and can be configured per-namespace or per-pod (with per-pod egress annotations overriding namespace annotations). | Disabled,
EnabledPerNamespace,
EnabledPerNamespaceOrPerPod | string | `Disabled` | -| egressIPVXLANPort | Port to use for egress gateway VXLAN traffic. A value of `0` means "use the kernel default". | int | int | `4790` | -| egressIPVXLANVNI | Virtual network ID to use for egress gateway VXLAN traffic. A value of `0` means "use the kernel default". | int | int | `4097` | -| egressIPRoutingRulePriority | Controls the priority value to use for the egress gateway routing rule. | int | int | `100` | -| egressGatewayPollInterval | Controls the interval at which Felix will poll remote egress gateways to check their health. Only Egress Gateways with a named "health" port will be polled in this way. Egress Gateways that fail the health check will be taken our of use as if they have been deleted. | `5s`, `10s`, `1m` etc. | duration | `10s` | -| egressGatewayPollFailureCount | Controls the minimum number of poll failures before a remote Egress Gateway is considered to have failed. | int | int | `3` | -| captureDir | Controls the directory where packet capture files are stored. | string | string | `/var/log/calico/pcap` | -| captureMaxSizeBytes | Controls the maximum size in bytes for a packet capture file before rotation. | int | int | `10000000` | -| captureRotationSeconds | Controls the rotation period in seconds for a packet capture file. | int | int | `3600` | -| captureMaxFiles | Controls the maximum number rotated packet capture files. | int | int | `2` | - -\* When `dropActionOverride` is set to `LogAndDrop` or `LogAndAccept`, the `syslog` entries look something like the following. +#### Datastore connection -``` -May 18 18:42:44 ubuntu kernel: [ 1156.246182] calico-drop: IN=tunl0 OUT=cali76be879f658 MAC= SRC=192.168.128.30 DST=192.168.157.26 LEN=60 TOS=0x00 PREC=0x00 TTL=62 ID=56743 DF PROTO=TCP SPT=56248 DPT=80 WINDOW=29200 RES=0x00 SYN URGP=0 MARK=0xa000000 -``` + + +#### Process: Feature detection/overrides + + + +#### Process: Go runtime + + + +#### Process: Health port and timeouts + + + +#### Process: Logging + + + +#### Process: Prometheus metrics + + + +#### Dataplane: Common + + + +#### Dataplane: iptables + + + +#### Dataplane: nftables + + + +#### Dataplane: eBPF + + + +#### Dataplane: Windows + + + +#### Dataplane: OpenStack support + + + +#### Dataplane: XDP acceleration for iptables dataplane + + + +#### Overlay: VXLAN overlay + + + +#### Overlay: IP-in-IP -\*\* Duration is denoted by the numerical amount followed by the unit of time. Valid units of time include nanoseconds (ns), microseconds (µs), milliseconds (ms), seconds (s), minutes (m), and hours (h). Units of time can also be used together e.g. `3m30s` to represent 3 minutes and 30 seconds. Any amounts of time that can be converted into larger units of time will be converted e.g. `90s` will become `1m30s`. + -
+#### Overlay: Wireguard -`genericXDPEnabled` and `xdpRefreshInterval` are only relevant when `bpfEnabled` is `false` and -`xdpEnabled` is `true`; in other words when XDP is being used to accelerate denial-of-service -prevention policies in the iptables data plane. + -When `bpfEnabled` is `true` the "xdp" settings all have no effect; in BPF mode the implementation of -policy is always accelerated, using the best available BPF technology. +#### Overlay: IPSec + + + +#### Flow logs: Prometheus reports + + + +#### Flow logs: Syslog reports + + + +#### Flow logs: file reports + + + +#### DNS logs / policy + + + +#### L7 logs + + + +#### AWS integration + + + +#### Egress gateway + + + +#### External network support + + + +#### Packet capture + + + +#### L7 proxy + + +#### Debug/test-only (generally unsupported) + + + +#### Usage reporting + + ### Health Timeout Overrides Felix has internal liveness and readiness watchdog timers that monitor its various loops. If a loop fails to "check in" within the allotted timeout then Felix will report non-Ready -or non-Live on its health port (which is monitored by kubelet in a Kubernetes system). +or non-Live on its health port (which is monitored by Kubelet in a Kubernetes system). If Felix reports non-Live, this can result in the Pod being restarted. In Kubernetes, if you see the calico-node Pod readiness or liveness checks fail @@ -246,7 +266,7 @@ overall health status (the list of components will depend on which features are If some health timeouts show as "timed out" it may help to apply an override using the `healthTimeoutOverrides` field: -```yaml +```yaml noValidation ... spec: healthTimeoutOverrides: @@ -296,27 +316,20 @@ failsafeOutboundHostPorts: ### DNSPolicyMode -| Value | Description | -| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Inline (default) | Felix parses DNS response inline with DNS response packet processing within IPTables. This guarantees the DNS rules reflect any change immediately. This mode works for iptables only and matches the same mode for `BPFDNSPolicyMode`.| -| DelayDeniedPacket | Felix delays any denied packet that traversed a policy that included egress domain matches, but did not match. The packet is released after a fixed time, or after the destination IP address was programmed. | -| DelayDNSResponse | Felix delays any DNS response until related IPSets are programmed. This introduces some latency to all DNS packets (even when no IPSet programming is required), but it ensures policy hit statistics are accurate. This is the recommended setting when you are making use of staged policies or policy rule hit statistics. | -| NoDelay | Felix does not introduce any delay to the packets. DNS rules may not have been programmed by the time the first packet traverses the policy rules. Client applications need to handle reconnection attempts if initial connection attempts fail. This may be problematic for some applications or for very low DNS TTLs. | - -On Windows, or when using the eBPF data plane, this setting is ignored and `NoDelay` is always used. - -This field has no effect in NFTables mode. Please use `NFTablesDNSPolicyMode` instead. - -A linux kernel version of 3.13 or greater is required to use `DelayDNSResponse`. For earlier kernel versions, this value is modified to `DelayDeniedPacket`. +| Value | Description | +| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| DelayDeniedPacket | Felix delays any denied packet that traversed a policy that included egress domain matches, but did not match. The packet is released after a fixed time, or after the destination IP address was programmed. | +| DelayDNSResponse | Felix delays any DNS response until related IPSets are programmed. This introduces some latency to all DNS packets (even when no IPSet programming is required), but it ensures policy hit statistics are accurate. This is the recommended setting when you are making use of staged policies or policy rule hit statistics. A Linux kernel version of 3.13 or greater is required to use `DelayDNSResponse`. For earlier kernel versions, this value is modified to `DelayDeniedPacket`. | +| NoDelay | Felix does not introduce any delay to the packets. DNS rules may not have been programmed by the time the first packet traverses the policy rules. Client applications need to handle reconnection attempts if initial connection attempts fail. This may be problematic for some applications or for very low DNS TTLs. | -### NFTablesDNSPolicyMode +On Windows, or when using the eBPF dataplane, this setting is ignored. Windows always uses `NoDelay` while eBPF has its own [BPFDNSPolicyMode](#bpfdnspolicymode) option. -| Value | Description | -| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| DelayDeniedPacket (default) | Felix delays any denied packet that traversed a policy that included egress domain matches, but did not match. The packet is released after a fixed time, or after the destination IP address was programmed. | -| DelayDNSResponse | Felix delays any DNS response until related IPSets are programmed. This introduces some latency to all DNS packets (even when no IPSet programming is required), but it ensures policy hit statistics are accurate. This is the recommended setting when you are making use of staged policies or policy rule hit statistics. | -| NoDelay | Felix does not introduce any delay to the packets. DNS rules may not have been programmed by the time the first packet traverses the policy rules. Client applications need to handle reconnection attempts if initial connection attempts fail. This may be problematic for some applications or for very low DNS TTLs. | +### BPFDNSPolicyMode +| Value | Description | +| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Inline | Felix does not introduce any delay to any packets. Felix's eBPF programs parse DNS responses and program policy rules immediately, before the DNS response is passed to the application. This only applies to wildcard prefixes: `*.x.y.z` will be processed in this manner, but wildcards such as `x.*.y.z` will not match. A Linux kernel version of 5.17 or greater is required. | +| NoDelay | Felix does not introduce any delay to the packets. DNS rules may not have been programmed by the time the first packet traverses the policy rules. Client applications need to handle reconnection attempts if initial connection attempts fail. This may be problematic for some applications or for very low DNS TTLs. | ### RouteTableRange diff --git a/calico-cloud_versioned_docs/version-22-2/reference/component-resources/node/felix/configuration.mdx b/calico-cloud_versioned_docs/version-22-2/reference/component-resources/node/felix/configuration.mdx index 9d2df35fca..72b8cce4ad 100644 --- a/calico-cloud_versioned_docs/version-22-2/reference/component-resources/node/felix/configuration.mdx +++ b/calico-cloud_versioned_docs/version-22-2/reference/component-resources/node/felix/configuration.mdx @@ -4,16 +4,7 @@ description: Configure Felix, the daemon that runs on every machine that provide # Configuring Felix -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - - - - -If you have installed Calico using the operator, you cannot modify the environment provided to felix directly. To configure felix, see the [FelixConfiguration](../../../resources/felixconfig.mdx) resource instead. - - - +import FelixConfig from '../../../../_includes/components/FelixConfig'; :::note @@ -39,338 +30,210 @@ defaults, and it should be rare to have to explicitly set them. The full list of parameters which can be set is as follows. -### General configuration - -| Configuration file parameter | Environment variable | Description | Schema | -| ----------------------------------- | ---------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | -| `DataplaneWatchdogTimeout` | `FELIX_DATAPLANEWATCHDOGTIMEOUT` | Deprecated: superseded by `HealthTimeoutOverrides`. Timeout before the main data plane goroutine is determined to have hung and Felix will report non-live and non-ready. Can be increased if the liveness check incorrectly fails (for example if Felix is running slowly on a heavily loaded system). [Default: `90`] | int | -| `AwsSrcDstCheck` | `FELIX_AWSSRCDSTCHECK` | Set the [source-destination-check](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCheck) when using AWS EC2 instances. Check [IAM role and profile configuration](../../../resources/felixconfig.mdx#aws-iam-rolepolicy-for-source-destination-check-configuration) for setting the necessary permission for this setting to work. [Default: `DoNothing`] | `DoNothing`, `Disable`, `Enable` | -| `AWSSecondaryIPSupport` | `FELIX_AWSSECONDARYIPSUPPORT` | Controls whether Felix will create secondary AWS ENIs for AWS-backed IP pools. This feature is documented in the [egress gateways on AWS guide](../../../../networking/egress/egress-gateway-aws.mdx). Should only be enabled on AWS. [Default: "Disabled"] | `Enabled`, `EnabledENIPerWorkload`, `Disabled` | -| `AWSSecondaryIPRoutingRulePriority` | `FELIX_AWSSECONDARYIPROUTINGRULEPRIORITY` | Controls the priority of the policy-based routing rules used to implement AWS-backed IP addresses. Should only be changed to avoid conflicts if your nodes have additional policy based routing rules. [Default: 101] | int | -| `AWSRequestTimeout` | `FELIX_AWSREQUESTTIMEOUT` Timeout used for communicating with the AWS API (seconds). [Default: "30"] | int | -| `DatastoreType` | `FELIX_DATASTORETYPE` | The datastore that Felix should read endpoints and policy information from. [Default: `etcdv3`] | `etcdv3`, `kubernetes` | -| `DeviceRouteSourceAddress` | `FELIX_DEVICEROUTESOURCEADDRESS` | IPv4 address to use as the source hint on device routes programmed by Felix [Default: No source hint is set on programmed routes and for local traffic from host to workload the source address will be chosen by the kernel.] | `` | -| `DeviceRouteSourceAddressIPv6` | `FELIX_DEVICEROUTESOURCEADDRESSIPV6` | IPv6 address to use as the source hint on device routes programmed by Felix [Default: No source hint is set on programmed routes and for local traffic from host to workload the source address will be chosen by the kernel.] | `` | -| `DeviceRouteProtocol` | `FELIX_DEVICEROUTEPROTOCOL` | This defines the route protocol added to programmed device routes. [Default: `RTPROT_BOOT`] | int | -| `DisableConntrackInvalidCheck` | `FELIX_DISABLECONNTRACKINVALIDCHECK` | Disable the dropping of packets that aren't either a valid handshake or part of an established connection. [Default: `false`] | boolean | -| `EndpointReportingDelaySecs` | `FELIX_ENDPOINTREPORTINGDELAYSECS` | Set the endpoint reporting delay between status check intervals, in seconds. Only used if endpoint reporting is enabled. [Default: `1`] | int | -| `EndpointReportingEnabled` | `FELIX_ENDPOINTREPORTINGENABLED` | Enable the endpoint status reporter. [Default: `false`] | boolean | -| `ExternalNodesCIDRList` | `FELIX_EXTERNALNODESCIDRLIST` | Comma-delimited list of IPv4 or CIDR of external-non-calico-nodes from which IPIP traffic is accepted by calico-nodes. [Default: ""] | string | -| `FailsafeInboundHostPorts` | `FELIX_FAILSAFEINBOUNDHOSTPORTS` | List of PortProto struct objects including UDP/TCP/SCTP ports and CIDRs that Felix will allow incoming traffic to host endpoints on irrespective of the security policy. This is useful to avoid accidentally cutting off a host with incorrect configuration. For backwards compatibility, if the protocol is not specified, it defaults to `"tcp"`. If a CIDR is not specified, it will allow traffic from all addresses. To disable all inbound host ports, use the value `[]`. The default value allows ssh access, DHCP, BGP, etcd and the Kubernetes API. [Default: `[{"port":22,"protocol":"tcp"},{"port":68,"protocol":"udp"},{"port":179,"protocol":"tcp"},{"port":2379,"protocol":"tcp"}, {"port":2380,"protocol":"tcp"}, {"port":5473,"protocol":"tcp"}, {"port":6443,"protocol":"tcp"}, {"port":6666,"protocol":"tcp"}, {"port":6667,"protocol":"tcp"}]`] | list | -| `FailsafeOutboundHostPorts` | `FELIX_FAILSAFEOUTBOUNDHOSTPORTS` | List of PortProto struct objects including UDP/TCP/SCTP ports and CIDRs that Felix will allow outgoing traffic from host endpoints to irrespective of the security policy. This is useful to avoid accidentally cutting off a host with incorrect configuration. For backwards compatibility, if the protocol is not specified, it defaults to `"tcp"`. If a CIDR is not specified, it will allow traffic from all addresses. To disable all outbound host ports, use the value `[]`. The default value opens etcd's standard ports to ensure that Felix does not get cut off from etcd as well as allowing DHCP, DNS, BGP and the Kubernetes API. [Default: `[{"port":53,"protocol":"udp"},{"port":67,"protocol":"udp"}, {"port":179,"protocol":"tcp"}, {"port":2379,"protocol":"tcp"}, {"port":2380,"protocol":"tcp"}, {"port":5473,"protocol":"tcp"}, {"port":6443,"protocol":"tcp"}, {"port": 6666,"protocol":"tcp"}, {"port":6667,"protocol":"tcp"}]`] | list | | -| `FelixHostname` | `FELIX_FELIXHOSTNAME` | The hostname Felix reports to the plugin. Should be used if the hostname Felix autodetects is incorrect or does not match what the plugin will expect. [Default: `socket.gethostname()`] | string | -| `HealthEnabled` | `FELIX_HEALTHENABLED` | When enabled, exposes felix health information via an http endpoint. | boolean | -| `HealthHost` | `FELIX_HEALTHHOST` | The address on which Felix will respond to health requests. [Default: `localhost`] | string | -| `HealthPort` | `FELIX_HEALTHPORT` | The port on which Felix will respond to health requests. [Default: `9099`] | int | -| `HealthTimeoutOverrides` | `FELIX_HEALTHTIMEOUTOVERRIDES` | Allows the internal watchdog timeouts of individual subcomponents to be overridden; example: "InternalDataplaneMainLoop=30s,CalculationGraph=2m". This is useful for working around "false positive" liveness timeouts that can occur in particularly stressful workloads or if CPU is constrained. For a list of active subcomponents, see Felix's logs. [Default: ``] | Comma-delimited list of key/value pairs where the values are durations: `1s`, `10s`, `5m`, etc. | -| `IpInIpEnabled` | `FELIX_IPINIPENABLED` | Optional, you shouldn't need to change this setting as Felix calculates if IPIP should be enabled based on the existing IP Pools. When set, this overrides whether Felix should configure an IPinIP interface on the host. When explicitly disabled in FelixConfiguration, Felix will not clean up addresses from the `tunl0` interface (use this if you need to add addresses to that interface and don't want to have them removed). [Default: unset] | optional boolean | -| `IpInIpMtu` | `FELIX_IPINIPMTU` | The MTU to set on the IPIP tunnel device. Zero value means auto-detect. See [Configuring MTU](../../../../networking/configuring/mtu.mdx) [Default: `0`] | int | -| `IPv4VXLANTunnelAddr` | | IP address of the IPv4 VXLAN tunnel. This is system configured and should not be updated manually. | string | -| `LogFilePath` | `FELIX_LOGFILEPATH` | The full path to the Felix log. Set to `none` to disable file logging. [Default: `/var/log/calico/felix.log`] | string | -| `LogSeverityFile` | `FELIX_LOGSEVERITYFILE` | The log severity above which logs are sent to the log file. [Default: `Info`] | `Debug`, `Info`, `Warning`, `Error`, `Fatal` | -| `LogSeverityScreen` | `FELIX_LOGSEVERITYSCREEN` | The log severity above which logs are sent to the stdout. [Default: `Info`] | `Debug`, `Info`, `Warning`, `Error`, `Fatal` | -| `LogSeveritySys` | `FELIX_LOGSEVERITYSYS` | The log severity above which logs are sent to the syslog. Set to `none` for no logging to syslog. [Default: `Info`] | `Debug`, `Info`, `Warning`, `Error`, `Fatal` | -| `LogDebugFilenameRegex` | `FELIX_LOGDEBUGFILENAMEREGEX` | Controls which source code files have their Debug log output included in the logs. Only logs from files with names that match the given regular expression are included. The filter only applies to Debug level logs. [Default: `""`] | regex | -| `PolicySyncPathPrefix` | `FELIX_POLICYSYNCPATHPREFIX` | File system path where Felix notifies services of policy changes over Unix domain sockets. This is required only if you're configuring [L7 logs](../../../../observability/elastic/l7/configure.mdx), or [egress gateways](../../../../networking/egress/index.mdx). Set to `""` to disable. [Default: `""`] | string | -| `PrometheusGoMetricsEnabled` | `FELIX_PROMETHEUSGOMETRICSENABLED` | Set to `false` to disable Go runtime metrics collection, which the Prometheus client does by default. This reduces the number of metrics reported, reducing Prometheus load. [Default: `true`] | boolean | -| `PrometheusMetricsEnabled` | `FELIX_PROMETHEUSMETRICSENABLED` | Set to `true` to enable the Prometheus metrics server in Felix. [Default: `false`] | boolean | -| `PrometheusMetricsHost` | `FELIX_PROMETHEUSMETRICSHOST` | TCP network address that the Prometheus metrics server should bind to. [Default: `""`] | string | -| `PrometheusMetricsPort` | `FELIX_PROMETHEUSMETRICSPORT` | TCP port that the Prometheus metrics server should bind to. [Default: `9091`] | int | -| `PrometheusProcessMetricsEnabled` | `FELIX_PROMETHEUSPROCESSMETRICSENABLED` | Set to `false` to disable process metrics collection, which the Prometheus client does by default. This reduces the number of metrics reported, reducing Prometheus load. [Default: `true`] | boolean | -| `PrometheusWireguardMetricsEnabled` | `FELIX_PROMETHEUSWIREGUARDMETRICSENABLED` | Set to `false` to disable WireGuard device metrics collection, which Felix does by default. [Default: `true`] | boolean | -| `RemoveExternalRoutes` | `FELIX_REMOVEEXTERNALROUTES` | Whether or not to remove device routes that have not been programmed by Felix. Disabling this will allow external applications to also add device routes. [Default: `true`] | bool | -| `ReportingIntervalSecs` | `FELIX_REPORTINGINTERVALSECS` | Interval at which Felix reports its status into the datastore. 0 means disabled and is correct for Kubernetes-only clusters. Must be non-zero in OpenStack deployments. [Default: `30`] | int | -| `ReportingTTLSecs` | `FELIX_REPORTINGTTLSECS` | Time-to-live setting for process-wide status reports. [Default: `90`] | int | -| `RouteTableRange` | `FELIX_ROUTETABLERANGE` | _deprecated in favor of `RouteTableRanges`_ Calico programs additional Linux route tables for various purposes. `RouteTableRange` specifies the indices of the route tables that Calico should use. [Default: `""`] | `-` | -| `RouteTableRanges` | `FELIX_ROUTETABLERANGES` | Calico programs additional Linux route tables for various purposes. `RouteTableRanges` specifies a set of table index ranges that Calico should use. Deprecates `RouteTableRange`, overrides `RouteTableRange`. [Default: `"1-250"`] | `-,-,...` | -| `RouteSyncDisabled` | `FELIX_ROUTESYNCDISABLED` | Set to `true` to disable Calico programming routes to local workloads. [Default: `false`] | boolean | -| `VXLANEnabled` | `FELIX_VXLANENABLED` | Optional, you shouldn't need to change this setting as Felix calculates if VXLAN should be enabled based on the existing IP Pools. When set, this overrides whether Felix should create the VXLAN tunnel device for VXLAN networking. [Default: unset] | optional boolean | -| `VXLANMTU` | `FELIX_VXLANMTU` | The MTU to set on the IPv4 VXLAN tunnel device. Zero value means auto-detect. Also controls NodePort MTU when eBPF enabled. See [Configuring MTU](../../../../networking/configuring/mtu.mdx) [Default: `0`] | int | -| `VXLANMTUV6` | `FELIX_VXLANMTUV6` | The MTU to set on the IPv6 VXLAN tunnel device. Zero value means auto-detect. Also controls NodePort MTU when eBPF enabled. See [Configuring MTU](../../../../networking/configuring/mtu.mdx) [Default: `0`] | int | -| `VXLANPort` | `FELIX_VXLANPORT` | The UDP port to use for VXLAN. [Default: `4789`] | int | -| `VXLANTunnelMACAddr` | | MAC address of the IPv4 VXLAN tunnel. This is system configured and should not be updated manually. | string | -| `VXLANVNI` | `FELIX_VXLANVNI` | The virtual network ID to use for VXLAN. [Default: `4096`] | int | -| `AllowVXLANPacketsFromWorkloads` | `FELIX_ALLOWVXLANPACKETSFROMWORKLOADS` | Set to `true` to allow VXLAN encapsulated traffic from workloads. [Default: `false`] | boolean | -| `AllowIPIPPacketsFromWorkloads` | `FELIX_ALLOWIPIPPACKETSFROMWORKLOADS` | Set to `true` to allow IPIP encapsulated traffic from workloads. [Default: `false`] | boolean | -| `TyphaAddr` | `FELIX_TYPHAADDR` | IPv4 address at which Felix should connect to Typha. [Default: none] | string | -| `TyphaK8sServiceName` | `FELIX_TYPHAK8SSERVICENAME` | Name of the Typha Kubernetes service | string | -| `Ipv6Support` | `FELIX_IPV6SUPPORT` | Enable $[prodname] networking and security for IPv6 traffic as well as for IPv4. | boolean | -| `RouteSource` | `FELIX_ROUTESOURCE` | Where Felix gets is routing information from for VXLAN and the BPF data plane. The CalicoIPAM setting is more efficient because it supports route aggregation, but it only works when Calico's IPAM or host-local IPAM is in use. Use the WorkloadIPs setting if you are using Calico's VXLAN or BPF data plane and not using Calico IPAM or host-local IPAM. [Default: "CalicoIPAM"] | 'CalicoIPAM', or 'WorkloadIPs' | -| `mtuIfacePattern` | `FELIX_MTUIFACEPATTERN` | Pattern used to discover the host's interface for MTU auto-detection. [Default: `^((en\|wl\|ww\|sl\|ib)[Pcopsvx].*\|(eth\|wlan\|wwan).*)`] | regex | -| `TPROXYMode` | `FELIX_TPROXYMODE` | Sets transparent proxying mode. [Default: "Disabled"] | 'Disabled', 'Enabled' | -| `TPROXYPort` | `FELIX_TPROXYPORT` | What local ports is the proxied traffic sent to. [Default: `16001`] | int | -| `FeatureDetectOverride` | `FELIX_FEATUREDETECTOVERRIDE` | Is used to override the feature detection. Values are specified in a comma separated list with no spaces, example; "SNATFullyRandom=true,MASQFullyRandom=false,RestoreSupportsLock=true,IPIPDeviceIsL3=true. "true" or "false" will force the feature, empty or omitted values are auto-detected. [Default: `""`] | string | - -### etcd datastore configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ----------------------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------- | -| `EtcdCaFile` | `FELIX_ETCDCAFILE` | Path to the file containing the root certificate of the certificate authority (CA) that issued the etcd server certificate. Configures Felix to trust the CA that signed the root certificate. The file may contain multiple root certificates, causing Felix to trust each of the CAs included. To disable authentication of the server by Felix, set the value to `none`. [Default: `/etc/ssl/certs/ca-certificates.crt`] | string | -| `EtcdCertFile` | `FELIX_ETCDCERTFILE` | Path to the file containing the client certificate issued to Felix. Enables Felix to participate in mutual TLS authentication and identify itself to the etcd server. Example: `/etc/felix/cert.pem` (optional) | string | -| `EtcdEndpoints` | `FELIX_ETCDENDPOINTS` | Comma-delimited list of etcd endpoints to connect to. Example: `http://127.0.0.1:2379,http://127.0.0.2:2379`. | `://:` | -| `EtcdKeyFile` | `FELIX_ETCDKEYFILE` | Path to the file containing the private key matching Felix's client certificate. Enables Felix to participate in mutual TLS authentication and identify itself to the etcd server. Example: `/etc/felix/key.pem` (optional) | string | - -### Kubernetes API datastore configuration - -The Kubernetes API datastore driver reads its configuration from Kubernetes-provided environment variables. - -### iptables data plane configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ------------------------------------ | ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- | -| `ChainInsertMode` | `FELIX_CHAININSERTMODE` | Controls whether Felix hooks the kernel's top-level iptables chains by inserting a rule at the top of the chain or by appending a rule at the bottom. `Insert` is the safe default since it prevents $[prodname]'s rules from being bypassed. If you switch to `Append` mode, be sure that the other rules in the chains signal acceptance by falling through to the $[prodname] rules, otherwise the $[prodname] policy will be bypassed. [Default: `Insert`] | `Insert`, `Append` | -| `DefaultEndpointToHostAction` | `FELIX_DEFAULTENDPOINTTOHOSTACTION` | This parameter controls what happens to traffic that goes from a workload endpoint to the host itself (after the traffic hits the endpoint egress policy). By default $[prodname] blocks traffic from workload endpoints to the host itself with an iptables `Drop` action. If you want to allow some or all traffic from endpoint to host, set this parameter to `Return` or `Accept`. Use `Return` if you have your own rules in the iptables "INPUT" chain; $[prodname] will insert its rules at the top of that chain, then `Return` packets to the "INPUT" chain once it has completed processing workload endpoint egress policy. Use `Accept` to unconditionally accept packets from workloads after processing workload endpoint egress policy. [Default: `Drop`] | `Drop`, `Return`, `Accept` | -| `GenericXDPEnabled` | `FELIX_GENERICXDPENABLED` | When enabled, Felix can fallback to the non-optimized `generic` XDP mode. This should only be used for testing since it doesn't improve performance over the non-XDP mode. [Default: `false`] | boolean | -| `InterfaceExclude` | `FELIX_INTERFACEEXCLUDE` | A comma-separated list of interface names that should be excluded when Felix is resolving host endpoints. The default value ensures that Felix ignores Kubernetes' internal `kube-ipvs0` device. If you want to exclude multiple interface names using a single value, the list supports regular expressions. For regular expressions you must wrap the value with `/`. For example having values `/^kube/,veth1` will exclude all interfaces that begin with `kube` and also the interface `veth1`. [Default: `kube-ipvs0`] | string | -| `IpsetsRefreshInterval` | `FELIX_IPSETSREFRESHINTERVAL` | Period, in seconds, at which Felix re-checks the IP sets in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable IP sets refresh. [Default: `10`] | int | -| `IptablesBackend` | `FELIX_IPTABLESBACKEND` | This parameter controls which variant of iptables Felix uses. Set this to `Auto` for auto detection of the backend. If a specific backend is needed then use `nft` for hosts using a netfilter backend or `Legacy` for others. [Default: `Auto`] | `Legacy`, `NFT`, `Auto` | -| `IptablesFilterAllowAction` | `FELIX_IPTABLESFILTERALLOWACTION` | This parameter controls what happens to traffic that is allowed by a Felix policy chain in the iptables filter table (i.e., a normal policy chain). The default will immediately `Accept` the traffic. Use `Return` to send the traffic back up to the system chains for further processing. [Default: `Accept`] | `Accept`, `Return` | -| `IptablesLockFilePath` | `FELIX_IPTABLESLOCKFILEPATH` | _Deprecated:_ For iptables versions prior to v1.6.2, location of the iptables lock file (later versions of iptables always use value "/run/xtables.lock"). You may need to change this if the lock file is not in its standard location (for example if you have mapped it into Felix's container at a different path). [Default: `/run/xtables.lock`] | string | -| `IptablesLockProbeIntervalMillis` | `FELIX_IPTABLESLOCKPROBEINTERVALMILLIS` | Time, in milliseconds, that Felix will wait between attempts to acquire the iptables lock if it is not available. Lower values make Felix more responsive when the lock is contended, but use more CPU. [Default: `50`] | int | -| `IptablesLockTimeoutSecs` | `FELIX_IPTABLESLOCKTIMEOUTSECS` | Time, in seconds, that Felix will wait for the iptables lock. Versions of iptables prior to v1.6.2 support disabling the iptables lock by setting this value to 0; v1.6.2 and above do not so Felix will default to 10s if a non-positive number is used. To use this feature, Felix must share the iptables lock file with all other processes that also take the lock. When running Felix inside a container, this typically requires the file /run/xtables.lock on the host to be mounted into the `$[nodecontainer]` or `calico/felix` container. [Default: `0` disabled for iptables <v1.6.2 or 10s for later versions] | int | -| `IptablesMangleAllowAction` | `FELIX_IPTABLESMANGLEALLOWACTION` | This parameter controls what happens to traffic that is allowed by a Felix policy chain in the iptables mangle table (i.e., a pre-DNAT policy chain). The default will immediately `Accept` the traffic. Use `Return` to send the traffic back up to the system chains for further processing. [Default: `Accept`] | `Accept`, `Return` | -| `IptablesMarkMask` | `FELIX_IPTABLESMARKMASK` | Mask that Felix selects its IPTables Mark bits from. Should be a 32 bit hexadecimal number with at least 8 bits set, none of which clash with any other mark bits in use on the system. When using $[prodname] with Kubernetes' `kube-proxy` in IPVS mode, [we recommend allowing at least 16 bits](#ipvs-bits). [Default: `0xffff0000`] | netmask | -| `IptablesNATOutgoingInterfaceFilter` | `FELIX_IPTABLESNATOUTGOINGINTERFACEFILTER` | This parameter can be used to limit the host interfaces on which Calico will apply SNAT to traffic leaving a Calico IPAM pool with "NAT outgoing" enabled. This can be useful if you have a main data interface, where traffic should be SNATted and a secondary device (such as the docker bridge) which is local to the host and doesn't require SNAT. This parameter uses the iptables interface matching syntax, which allows `+` as a wildcard. Most users will not need to set this. Example: if your data interfaces are eth0 and eth1 and you want to exclude the docker bridge, you could set this to `eth+` | string | -| `IptablesPostWriteCheckIntervalSecs` | `FELIX_IPTABLESPOSTWRITECHECKINTERVALSECS` | Period, in seconds, after Felix has done a write to the data plane that it schedules an extra read back to check the write was not clobbered by another process. This should only occur if another application on the system doesn't respect the iptables lock. [Default: `1`] | int | -| `IptablesRefreshInterval` | `FELIX_IPTABLESREFRESHINTERVAL` | Period, in seconds, at which Felix re-checks all iptables state to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable iptables refresh. [Default: `90`] | int | -| `LogPrefix` | `FELIX_LOGPREFIX` | The log prefix that Felix uses when rendering LOG rules. [Default: `calico-packet`] | string | -| `MaxIpsetSize` | `FELIX_MAXIPSETSIZE` | Maximum size for the ipsets used by Felix. Should be set to a number that is greater than the maximum number of IP addresses that are ever expected in a selector. [Default: `1048576`] | int | -| `NATPortRange` | `FELIX_NATPORTRANGE` | Port range used by iptables for port mapping when doing outgoing NAT. (Example: `32768:65000`). [Default: iptables maps source ports below 512 to other ports below 512: those between 512 and 1023 inclusive will be mapped to ports below 1024, and other ports will be mapped to 1024 or above. Where possible, no port alteration will occur.] | string | -| `NATOutgoingAddress` | `FELIX_NATOUTGOINGADDRESS` | Source address used by iptables for an SNAT rule when doing outgoing NAT. [Default: an iptables `MASQUERADE` rule is used for outgoing NAT which will use the address on the interface traffic is leaving on.] | `` | -| `NetlinkTimeoutSecs` | `FELIX_NETLINKTIMEOUTSECS` | Time, in seconds, that Felix will wait for netlink (i.e. routing table list/update) operations to complete before giving up and retrying. [Default: `10`] | float | -| `RouteRefreshInterval` | `FELIX_ROUTEREFRESHINTERVAL` | Period, in seconds, at which Felix re-checks the routes in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable route refresh. [Default: `90`] | int | -| `ServiceLoopPrevention` | `FELIX_SERVICELOOPPREVENTION` | When [service IP advertisement is enabled](../../../../networking/configuring/advertise-service-ips.mdx), prevent routing loops to service IPs that are not in use, by dropping or rejecting packets that do not get DNAT'd by kube-proxy. Unless set to "Disabled", in which case such routing loops continue to be allowed. [Default: `Drop`] | `Drop`, `Reject`, `Disabled` | -| `WorkloadSourceSpoofing` | `FELIX_WORKLOADSOURCESPOOFING` | Controls whether pods can enable source IP address spoofing with the `cni.projectcalico.org/allowedSourcePrefixes` annotation. When set to `Any`, pods can use this annotation to send packets from any IP address. [Default: `Disabled`] | `Any`, `Disabled` | -| `XDPRefreshInterval` | `FELIX_XDPREFRESHINTERVAL` | Period, in seconds, at which Felix re-checks the XDP state in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable XDP refresh. [Default: `90`] | int | -| `XDPEnabled` | `FELIX_XDPENABLED` | Enable XDP acceleration for host endpoint policies. [Default: `true`] | boolean | - -### eBPF data plane configuration - -eBPF data plane mode uses the Linux Kernel's eBPF virtual machine to implement networking and policy instead of iptables. When BPFEnabled is set to `true`, Felix will: - -- Require a v5.3 Linux kernel. -- Implement policy with eBPF programs instead of iptables. -- Activate its embedded implementation of `kube-proxy` to implement Kubernetes service load balancing. -- Disable support for IPv6. - -See [Enable the eBPF data plane](../../../../operations/ebpf/enabling-ebpf.mdx) for step-by step instructions to enable this feature. - -| Configuration parameter / Environment variable | Description | Schema | Default | -| ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | -------------- | ------ | ------ | ----------------- | -| BPFEnabled /
FELIX_BPFENABLED | Enable eBPF data plane mode. eBPF mode has a number of limitations, see the [HOWTO guide](../../../../operations/ebpf/enabling-ebpf.mdx). | true, false | false | -| BPFDisableUnprivileged /
FELIX_BPFDISABLEUNPRIVILEGED | If true, Felix sets the kernel.unprivileged_bpf_disabled sysctl to disable unprivileged use of BPF. This ensures that unprivileged users cannot access Calico's BPF maps and cannot insert their own BPF programs to interfere with the ones that $[prodname] installs. | true, false | true | -| BPFLogLevel /
FELIX_BPFLOGLEVEL | The log level used by the BPF programs. The logs are emitted to the BPF trace pipe, accessible with the command `tc exec BPF debug`. | Off,Info,Debug | Off | -| BPFDataIfacePattern /
FELIX_BPFDATAIFACEPATTERN | Controls which interfaces Felix should attach BPF programs to catch traffic to/from the external network. This needs to match the interfaces that Calico workload traffic flows over as well as any interfaces that handle incoming traffic to NodePorts and services from outside the cluster. It should not match the workload interfaces (usually named cali...).. | regular expression | `^(en[opsvx].\* | eth.\* | tunl0$ | wireguard.cali$)` | -| BPFConnectTimeLoadBalancingEnabled /
FELIX_BPFCONNECTTIMELOADBALANCINGENABLED | Controls whether Felix installs the connect-time load balancer. In the current release, the connect-time load balancer is required for the host to reach kubernetes services. | true,false | true | -| BPFExternalServiceMode /
FELIX_BPFEXTERNALSERVICEMODE | Controls how traffic from outside the cluster to NodePorts and ClusterIPs is handled. In Tunnel mode, packet is tunneled from the ingress host to the host with the backing pod and back again. In DSR mode, traffic is tunneled to the host with the backing pod and then returned directly; this requires a network that allows direct return. | Tunnel,DSR | Tunnel | -| BPFExtToServiceConnmark /
FELIX_BPFEXTTOSERVICECONNMARK | Controls a 32bit mark that is set on connections from an external client to a local service. This mark allows us to control how packets of that connection are routed within the host and how is routing interpreted by RPF check. | int | 0 | -| BPFKubeProxyIptablesCleanupEnabled /
FELIX_BPFKUBEPROXYIPTABLESCLEANUPENABLED | Controls whether Felix will clean up the iptables rules created by the Kubernetes `kube-proxy`; should only be enabled if `kube-proxy` is not running. | true,false | true | -| BPFKubeProxyMinSyncPeriod /
FELIX_BPFKUBEPROXYMINSYNCPERIOD | Controls the minimum time between data plane updates for Felix's embedded `kube-proxy` implementation. | seconds | `1` | -| BPFKubeProxyEndpointSlicesEnabled /
FELIX_BPFKUBEPROXYENDPOINTSLICESENABLED | Controls whether Felix's embedded kube-proxy derives its services from Kubernetes' EndpointSlices resources. Using EndpointSlices is more efficient but it requires EndpointSlices support to be enabled at the Kubernetes API server. | true,false | false | -| BPFMapSizeConntrack /
FELIX_BPFMapSizeConntrack | Controls the size of the conntrack map. This map must be large enough to hold an entry for each active connection. Warning: changing the size of the conntrack map can cause disruption. | int | 512000 | -| BPFMapSizeNATFrontend /
FELIX_BPFMapSizeNATFrontend | Controls the size of the NAT frontend map. FrontendMap should be large enough to hold an entry for each nodeport, external IP and each port in each service. | int | 65536 | -| BPFMapSizeNATBackend /
FELIX_BPFMapSizeNATBackend | Controls the size of the NAT backend map. This is the total number of endpoints. This is mostly more than the size of the number of services. | int | 262144 | -| BPFMapSizeNATAffinity /
FELIX_BPFMapSizeNATAffinity | Controls the size of the NAT affinity map. | int | 65536 | -| BPFMapSizeIPSets /
FELIX_BPFMapSizeIPSets | Controls the size of the IPSets map. The IP sets map must be large enough to hold an entry for each endpoint matched by every selector in the source/destination matches in network policy. Selectors such as "all()" can result in large numbers of entries (one entry per endpoint in that case). | int | 1048576 | -| BPFMapSizeRoute /
FELIX_BPFMapSizeRoute | Controls the size of the route map. The routes map should be large enough to hold one entry per workload and a handful of entries per host (enough to cover its own IPs and tunnel IPs). | int | 262144 | -| BPFHostConntrackBypass /
FELIX_BPFHostConntrackBypass | Controls whether to bypass Linux conntrack in BPF mode for workloads and services. | true,false | true | -| BPFPolicyDebugEnabled /
FELIX_BPFPOLICYDEBUGENABLED | In eBPF data plane mode, Felix records detailed information about the BPF policy programs, which can be examined with the calico-bpf command-line tool. | true, false | true | - -### Windows-specific configuration - -| Configuration parameter | Environment variable | Description | Schema | Default | -| ------------------------------- | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ------------------------------------------- | -| windowsFlowLogsFileDirectory |
FELIX_WINDOWSFLOWLOGSFILEDIRECTORY | Set the directory where flow logs files are stored on Windows nodes. This parameter only takes effect when `flowLogsFileEnabled` is set to `true`. | string | `c:\\TigeraCalico\\flowlogs` | -| windowsFlowLogsPositionFilePath |
FELIX_WINDOWSFLOWLOGSPOSITIONFILEPATH | Specify the position of the external pipeline that reads flow logs on Windows nodes. This parameter only takes effect when `FlowLogsDynamicAggregationEnabled` is set to `true`. | string | `c:\\TigeraCalico\\flowlogs\\flows.log.pos` | -| windowsStatsDumpFilePath |
FELIX_WINDOWSTATSDUMPFILEPATH | Specify the position of the file used for dumping flow log statistics on Windows nodes. Note this is an internal setting that users shouldn't need to modify. | string | `c:\\TigeraCalico\\stats\\dump` | -| WindowsDNSCacheFile |
FELIX_WINDOWSDNSCACHEFILE | Specify the name of the file that Felix uses to preserve learned DNS information when restarting. | string | `c:\\TigeraCalico\\felix-dns-cache.txt` | -| WindowsDNSExtraTTL |
FELIX_WINDOWSDNSEXTRATTL | Specify extra time in seconds to keep IPs and alias names that are learned from DNS, in addition to each name or IP's advertised TTL. | seconds | `120` | - -### Kubernetes-specific configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ----------------------- | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ | -| `KubeNodePortRanges` | `FELIX_KUBENODEPORTRANGES` | A list of port ranges that Felix should treat as Kubernetes node ports. Only when `kube-proxy` is configured to use IPVS mode: Felix assumes that traffic arriving at the host of one of these ports will ultimately be forwarded instead of being terminated by a host process. [Default: `30000:32767`] | Comma-delimited list of `:` port ranges or single ports. | -| `KubeMasqueradeBit` | `FELIX_KUBEMASQUERADEBIT` | KubeMasqueradeBit should be set to the same value as --iptables-masquerade-bit of kube-proxy when TPROXY is used. This defaults to the corresponding kube-proxy default value so it only needs to change if kube-proxy is using a non-standard setting. Must be within the range of 0-31. OpenShift sets the bit to 0 by default. [Default: 14] | integer | +## Spec -:::note +### Datastore connection - When using $[prodname] with Kubernetes' `kube-proxy` in IPVS mode, $[prodname] uses additional -iptables mark bits to store an ID for each local $[prodname] endpoint. For example, the default `IptablesMarkMask` value, -`0xffff0000` gives $[prodname] 16 bits, up to 6 of which are used for internal purposes, leaving 10 bits for endpoint -IDs. 10 bits is enough for 1024 different values and $[prodname] uses 2 of those for internal purposes, leaving enough -for 1022 endpoints on the host. + -::: +### Process: Feature detection/overrides + + + +### Process: Go runtime + + + +### Process: Health port and timeouts + + + +### Process: Logging + + + +### Process: Prometheus metrics + + + +### Dataplane: Common + + + +### Dataplane: iptables + + + +### Dataplane: nftables + + + +### Dataplane: eBPF + + + +### Dataplane: Windows + + + +### Dataplane: OpenStack support + + + +### Dataplane: XDP acceleration for iptables dataplane + + + +### Overlay: VXLAN overlay + + + +### Overlay: IP-in-IP + + -### Bare metal specific configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ----------------------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | -| `InterfacePrefix` | `FELIX_INTERFACEPREFIX` | The interface name prefix that identifies workload endpoints and so distinguishes them from host endpoint interfaces. Accepts more than one interface name prefix in comma-delimited format, e.g., `tap,cali`. Note: in environments other than bare metal, the orchestrators configure this appropriately. For example our Kubernetes and Docker integrations set the `cali` value, and our OpenStack integration sets the `tap` value. [Default: `cali`] | string | - -### $[prodname] specific configuration - -| Setting | Environment variable | Default | Meaning | -| --------------------------------------- | --------------------------------------------- | ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `DropActionOverride` | `FELIX_DROPACTIONOVERRIDE` | `Drop` | How to treat packets that are disallowed by the current $[prodname] policy. For more detail please see below. | -| `LogDropActionOverride` | `FELIX_LOGDROPACTIONOVERRIDE` | `false` | Set to `true` to add the `DropActionOverride` to the syslog entries. For more detail please see below. | -| `PrometheusReporterEnabled` | `FELIX_PROMETHEUSREPORTERENABLED` | `false` | Set to `true` to enable Prometheus reporting of denied packet metrics. For more detail please see below. | -| `PrometheusReporterPort` | `FELIX_PROMETHEUSREPORTERPORT` | `9092` | The TCP port on which to report denied packet metrics. | -| `PrometheusReporterCertFile` | `FELIX_PROMETHEUSREPORTERCERTFILE` | None | Certificate for encrypting Prometheus denied packet metrics. | -| `PrometheusReporterKeyFile` | `FELIX_PROMETHEUSREPORTERKEYFILE` | None | Private key for encrypting Prometheus denied packet metrics. | -| `PrometheusReporterCAFile` | `FELIX_PROMETHEUSREPORTERCAFILE` | None | Trusted CA file for clients attempting to read Prometheus denied packet metrics. | -| `PrometheusMetricsCertFile` | `FELIX_PROMETHEUSMETRICSCERTFILE` | None | Certificate for encrypting general Felix Prometheus metrics. | -| `PrometheusMetricsKeyFile` | `FELIX_PROMETHEUSMETRICSKEYFILE` | None | Private key for encrypting general Felix Prometheus metrics. | -| `PrometheusMetricsCAFile` | `FELIX_PROMETHEUSMETRICSCAFILE` | None | Trusted CA file for clients attempting to read general Felix Prometheus metrics. | -| `IPSecMode` | `FELIX_IPSECMODE` | None | Controls which mode IPsec is operating on. The only supported value is `PSK`. An empty value means IPsec is not enabled. | -| `IPSecAllowUnsecuredTraffic` | `FELIX_IPSECALLOWUNSECUREDTRAFFIC` | `false` | When set to false, only IPsec-protected traffic will be allowed on the packet paths where IPsec is supported. When set to true, IPsec will be used but non-IPsec traffic will be accepted. In general, setting this to `true` is less safe since it allows an attacker to inject packets. However, it is useful when transitioning from non-IPsec to IPsec since it allows traffic to flow while the cluster negotiates the IPsec mesh. | -| `IPSecIKEAlgorithm` | `FELIX_IPSECIKEALGORITHM` | `aes128gcm16-prfsha256-ecp256` | IPsec IKE algorithm. Default is NIST suite B recommendation. | -| `IPSecESPAlgorithm` | `FELIX_IPSECESPALGORITHM` | `aes128gcm16-ecp256` | IPsec ESP algorithm. Default is NIST suite B recommendation. | -| `IPSecLogLevel` | `FELIX_IPSECLOGLEVEL` | `Info` | Controls log level for IPsec components. Set to `None` for no logging. Other valid values are `Notice`, `Info`, `Debug` and `Verbose`. | -| `IPSecPSKFile` | `FELIX_IPSECPSKFILE` | None | The path to the pre shared key file for IPsec. | -| `FlowLogsFileEnabled` | `FELIX_FLOWLOGSFILEENABLED` | `false` | Set to `true`, enables flow logs. If set to `false` no flow logging will occur. Flow logs are written to a file `flows.log` and sent to Elasticsearch. The location of this file can be configured using the `FlowLogsFileDirectory` field. File rotation settings for this `flows.log` file can be configured using the fields `FlowLogsFileMaxFiles` and `FlowLogsFileMaxFileSizeMB`. Note that flow log exports to Elasticsearch are dependent on flow logs getting written to this file. Setting this parameter to `false` will disable flow logs. | -| `FlowLogsFileIncludeLabels` | `FELIX_FLOWLOGSFILEINCLUDELABELS` | `false` | Set to `true` to include endpoint label information in flow logs. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFileIncludePolicies` | `FELIX_FLOWLOGSFILEINCLUDEPOLICIES` | `false` | Set to `true` to include policy match information in flow logs. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFileIncludeService` | `FELIX_FLOWLOGSFILEINCLUDESERVICE` | `false` | Set to `true` to include destination service information in flow logs. The service information is derived from pre-DNAT destination IP and is therefore only available on the node where DNAT occurs. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFileDirectory` | `FELIX_FLOWLOGSFILEDIRECTORY` | `/var/log/calico/flowlogs` | The directory where flow logs files are stored. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFileMaxFiles` | `FELIX_FLOWLOGSFILEMAXFILES` | `5` | The number of files to keep when rotating flow log files. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFileMaxFileSizeMB` | `FELIX_FLOWLOGSFILEMAXFILESIZEMB` | `100` | The max size in MB of flow logs files before rotation. This parameter only takes effect when `FlowLogsFileEnabled` is set to `true`. | -| `FlowLogsFlushInterval` | `FELIX_FLOWLOGSFLUSHINTERVAL` | `300` | The period, in seconds, at which Felix exports the flow logs. | -| `FlowLogsEnableNetworkSets` | `FELIX_FLOWLOGSENABLENETWORKSETS` | `false` | Whether to specify the network set a flow log originates from. | -| `FlowLogsFileAggregationKindForAllowed` | `FELIX_FLOWLOGSFILEAGGREGATIONKINDFORALLOWED` | `2` | How much to aggregate the flow logs sent to Elasticsearch for allowed traffic. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. `0` means no aggregation, `1` means aggregate all flows that share a source port on each node, `2` means aggregate all flows that share source ports or are from the same ReplicaSet and `3` means aggregate all flows that share destination and source ports and are from the same ReplicateSet | -| `FlowLogsFileAggregationKindForDenied` | `FELIX_FLOWLOGSFILEAGGREGATIONKINDFORDENIED` | `1` | How much to aggregate the flow logs sent to Elasticsearch for denied traffic. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. `0` means no aggregation, `1` means aggregate all flows that share a source port on each node, and `2` means aggregate all flows that share source ports or are from the same ReplicaSet and `3` means aggregate all flows that share destination and source ports and are from the same ReplicateSet. | -| `FlowLogsDynamicAggregationEnabled` | `FELIX_FLOWLOGSDYNAMICAGGREGATIONENABLED` | `false` | Enable dynamic aggregation for flow logs. This will increase aggregation up to the maximum level allowed (which is 3 and means aggregate all flows that share destination and source ports and are from the same ReplicateSet) when it detects the pipeline for reading flow logs is stalled. It will revert to its initial aggregation level when this condition changes. The initial aggregation level can be specified using `FlowLogsFileAggregationKindForAllowed` and `FlowLogsFileAggregationKindForDenied`. If these values are not specified, default values of `2` and `1` will be used. | -| `FlowLogsPositionFilePath` | `FELIX_FLOWLOGSPOSITIONPATH` | `/var/log/calico/flows.log.pos` | Default path of the position file. It is used to read the current state of pipeline for flow logs. This parameter will be used only when `FlowLogsDynamicAggregationEnabled` is set to `true` | -| `FlowLogsAggregationThresholdBytes` | `FELIX_FLOWLOGSAGGREGATIONTHRESHOLDBYTES` | `8192` | Default threshold to determine how far behind the pipeline for flow logs can get before aggregation starts in. Detecting a difference of 8192 bytes means increase 1 level, while a difference of 16384 means increasing two levels. This parameter will be used only when `FlowLogsDynamicAggregationEnabled` is set to `true`. | -| `FlowLogsCollectProcessInfo` | `FELIX_FLOWLOGSCOLLECTPROCESSINFO` | `true` | If enabled Felix will load the kprobe BPF programs to collect process info. | -| `FlowLogsCollectTcpStats` | `FELIX_FLOWLOGSCOLLECTTCPSTATS` | `true` | If enabled Felix will collect TCP socket stats using BPF and requires a recent kernel that supports BPF | -| `FlowLogsCollectProcessPath` | `FELIX_FLOWLOGSCOLLECTPROCESSPATH` | `true` | If enabled, along with FlowLogsCollectProcessInfo, each flow log will contain the full path of the process executable and the arguments with which the executable was invoked. If path or arguments cannot be determined, Felix will fallback to using task names and arguments will be empty. For full functionality, this feature should be enabled via operator see [Enabling process path](../../../../observability/elastic/flow/processpath.mdx) | -| `FlowLogsFilePerFlowProcessLimit` | `FELIX_FLOWLOGSFILEPERFLOWPROCESSLIMIT` | `2` | Specify the maximum number of flow log entries with distinct process information beyond which process information will be aggregated. | -| `FlowLogsFilePerFlowProcessArgsLimit` | `FELIX_FLOWLOGSFILEPERFLOWPROCESSARGSLIMIT` | `5` | Specify the maximum number of arguments beyond which the process arguments will be aggregated. | -| `FlowLogsPolicyEvaluationMode` | `FELIX_FLOWLOGSPOLICYEVALUATIONMODE` | `Continuous` | Defines how policies are evaluated and reflected in flow logs. `OnNewConnection`: In this mode, staged policies are only evaluated when new connections are made in the data plane. Staged/active policy changes will not be reflected in the `pending_policies` field of flow logs for long lived connections. `Continuous`: Felix evaluates active flows on a regular basis to determine the rule traces in the flow logs. Any policy updates that impact a flow will be reflected in the `pending_policies` field, offering a near-real-time view of policy changes across flows. | -| `DNSCacheFile` | `FELIX_DNSCACHEFILE` | `/var/run/calico/felix-dns-cache.txt` | The name of the file that Felix uses to preserve learned DNS information when restarting. | -| `DNSCacheSaveInterval` | `FELIX_DNSCACHESAVEINTERVAL` | `60` | The periodic interval at which Felix saves learned DNS information to the cache file. | -| `DNSCacheEpoch` | `FELIX_DNSCACHEEPOCH` | `0` | An arbitrary number that can be changed, at runtime, to tell Felix to discard all its learned DNS information. | -| `DNSExtraTTL` | `FELIX_DNSEXTRATTL` | `0` | Extra time, in seconds, to keep IPs and alias names that are learned from DNS, in addition to each name or IP's advertised TTL. | -| `DNSTrustedServers` | `FELIX_DNSTRUSTEDSERVERS` | `k8s-service:kube-dns` | The DNS servers that Felix should trust. Each entry here must be `[:]` - indicating an explicit DNS server IP - or `k8s-service:[/][:port]` - indicating a Kubernetes DNS service. `` defaults to the first service port, or 53 for an IP, and `` to `kube-system`. An IPv6 address with a port must use the square brackets convention, for example `[fd00:83a6::12]:5353`. Note that Felix (calico-node) will need RBAC permission to read the details of each service specified by a `k8s-service:...` form. | -| `DNSLogsFileEnabled` | `FELIX_DNSLOGSFILEENABLED` | `false` | Set to `true`, enables DNS logs. If set to `false` no DNS logging will occur. DNS logs are written to a file `dns.log` and sent to Elasticsearch. The location of this file can be configured using the `DNSLogsFileDirectory` field. File rotation settings for this `dns.log` file can be configured using the fields `DNSLogsFileMaxFiles` and `DNSLogsFileMaxFileSizeMB`. Note that DNS log exports to Elasticsearch are dependent on DNS logs getting written to this file. Setting this parameter to `false` will disable DNS logs. | -| `DNSLogsFileDirectory` | `FELIX_DNSLOGSFILEDIRECTORY` | `/var/log/calico/dnslogs` | The directory where DNS logs files are stored. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | -| `DNSLogsFileMaxFiles` | `FELIX_DNSLOGSFILEMAXFILES` | `5` | The number of files to keep when rotating DNS log files. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | -| `DNSLogsFileMaxFileSizeMB` | `FELIX_DNSLOGSFILEMAXFILESIZEMB` | `100` | The max size in MB of DNS log files before rotation. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | -| `DNSLogsFlushInterval` | `FELIX_DNSLOGSFLUSHINTERVAL` | `300` | The period, in seconds, at which Felix exports DNS logs. | -| `DNSLogsFileAggregationKind` | `FELIX_DNSLOGSFILEAGGREGATIONKIND` | `1` | How much to aggregate DNS logs. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. `0` means no aggregation, `1` means aggregate similar DNS logs from workloads in the same ReplicaSet. | -| `DNSLogsFileIncludeLabels` | `FELIX_DNSLOGSFILEINCLUDELABELS` | `true` | Whether to include client and server workload labels in DNS logs. | -| `DNSLogsFilePerNodeLimit` | `FELIX_DNSLOGSFILEPERNODELIMIT` | `0` (no limit) | Limit on the number of DNS logs that can be emitted within each flush interval. When this limit has been reached, Felix counts the number of unloggable DNS responses within the flush interval, and emits a WARNING log with that count at the same time as it flushes the buffered DNS logs. | -| `DNSLogsLatency` | `FELIX_DNSLOGSLATENCY` | `true` | Indicates to include measurements of DNS request/response latency in each DNS log. | -| `EgressIPSupport` | `FELIX_EGRESSIPSUPPORT` | `Disabled` | Defines three different support modes for egress gateway function. `Disabled` means egress gateways are not supported. `EnabledPerNamespace` means egress gateway function is enabled and can be configured on a per-namespace basis (but per-pod egress annotations are ignored). `EnabledPerNamespaceOrPerPod` means egress gateway function is enabled and can be configured per-namespace or per-pod (with per-pod egress annotations overriding namespace annotations). | -| `EgressIPVXLANPort` | `FELIX_EGRESSIPVXLANPORT` | `4097` | Port to use for egress gateway VXLAN traffic. A value of `0` means "use the kernel default". | -| `EgressIPVXLANVNI` | `FELIX_EGRESSIPVXLANVNI` | `4790` | Virtual network ID to use for egress gateway VXLAN traffic. A value of `0` means "use the kernel default". | -| `EgressIPRoutingRulePriority` | `FELIX_EGRESSIPROUTINGRULEPRIORITY` | `100` | Priority value to use for the egress gateway routing rule. | -| `L7LogsFileEnabled` | `FELIX_L7LOGSFILEENABLED` | `true` | If set to `false` no L7 logging will occur. L7 logs are written to a file `l7.log` and sent to Elasticsearch. The location of this file can be configured using the `L7LogsFileDirectory` field. File rotation settings for this `l7.log` file can be configured using the fields `L7LogsFileMaxFiles` and `L7LogsFileMaxFileSizeMB`. Note that L7 log exports to Elasticsearch are dependent on L7 logs getting written to this file. | -| `L7LogsFileDirectory` | `FELIX_L7LOGSFILEDIRECTORY` | `/var/log/calico/l7logs` | The directory where L7 log files are stored. This parameter only takes effect when `L7LogsFileEnabled` is `true`. | -| `L7LogsFileMaxFiles` | `FELIX_L7LOGSFILEMAXFILES` | `5` | The number of files to keep when rotating L7 log files. This parameter only takes effect when `L7LogsFileEnabled` is `true`. | -| `L7LogsFileMaxFileSizeMB` | `FELIX_L7LOGSFILEMAXFILESIZEMB` | `100` | The max size in MB of L7 log files before rotation. This parameter only takes effect when `L7LogsFileEnabled` is `true`. | -| `L7LogsFlushInterval` | `FELIX_L7LOGSFLUSHINTERVAL` | `300` | The period, in seconds, at which Felix exports L7 logs. | -| `L7LogsFileAggregationHTTPHeaderInfo` | `FELIX_L7LOGSFILEAGGREGATIONHTTPHEADERINFO` | `ExcludeL7HTTPHeaderInfo` | How to handle HTTP header information for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7HTTPHeaderInfo` and `IncludeL7HTTPHeaderInfo`. | -| `L7LogsFileAggregationHTTPMethod` | `FELIX_L7LOGSFILEAGGREGATIONHTTPMETHOD` | `IncludeL7HTTPMethod` | How to handle HTTP method data for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7HTTPMethod` and `IncludeL7HTTPMethod`. | -| `L7LogsFileAggregationServiceInfo` | `FELIX_L7LOGSFILEAGGREGATIONSERVICEINFO` | `IncludeL7ServiceInfo` | How to handle service information for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7ServiceInfo` and `IncludeL7ServiceInfo`. | -| `L7LogsFileAggregationDestinationInfo` | `FELIX_L7LOGSFILEAGGREGATIONDESTINATIONINFO` | `IncludeL7DestinationInfo` | How to handle destination metadata for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7DestinationInfo` and `IncludeL7DestinationInfo`. | -| `L7LogsFileAggregationSourceInfo` | `FELIX_L7LOGSFILEAGGREGATIONSOURCEINFO` | `IncludeL7SourceInfoNoPort` | How to handle source metadata for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7SourceInfo`, `IncludeL7SourceInfoNoPort`, and `IncludeL7SourceInfo`. | -| `L7LogsFileAggregationResponseCode` | `FELIX_L7LOGSFILEAGGREGATIONRESPONSECODE` | `IncludeL7ResponseCode` | How to handle response code data for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7ResponseCode` and `IncludeL7ResponseCode`. | -| `L7LogsFileAggregationTrimURL` | `FELIX_L7LOGSFILEAGGREGATIONTRIMURL` | `IncludeL7FullURL` | How to handle URL data for aggregating L7 logs. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Possible values include `ExcludeL7URL`, `TrimURLQuery`, `TrimURLQueryAndPath`, and `IncludeL7FullURL`. | -| `L7LogsFileAggregationNumURLPath` | `FELIX_L7LOGSFILEAGGREGATIONNUMURLPATH` | `5` | How many components in the path to limit the URL by. This parameter only takes effect when `L7LogsFileAggregationTrimURL` is set to `IncludeL7FullURL`. Bear in mind that changing this value may have a dramatic impact on the volume of L7 logs sent to Elasticsearch. Negative values set the limit to infinity. | - -DropActionOverride controls what happens to each packet that is denied by -the current $[prodname] policy - i.e. by the ordered combination of all the -configured policies and profiles that apply to that packet. It may be -set to one of the following values: - -- `Drop` -- `Accept` -- `LogAndDrop` -- `LogAndAccept` - -Normally the `Drop` or `LogAndDrop` value should be used, as dropping a -packet is the obvious implication of that packet being denied. However when -experimenting, or debugging a scenario that is not behaving as you expect, the -`Accept` and `LogAndAccept` values can be useful: then the packet will be -still be allowed through. - -When set to `LogAndDrop` or `LogAndAccept`, each denied packet is logged in -syslog, with an entry like this: - -``` -May 18 18:42:44 ubuntu kernel: [ 1156.246182] calico-drop: IN=tunl0 OUT=cali76be879f658 MAC= SRC=192.168.128.30 DST=192.168.157.26 LEN=60 TOS=0x00 PREC=0x00 TTL=62 ID=56743 DF PROTO=TCP SPT=56248 DPT=80 WINDOW=29200 RES=0x00 SYN URGP=0 MARK=0xa000000 -``` - -If the `LogDropActionOverride` flag is set, then the `DropActionOverride` will also appear in the syslog entry: - -``` -May 18 18:42:44 ubuntu kernel: [ 1156.246182] calico-drop LOGandDROP: IN=tunl0 OUT=cali76be879f658 MAC= SRC=192.168.128.30 DST=192.168.157.26 LEN=60 TOS=0x00 PREC=0x00 TTL=62 ID=56743 DF PROTO=TCP SPT=56248 DPT=80 WINDOW=29200 RES=0x00 SYN URGP=0 MARK=0xa000000 -``` - -When the reporting of denied packet metrics is enabled, Felix keeps counts of -recently denied packets and publishes these as Prometheus metrics on the port -configured by the `PrometheusReporterPort` setting. - -Note that denied packet metrics are independent of the DropActionOverride -setting. Specifically, if packets that would normally be denied are being -allowed through by a setting of `Accept` or `LogAndAccept`, those packets -still contribute to the denied packet metrics as just described. - -### Felix-Typha Configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ----------------------- | --------------------------- | ----------------------------------------------------------------------------------------- | ------ | -| `TyphaAddr` | `FELIX_TYPHAADDR` | Address of the Typha Server when running outside a K8S Cluster, in the format IP:PORT | string | -| `TyphaK8sServiceName` | `FELIX_TYPHAK8SSERVICENAME` | Service Name of Typha Deployment when running inside a K8S Cluster | string | -| `TyphaK8sNamespace` | `FELIX_TYPHAK8SNAMESPACE` | Namespace of Typha Deployment when running inside a K8S Cluster. [Default: `kube-system`] | string | -| `TyphaReadTimeout` | `FELIX_TYPHAREADTIMEOUT` | Timeout of Felix when reading information from Typha, in seconds. [Default: 30] | int | -| `TyphaWriteTimeout` | `FELIX_TYPHAWRITETIMEOUT` | Timeout of Felix when writing information to Typha, in seconds. [Default: 30] | int | - -### Felix-Typha TLS configuration - -| Configuration parameter | Environment variable | Description | Schema | -| ----------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | -| `TyphaCAFile` | `FELIX_TYPHACAFILE` | Path to the file containing the root certificate of the CA that issued the Typha server certificate. Configures Felix to trust the CA that signed the root certificate. The file may contain multiple root certificates, causing Felix to trust each of the CAs included. Example: `/etc/felix/ca.pem` | string | -| `TyphaCertFile` | `FELIX_TYPHACERTFILE` | Path to the file containing the client certificate issued to Felix. Enables Felix to participate in mutual TLS authentication and identify itself to the Typha server. Example: `/etc/felix/cert.pem` | string | -| `TyphaCN` | `FELIX_TYPHACN` | If set, the `Common Name` that Typha's certificate must have. If you have enabled TLS on the communications from Felix to Typha, you must set a value here or in `TyphaURISAN`. You can set values in both, as well, such as to facilitate a migration from using one to the other. If either matches, the communication succeeds. [Default: none] | string | -| `TyphaKeyFile` | `FELIX_TYPHAKEYFILE` | Path to the file containing the private key matching the Felix client certificate. Enables Felix to participate in mutual TLS authentication and identify itself to the Typha server. Example: `/etc/felix/key.pem` (optional) | string | -| `TyphaURISAN` | `FELIX_TYPHAURISAN` | If set, a URI SAN that Typha's certificate must have. We recommend populating this with a [SPIFFE](https://github.com/spiffe/spiffe/blob/master/standards/SPIFFE-ID.md#2-spiffe-identity) string that identifies Typha. All Typha instances should use the same SPIFFE ID. If you have enabled TLS on the communications from Felix to Typha, you must set a value here or in `TyphaCN`. You can set values in both, as well, such as to facilitate a migration from using one to the other. If either matches, the communication succeeds. [Default: none] | string | - -For more information on how to use and set these variables, refer to -[Connections from Node to Typha (Kubernetes)](../../../../operations/comms/crypto-auth.mdx#connections-from-node-to-typha-kubernetes). - -### PacketCapture configuration - -The following parameters fine tune packet capture rotation: - -| Configuration parameter | Environment variable | Description | Schema | -| ------------------------ | --------------------------------- | --------------------------------------------------------------------------------------------- | ------ | -| `CaptureDir` | `FELIX_CAPTUREDIR` | Controls the directory where packet capture files are stored. Example: `/var/log/calico/pcap` | string | -| `CaptureMaxSizeBytes` | `FELIX_CAPTUREMAXSIZEBYTES` | Controls the maximum size in bytes for a packet capture file before rotation. | int | -| `CaptureRotationSeconds` | `FELIX_CAPTUREMAXROTATIONSECONDS` | Controls the rotation period in seconds for a packet capture file. | int | -| `CaptureMaxFiles` | `FELIX_CAPTUREMAXFILES` | Controls the maximum number rotated packet capture files. | int | - -### WireGuard configuration - -| Configuration parameter | Environment variable | Description | Schema | | -| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------- | ------- | -------------- | -| wireguardEnabled | Enable encryption for IPv4 on WireGuard supported nodes in cluster. When enabled, pod to pod traffic will be sent over encrypted tunnels between the nodes. | `true`, `false` | boolean | `false` | -| wireguardEnabledV6 | Enable encryption for IPv6 on WireGuard supported nodes in cluster. When enabled, pod to pod traffic will be sent over encrypted tunnels between the nodes. | `true`, `false` | boolean | `false` | -| wireguardInterfaceName | Name of the IPv4 WireGuard interface created by Felix. If you change the name, and want to clean up the previously-configured interface names on each node, this is a manual process. | string | string | wireguard.cali | -| wireguardInterfaceNameV6 | Name of the IPv6 WireGuard interface created by Felix. If you change the name, and want to clean up the previously-configured interface names on each node, this is a manual process. | string | string | wg-v6.cali | -| wireguardListeningPort | Port used by IPv4 WireGuard tunnels. Felix sets up an IPv4 WireGuard tunnel on each node specified by this port. Available for configuration only in the global FelixConfiguration resource; setting it per host, config-file or environment variable will not work. | 1-65535 | int | 51820 | -| wireguardListeningPortV6 | Port used by IPv6 WireGuard tunnels. Felix sets up an IPv6 WireGuard tunnel on each node specified by this port. Available for configuration only in the global FelixConfiguration resource; setting it per host, config-file or environment variable will not work. | 1-65535 | int | 51821 | -| wireguardMTU | MTU set on the IPv4 WireGuard interface created by Felix. Zero value means auto-detect. See [Configuring MTU](../../../../networking/configuring/mtu.mdx). | int | int | 0 | -| wireguardMTUV6 | MTU set on the IPv6 WireGuard interface created by Felix. Zero value means auto-detect. See [Configuring MTU](../../../../networking/configuring/mtu.mdx). | int | int | 0 | -| wireguardRoutingRulePriority | WireGuard routing rule priority value set up by Felix. If you change the default value, set it to a value most appropriate to routing rules for your nodes. | 1-32765 | int | 99 | -| wireguardHostEncryptionEnabled | **Experimental**: Adds host-namespace workload IP's to WireGuard's list of peers. Should **not** be enabled when WireGuard is enabled on a cluster's control plane node, as networking deadlock can occur. | true, false | boolean | false | -| wireguardKeepAlive | WireguardKeepAlive controls WireGuard PersistentKeepalive option. Set 0 to disable. [Default: 0] | int | int | 25 | - -For more information on encrypting in-cluster traffic with WireGuard, refer to -[Encrypt cluster pod traffic](../../../../compliance/encrypt-cluster-pod-traffic.mdx) +### Overlay: Wireguard + + + +### Overlay: IPSec + + + +### Flow logs: Prometheus reports + + + +### Flow logs: Syslog reports + + + +### Flow logs: file reports + + + +### DNS logs / policy + + + +### L7 logs + + + +### AWS integration + + + +### Egress gateway + + + +### External network support + + + +### Packet capture + + + +### L7 proxy + + + +### Debug/test-only (generally unsupported) + + + +### Usage reporting + + ## Environment variables @@ -383,10 +246,6 @@ include `FELIX_ETCDSCHEME`, `FELIX_ETCDKEYFILE`, `FELIX_ETCDCERTFILE`, `FELIX_ETCDCAFILE`, `FELIX_FELIXHOSTNAME`, `FELIX_LOGFILEPATH` and `FELIX_METADATAADDR`. -:::note -To set a parameter to an empty value using an environment variable, assign an empty string to it (e.g., `FELIX_FAILSAFEINBOUNDHOSTPORTS=""`). -::: - ## Configuration file On startup, Felix reads an ini-style configuration file. The path to @@ -432,7 +291,3 @@ a global setting and a per-host override. ``` For more information, see [Felix Configuration Resource](../../../resources/felixconfig.mdx). - - - - diff --git a/calico-cloud_versioned_docs/version-22-2/reference/resources/felixconfig.mdx b/calico-cloud_versioned_docs/version-22-2/reference/resources/felixconfig.mdx index f8671ef5f2..fc3aeb5f1d 100644 --- a/calico-cloud_versioned_docs/version-22-2/reference/resources/felixconfig.mdx +++ b/calico-cloud_versioned_docs/version-22-2/reference/resources/felixconfig.mdx @@ -1,7 +1,10 @@ --- description: API for this Calico Enterprise resource. +toc_max_heading_level: 4 --- +import FelixConfig from '../../_includes/components/FelixConfig'; + # Felix configuration A Felix configuration resource (`FelixConfiguration`) represents Felix configuration options for the cluster. @@ -37,196 +40,213 @@ spec: ### Spec -| Field | Description | Accepted Values | Schema | Default | -| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| awsSrcDstCheck | Controls automatically setting [source-destination-check](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCheck) on an AWS EC2 instance running Felix. Setting the value to `Enable` will set the check value in the instance description to `true`. For `Disable`, the check value will be `false`. Setting must be `Disable` if you want the EC2 instance to process traffic not matching the host interface IP address. For example, EKS cluster using Calico CNI with `VXLANMode=CrossSubnet`. Check [IAM role and profile configuration](#aws-iam-rolepolicy-for-source-destination-check-configuration) for setting the necessary permission for this setting to work. | DoNothing, Enable, Disable | string | `DoNothing` | -| awsSecondaryIPSupport | Controls whether Felix will create secondary AWS ENIs for AWS-backed IP pools. This feature is documented in the [egress gateways on AWS guide](../../networking/egress/egress-gateway-aws.mdx). Should only be enabled on AWS. | `Enabled`, `EnabledENIPerWorkload`, `Disabled` | string | `Disabled` | -| awsSecondaryIPRoutingRulePriority | Controls the priority of the policy-based routing rules used to implement AWS-backed IP addresses. Should only be changed to avoid conflicts if your nodes have additional policy based routing rules. | 0-4294967295 | int | 101 | -| awsRequestTimeout | Timeout used for communicating with the AWS API. | `5s`, `10s`, `1m` etc. | duration | `30s` | -| dropActionOverride | Controls what happens to each packet that is denied by the current $[prodname] policy. Normally the `Drop` or `LogAndDrop` value should be used. However when experimenting or debugging a scenario that is not behaving as you expect, the `Accept` and `LogAndAccept` values can be useful: then the packet will be still be allowed through. When one of the `LogAnd...` values is set, each denied packet is logged in syslog.\* | `Drop`, `Accept`, `LogAndDrop`, `LogAndAccept` | string | `Drop` | -| chainInsertMode | Controls whether Felix hooks the kernel's top-level iptables chains by inserting a rule at the top of the chain or by appending a rule at the bottom. `Insert` is the safe default since it prevents $[prodname]'s rules from being bypassed. If you switch to `Append` mode, be sure that the other rules in the chains signal acceptance by falling through to the $[prodname] rules, otherwise the $[prodname] policy will be bypassed. | `Insert`, `Append` | string | `Insert` | -| healthTimeoutOverrides | A list of overrides for Felix's internal liveness/readiness timeouts. | see [below](#health-timeout-overrides) | List of `HealthTimeoutOverride` objects | `[]` | -| dataplaneWatchdogTimeout | Deprecated, use `healthTimeoutOverrides` instead. Timeout before the main data plane goroutine is determined to have hung and Felix will report non-live and non-ready. Can be increased if the liveness check incorrectly fails (for example if Felix is running slowly on a heavily loaded system). | `90s`, `120s`, `10m` etc. | duration | `90s` | -| defaultEndpointToHostAction | This parameter controls what happens to traffic that goes from a workload endpoint to the host itself (after the traffic hits the endpoint egress policy). By default $[prodname] blocks traffic from workload endpoints to the host itself with an iptables "DROP" action. If you want to allow some or all traffic from endpoint to host, set this parameter to `Return` or `Accept`. Use `Return` if you have your own rules in the iptables "INPUT" chain; $[prodname] will insert its rules at the top of that chain, then `Return` packets to the "INPUT" chain once it has completed processing workload endpoint egress policy. Use `Accept` to unconditionally accept packets from workloads after processing workload endpoint egress policy. | Drop, Return, Accept | string | `Drop` | -| deviceRouteSourceAddress | IPv4 address to set as the source hint for routes programmed by Felix. When not set the source address for local traffic from host to workload will be determined by the kernel. | IPv4 | string | `""` | -| deviceRouteSourceAddressIPv6 | IPv6 address to set as the source hint for routes programmed by Felix. When not set the source address for local traffic from host to workload will be determined by the kernel. | IPv6 | string | `""` | -| deviceRouteProtocol | This defines the route protocol added to programmed device routes. | Protocol | int | RTPROT_BOOT | -| externalNodesCIDRList | A comma-delimited list of CIDRs of external non-calico nodes that can source tunnel traffic for acceptance by calico-nodes. | IPv4 | string | `""` | -| failsafeInboundHostPorts | UDP/TCP/SCTP protocol/cidr/port groupings that Felix will allow incoming traffic to host endpoints on irrespective of the security policy. This is useful to avoid accidentally cutting off a host with incorrect configuration. The default value allows SSH access, etcd, BGP, DHCP and the Kubernetes API. | | List of [ProtoPort](#protoport) |

- protocol: tcp
  port: 22
- protocol: udp
  port: 68
- protocol: tcp
  port: 179
- protocol: tcp
  port: 2379
- protocol: tcp
  port: 2380
- protocol: tcp
  port: 5473
- protocol: tcp
  port: 6443
- protocol: tcp
  port: 6666
- protocol: tcp
  port: 6667

| -| failsafeOutboundHostPorts | UDP/TCP/SCTP protocol/port groupings that Felix will allow outgoing traffic from host endpoints to irrespective of the security policy. This is useful to avoid accidentally cutting off a host with incorrect configuration. The default value opens etcd's standard ports to ensure that Felix does not get cut off from etcd as well as allowing DHCP, DNS, BGP and the Kubernetes API. | | List of [ProtoPort](#protoport) |

- protocol: udp
  port: 53
- protocol: udp
  port: 67
- protocol: tcp
  port: 179
- protocol: tcp
  port: 2379
- protocol: tcp
  port: 2380
- protocol: tcp
  port: 5473
- protocol: tcp
  port: 6443
- protocol: tcp
  port: 6666
- protocol: tcp
  port: 6667

| -| featureDetectOverride | Is used to override the feature detection. Values are specified in a comma separated list with no spaces, example; "SNATFullyRandom=true,MASQFullyRandom=false,RestoreSupportsLock=". "true" or "false" will force the feature, empty or omitted values are auto-detected. | string | string | `""` | -| genericXDPEnabled | When enabled, Felix can fallback to the non-optimized `generic` XDP mode. This should only be used for testing since it doesn't improve performance over the non-XDP mode. | true,false | boolean | `false` | -| interfaceExclude | A comma-separated list of interface names that should be excluded when Felix is resolving host endpoints. The default value ensures that Felix ignores Kubernetes' internal `kube-ipvs0` device. If you want to exclude multiple interface names using a single value, the list supports regular expressions. For regular expressions you must wrap the value with `/`. For example having values `/^kube/,veth1` will exclude all interfaces that begin with `kube` and also the interface `veth1`. | string | string | `kube-ipvs0` | -| interfacePrefix | The interface name prefix that identifies workload endpoints and so distinguishes them from host endpoint interfaces. Note: in environments other than bare metal, the orchestrators configure this appropriately. For example our Kubernetes and Docker integrations set the 'cali' value, and our OpenStack integration sets the 'tap' value. | string | string | `cali` | -| ipipEnabled | Optional, you shouldn't need to change this setting as Felix calculates if IPIP should be enabled based on the existing IP Pools. When set, this overrides whether Felix should configure an IPinIP interface on the host. When explicitly disabled in FelixConfiguration, Felix will not clean up addresses from the `tunl0` interface (use this if you need to add addresses to that interface and don't want to have them removed). | `true`, `false`, unset | optional boolean | unset | -| ipipMTU | The MTU to set on the tunnel device. Zero value means auto-detect. See [Configuring MTU](../../networking/configuring/mtu.mdx) | int | int | `0` | -| ipsetsRefreshInterval | Period at which Felix re-checks the IP sets in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable IP sets refresh. | `5s`, `10s`, `1m` etc. | duration | `10s` | -| iptablesFilterAllowAction | This parameter controls what happens to traffic that is accepted by a Felix policy chain in the iptables filter table (i.e. a normal policy chain). The default will immediately `Accept` the traffic. Use `Return` to send the traffic back up to the system chains for further processing. | Accept, Return | string | `Accept` | -| iptablesBackend | This parameter controls which variant of iptables Felix uses. If using Felix on a system that uses the netfilter-backed iptables binaries, set this to `nft`. | Legacy, nft | string | automatic detection | -| iptablesLockFilePath | Location of the iptables lock file. You may need to change this if the lock file is not in its standard location (for example if you have mapped it into Felix's container at a different path). | string | string | `/run/xtables.lock` | -| iptablesLockProbeInterval | Time that Felix will wait between attempts to acquire the iptables lock if it is not available. Lower values make Felix more responsive when the lock is contended, but use more CPU. | `5s`, `10s`, `1m` etc. | duration | `50ms` | -| iptablesLockTimeout | Time that Felix will wait for the iptables lock, or 0, to disable. To use this feature, Felix must share the iptables lock file with all other processes that also take the lock. When running Felix inside a container, this requires the /run directory of the host to be mounted into the $[nodecontainer] or calico/felix container. | `5s`, `10s`, `1m` etc. | duration | `0` (Disabled) | -| iptablesMangleAllowAction | This parameter controls what happens to traffic that is accepted by a Felix policy chain in the iptables mangle table (i.e. a pre-DNAT policy chain). The default will immediately `Accept` the traffic. Use `Return` to send the traffic back up to the system chains for further processing. | `Accept`, `Return` | string | `Accept` | -| iptablesMarkMask | Mask that Felix selects its IPTables Mark bits from. Should be a 32 bit hexadecimal number with at least 8 bits set, none of which clash with any other mark bits in use on the system. | netmask | netmask | `0xffff0000` | -| iptablesNATOutgoingInterfaceFilter | This parameter can be used to limit the host interfaces on which Calico will apply SNAT to traffic leaving a Calico IPAM pool with "NAT outgoing" enabled. This can be useful if you have a main data interface, where traffic should be SNATted and a secondary device (such as the docker bridge) which is local to the host and doesn't require SNAT. This parameter uses the iptables interface matching syntax, which allows `+` as a wildcard. Most users will not need to set this. Example: if your data interfaces are eth0 and eth1 and you want to exclude the docker bridge, you could set this to `eth+` | string | string | `""` | -| iptablesPostWriteCheckInterval | Period after Felix has done a write to the data plane that it schedules an extra read back to check the write was not clobbered by another process. This should only occur if another application on the system doesn't respect the iptables lock. | `5s`, `10s`, `1m` etc. | duration | `1s` | -| iptablesRefreshInterval | Period at which Felix re-checks all iptables state to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable iptables refresh. | `5s`, `10s`, `1m` etc. | duration | `90s` | -| ipv6Support | IPv6 support for Felix | `true`, `false` | boolean | `true` | -| logFilePath | The full path to the Felix log. Set to `none` to disable file logging. | string | string | `/var/log/calico/felix.log` | -| logPrefix | The log prefix that Felix uses when rendering LOG rules. | string | string | `calico-packet` | -| logSeverityFile | The log severity above which logs are sent to the log file. | Same as logSeveritySys | string | `Info` | -| logSeverityScreen | The log severity above which logs are sent to the stdout. | Same as logSeveritySys | string | `Info` | -| logSeveritySys | The log severity above which logs are sent to the syslog. Set to `none` for no logging to syslog. | Debug, Info, Warning, Error, Fatal | string | `Info` | -| logDebugFilenameRegex | controls which source code files have their Debug log output included in the logs. Only logs from files with names that match the given regular expression are included. The filter only applies to Debug level logs. | regex | string | `""` | -| maxIpsetSize | Maximum size for the ipsets used by Felix. Should be set to a number that is greater than the maximum number of IP addresses that are ever expected in a selector. | int | int | `1048576` | -| metadataAddr | The IP address or domain name of the server that can answer VM queries for cloud-init metadata. In OpenStack, this corresponds to the machine running nova-api (or in Ubuntu, nova-api-metadata). A value of `none` (case insensitive) means that Felix should not set up any NAT rule for the metadata path. | IPv4, hostname, none | string | `127.0.0.1` | -| metadataPort | The port of the metadata server. This, combined with global.MetadataAddr (if not 'None'), is used to set up a NAT rule, from 169.254.169.254:80 to MetadataAddr:MetadataPort. In most cases this should not need to be changed. | int | int | `8775` | -| natOutgoingAddress | The source address to use for outgoing NAT. By default an iptables MASQUERADE rule determines the source address which will use the address on the host interface the traffic leaves on. | IPV4 | string | `""` | -| policySyncPathPrefix | File system path where Felix notifies services of policy changes over Unix domain sockets. This is required only if you're configuring [L7 logs](../../observability/elastic/l7/configure.mdx), or [egress gateways](../../networking/egress/index.mdx). Set to `""` to disable. | string | string | `""` | -| prometheusGoMetricsEnabled | Set to `false` to disable Go runtime metrics collection, which the Prometheus client does by default. This reduces the number of metrics reported, reducing Prometheus load. | boolean | boolean | `true` | -| prometheusMetricsEnabled | Set to `true` to enable the experimental Prometheus metrics server in Felix. | boolean | boolean | `false` | -| prometheusMetricsHost | TCP network address that the Prometheus metrics server should bind to. | IPv4, IPv6, Hostname | string | `""` | -| prometheusMetricsPort | TCP port that the Prometheus metrics server should bind to. | int | int | `9091` | -| prometheusProcessMetricsEnabled | Set to `false` to disable process metrics collection, which the Prometheus client does by default. This reduces the number of metrics reported, reducing Prometheus load. | boolean | boolean | `true` | -| prometheusReporterEnabled | Set to `true` to enable configure Felix to keep count of recently denied packets and publish these as Prometheus metrics. Note that denied packet metrics are independent of the `dropActionOverride` setting. Specifically, if packets that would normally be denied are being allowed through by a setting of `Accept` or `LogAndAccept`, those packets still get counted as denied packets. | `true`, `false` | boolean | `false` | -| prometheusReporterPort | The TCP port on which to report denied packet metrics, if `prometheusReporterEnabled` is set to `true`. | | | `9092` | -| removeExternalRoutes | Whether or not to remove device routes that have not been programmed by Felix. Disabling this will allow external applications to also add device routes. | bool | boolean | `true` | -| reportingInterval | Interval at which Felix reports its status into the datastore. 0 means disabled and is correct for Kubernetes-only clusters. Must be non-zero in OpenStack deployments. | `5s`, `10s`, `1m` etc. | duration | `30s` | -| reportingTTL | Time-to-live setting for process-wide status reports. | `5s`, `10s`, `1m` etc. | duration | `90s` | -| routeRefreshInterval | Period at which Felix re-checks the routes in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable route refresh. | `5s`, `10s`, `1m` etc. | duration | `90s` | -| ipsecMode | Controls which mode IPsec is operating on. The only supported value is `PSK`. An empty value means IPsec is not enabled. | PSK | string | `""` | -| ipsecAllowUnsecuredTraffic | When set to `false`, only IPsec-protected traffic will be allowed on the packet paths where IPsec is supported. When set to `true`, IPsec will be used but non-IPsec traffic will be accepted. In general, setting this to `true` is less safe since it allows an attacker to inject packets. However, it is useful when transitioning from non-IPsec to IPsec since it allows traffic to flow while the cluster negotiates the IPsec mesh. | `true`, `false` | boolean | `false` | -| ipsecIKEAlgorithm | IPsec IKE algorithm. Default is NIST suite B recommendation. | string | string | `aes128gcm16-prfsha256-ecp256` | -| ipsecESPAlgorithm | IPsec ESP algorithm. Default is NIST suite B recommendation. | string | string | `aes128gcm16-ecp256` | -| ipsecLogLevel | Controls log level for IPsec components. Set to `None` for no logging. | `None`, `Notice`, `Info`, `Debug`, `Verbose` | string | `Info` | -| ipsecPSKFile | The path to the pre shared key file for IPsec. | string | string | `""` | -| flowLogsFileEnabled | Set to `true`, enables flow logs. If set to `false` no flow logging will occur. Flow logs are written to a file `flows.log` and sent to Elasticsearch. The location of this file can be configured using the `flowLogsFileDirectory` field. File rotation settings for this `flows.log` file can be configured using the fields `flowLogsFileMaxFiles` and `flowLogsFileMaxFileSizeMB`. Note that flow log exports to Elasticsearch are dependent on flow logs getting written to this file. Setting this parameter to `false` will disable flow logs. | `true`, `false` | boolean | `false` | -| flowLogsFileDirectory | Set the directory where flow logs files are stored on Linux nodes. This parameter only takes effect when `flowLogsFileEnabled` is set to `true`. | string | string | `/var/log/calico/flowlogs` | -| flowLogsPositionFilePath | Specify the position of the external pipeline that reads flow logs on Linux nodes. This parameter only takes effect when `FlowLogsDynamicAggregationEnabled` is set to `true`. | string | string | `/var/log/calico/flows.log.pos` | -| flowLogsFileMaxFiles | Set the number of log files to keep. This parameter only takes effect when `flowLogsFileEnabled` is set to `true`. | int | int | `5` | -| flowLogsFileMaxFileSizeMB | Set the max size in MB of flow logs files before rotation. This parameter only takes effect when `flowLogsFileEnabled` is set to `true`. | int | int | `100` | -| flowLogsFlushInterval | The period, in seconds, at which Felix exports the flow logs. | int | int | `300s` | -| flowLogsFileAggregationKindForAllowed | How much to aggregate the flow logs sent to Elasticsearch for allowed traffic. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. | 0-2 | [AggregationKind](#aggregationkind) | 2 | -| flowLogsFileAggregationKindForDenied | How much to aggregate the flow logs sent to Elasticsearch for denied traffic. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. | 0-2 | [AggregationKind](#aggregationkind) | 1 | -| flowLogsFileIncludeService | When set to `true`, include destination service information in the aggregated flow log. Note that service information will only be included when the flow can be explicitly determined to be bound to a service (e.g. pre-DNAT destination matches a service ClusterIP). | `true`, `false` | boolean | `false` | -| flowLogsFileIncludeLabels | When set to `true`, include source and destination endpoint labels in the aggregated flow log. Note that only Kubernetes endpoints or network sets are included; arbitrary networks do not contain labels. | `true`, `false` | boolean | `false` | -| flowLogsFileIncludePolicies | When set to `true`, include all policies in the aggregated flow logs that acted upon and matches the flow log traffic. | `true`, `false` | boolean | `false` | -| flowLogsDestDomainsByClient | When set to true, top-level domains are strictly associated with the source IP that originally queried the domains. (default: true) -| flowLogsEnableNetworkSets | When set to `true`, include an arbitrary network set in the aggregated flow log that matches the IP address of the flow log endpoint. | `true`, `false` | boolean | `false` | -| flowLogsCollectProcessInfo | When set to `true`, Felix will load the kprobe BPF programs to collect process info. | `true`, `false` | boolean | `false` | -| flowLogsCollectTcpStats | When set to `true`, Felix will collect the TCP socket stats. | `true`, `false` | boolean | `true` | -| flowLogsCollectProcessPath | When set to `true`, along with flowLogsCollectProcessInfo, each flow log will include the full path of the executable and the arguments with which the executable was invoked. | `true`, `false` | boolean | `false` | -| flowLogsFilePerFlowProcessLimit | Specify the maximum number of flow log entries with distinct process information beyond which process information will be aggregated | int | int | `2` | -| flowLogsFileNatOutgoingPortLimit | Specify the maximum number of distinct post SNAT ports that will appear in the flowLogs | int | int | `3` | -| flowLogsFilePerFlowProcessArgsLimit | Specify the maximum number of unique arguments in the flowlogs beyond which process arguments will be aggregated | int | int | `5` | -| flowLogsFileDomainsLimit | Specify the maximum number of top-level domains to include in a flow log. This only applies to source reported flows to destinations external to the cluster. | int | int | `5` | -| flowLogsPolicyEvaluationMode | Defines how policies are evaluated and reflected in flow logs. `OnNewConnection`: In this mode, staged policies are only evaluated when new connections are made in the data plane. Staged/active policy changes will not be reflected in the `pending_policies` field of flow logs for long lived connections. `Continuous`: Felix evaluates active flows on a regular basis to determine the rule traces in the flow logs. Any policy updates that impact a flow will be reflected in the `pending_policies` field, offering a near-real-time view of policy changes across flows. | `Continuous`, `OnNewConnection` | string | `Continuous` | -| statsDumpFilePath | Specify the position of the file used for dumping flow log statistics on Linux nodes. Note this is an internal setting that users shouldn't need to modify. | string | string | `/var/log/calico/stats/dump` | -| routeTableRange | _deprecated in favor of `RouteTableRanges`_ Calico programs additional Linux route tables for various purposes. `RouteTableRange` specifies the indices of the route tables that Calico should use. | | [RouteTableRanges](#routetablerange) | `""` | -| routeTableRanges | Calico programs additional Linux route tables for various purposes. `RouteTableRanges` specifies a set of table index ranges that Calico should use. Deprecates `RouteTableRange`, overrides `RouteTableRange` | | [RouteTableRanges](#routetableranges) | `[{"min": 1, "max": 250}]` | -| routeSyncDisabled | Set to `true` to disable Calico programming routes to local workloads. | boolean | boolean | `false` | -| serviceLoopPrevention | When [service IP advertisement is enabled](../../networking/configuring/advertise-service-ips.mdx), prevent routing loops to service IPs that are not in use, by dropping or rejecting packets that do not get DNAT'd by kube-proxy. Unless set to "Disabled", in which case such routing loops continue to be allowed. | `Drop`, `Reject`, `Disabled` | string | `Drop` | -| workloadSourceSpoofing | Controls whether pods can enable source IP address spoofing with the `cni.projectcalico.org/allowedSourcePrefixes` annotation. When set to `Any`, pods can use this annotation to send packets from any IP address. | `Any`, `Disabled` | string | `Disabled` | -| vxlanEnabled | Optional, you shouldn't need to change this setting as Felix calculates if VXLAN should be enabled based on the existing IP Pools. When set, this overrides whether Felix should create the VXLAN tunnel device for VXLAN networking. | `true`, `false`, unset | optional boolean | unset | -| vxlanMTU | MTU to use for the IPv4 VXLAN tunnel device. Zero value means auto-detect. Also controls NodePort MTU when eBPF enabled. | int | int | `0` | -| vxlanMTUV6 | MTU to use for the IPv6 VXLAN tunnel device. Zero value means auto-detect. Also controls NodePort MTU when eBPF enabled. | int | int | `0` | -| vxlanPort | Port to use for VXLAN traffic. A value of `0` means "use the kernel default". | int | int | `4789` | -| vxlanVNI | Virtual network ID to use for VXLAN traffic. A value of `0` means "use the kernel default". | int | int | `4096` | -| allowVXLANPacketsFromWorkloads | Set to `true` to allow VXLAN encapsulated traffic from workloads. | boolean | boolean | `false` | -| allowIPIPPacketsFromWorkloads | Set to `true` to allow IPIP encapsulated traffic from workloads. | boolean | boolean | `false` | -| windowsFlowLogsFileDirectory | Set the directory where flow logs files are stored on Windows nodes. This parameter only takes effect when `flowLogsFileEnabled` is set to `true`. | string | string | `c:\\TigeraCalico\\flowlogs` | -| windowsFlowLogsPositionFilePath | Specify the position of the external pipeline that reads flow logs on Windows nodes. This parameter only takes effect when `FlowLogsDynamicAggregationEnabled` is set to `true`. | string | string | `c:\\TigeraCalico\\flowlogs\\flows.log.pos` | -| windowsStatsDumpFilePath | Specify the position of the file used for dumping flow log statistics on Windows nodes. Note this is an internal setting that users shouldn't need to modify. | string | string | `c:\\TigeraCalico\\stats\\dump` | -| windowsDNSCacheFile | Specify the name of the file that Calico uses to preserve learnt DNS information when restarting. | string | string | `c:\\TigeraCalico\\felix-dns-cache.txt` | -| windowsDNSExtraTTL | Specify extra time in seconds to keep IPs and alias names that are learnt from DNS, in addition to each name or IP's advertised TTL. | int | int | `120` | -| wireguardEnabled | Enable encryption on WireGuard supported nodes in cluster. When enabled, pod to pod traffic will be sent over encrypted tunnels between the nodes. | `true`, `false` | boolean | `false` | -| wireguardEnabledV6 | Enable encryption for IPv6 on WireGuard supported nodes in cluster. When enabled, pod to pod traffic will be sent over encrypted tunnels between the nodes. | `true`, `false` | boolean | `false` | -| wireguardInterfaceName | Name of the WireGuard interface created by Felix. If you change the name, and want to clean up the previously-configured interface names on each node, this is a manual process. Felix expect the name to end with either .cali or .calico suffix. | string | string | wireguard.cali | -| wireguardInterfaceNameV6 | Name of the IPv6 WireGuard interface created by Felix. If you change the name, and want to clean up the previously-configured interface names on each node, this is a manual process. Felix expect the name to end with either .cali or .calico suffix. | string | string | wg-v6.cali | -| wireguardListeningPort | Port used by WireGuard tunnels. Felix sets up WireGuard tunnel on each node specified by this port. Available for configuration only in the global FelixConfiguration resource; setting it per host, config-file or environment variable will not work. | 1-65535 | int | 51820 | -| wireguardListeningPortV6 | Port used by IPv6 WireGuard tunnels. Felix sets up an IPv6 WireGuard tunnel on each node specified by this port. Available for configuration only in the global FelixConfiguration resource; setting it per host, config-file or environment variable will not work. | 1-65535 | int | 51821 | -| wireguardMTU | MTU set on the WireGuard interface created by Felix. Zero value means auto-detect. See [Configuring MTU](../../networking/configuring/mtu.mdx). | int | int | 0 | -| wireguardMTUV6 | MTU set on the IPv6 WireGuard interface created by Felix. Zero value means auto-detect. See [Configuring MTU](../../networking/configuring/mtu.mdx). | int | int | 0 | -| wireguardRoutingRulePriority | WireGuard routing rule priority value set up by Felix. If you change the default value, set it to a value most appropriate to routing rules for your nodes. | 1-32765 | int | 99 | -| wireguardHostEncryptionEnabled | **Experimental**: Adds host-namespace workload IP's to WireGuard's list of peers. Should **not** be enabled when WireGuard is enabled on a cluster's control plane node, as networking deadlock can occur. | true, false | boolean | false | -| wireguardKeepAlive | WireguardKeepAlive controls WireGuard PersistentKeepalive option. Set 0 to disable. [Default: 0] | `5s`, `10s`, `1m` etc. | duration | `0` | -| xdpRefreshInterval | Period at which Felix re-checks the XDP state in the data plane to ensure that no other process has accidentally broken $[prodname]'s rules. Set to 0 to disable XDP refresh. | `5s`, `10s`, `1m` etc. | duration | `90s` | -| xdpEnabled | When `bpfEnabled` is `false`: enable XDP acceleration for host endpoint policies. When `bpfEnabled` is `true`, XDP is automatically used for Calico policy where that makes sense, regardless of this setting. [Default: `true`] | true,false | boolean | `true` | -| dnsCacheFile | The name of the file that Felix uses to preserve learnt DNS information when restarting. | file name | string | `/var/run/calico/felix-dns-cache.txt` | -| dnsCacheSaveInterval | The period, in seconds, at which Felix saves learnt DNS information to the cache file. | `5s`, `10s`, `1m` etc. | duration | `60s` | -| dnsCacheEpoch | An arbitrary number that can be changed, at runtime, to tell Felix to discard all its learnt DNS information. | int | int | `0` | -| dnsExtraTTL | Extra time to keep IPs and alias names that are learnt from DNS, in addition to each name or IP's advertised TTL. | `5s`, `10s`, `1m` etc. | duration | `0s` | -| dnsTrustedServers | The DNS servers that Felix should trust. Each entry here must be `[:]` - indicating an explicit DNS server IP - or `k8s-service:[/][:port]` - indicating a Kubernetes DNS service. `` defaults to the first service port, or 53 for an IP, and `` to `kube-system`. An IPv6 address with a port must use the square brackets convention, for example `[fd00:83a6::12]:5353`. Note that Felix (calico-node) will need RBAC permission to read the details of each service specified by a `k8s-service:...` form. | IPs or service names | comma-separated strings | `k8s-service:kube-dns` | -| dnsLogsFileEnabled | Set to `true`, enables DNS logs. If set to `false` no DNS logging will occur. DNS logs are written to a file `dns.log` and sent to Elasticsearch. The location of this file can be configured using the `DNSLogsFileDirectory` field. File rotation settings for this `dns.log` file can be configured using the fields `DNSLogsFileMaxFiles` and `DNSLogsFileMaxFileSizeMB`. Note that DNS log exports to Elasticsearch are dependent on DNS logs getting written to this file. Setting this parameter to `false` will disable DNS logs. | `true`, `false` | boolean | `false` | -| dnsLogsFileDirectory | The directory where DNS logs files are stored. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | directory | string | `/var/log/calico/dnslogs` | -| dnsLogsFileMaxFiles | The number of files to keep when rotating DNS log files. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | int | int | `5` | -| dnsLogsFileMaxFileSizeMB | The max size in MB of DNS log files before rotation. This parameter only takes effect when `DNSLogsFileEnabled` is `true`. | int | int | `100` | -| dnsLogsFlushInterval | The period, in seconds, at which Felix exports DNS logs. | int | int | `300s` | -| dnsLogsFileAggregationKind | How much to aggregate DNS logs. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. `0` means no aggregation, `1` means aggregate similar DNS logs from workloads in the same ReplicaSet. | `0`,`1` | int | `1` | -| dnsLogsFileIncludeLabels | Whether to include client and server workload labels in DNS logs. | `true`, `false` | boolean | `true` | -| dnsLogsFilePerNodeLimit | Limit on the number of DNS logs that can be emitted within each flush interval. When this limit has been reached, Felix counts the number of unloggable DNS responses within the flush interval, and emits a WARNING log with that count at the same time as it flushes the buffered DNS logs. | int | int | `0` (no limit) | -| dnsLogsLatency | Indicates to include measurements of DNS request/response latency in each DNS log. | `true`, `false` | boolean | `true` | -| dnsPolicyMode | DNSPolicyMode specifies how DNS policy programming will be handled. | `Inline`, `NoDelay`, `DelayDNSResponse`, `DelayDeniedPacket` | [DNSPolicyMode](#dnspolicymode) | `Inline` | -| bpfDNSPolicyMode | BPFDNSPolicyMode specifies how DNS policy programming will be handled. | `NoDelay`, `Inline` | [BPFDNSPolicyMode](#bpfdnspolicymode) | `Inline` | -| nftablesDNSPolicyMode | Specifies how DNS policy programming will be handled for NFTables. Options include `NoDelay`, `DelayDNSResponse`, and `DelayDeniedPacket`. See [NFTablesDNSPolicyMode](#nftablesdnspolicymode) for details. | `NoDelay`, `DelayDNSResponse`, `DelayDeniedPacket` | [NFTablesDNSPolicyMode](#nftablesdnspolicymode) | `DelayDeniedPacket` | -| dnsPolicyNfqueueID | DNSPolicyNfqueueID is the NFQUEUE ID to use for DNS Policy re-evaluation when the domains IP hasn't been programmed to ipsets yet. This value can be changed to avoid conflicts with other users of NFQUEUEs. Used when `DNSPolicyMode` is `DelayDeniedPacket`. | 0-65535 | int | `100` | -| dnsPolicyNfqueueSize | DNSPolicyNfqueueID is the size of the NFQUEUE for DNS policy re-evaluation. This is the maximum number of denied packets that may be queued up pending re-evaluation. Used when `DNSPolicyMode` is `DelayDeniedPacket`. | 0-65535 | int | `100` | -| dnsPacketsNfqueueID | DNSPacketsNfqueueID is the NFQUEUE ID to use for capturing DNS packets to ensure programming IPSets occurs before the response is released. Used when `DNSPolicyMode` is `DelayDNSResponse`. | 0-65535 | int | `101` | -| dnsPacketsNfqueueSize | DNSPacketsNfqueueSize is the size of the NFQUEUE for captured DNS packets. This is the maximum number of DNS packets that may be queued awaiting programming in the data plane. Used when `DNSPolicyMode` is `DelayDNSResponse`. | 0-65535 | int | `100` | -| dnsPacketsNfqueueMaxHoldDuration | DNSPacketsNfqueueMaxHoldDuration is the max length of time to hold on to a DNS response while waiting for the data plane to be programmed. Used when `DNSPolicyMode` is `DelayDNSResponse`. | `5s`, `10s`, `1m` etc. | duration | `3s` | -| bpfEnabled | Enable eBPF data plane mode. eBPF mode has some limitations, see the [HOWTO guide](../../operations/ebpf/enabling-ebpf.mdx) for more details. | true, false | boolean | false | -| bpfDisableUnprivileged | If true, Felix sets the kernel.unprivileged_bpf_disabled sysctl to disable unprivileged use of BPF. This ensures that unprivileged users cannot access Calico's BPF maps and cannot insert their own BPF programs to interfere with the ones that $[prodname] installs. | true, false | boolean | true | -| bpfLogLevel | In eBPF data plane mode, the log level used by the BPF programs. The logs are emitted to the BPF trace pipe, accessible with the command `tc exec bpf debug`. This is a tech preview feature and subject to change in future releases. | Off,Info,Debug | string | Off | -| bpfDataIfacePattern | In eBPF data plane mode, controls which interfaces Felix should attach BPF programs to catch traffic to/from the external network. This needs to match the interfaces that Calico workload traffic flows over as well as any interfaces that handle incoming traffic to NodePorts and services from outside the cluster. It should not match the workload interfaces (usually named cali...).. This is a tech preview feature and subject to change in future releases. | regular expression | string | ^(en.*|eth.*|tunl0$) | -| bpfConnectTimeLoadBalancingEnabled | In eBPF data plane mode, controls whether Felix installs the connect-time load balancer. In the current release, the connect-time load balancer is required for the host to reach kubernetes services. This is a tech preview feature and subject to change in future releases. | true,false | boolean | true | -| bpfExternalServiceMode | In eBPF data plane mode, controls how traffic from outside the cluster to NodePorts and ClusterIPs is handled. In Tunnel mode, packet is tunneled from the ingress host to the host with the backing pod and back again. In DSR mode, traffic is tunneled to the host with the backing pod and then returned directly; this requires a network that allows direct return. | Tunnel,DSR | string | Tunnel | -| bpfKubeProxyIptablesCleanupEnabled | In eBPF data plane mode, controls whether Felix will clean up the iptables rules created by the Kubernetes `kube-proxy`; should only be enabled if `kube-proxy` is not running. This is a tech preview feature and subject to change in future releases. | true,false | boolean | true | -| bpfKubeProxyMinSyncPeriod | In eBPF data plane mode, controls the minimum time between data plane updates for Felix's embedded `kube-proxy` implementation. | `5s`, `10s`, `1m` etc. | duration | `1s` | -| BPFKubeProxyEndpointSlicesEnabled | In eBPF data plane mode, controls whether Felix's embedded kube-proxy derives its services from Kubernetes' EndpointSlices resources. Using EndpointSlices is more efficient but it requires EndpointSlices support to be enabled at the Kubernetes API server. | true,false | boolean | false | -| bpfMapSizeConntrack | In eBPF data plane mode, controls the size of the conntrack map. | int | int | 512000 | -| bpfMapSizeIPSets | In eBPF data plane mode, controls the size of the ipsets map. | int | int | 1048576 | -| bpfMapSizeNATAffinity | In eBPF data plane mode, controls the size of the NAT affinity map. | int | int | 65536 | -| bpfMapSizeNATFrontend | In eBPF data plane mode, controls the size of the NAT front end map. | int | int | 65536 | -| bpfMapSizeNATBackend | In eBPF data plane mode, controls the size of the NAT back end map. | int | int | 262144 | -| bpfMapSizeRoute | In eBPF data plane mode, controls the size of the route map. | int | int | 262144 | -| bpfPolicyDebugEnabled | In eBPF data plane mode, controls whether felix will collect policy dump for each interface. | true, false | boolean | true | -| routeSource | Where Felix gets is routing information from for VXLAN and the BPF data plane. The CalicoIPAM setting is more efficient because it supports route aggregation, but it only works when Calico's IPAM or host-local IPAM is in use. Use the WorkloadIPs setting if you are using Calico's VXLAN or BPF data plane and not using Calico IPAM or host-local IPAM. | CalicoIPAM,WorkloadIPs | string | `CalicoIPAM` | -| mtuIfacePattern | Pattern used to discover the host's interface for MTU auto-detection. | regex | string | ^((en|wl|ww|sl|ib)[opsvx].*|(eth|wlan|wwan).*) | -| bpfForceTrackPacketsFromIfaces | Forces traffic from these interfaces in BPF mode to skip Calico's iptables NOTRACK rule, allowing traffic from those interfaces to be tracked by Linux conntrack. Use only for interfaces that are not used for the Calico fabric, for example, a docker bridge device for non-Calico-networked containers. | A list of strings | A list of strings | docker+ | -| bpfDisableGROForIfaces | BPFDisableGROForIfaces is a regular expression that controls which interfaces Felix should disable the Generic Receive Offload [GRO] option. It should not match the workload interfaces (usually named cali...). | regex | string | "" | -| egressIPSupport | Defines three different support modes for egress gateway function. `Disabled` means egress gateways are not supported. `EnabledPerNamespace` means egress gateway function is enabled and can be configured on a per-namespace basis (but per-pod egress annotations are ignored). `EnabledPerNamespaceOrPerPod` means egress gateway function is enabled and can be configured per-namespace or per-pod (with per-pod egress annotations overriding namespace annotations). | Disabled,
EnabledPerNamespace,
EnabledPerNamespaceOrPerPod | string | `Disabled` | -| egressIPVXLANPort | Port to use for egress gateway VXLAN traffic. A value of `0` means "use the kernel default". | int | int | `4790` | -| egressIPVXLANVNI | Virtual network ID to use for egress gateway VXLAN traffic. A value of `0` means "use the kernel default". | int | int | `4097` | -| egressIPRoutingRulePriority | Controls the priority value to use for the egress gateway routing rule. | int | int | `100` | -| egressGatewayPollInterval | Controls the interval at which Felix will poll remote egress gateways to check their health. Only Egress Gateways with a named "health" port will be polled in this way. Egress Gateways that fail the health check will be taken our of use as if they have been deleted. | `5s`, `10s`, `1m` etc. | duration | `10s` | -| egressGatewayPollFailureCount | Controls the minimum number of poll failures before a remote Egress Gateway is considered to have failed. | int | int | `3` | -| captureDir | Controls the directory where packet capture files are stored. | string | string | `/var/log/calico/pcap` | -| captureMaxSizeBytes | Controls the maximum size in bytes for a packet capture file before rotation. | int | int | `10000000` | -| captureRotationSeconds | Controls the rotation period in seconds for a packet capture file. | int | int | `3600` | -| captureMaxFiles | Controls the maximum number rotated packet capture files. | int | int | `2` | - -\* When `dropActionOverride` is set to `LogAndDrop` or `LogAndAccept`, the `syslog` entries look something like the following. +#### Datastore connection -``` -May 18 18:42:44 ubuntu kernel: [ 1156.246182] calico-drop: IN=tunl0 OUT=cali76be879f658 MAC= SRC=192.168.128.30 DST=192.168.157.26 LEN=60 TOS=0x00 PREC=0x00 TTL=62 ID=56743 DF PROTO=TCP SPT=56248 DPT=80 WINDOW=29200 RES=0x00 SYN URGP=0 MARK=0xa000000 -``` + + +#### Process: Feature detection/overrides + + + +#### Process: Go runtime + + + +#### Process: Health port and timeouts + + + +#### Process: Logging + + + +#### Process: Prometheus metrics + + + +#### Dataplane: Common + + + +#### Dataplane: iptables + + + +#### Dataplane: nftables + + + +#### Dataplane: eBPF + + + +#### Dataplane: Windows + + + +#### Dataplane: OpenStack support + + + +#### Dataplane: XDP acceleration for iptables dataplane + + + +#### Overlay: VXLAN overlay + + + +#### Overlay: IP-in-IP -\*\* Duration is denoted by the numerical amount followed by the unit of time. Valid units of time include nanoseconds (ns), microseconds (µs), milliseconds (ms), seconds (s), minutes (m), and hours (h). Units of time can also be used together e.g. `3m30s` to represent 3 minutes and 30 seconds. Any amounts of time that can be converted into larger units of time will be converted e.g. `90s` will become `1m30s`. + -
+#### Overlay: Wireguard -`genericXDPEnabled` and `xdpRefreshInterval` are only relevant when `bpfEnabled` is `false` and -`xdpEnabled` is `true`; in other words when XDP is being used to accelerate denial-of-service -prevention policies in the iptables data plane. + -When `bpfEnabled` is `true` the "xdp" settings all have no effect; in BPF mode the implementation of -policy is always accelerated, using the best available BPF technology. +#### Overlay: IPSec + + + +#### Flow logs: Prometheus reports + + + +#### Flow logs: Syslog reports + + + +#### Flow logs: file reports + + + +#### DNS logs / policy + + + +#### L7 logs + + + +#### AWS integration + + + +#### Egress gateway + + + +#### External network support + + + +#### Packet capture + + + +#### L7 proxy + + +#### Debug/test-only (generally unsupported) + + + +#### Usage reporting + + ### Health Timeout Overrides Felix has internal liveness and readiness watchdog timers that monitor its various loops. If a loop fails to "check in" within the allotted timeout then Felix will report non-Ready -or non-Live on its health port (which is monitored by kubelet in a Kubernetes system). +or non-Live on its health port (which is monitored by Kubelet in a Kubernetes system). If Felix reports non-Live, this can result in the Pod being restarted. In Kubernetes, if you see the calico-node Pod readiness or liveness checks fail @@ -246,7 +266,7 @@ overall health status (the list of components will depend on which features are If some health timeouts show as "timed out" it may help to apply an override using the `healthTimeoutOverrides` field: -```yaml +```yaml noValidation ... spec: healthTimeoutOverrides: @@ -296,27 +316,20 @@ failsafeOutboundHostPorts: ### DNSPolicyMode -| Value | Description | -| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Inline (default) | Felix parses DNS response inline with DNS response packet processing within IPTables. This guarantees the DNS rules reflect any change immediately. This mode works for iptables only and matches the same mode for `BPFDNSPolicyMode`.| -| DelayDeniedPacket | Felix delays any denied packet that traversed a policy that included egress domain matches, but did not match. The packet is released after a fixed time, or after the destination IP address was programmed. | -| DelayDNSResponse | Felix delays any DNS response until related IPSets are programmed. This introduces some latency to all DNS packets (even when no IPSet programming is required), but it ensures policy hit statistics are accurate. This is the recommended setting when you are making use of staged policies or policy rule hit statistics. | -| NoDelay | Felix does not introduce any delay to the packets. DNS rules may not have been programmed by the time the first packet traverses the policy rules. Client applications need to handle reconnection attempts if initial connection attempts fail. This may be problematic for some applications or for very low DNS TTLs. | - -On Windows, or when using the eBPF data plane, this setting is ignored and `NoDelay` is always used. - -This field has no effect in NFTables mode. Please use `NFTablesDNSPolicyMode` instead. - -A linux kernel version of 3.13 or greater is required to use `DelayDNSResponse`. For earlier kernel versions, this value is modified to `DelayDeniedPacket`. +| Value | Description | +| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| DelayDeniedPacket | Felix delays any denied packet that traversed a policy that included egress domain matches, but did not match. The packet is released after a fixed time, or after the destination IP address was programmed. | +| DelayDNSResponse | Felix delays any DNS response until related IPSets are programmed. This introduces some latency to all DNS packets (even when no IPSet programming is required), but it ensures policy hit statistics are accurate. This is the recommended setting when you are making use of staged policies or policy rule hit statistics. A Linux kernel version of 3.13 or greater is required to use `DelayDNSResponse`. For earlier kernel versions, this value is modified to `DelayDeniedPacket`. | +| NoDelay | Felix does not introduce any delay to the packets. DNS rules may not have been programmed by the time the first packet traverses the policy rules. Client applications need to handle reconnection attempts if initial connection attempts fail. This may be problematic for some applications or for very low DNS TTLs. | -### NFTablesDNSPolicyMode +On Windows, or when using the eBPF dataplane, this setting is ignored. Windows always uses `NoDelay` while eBPF has its own [BPFDNSPolicyMode](#bpfdnspolicymode) option. -| Value | Description | -| ----------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| DelayDeniedPacket (default) | Felix delays any denied packet that traversed a policy that included egress domain matches, but did not match. The packet is released after a fixed time, or after the destination IP address was programmed. | -| DelayDNSResponse | Felix delays any DNS response until related IPSets are programmed. This introduces some latency to all DNS packets (even when no IPSet programming is required), but it ensures policy hit statistics are accurate. This is the recommended setting when you are making use of staged policies or policy rule hit statistics. | -| NoDelay | Felix does not introduce any delay to the packets. DNS rules may not have been programmed by the time the first packet traverses the policy rules. Client applications need to handle reconnection attempts if initial connection attempts fail. This may be problematic for some applications or for very low DNS TTLs. | +### BPFDNSPolicyMode +| Value | Description | +| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Inline | Felix does not introduce any delay to any packets. Felix's eBPF programs parse DNS responses and program policy rules immediately, before the DNS response is passed to the application. This only applies to wildcard prefixes: `*.x.y.z` will be processed in this manner, but wildcards such as `x.*.y.z` will not match. A Linux kernel version of 5.17 or greater is required. | +| NoDelay | Felix does not introduce any delay to the packets. DNS rules may not have been programmed by the time the first packet traverses the policy rules. Client applications need to handle reconnection attempts if initial connection attempts fail. This may be problematic for some applications or for very low DNS TTLs. | ### RouteTableRange