-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Blueprints evpn-udn docs #109142
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Blueprints evpn-udn docs #109142
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * networking/advanced_networking/bgp_evpn_udn/about-bgp-evpn-user-defined-networks.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="nw-bgp-evpn-about_{context}"] | ||
| = Overview of BGP EVPN with OVN-Kubernetes | ||
|
|
||
| [role="_abstract"] | ||
| Border Gateway Protocol (BGP) Ethernet Virtual Private Network (EVPN) extends BGP with address families that distribute Ethernet and IP VPN routes, such as MAC reachability in a MAC-VRF and IP prefixes in an IP-VRF. | ||
|
|
||
| With the OVN-Kubernetes network plugin on {product-title}, you can attach primary cluster user-defined networks (CUDNs) to an external EVPN fabric. This integration lets you extend segmentation and reachability for tenant namespaces beyond the cluster boundary by using the same BGP-based standards as your physical or virtual network infrastructure. | ||
|
|
||
| OVN-Kubernetes drives EVPN integration with FRR-K8s: FRR establishes BGP EVPN sessions toward your fabric, while OVN-Kubernetes programs overlay and underlay state for the selected CUDNs. A `VTEP` custom resource defines how the cluster exposes virtual tunnel endpoints (VTEPs) for EVPN encapsulation, and `RouteAdvertisements` objects select which primary CUDNs are advertised and which FRR configuration to use. | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| For primary CUDNs that use EVPN transport, traffic carried over the EVPN fabric typically uses VXLAN encapsulation rather than the Geneve overlay that OVN-Kubernetes commonly uses for east-west traffic on user-defined networks without EVPN. The cluster default pod network is not modeled as an EVPN VPN in this design; EVPN applies to primary CUDNs that you configure for EVPN. | ||
| ==== | ||
|
|
||
| [id="bgp-evpn-primary-use-cases_{context}"] | ||
| == BGP EVPN with primary CUDN use cases | ||
|
|
||
| Common reasons to deploy BGP EVPN with primary CUDNs include the following: | ||
|
|
||
| Fabric integration:: Map tenant networks to the same route-target and VRF semantics that you already use on an external BGP EVPN fabric. | ||
|
|
||
| Layer 2 stretch for mobility:: Extend a layer 2 segment into the cluster so that virtual machines or pods can keep addresses stable. EVPN MAC mobility can help when workloads move between nodes. | ||
|
|
||
| Simpler attachment than per-node local wiring:: Reduce dependence on host-specific local attachment patterns when the fabric can carry many tenant VNIs over shared uplinks. | ||
|
|
||
| Multi-cluster extension:: Stretching the same logical segment across clusters might be possible in some designs; treat this as advanced and confirm support and caveats for your release. | ||
|
|
||
| [id="about-bgp-evpn-other-features_{context}"] | ||
| == About BGP EVPN, BGP routing, and route advertisements | ||
|
|
||
| BGP EVPN for primary CUDNs is not a separate routing stack. It requires the same platform pieces as OVN-Kubernetes BGP: | ||
|
|
||
| * FRR-K8s must be deployed through Cluster Network Operator `additionalRoutingCapabilities` so that `FRRConfiguration` objects can express BGP and EVPN toward peers on the provider network. | ||
|
|
||
| * Route advertisements must be enabled in the OVN-Kubernetes configuration so that `RouteAdvertisements` CRs can associate fabric-facing FRR configuration with primary CUDNs that use EVPN transport. | ||
|
|
||
| You configure BGP neighbors, address families, and related policies in `FRRConfiguration` CRs. You then use `RouteAdvertisements` to tie those configurations to `ClusterUserDefinedNetwork` objects that are labeled or selected according to your design. | ||
|
|
||
| Plan BGP sessions to the underlay or route reflectors (typically on every node that participates) before you rely on EVPN reachability. If VTEP addresses are not taken from the machine network, ensure that those addresses, such as loopbacks, are routable on the underlay so that remote VTEPs can encapsulate to them. | ||
|
|
||
| [id="bgp-evpn-gateway-mode_{context}"] | ||
| == Gateway mode and node forwarding | ||
|
|
||
| EVPN for primary CUDNs is supported when the cluster uses local gateway mode. The `spec.defaultNetwork.ovnKubernetesConfig.gatewayConfig.routingViaHost` field must be `true` in the `Network` Operator API. Invalid combinations are rejected through resource status conditions. | ||
|
|
||
| [id="bgp-evpn-supported-platforms_{context}"] | ||
| == Supported platforms | ||
|
|
||
| BGP EVPN for primary CUDNs is intended for on-premises deployments where nodes peer with an external BGP EVPN fabric. Bare-metal clusters are the common case. | ||
|
|
||
| [id="bgp-evpn-compatible-features_{context}"] | ||
| == Compatible features | ||
|
|
||
| The following cluster networking features are compatible with EVPN on primary CUDNs: | ||
|
|
||
| * Egress Firewall | ||
| * Egress QoS | ||
| * Network policy, including AdminNetworkPolicy | ||
|
|
||
| Kubernetes `Service` objects need special consideration on EVPN: | ||
|
|
||
| Cluster IP:: Pods on the CUDN can use the cluster IP of services in their namespaces as usual. | ||
|
|
||
| NodePort, External IP, and LoadBalancer:: Support is limited compared with cluster-internal service access. MetalLB does not provide per-VRF advertisements to the same BGP peer in all configurations, so advertising external IPs or `LoadBalancer` IPs onto an EVPN fabric might require manual `FRRConfiguration` in FRR-K8s. Advertising cluster IPs into the EVPN fabric can also require explicit FRR-K8s configuration. | ||
|
|
||
| Multicast:: Multicast in a MAC-VRF (layer 2) context is supported with the expected EVPN multicast route behavior. Multicast in an IP-VRF (layer 3) context is not supported because kernel routing is not configured for that case. | ||
|
|
||
| [id="bgp-evpn-limitations_{context}"] | ||
| == Limitations | ||
|
|
||
| The following cluster networking features are incompatible with EVPN on primary CUDNs: | ||
|
|
||
| * Multiple external gateways (MEG). MEG requires shared gateway mode and EVPN is supported only in local gateway mode. | ||
| * EgressIP | ||
| * Egress Service | ||
| * IPsec encryption for OVN-Kubernetes | ||
| * No-overlay mode | ||
| * IPv6 VTEPs are not supported | ||
|
|
||
| [id="bgp-evpn-cudn-lifecycle_{context}"] | ||
| == Cluster user-defined network lifecycle | ||
|
|
||
| You enable EVPN by setting EVPN-related fields on a primary `ClusterUserDefinedNetwork` resource, for example, EVPN transport, VNI, and optional route target settings. If you omit a route target, implementations can default it from your autonomous system number and VNI. You cannot retroactively enable EVPN on an existing CUDN whose specification is already immutable. You must plan EVPN when you create the network or create a new CUDN for EVPN workloads. | ||
|
|
||
| [id="bgp-evpn-custom-resources_{context}"] | ||
| == Custom resources involved in EVPN | ||
|
|
||
| The following custom resources are commonly used in an EVPN deployment: | ||
|
|
||
| `FRRConfiguration`:: Defines BGP peers, autonomous system numbers, and EVPN-related address-family configuration toward the provider fabric. Namespaced; used with FRR-K8s. | ||
|
|
||
| `VTEP`:: Describes Virtual Tunnel Endpoint parameters (for example, a CIDR and `managed` or `unmanaged` mode). In `managed` mode, OVN-Kubernetes assigns a VTEP address from the range to each node. In `unmanaged` mode, each node uses an address on the host that falls inside the CIDR. You can configure host addressing with a `NodeNetworkConfigurationPolicy` resource from the Kubernetes NMState Operator, if you use it. The CIDR is often not the machine network. If you use a dedicated range, ensure that it is reachable on the underlay. | ||
|
|
||
| `RouteAdvertisements`:: Selects which networks to advertise (including primary CUDNs), which `FRRConfiguration` instances to use, and optional node selectors. | ||
|
|
||
| `ClusterUserDefinedNetwork`:: Defines the primary CUDN topology (layer 2 or layer 3), attaches namespaces, and carries EVPN transport settings that link the logical network to the configured VTEP and VPN identifiers. | ||
|
|
||
| Together, these objects separate fabric peering policy (FRR), what to advertise (`RouteAdvertisements`), tunnel endpoint context (`VTEP`), and tenant network definition (CUDN). | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * networking/bgp_routing/enabling-bgp-evpn-primary-cudn.adoc | ||
|
|
||
| :_mod-docs-content-type: PROCEDURE | ||
| [id="nw-bgp-evpn-routing-config_{context}"] | ||
| = Enabling Border Gateway Protocol (BGP) EVPN routing | ||
stevsmit marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| [role="_abstract"] | ||
| As a cluster administrator, you can. . . | ||
|
|
||
| .Prerequisites | ||
|
|
||
| .Procedure | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../../_attributes |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| :_mod-docs-content-type: ASSEMBLY | ||
| [id="about-bgp-evpn-user-defined-networks"] | ||
| = About BGP EVPN for primary cluster user-defined networks | ||
stevsmit marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| include::_attributes/common-attributes.adoc[] | ||
| :context: about-bgp-evpn-user-defined-networks | ||
|
|
||
| toc::[] | ||
|
|
||
| [role="_abstract"] | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Lines 1-14 are based on the following sections from the OKEP-5088 resource: EVPN/UDN background (MAC-VRF / IP-VRF / pod) Not everything is verbatim from those resources. Information was compressed (e.g., the second sentence of line 10 layer 2 mac VRF (switched/bridged behavior), layer 3 IP-VRF (routed behavior) and other information (VPN overlay and VXlAN information included in the EVPN/UDN background section). This is a dense feature @_@ |
||
| Ethernet Virtual Private Network (EVPN) is a Border Gateway Protocol (BGP) control-plane technology commonly used in data center fabrics. When you enable this capability on {product-title}, you can connect primary cluster user-defined networks (CUDNs) to an external BGP EVPN fabric so that pod and virtual machine traffic on those networks participates in the same routing and bridging semantics as the rest of your network. | ||
|
|
||
| Integrating BGP EVPN with primary CUDNs lets you align tenant segmentation with your existing BGP operations and reduce reliance on per-node static route configuration on hosts when you combine EVPN with route advertisements and local gateway mode. The feature builds on the same OVN-Kubernetes components as BGP routing and route advertisements: FRR-K8s, `RouteAdvertisements`, and `ClusterUserDefinedNetwork` objects configured for EVPN transport. | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the following true?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Obtained from enhancement limits the scope of EVPN support to local gateway mode and will specify what routes should be advertised via EVPN |
||
|
|
||
| Network administrators might use BGP EVPN with primary CUDNs to include layer 2 extension (MAC-VRF) for workloads that share a broadcast domain, layer 3 extension (IP-VRF) for routed VPN segments, and attachment to fabrics that already run BGP EVPN for redundancy and multihoming patterns such as BFD on the underlay. | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Taken from slide 16 of the Enablement deck (I won't link like that here unless given permission). |
||
|
|
||
| [IMPORTANT] | ||
| ==== | ||
| If you are using the MetalLB Operator and there are existing `FRRConfiguration` custom resources (CRs) in the `metallb-system` namespace that were created by cluster administrators or by third-party components other than the MetalLB Operator, you must ensure that they are copied to the `openshift-frr-k8s` namespace or that those components use the `openshift-frr-k8s` namespace. For more information, see xref:../../../networking/advanced_networking/bgp_routing/migrating-frr-k8s-resources.adoc#migrating-frr-k8s-resources[Migrating FRR-K8s resources]. | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This note is taken from a note that is currently in our documentation. |
||
| ==== | ||
|
|
||
| include::modules/nw-bgp-evpn-about.adoc[leveloffset=+1] | ||
|
|
||
| [role="_additional-resources"] | ||
| [id="additional-resources_about-bgp-evpn-user-defined-networks"] | ||
| == Additional resources | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ensure relevancy of included Additional resources. |
||
|
|
||
| * xref:../../../networking/advanced_networking/bgp_routing/about-bgp-routing.adoc#about-bgp-routing[About BGP routing] | ||
|
|
||
| * xref:../../../networking/advanced_networking/route_advertisements/about-route-advertisements.adoc#about-route-advertisements[About route advertisements] | ||
|
|
||
| * xref:../../../networking/multiple_networks/primary_networks/about-user-defined-networks.adoc#about-user-defined-networks[About user-defined networks] | ||
|
|
||
| * link:https://docs.frrouting.org/en/latest/bgp.html[FRRouting User Guide: BGP] | ||
|
|
||
| * link:https://docs.frrouting.org/en/latest/evpn.html[FRRouting User Guide: EVPN] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| :_mod-docs-content-type: ASSEMBLY | ||
| [id="enabling-bgp-evpn-primary-cudn"] | ||
| = Enabling BGP EVPN for primary cluster user-defined networks | ||
stevsmit marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| include::_attributes/common-attributes.adoc[] | ||
| :context: enabling-bgp-evpn-routing | ||
|
|
||
| toc::[] | ||
|
|
||
| [role="_abstract"] | ||
| As a cluster administrator, you can enable OVN-Kubernetes EVPN Border Gateway Protocol (BGP) routing support for your cluster. | ||
|
|
||
| include::modules/nw-bgp-evpn-enable.adoc[leveloffset=+1] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../../images |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../../modules |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../../snippets |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The abstract here was compiled from the following resources:
EVPN/UPN background
OVN Kubernetes Support for EVPN