Static NAT + Masquerading / Static NAT + Port Forwarding by qmonnet · Pull Request #1548 · githedgehog/dataplane

qmonnet · 2026-05-20T20:36:52Z

The first commit are some preparatory clean-ups for the static NAT code:

refactor(nat): Move static NAT network function to dedicated file
refactor(nat): Move unicast check closer to source IP mapping lookup
refactor(nat): Handle zero-port check at port mapping lookup
feat(net): Add wrappers for transport with types (TCP/UDP)
refactor(nat): Use TcpUdp view in static NAT to simplify port update
refactor(net): Simplify metadata flag toggling

Then we add some finer control for running static NAT on a packet:

chore(flow-filter): Split requires_stateless_nat() into source/dest
feat(flow-filter): Tag packets for src/dst static NAT
feat(nat): Mark packets as NAT-ed for source or destination
feat(nat): For static NAT, only apply NAT for relevant direction(s)

We proceed to the changes we need to support combinations of NAT modes:

feat(dataplane): Move static NAT stage before port forwarding
feat(flow-filter,nat): Store static NAT requirements in flow info
refactor(net): Make PacketMeta's src_vpcd private
feat(net): Embed flow key in packet metadata
feat(nat): Use initial IP addresses for forward flow table entry

We update the validation step and add tests:

feat(config): Allow static + stateful NAT, on opposite ends
test(nat): Add tests for static NAT + masquerade

Please refer to individual commit description for details.

Copilot

Pull request overview

This PR extends the dataplane NAT pipeline to support combinations like static NAT + masquerading and static NAT + port forwarding by (a) making static NAT direction-aware, (b) persisting “initial” flow identity in packet metadata for later flow creation, and (c) reordering/propagating NAT requirements consistently through flow lookup/filtering and flow state.

Changes:

Split static NAT requirements into source vs destination (FlowFilter + PacketMeta flags) and track NAT application as src/dst NATed.
Persist an initial FlowKey on PacketMeta and propagate static-NAT requirement flags into FlowInfo so later packets rehydrate requirements from the flow table.
Refactor stateless NAT into a dedicated NF module, add TCP/UDP transport wrappers, and move static NAT stage earlier in the dataplane pipeline.

Reviewed changes

Copilot reviewed 37 out of 38 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
stats/src/dpstats.rs	Updated access to `src_vpcd` via accessor after metadata encapsulation changes.
net/src/tcp_udp.rs	Added TCP/UDP protocol-agnostic wrapper types (`TcpUdp`, `TcpUdpMut`).
net/src/packet/test_utils.rs	Ensured test packet builders update embedded flow key after port changes.
net/src/packet/mod.rs	Added `Packet::update_flow_key()` and initialize `meta.flow_key` on parse.
net/src/packet/meta.rs	Split NAT flags into src/dst, privatized `src_vpcd`, added flow key + flow-flag computation helpers.
net/src/packet/display.rs	Updated metadata display for new NAT flags + `src_vpcd()` accessor.
net/src/lib.rs	Exported new `tcp_udp` module.
net/src/headers/mod.rs	Added `TryTcpUdp`/`TryTcpUdpMut` traits and delegated implementations.
net/src/flows/flow_key.rs	Updated flow-key derivation to use `PacketMeta::src_vpcd()` accessor.
net/src/flows/flow_info.rs	Introduced `FlowInfoFlags` and extended `related_pair()` to store per-flow flags.
nat/src/test.rs	Added end-to-end tests for static NAT + masquerade and static NAT + port forwarding.
nat/src/stateless/test.rs	Updated tests to use new static NAT direction flags + `set_src_vpcd`.
nat/src/stateless/setup/tables.rs	Tightened mapping lookups (unicast check; reject port 0) and updated types.
nat/src/stateless/setup/mod.rs	Adjusted imports after stateless NAT refactor.
nat/src/stateless/nf.rs	New stateless NAT NF implementation supporting directional static NAT + ICMP inner translation.
nat/src/stateless/mod.rs	Refactored stateless NAT module to re-export new NF implementation.
nat/src/stateful/test.rs	Updated tests for `src_vpcd()` accessor + `set_src_vpcd`.
nat/src/stateful/packet.rs	Mark packets as src/dst NATed when stateful NAT modifies them.
nat/src/stateful/nf.rs	Use initial vs current flow keys for session creation; store static-NAT requirement flags into flows.
nat/src/stateful/icmp_handling.rs	Updated `src_vpcd()` accessor usage.
nat/src/portfw/test.rs	Updated tests to use `set_src_vpcd`.
nat/src/portfw/portfwtable/access.rs	Added helper to rebuild port-forwarding rules directly from validated VPC table.
nat/src/portfw/packet.rs	Added debug asserts to prevent double NAT; (see comments re: metadata marking/message).
nat/src/portfw/nf.rs	Updated `src_vpcd()` usage and threaded static-NAT flow flags into `FlowInfo::related_pair`.
nat/src/portfw/icmp_handling.rs	Updated `src_vpcd()` accessor usage.
nat/src/portfw/flow_state.rs	Build port-forwarding flow keys using stored initial flow key and current packet key.
nat/src/lib.rs	Included new NAT test module.
nat/src/icmp_handler/nf.rs	Updated `src_vpcd()` accessor usage.
flow-filter/src/tests.rs	Updated tests to reference `requires_static_nat()` semantics and `set_src_vpcd`.
flow-filter/src/tables.rs	Split “requires stateless nat” into src vs dst static-NAT requirements.
flow-filter/src/lib.rs	Set directional static NAT flags on packets and rehydrate static-NAT requirements from `FlowInfoFlags`.
flow-filter/Cargo.toml	Added `bitflags` dependency for flow info flags.
flow-entry/src/flow_table/nf_lookup.rs	Updated flow creation helper calls for new `related_pair()` signature.
dataplane/src/packet_processor/mod.rs	Reordered pipeline so static NAT runs before port forwarding.
dataplane/src/packet_processor/ipforward.rs	Updated VNI annotation to use `set_src_vpcd`.
config/src/external/overlay/vpcpeering.rs	Removed unused `has_nat()` helper.
config/src/external/overlay/vpc.rs	Updated NAT-mode compatibility validation to allow static NAT alongside other modes on the opposite side.
Cargo.lock	Added transitive lockfile updates for new dependency usage.

Comments suppressed due to low confidence (1)

nat/src/portfw/packet.rs:103

confidence: 8
tags: [logic]

`dnat_packet` can modify the packet's destination IP/port, but it never sets `PacketMeta`'s `dst_natted` flag when it does so. This makes `PacketMeta::is_dst_natted()` / display output inaccurate and weakens the intended "double NAT" protections. Set `dst_natted(true)` when `modified` is true.

debug_assert!(
    !packet.meta().is_dst_natted(),
    "Trying to apply double source NAT to packet!"
);

</details>

qmonnet · 2026-05-20T23:21:32Z

Adding the flow key to the packet metadata increases the size of the metadata and hence the packet. With its 1,000 stages each adding a copy to the stack, long_dyn_pipeline() exhausts its stack. Here's the difference:

  ┌────────────────────┬────────┬────────┐
  │      size_of       │ before │ after  │
  ├────────────────────┼────────┼────────┤
  │ Packet<TestBuffer> │ 544 B  │ 624 B  │
  ├────────────────────┼────────┼────────┤
  │ PacketMeta         │ 72 B   │ 152 B  │
  ├────────────────────┼────────┼────────┤
  │ Option<FlowKey>    │ —      │ 80 B   │
  └────────────────────┴────────┴────────┘

I'll look into reducing the size of the metadata tomorrow.

For consistency with masquerading and port forwarding, move the code for the network function and for the main logic for static NAT to a dedicated nf.rs file, only leaving dependency inclusion and re-exports in mod.rs. Signed-off-by: Quentin Monnet <qmo@qmon.net>

Avoid running the transformation of the IP into a validated unicast address at the last moment; do it as soon as we return the mapping from the table for static NAT. This allows working with a UnicastIpAddr all along, ensuring we've got the right type. Signed-off-by: Quentin Monnet <qmo@qmon.net>

Move the check on ports being non-zero for PAT, and pass NonZero<u16> to follow-up methods rather than working with u16 that have a risk of being zero. This allows some minor simplification of the translation code. Signed-off-by: Quentin Monnet <qmo@qmon.net>

I wanted a view on a transport header that would use ports (meaning, TCP or UDP, only, at the moment). After various manual attempts, I just let Claude run its magic. We get a TcpUdp enum that is very similar to the IcmpAny objects. We can get ports, and set them too for the mutable variant. This should help simplify some portions of the NAT code. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Quentin Monnet <qmo@qmon.net>

Use the new TcpUdp view to simplify port translation in static NAT, and get rid of some unnecessary error variants. Signed-off-by: Quentin Monnet <qmo@qmon.net>

Instead of repeating the "if () insert else remove" for each flag, let's add a set_flag() private method to PacketMeta, to simplify setting the different flags. Signed-off-by: Quentin Monnet <qmo@qmon.net>

In preparation for finer-grained flagging for static NAT, with separation for source and destination NAT, split requires_stateless_nat() into a source- and a destination-based version. Signed-off-by: Quentin Monnet <qmo@qmon.net>

Instead of just marking packets for static NAT, extend the flags to specify individually for source and destination whether the packet should go through static NAT. This will allow to handle packets more efficiently when using static NAT in addition to masquerading or port forwarding in the future. Signed-off-by: Quentin Monnet <qmo@qmon.net>

Use a finer-grained marking for packets after NAT-ing them, to account for whether the source or destination address has been translated. This will help processing packets when supporting static NAT in addition to masquerading or port forwarding in the future. Signed-off-by: Quentin Monnet <qmo@qmon.net>

We mark the packet independently for source or destination static NAT, so we can leverage this in the static NAT code to only apply the translation to the relevant direction(s), and only if translation has not been applied for that direction already. Signed-off-by: Quentin Monnet <qmo@qmon.net>

So far, using NAT on both sides of a peering was supported only if both sides used static NAT. Combining static NAT and masquerading or port forwarding for a connection was not possible. We want to support it now, and for this, we need to have the static NAT stage coming first. Example setup ------------- Here are some detailed explanations, and that the packet goes through. Let's imagine two endpoints communicating through the gateway, and both using NAT. For this connection, endpoint 1 has IP address "a", NAT-ed to "A"; and endpoint 2 has IP address "b", exposed with NAT as "B". Here's a simple diagram: +------------+ +---------+ +------------+ | | a A | | B b | | | Endpoint 1 |------------| Gateway |------------| Endpoint 2 | | | | | | | +------------+ +---------+ +------------+ Let's look at a packet leaving endpoint 1 for endpoint 2: - Initially: src: a, dst: B - After translation: src: A, dst: b Now the reply: - Initially: src: b, dst: A - After translation: src: B, dst: a With static NAT on both sides, this doesn't present any major difficulty, we can just do source and destination NAT in the static NAT pipeline stage, the order is not really important. Port forwarding then static NAT ------------------------------- Now if we want to have masquerading or port forwarding on one end of the peering, and static NAT on the other, the situation is different. Let's look at the case where endpoint 1 uses port forwarding, and endpoint 2 uses static NAT. Let's consider the first packet. If we apply port forwarding first, then the port forwarding stage will create the following entries in the flow table: - Key: (src: a, dst: B), translation value: (src: a, dst: b) (forward) - Key: (src: b, dst: a), translation value: (src: B, dst: a) (reverse) But this is not what we want! The reply will come as (src: b, dst: A) and the packet will go through the flow table lookup stage with these IPs, before any NAT operation. The lookup will fail, given that the table entry has "a" as destination address and not "A". Static NAT then port forwarding ------------------------------- If we do static NAT first, the situation doesn't look better. The port forwarding stage creates the flows based on the packet with the translated source address, so we get: - Key: (src: A, dst: B), translation value: (src: A, dst: b) (forward) - Key: (src: b, dst: A), translation value: (src: B, dst: A) (reverse) This works for replies, but not for subsequent packets in the forward direction. However, we do want static NAT first: in a follow-up commit we will keep track of the initial source and destination IP for the packets so that the port forwarding stage can create the relevant flow for the reverse direction. We'll get (after this follow-up commit): - Key: (src: B, dst: A), translation value: (src: B, dst: a) (reverse) However, for the stage to be able to use the translated address and to create the right entry for the forward direction, it needs to know "A", in other words: it needs to process the packet _after_ static NAT has been applied. Then we'll get (after this follow-up commit): - Key: (src: a, dst: B), translation value: (src: A, dst: B) (forward) The same also applies to masquerading with static NAT (swapping source and destination), but static NAT already comes before masquerading. Changes ------- So here we move the static NAT stage before the port forwarding stage, so that in the future we can make sure to create the write flow table entries when combining NAT modes. We also add debug asserts to ensure that we never attempt to NAT an IP address twice. We also allow the masquerading stage to process packets that have already been through other NAT modes (in practice, this can only be static NAT, because we never mark packets as requiring both masquerade and port forwarding in flow-filter). Signed-off-by: Quentin Monnet <qmo@qmon.net>

We want to support static NAT in addition with masquerading (or port forwarding), on opposite ends of a peering. But we've got an issue: when a stateful flow is established, the flow-filter stage bypass mechanism derives the NAT requirements from the existing flow information, without going through the flow-filter table lookup. As a result we don't mark the packet for static NAT, and the packet does not go through the static NAT stage! As a workaround, store the static NAT requirements as new bitflag fields for the flow information, and use these fields when setting NAT requirements from the flow information to packet metadata. Signed-off-by: Quentin Monnet <qmo@qmon.net>

To handle some NAT mode combinations, we need to keep track of the initial packet's addresses before the first NAT pass, when no flow information was found for the packet. To do this, we optionally attach a flow key to the packet's metadata. We could attach the flow key unconditionally, but this would grow the size of the metadata quite significantly (80 bytes estimated). Instead, we Box<> it, and only allocate when it's relevant: when we have a combination of static NAT with masquerading or port forwarding, and when no flow table entry was found for the packet. Note that adding even the boxed flow key makes the long_dyn_pipeline() test fail with its 1,000 stages each adding a copy of the metadata to the stack, resulting in stack overflow for the thread. Interestingly, lowering down to 999 stages seems to be enough to make it pass again. Signed-off-by: Quentin Monnet <qmo@qmon.net>

So far, using NAT on both sides of a peering was supported only if both sides used static NAT. Combining static NAT and masquerading or port forwarding for a connection was not possible. We want to support it now, and for this, we need to change the way we create flow table entries. Here are some detailed explanations, and that the packet goes through. Let's imagine two endpoints communicating through the gateway, and both using NAT. For this connection, endpoint 1 has IP address "a", NAT-ed to "A"; and endpoint 2 has IP address "b", exposed with NAT as "B". Here's a simple diagram: +------------+ +---------+ +------------+ | | a A | | B b | | | Endpoint 1 |------------| Gateway |------------| Endpoint 2 | | | | | | | +------------+ +---------+ +------------+ Let's look at a packet leaving endpoint 1 for endpoint 2: - Initially: src: a, dst: B - After translation: src: A, dst: b Now the reply: - Initially: src: b, dst: A - After translation: src: B, dst: a Let's consider a setup where endpoint 1 uses static NAT, and endpoint 2 uses port forwarding. When we process the packet in the port forwarding stage, we want to create the following flow table entries: - Key: (src: a, dst: B), translation value: (src: A, dst: b) (forward) - Key: (src: b, dst: A), translation value: (src: B, dst: a) (reverse) However, because the packet has already been through the static NAT stage before reaching the port forwarding stage, we collect the translated source IP address from the packet and create the following entries: - Key: (src: A, dst: B), translation value: (src: A, dst: b) (forward) - Key: (src: b, dst: A), translation value: (src: B, dst: A) (reverse) This is fine for the return path, but subsequent packets in the forward direction will not match: the flow lookup stage comes before the static NAT stage and "a" hasn't been translated to "A" yet before the flow lookup. Flow lookup fails, we find no flow. So instead, we want to create the forward-direction entry based not on the address observed by the port forwarding stage, but on the initial source IP address that the packet came with. In order to keep track of that address, we associate a flow key for the packet's metadata. We can use this flow key to create flow table entries for the forward direction, but we keep using the addresses from the packet, after static NAT has occurred, for the reverse direction. Signed-off-by: Quentin Monnet <qmo@qmon.net>

We've stopped using this error for the case where no mapping is found for the source and destination IP addresses for the packet; we should stop using it when no mapping is found for the addresses in the inner packet for ICMP Error messages, too. We don't want to raise an error when no mapping is found, it may be that no static NAT is required for the address. At the moment this should not happen because we tag packets that should be processed with static NAT, but it may happen in the future (for ICMP Error message handling for some NAT modes combinations, when packets haven't been through the flow-filter stage yet). Signed-off-by: Quentin Monnet <qmo@qmon.net>

This is not completely ideal because we'd like to keep the possibility to rework this code in the future to enter() just once for a batch of packets, but this will help reuse the process_packet() function in the ICMP Error handler in a follow-up commit. Signed-off-by: Quentin Monnet <qmo@qmon.net>

This requires deriving the Clone trait for the factory. Passing the factory will allow us to instantiate and run a StatelessNat processor from the IcmpErrorHandler network function in a follow-up commit. But we keep it optional, because it's not relevant in the case where no static NAT pipeline stage is created. Signed-off-by: Quentin Monnet <qmo@qmon.net>

We've recently added support for static NAT in combination with masquerade or with port forwarding, on the opposite ends of a peering. But we haven't taken care of ICMP yet! In particular, the change means that ICMP Error messages may require to go through static NAT after they have been masqueraded or port-forwarded. To do so, we instantiate a StatelessNat object in the IcmpErrorHandler, mark the packet as requiring static NAT, and attempt to translate the addresses (and ports, if relevant). Whether or not the packet actually requires NAT doesn't really matter here. We don't have a good way to tell, because the packet hasn't been through the flow-filter stage yet, so we need to translate as a best-effort. However, we do mark packets as source-NAT-ed or dest-NAT-ed when we process them for masquerade or port forwarding, and the static NAT stage does not re-apply translation for the indicated address, so we there's no risk to incur double translations. Signed-off-by: Quentin Monnet <qmo@qmon.net>

Now that we support using static NAT on one side of a peering and masquerading or port forwarding on the other end, let's allow users to deploy such configurations. Also adjust relevant tests accordingly. Signed-off-by: Quentin Monnet <qmo@qmon.net>

Add unit tests to validate the use in a pipeline of static NAT + masquerade, or static NAT + port forwarding, on opposite sides of a peering. We set up a pipeline, create a packet, process it, and check for its output; then we create a fake reply, process it, check as well; at last, we re-process the original packet, to make sure that the connection still works after the flow table entry has been created. Signed-off-by: Quentin Monnet <qmo@qmon.net>

qmonnet requested review from Fredi-raspall and mvachhar May 20, 2026 20:36

qmonnet self-assigned this May 20, 2026

qmonnet requested a review from a team as a code owner May 20, 2026 20:36

Copilot AI review requested due to automatic review settings May 20, 2026 20:36

qmonnet added the area/nat Related to Network Address Translation (NAT) label May 20, 2026

Copilot started reviewing on behalf of qmonnet May 20, 2026 20:37 View session

This was linked to issues May 20, 2026

NAT: Investigate stateless + stateful NAT #998

Closed

Split stateless NAT stage into two #1273

Open

Copilot AI reviewed May 20, 2026

View reviewed changes

Comment thread nat/src/portfw/packet.rs Outdated

Comment thread nat/src/portfw/packet.rs

Comment thread nat/src/stateless/setup/tables.rs

qmonnet mentioned this pull request May 20, 2026

Split stateless NAT stage into two #1273

Open

qmonnet force-pushed the pr/qmonnet/split-static-nat branch from e71a365 to 41463b9 Compare May 20, 2026 21:09

qmonnet mentioned this pull request May 20, 2026

net/nat clean-ups #1550

Open

qmonnet force-pushed the pr/qmonnet/split-static-nat branch from 41463b9 to 1bee3c4 Compare May 20, 2026 22:25

daniel-noland mentioned this pull request May 21, 2026

thoughts on pr #1548 #1553

Draft

qmonnet force-pushed the pr/qmonnet/split-static-nat branch 2 times, most recently from 4bea4c4 to 1d0ea0e Compare May 21, 2026 17:00

qmonnet force-pushed the pr/qmonnet/split-static-nat branch from 1d0ea0e to 55f2af7 Compare May 30, 2026 02:41

qmonnet and others added 11 commits May 30, 2026 03:44

refactor(nat): Use TcpUdp view in static NAT to simplify port update

df886e8

Use the new TcpUdp view to simplify port translation in static NAT, and get rid of some unnecessary error variants. Signed-off-by: Quentin Monnet <qmo@qmon.net>

refactor(net): Simplify metadata flag toggling

ff26207

Instead of repeating the "if () insert else remove" for each flag, let's add a set_flag() private method to PacketMeta, to simplify setting the different flags. Signed-off-by: Quentin Monnet <qmo@qmon.net>

qmonnet added 9 commits May 30, 2026 03:44

qmonnet force-pushed the pr/qmonnet/split-static-nat branch from 55f2af7 to 76c39fc Compare May 30, 2026 02:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Static NAT + Masquerading / Static NAT + Port Forwarding#1548

Static NAT + Masquerading / Static NAT + Port Forwarding#1548
qmonnet wants to merge 20 commits into
mainfrom
pr/qmonnet/split-static-nat

qmonnet commented May 20, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

qmonnet commented May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

qmonnet commented May 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

qmonnet commented May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants