Skip to content

feat: optional reverse TCP tunnel for WDA in NAT-restricted environments#1128

Open
dankefox wants to merge 4 commits intoappium:masterfrom
dankefox:feat/reverse-tcp-tunnel
Open

feat: optional reverse TCP tunnel for WDA in NAT-restricted environments#1128
dankefox wants to merge 4 commits intoappium:masterfrom
dankefox:feat/reverse-tcp-tunnel

Conversation

@dankefox
Copy link
Copy Markdown

@dankefox dankefox commented Apr 25, 2026

Summary

Add an optional reverse TCP tunnel mode that allows WDA to actively connect outbound to an external relay server. This enables remote control of iOS devices in network environments where inbound connections to port 8100 are not feasible.

Problem

WDA defaults to listening on port 8100 for inbound HTTP connections. However, in many real-world environments, inbound connections to the iOS device are blocked or unreachable:

  • Symmetric NAT — common in corporate and campus networks
  • Multi-layer firewalls — enterprise security policies blocking inbound traffic to mobile devices
  • VPN tunnels — overlay networks where iOS restricts inbound TCP on the tunnel interface
  • Remote test labs — devices behind NAT gateways with no port forwarding

In these scenarios, the standard http://<device-ip>:8100 approach simply does not work.

Solution

Instead of requiring the client to connect in to WDA, this PR lets WDA connect out to a relay server. The relay bridges HTTP clients on one side and the WDA reverse connection on the other.

How it works

  1. Set WDA_RELAY_HOST and optionally WDA_RELAY_PORT (default: 8201) as environment variables when launching WDA
  2. On startup, WDA opens an outbound TCP connection to the relay
  3. The relay forwards HTTP requests through this connection and returns responses
  4. If the connection drops, WDA automatically reconnects after 5 seconds

When not configured

Zero impact. If WDA_RELAY_HOST is not set, the feature is completely inactive. No code paths are touched, no connections are made, existing behavior is identical.

Changes

File Change
FBConfiguration.h/m Add relayHost and relayPort accessors reading from WDA_RELAY_HOST / WDA_RELAY_PORT env vars
FBWebServer.m Add reverse tunnel client using Apple's Network.framework (nw_connection), with auto-reconnect on failure
Scripts/wda-relay-server.js Example Node.js relay server (~100 lines) that bridges HTTP ↔ reverse tunnel

Usage

# 1. Start the relay server on a reachable host
node Scripts/wda-relay-server.js  # listens on 8201 (relay) and 8100 (HTTP proxy)

# 2. Launch WDA with relay configured
WDA_RELAY_HOST=192.168.1.100 WDA_RELAY_PORT=8201 xcodebuild test-without-building \
  -project WebDriverAgent.xcodeproj \
  -scheme WebDriverAgentRunner \
  -destination 'id=<device-udid>'

# 3. Use WDA as usual — requests go through the relay
curl http://localhost:8100/status

Design decisions

  • Opt-in via environment variables — consistent with existing USE_PORT, USE_IP, and MJPEG_SERVER_PORT patterns in FBConfiguration
  • Network.framework — uses Apple's modern networking API (nw_connection) for reliable connection management
  • 4-byte length-prefixed framing — simple binary protocol for multiplexing HTTP over a single TCP connection
  • Auto-reconnect — 5-second backoff on connection failure, no manual intervention needed
  • Relay server as example script — included in Scripts/ for easy adoption, not a required component

Testing

Tested on:

  • iPhone 16 Pro Max, iOS 26.3.1
  • Xcode 26.4
  • macOS Tahoe 26.4
  • Relay server running on a separate host behind NAT

Verified: /status, /session, tap, swipe, screenshot, and other WDA endpoints all work correctly through the reverse tunnel.

@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla Bot commented Apr 25, 2026

CLA Signed

The committers listed above are authorized under a signed CLA.

Comment thread docs/wda-relay-server.mjs
Comment thread WebDriverAgentLib/Routing/FBWebServer.m Outdated

#pragma mark - Reverse TCP Tunnel

- (void)startReverseTunnel
Copy link
Copy Markdown

@mykola-mokhnach mykola-mokhnach Apr 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this functionality must be extracted to a separate module and covered by integration tests

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I'll extract the reverse tunnel into its own module (e.g., FBReverseTunnel.h/m) so it's cleanly separated from FBWebServer. FBWebServer would just call [FBReverseTunnel startWithHost:port:] if the env vars are set.

Will also add integration test coverage for the module.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't observe any tests, neither in WDA objc sources nor in nodejs sources

Comment thread WebDriverAgentLib/Routing/FBWebServer.m Outdated
Comment thread WebDriverAgentLib/Routing/FBWebServer.m Outdated
@mykola-mokhnach
Copy link
Copy Markdown

I'm not sure why to reinvent a custom protocol if the project already uses CocoaHTTPServer

@dankefox
Copy link
Copy Markdown
Author

Thank you for the review @mykola-mokhnach!

Regarding CocoaHTTPServer — it serves as an inbound HTTP server (listening on a port for incoming connections). The core problem this PR addresses is precisely that inbound connections to the iOS device are blocked in NAT-restricted environments.

The reverse tunnel requires WDA to initiate an outbound TCP connection to an external relay. During development, I tested three approaches for the outbound connection:

  1. NSStream — failed to establish reliable connections over VPN/tunnel interfaces
  2. POSIX sockets — also unreliable on iOS for outbound TCP over non-standard network paths
  3. nw_connection (Network.framework) — worked reliably, and is Apple's recommended API for modern network connections

The 4-byte length-prefixed framing is intentionally minimal — just enough to multiplex HTTP request/response pairs over a single persistent TCP connection. I considered using HTTP itself as the transport (e.g., long-polling or WebSocket), but a raw TCP connection with simple framing has significantly lower overhead and latency for this use case.

That said, I'm absolutely open to alternative approaches if you have a preferred pattern in mind. Would it make sense to, for example, use CocoaHTTPServer as an HTTP client (reverse proxy) that forwards to a relay via HTTP? I'd be happy to explore that if it fits better with the project's architecture.

@dankefox dankefox force-pushed the feat/reverse-tcp-tunnel branch from 7426e66 to 7b77d6d Compare April 26, 2026 05:16
@dankefox
Copy link
Copy Markdown
Author

Updated the PR based on all review feedback:

  • Extracted to separate module: All reverse tunnel logic is now in FBReverseTunnel.h/m. FBWebServer.m has a single one-line call: [FBReverseTunnel startIfConfiguredWithLocalPort:]
  • Split complex methods: FBReverseTunnel.m is organized into focused methods — parseHTTPRequest:, buildLocalRequestWithMethod:path:body:, buildFramedResponse:statusCode:, sendData:throughConnection:, etc.
  • Named constants: Extracted FBReverseTunnelMaxPayloadSize (10 MB) and FBReverseTunnelReconnectDelay (5s) with documenting comments
  • Relay script: Kept in Scripts/ as it is a required counterpart (users need a relay server on the other end). Happy to move to a different directory if preferred.

Regarding the CLA — I'll get that signed. The commit email needs to be linked to my GitHub account.

Ready for another round of review!

Add opt-in reverse TCP tunnel mode that allows WDA to actively connect
outbound to an external relay server, enabling remote control in
environments where inbound connections to the iOS device are not feasible
(symmetric NAT, multi-layer firewalls, corporate VPNs, etc.).

Controlled via environment variables (disabled by default):
- WDA_RELAY_HOST: relay server address
- WDA_RELAY_PORT: relay server port (default 8201)

When not configured, WDA behavior is completely unchanged.

Changes based on review feedback:
- Extracted reverse tunnel into dedicated FBReverseTunnel module
- Split complex methods into focused, single-responsibility functions
- Extracted magic numbers into named constants
- FBWebServer only has a single-line call to FBReverseTunnel

Implementation notes:
- Uses Network.framework (nw_connection) for outbound TCP — tested and
  verified after NSStream and POSIX sockets both proved unreliable for
  outbound connections over VPN/tunnel interfaces on iOS
- 4-byte big-endian length-prefixed framing for minimal overhead
- Auto-reconnect with configurable delay on connection failure

Includes:
- FBConfiguration: relay host/port accessors from env vars
- FBReverseTunnel: standalone reverse tunnel module
- Scripts/wda-relay-server.js: reference relay server implementation
  (required counterpart — users need a relay to connect to)
@dankefox dankefox force-pushed the feat/reverse-tcp-tunnel branch from 7b77d6d to 24f73f1 Compare April 26, 2026 05:17
@KazuCocoa
Copy link
Copy Markdown
Member

KazuCocoa commented Apr 26, 2026

So, this is when a client wants to connect to the WDA over ip:port (which is not though local proxy forward appium xcuitest driver generally does,) correct? For example, this PR + appium:webDriverAgentUrl (to the relay), configuration? - just to understand the usage properly

Comment thread docs/wda-relay-server.mjs
Comment thread WebDriverAgentLib/Utilities/FBConfiguration.m Outdated
@dankefox
Copy link
Copy Markdown
Author

Yes, exactly! The primary use case is:

  1. WDA runs on an iOS device behind NAT/firewall (cellular network, corporate firewall, etc.) where inbound TCP is blocked
  2. A relay server runs on a publicly accessible host
  3. WDA initiates an outbound connection to the relay and establishes a reverse tunnel
  4. The Appium client connects to the relay using appium:webDriverAgentUrl pointed at the relay address

So the flow is: Appium Client → Relay Server ← WDA (reverse tunnel)

This is particularly useful for remote device farms, CI/CD pipelines with devices on cellular networks, or any environment where direct port access to the iOS device isn't possible.

Comment thread Scripts/wda-relay-server.js Outdated
Comment thread docs/wda-relay-server.mjs
let requestCounter = 0;

// --- Relay server: accepts reverse connection from WDA ---
const relayServer = net.createServer((socket) => {
Copy link
Copy Markdown

@mykola-mokhnach mykola-mokhnach Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider restructuring this example, so corresponding code pieces are organised into classes. I am not a big fan of global module variables (especially mutable ones) unless it could be proven they are required

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point. For the relay server (which lives in docs/ as a reference implementation), I've kept it straightforward since users will likely adapt it to their own infrastructure. The relay is intentionally minimal — the important complexity lives on the WDA side in FBReverseTunnel.

Happy to restructure if you feel strongly about it.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not about minimalism, but rather about designing it properly, so entities belonging to different domains are not mixed together. Also, properly designed architecture makes it easier to adapt/modify it for different purposes as it is easier to parse and understand.
I can observe you are anyway extensively using AI, so it must not be a complicated task to supply it with an appropriate prompt.

… port constant

- Move wda-relay-server.js from Scripts/ to docs/ as reference implementation
- Convert to ESM format (.mjs) with node: protocol imports
- Replace Buffer.slice() with Buffer.subarray() (non-deprecated)
- Extract DefaultRelayPort constant in FBConfiguration.m

Addresses review feedback from @KazuCocoa and @mykola-mokhnach
Comment thread WebDriverAgentLib/Routing/FBReverseTunnel.h Outdated
Comment thread WebDriverAgentLib/Routing/FBReverseTunnel.m Outdated
Comment thread WebDriverAgentLib/Routing/FBReverseTunnel.m Outdated
Comment thread WebDriverAgentLib/Routing/FBReverseTunnel.m Outdated
Comment thread WebDriverAgentLib/Routing/FBReverseTunnel.m Outdated
return result;
}

#pragma mark - HTTP Request Parsing
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use existing HTTP vendor libs for this purpose?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tunnel relays raw HTTP bytes — the payload from the relay is a complete HTTP request (method line + headers + body) that the local WDA server parses directly.

Using POSIX socket lets us forward these raw bytes to 127.0.0.1:port without parsing/reconstructing the HTTP message. An HTTP client library (like NSURLSession) would require parsing the raw request into a structured NSURLRequest first, which adds complexity and potential for subtle differences in header handling.

The raw forwarding approach is simpler, produces byte-identical requests to the local server, and has been tested extensively in production.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it does not make sense to replace everything, but for example the below stuff

  const char *err = "HTTP/1.1 502 Bad Gateway\r\n\r\nLocal WDA unreachable";
  [response appendBytes:err length:strlen(err)];

is confusing and unoptimal. I would rather supply a proper http message there, with content length, content type, etc. headers

- Use 8-byte header protocol (4-byte length + 4-byte request ID) for
  reliable request-response correlation, matching tested implementation
- Use POSIX socket forwarding to localhost for raw HTTP relay
- Accept host/port as explicit parameters instead of reading FBConfiguration
  internally (addresses @mykola-mokhnach feedback on module coupling)
- Increase max payload size from 10 MB to 1 GB to match Appium HTTP server
  limit, supporting large screenshots and video captures
- Add SIGTERM/SIGHUP signal handling in FBWebServer to survive IDE
  disconnection (enables wireless-only operation)
- Add network path monitor to automatically restart HTTP server on
  network interface changes (WiFi ↔ cellular transitions)
- Update relay server to match 8-byte header protocol with reqId-based
  response routing

Map is the correct data structure for pendingRequests because responses
are now routed by request ID rather than FIFO order.
- Add exponential backoff for reconnection (5s → 10s → ... → 60s cap),
  resets on successful connection (circuit breaker)
- Extract all magic numbers to named constants:
  FBReverseTunnelHeaderSize, FBReverseTunnelRecvBufferSize,
  FBReverseTunnelInitialReconnectDelay, FBReverseTunnelMaxReconnectDelay
- Add docs/reverse-tunnel.md with feature overview, architecture diagram,
  configuration guide, protocol specification, and resilience details
- Use POSIX socket for HTTP forwarding: the tunnel relays raw HTTP bytes
  directly to the local WDA server, avoiding unnecessary parsing/reconstruction
  that an HTTP client library would require
Comment thread docs/reverse-tunnel.md
@@ -0,0 +1,106 @@
# Reverse TCP Tunnel for NAT-Restricted Environments
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you leave one security notice for the user? So, "you'll need to be responsible for the network security by yourself"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, NW_PARAMETERS_DISABLE_PROTOCOL is provided in the FBReverseTunnel.m, which will work for the plain TCP replay server only (not a full TLS handshake). Architecture section probably can address this behavior as well

@KazuCocoa
Copy link
Copy Markdown
Member

These might have issues in CI:

  • npm run lint
  • unit test in WDA project

@param port The relay server port
@param localPort The local WDA HTTP server port to forward requests to
*/
+ (void)startWithHost:(NSString *)host
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDA also provides shutdown endpoint. Consider stopping this server as well when the main web server is stopped

/** Maximum reconnect delay (exponential backoff cap) */
static const uint64_t FBReverseTunnelMaxReconnectDelay = 60; // seconds

static NSString *_relayHost;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider making these to class properties. Avoid using static as it makes the entity less flexible


- (void)startServing
{
// Ignore SIGTERM/SIGHUP to survive IDE disconnection (enables wireless operation)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a breaking change, please revert

[self initScreenshotsBroadcaster];

// Start reverse tunnel if configured
NSString *relayHost = FBConfiguration.relayHost;
Copy link
Copy Markdown

@mykola-mokhnach mykola-mokhnach Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please extract this part into a separate private method

localPort:FBConfiguration.bindingPortRange.location];
}

// Network change monitor - restart HTTP server on interface changes
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is also a breaking change, please revert

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants