“Deadline has elapsed.”
That’s the error I kept getting when trying to connect to a family member’s computer through my own RustDesk server. The server that I set up. Running in my house. On my network.
Meanwhile, someone 100 miles away could connect to them just fine. Using my server. The one that wouldn’t let me connect.
This is the story of self-hosting RustDesk—the open-source TeamViewer alternative—and the networking nightmare that taught me more about NAT hairpinning than I ever wanted to know.
Why RustDesk?
TeamViewer and AnyDesk both want monthly subscriptions for commercial use. Even personal use has gotten increasingly restricted. AnyDesk used to be free and unlimited; now it nags you about commercial activity if you connect too often.
RustDesk is open source. You can run your own server. No subscriptions. No “detected commercial use” warnings. No mystery algorithms deciding when to throttle your connections.
The self-hosted model means you control the infrastructure. Your connection data stays on your network. And when something breaks, you can actually debug it—which is both a feature and, as I learned, a curse.
The Setup
RustDesk requires two server components:
- hbbs (ID/Rendezvous server): Handles initial client registration and peer discovery
- hbbr (Relay server): Routes traffic when direct P2P connections fail
I deployed both on an existing container host at 10.42.0.199, exposed ports 21115-21119, and set up DNS at rustdesk.myserver.com.
# Docker compose for RustDesk server
docker run -d \
--name hbbs \
-p 21115:21115 -p 21116:21116 -p 21116:21116/udp -p 21118:21118 \
rustdesk/rustdesk-server hbbs
docker run -d \
--name hbbr \
-p 21117:21117 -p 21119:21119 \
rustdesk/rustdesk-server hbbrPort 21116/UDP is critical. RustDesk uses UDP for signaling and peer-to-peer hole punching. If UDP is blocked anywhere in the path, connections silently fail—TCP might connect, but the actual session never establishes.
After setting up the server, I configured all the family’s computers with:
- ID Server:
rustdesk.myserver.com - Key: (generated during server setup)
A family member 100 miles away tested first. Connected instantly to another family member’s computer. Perfect. Ship it.
Then I tried to connect.
“Deadline has elapsed.”
The Debugging Spiral
First, I checked the obvious. RustDesk client settings—correct server address, correct key. Target computer was online, showing as “Ready” in RustDesk. I’d successfully connected to the same computer weeks earlier.
Second, I tested network connectivity:
nc -zv 10.42.0.199 21116
# Connection to 10.42.0.199 21116 port [tcp/*] succeeded!
nc -zv 10.42.0.199 21117
# Connection to 10.42.0.199 21117 port [tcp/*] succeeded!
nc -zvu 10.42.0.199 21116
# Connection to 10.42.0.199 21116 port [udp/*] succeeded!All ports open. TCP and UDP both succeeded. Ping worked. DNS resolved correctly:
nslookup rustdesk.myserver.com
# Server: 10.42.0.1
# Address: 10.42.0.1#53
# Name: rustdesk.myserver.com
# Address: 10.42.0.199My DNS server was returning the local IP. Everything looked correct.
But the connection still failed. Every device I tried—Linux desktop, Windows laptop, another Linux machine—all failed with “deadline elapsed.” Meanwhile, someone connecting from outside my network had zero problems.
NAT Hairpinning: The Enemy
After three days of intermittent debugging, the pattern finally clicked.
I’m inside my network. The RustDesk server is inside my network. But I’m connecting through a domain name that—to the rest of the internet—resolves to my public IP.
The connection path looked like this:
- My computer resolves
rustdesk.myserver.com - My DNS server (OPNsense) returns
10.42.0.199(split DNS override—good) - RustDesk connects to
10.42.0.199(local IP—good) - Server tells my client to connect to another peer
- The peer connection tries to use… my public IP
That last step is where it breaks. When RustDesk tries to establish peer-to-peer connections, it uses the public IP addresses it knows about. My router (OPNsense) has NAT reflection disabled by default—which means traffic from inside my network, addressed to my own public IP, gets dropped.
The person 100 miles away doesn’t have this problem. Their traffic goes out to the internet, hits my public IP, and routes through properly. They’re not trying to hairpin through my NAT.
The Fix: Split DNS Done Right
OPNsense has DNS overrides in Unbound (the DNS resolver). I’d already configured rustdesk.myserver.com to resolve to the local IP 10.42.0.199. That’s why my initial connection to the server worked.
But I hadn’t configured the relay server separately. And there’s a subtlety with RustDesk: the relay server configuration matters even if you think you’re not using relays.
The complete DNS override configuration:
Firewall → Services → Unbound DNS → Overrides → Host Overrides:
| Host | Domain | IP |
|---|---|---|
| rustdesk | myserver.com | 10.42.0.199 |
| rustdesk-relay | myserver.com | 10.42.0.199 |
Both entries pointing to the local IP.
Then in the RustDesk client configuration:
- ID Server:
rustdesk.myserver.com - Relay Server:
rustdesk-relay.myserver.com - Key: (your server’s key)
The relay server field was empty in my original configuration. I’d assumed relays were only needed if direct P2P failed. But RustDesk uses the relay for initial connection negotiation even when peers might be able to connect directly.
Alternative: Enable NAT Reflection
The other fix is enabling NAT reflection (hairpinning) in OPNsense:
Firewall → NAT → Port Forward → (edit your RustDesk rule):
- NAT reflection: Enable
- NAT reflection mode: Pure NAT
This tells the firewall to properly route traffic that originates from inside the network, is addressed to the public IP, and should be redirected back inside. Some people call this “loopback NAT.”
I went with split DNS instead because it’s cleaner—traffic stays local without ever touching the public interface. Less latency, less complexity in the routing path.
The Oracle Cloud Detour
Before I figured out the NAT hairpinning issue, I tried setting up a relay server on Oracle Cloud’s free tier. The theory: if the problem is my local network, maybe an external relay would bypass it.
Oracle Cloud has a generous free tier with a small VM that’s perfect for running relay servers. I deployed RustDesk there and… couldn’t connect to that either.
Test-NetConnection -ComputerName 129.146.52.177 -Port 21115
# WARNING: TCP connect to (129.146.52.177 : 21115) failedThis turned into a separate debugging session involving OPNsense outbound rules, Oracle Cloud security lists (they have their own firewall layer), and ultimately discovering that some ISPs selectively block Oracle Cloud IP ranges due to their popularity with abuse.
The Oracle detour wasn’t wasted time—I learned a lot about cloud networking—but it wasn’t the actual solution either. The problem was always local.
What I Should Have Done First
If I’d understood the symptoms from the start, the fix would have taken minutes:
- Everyone outside my network can connect: External routing works
- Nobody inside my network can connect: Internal routing is broken
- The server is inside my network: Classic NAT hairpinning scenario
The debugging tools that would have helped:
# Test if DNS resolves to local IP (split DNS working?)
nslookup rustdesk.myserver.com
# Test raw connectivity to server
nc -zvu 10.42.0.199 21116
# Try connecting with explicit local IP in client (bypasses DNS entirely)
# ID Server: 10.42.0.199That third test is the killer. If connecting with the explicit local IP works but the domain name doesn’t, you’ve got a DNS or NAT reflection issue. If even the local IP fails, the problem is elsewhere.
The Lessons
Self-hosting isn’t hard, but debugging self-hosted services is. When TeamViewer doesn’t connect, you click “retry” and hope. When your own RustDesk server doesn’t connect, you’re on your own.
NAT hairpinning will get you eventually. Any time you’re inside a network, connecting to a service that’s also inside that network, but using a public address—you’re hairpinning. Either enable NAT reflection or use split DNS to return local addresses.
Split DNS solves most internal routing problems. Configure your DNS server to return local IPs for local services when the query comes from inside the network. OPNsense, pfSense, Pi-hole, and Unbound all support this.
UDP matters for RustDesk. TCP connectivity isn’t enough. Port 21116/UDP is required for signaling. If you’re testing with nc -zv (TCP only), also test with nc -zvu (UDP).
The relay server field isn’t optional. Even if you think you don’t need relays, fill it in. RustDesk uses it for connection negotiation.
Was It Worth It?
Three days of debugging for a connection timeout error. Was self-hosting RustDesk worth the trouble?
Yes.
Now I have remote access infrastructure that I control. No subscription fees. No commercial use detection. No wondering if TeamViewer is scanning my connections or training AI on my screen shares.
When someone in the family needs help with their computer, I connect through my own server. The connection is fast—faster than TeamViewer ever was, actually, because traffic routes through my infrastructure instead of bouncing through German datacenters.
And now that I understand NAT hairpinning at a visceral level, I’ll never waste three days on this particular problem again.
That’s the real value of self-hosting. You learn things the hard way, but you actually learn them.
The Oracle Cloud relay server is still running. Turns out it’s useful for friends who aren’t on my Tailscale network and need a relay that’s not inside my house.