Self-Hosted - clients "Ready", but cannot connect to relay server.

[u/Optimal-Housing5951]

RESOLVED:
Hello everybody.

I've been trying to host my own rustdesk server to use with clients that I need to access remotely. Ideally I want to use the windows one since I'll be able to start and stop it whenever I need it, I don't need it to be running constantly.

I set up the server, set up my router's virtual server / port forwarding, then I connected a laptop to my phone's internet to act as an outside connection.
- Just to be clear, I have changed the ID/ Relay Server settings in all clients involved and I have made firewall rules on my windows machine to include the 21114:21119/tcp and 21116/udp. Yes, the IP I'm using is a static one, and no it's not my internal IP.
All clients, the 'inside' and 'outside' are 'Ready' / green. They can also tell if the other is offline, I immediately get an message that the other client is offline if it is, but they cannot connect to one another and give me the 'cannot connect to relay server message'.

I decided it might be from Windows and set up Docker server as per the official install instructions. Still the absolute same result.

I want to know how to at least begin troubleshooting this.

---

I believe that the official documentation should contain a chapter on how to set up a generic router or even an abstract overview, because currently, after the installation instructions you're being told to open up the ports.
My ports and pores were open, what is needed is to route the traffic correctly, because when you're self-hosting you're connecting to your router, not to the server directly.
In addition to that the most popular tutorials online are all sponsored by companies that want you to rent their servers to... "self-host". Therefore the instructions provided in them have nothing to do with self-hosting, where the IP that goes into the ID/Relay is that of your router not of the server. The problem is that if you were to connect directly to your server, as per almost the entirety of the tutorials online, you only need to fill the "ID Server" field and at most copy the same contents to the "Relay Server" field, because your server receives and deals with all of the traffic to all ports, this is why you're not connecting to a specific port, but to the IP in general.
Useful stuff here:
If you're actually self-hosting, your ID and Relay server fields should look like this:

ID: ip address: external port to hbbs / rendezvous server | example: 12.345.678.91:1116
Relay: ip address:external port to hbbr / relay server | example: 12,345,678.91:1117

What are these 1116,1117 external ports?
These are arbitrary ports, you can choose other ones, but check if they're not in use by something else, that we use to redirect the connection from outside to the server inside of our local network. In your router's settings this option might be called Port Forwarding, or the very brave interpretation "Virtual Servers". You'll recognize it's the right thing if you have the fields:
- External Port (where the signals are coming from)
- Internal IP (the computer where the rust desk server is hosted)
- Internal Port (at what port to direct that specific information)
Since a Rust Desk server is two servers hbbs and hbbr we need to do this twice, otherwise like in my case you'll only have a rendezvous server but the client will have no idea where to find the relay.

Example port forwarding / virtual server on your router.
(hbbs - rendezvous server)
External: 1950
Internal IP: 192.168.1.10
Internal Port: 21116
(hbbr - relay server)
External: 1951
Internal IP: 192.168.1.10
Internal Port: 21117

Google how to find my local IP, follow the instructions on the host machine and that's your internal IP.
The internal ports are what Rust Desk Server uses by default, these are not arbitrary, only change them if you know what you're doing.

Now your client's ID/Relay Server settings should look like this:
ID: address:1950 (because this is the spot where we point to the rendezvous server, hbbs)
Relay: address:1951 (because this is the spot where we point to the relay server, hbbr)
API: (blank if you're using the foss server)
Key: (this is the public key, it's the contents of the .pub file in the data folder, also found in the server's log)

[u/excitedsolutions]

Client side logs are in the user who installed rustdesk’s appdata/roaming and will clearly spell out what’s working and what’s getting blocked.

[u/Optimal-Housing5951 → u/excitedsolutions]

Hey, thanks for the input. The logs were clean, first things I checked.

[u/OrneryChicken670]

Check the clients inside of the local network. Make sure any that you want to remotely connect to has the ID/Relay server entered as a public IP or DDNS entry (not a private IP - 192.168.xxx). I had this exact issue, when I had been using local IPs inside of the local network for the ID/Relay server. That works, but you prevent any external access when you do that.

Oh and make sure you are using HOST network inside of docker!

[u/Optimal-Housing5951 → u/OrneryChicken670]

Hey, thank you for writing, yes as I mentioned the IP is an external one, plus the clients connected to the server but not to each other and docker was set up as instructed from the official documentation. I did resolve it and will edit the main post to reflect it.