CreatePermission 403 Forbidden

[lherman-cs]

This project is awesome! I had a nice time reading the codebase and particularly enjoyed the separation of turn vs `turn-server.

Describe the bug
I was able to receive the reflexive candidate from the TURN service. However, when my client tried to create a binding or CreatePermission request, The turn server responded with 403 Forbidden.

To Reproduce
My TURN setup currently is the following:

1. Compile from the latest main: f113b2b
2. Configure so the bind port listens to 0.0.0.0:3478 and the external IP is set to <external>:3478. The transport is set to UDP.
3. Static auth config is demo=demo.
4. The server ran without any containerization, just a barebone binary. The instance also has direct access to an external IP.
5. Use either https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/ or https://github.com/pion/turn/tree/master/examples/turn-client/udp to run a ping test.

Expected behavior
TURN server to allow the allocation and let the media flow.

Additional context
From tracing the codebase, these lines caused the issue, https://github.com/mycrl/turn-rs/blob/main/turn/src/processor/create_permission.rs#L93-L99.

I didn't see anything in RFC that defines this behavior as a MUST. So, I'm expecting that there are some specific reasons for this implementation.

Regarding ip_is_local:

ctx contains my server's external IPs, whereas peer contains the reflexive IP. But, with !ip_is_local (as the name implies), the TURN server will always reject external IPs.

Then, I tried commenting out ip_is_local check:

I found that the bind_port look ups for the peer's port whereas the allocation used the server's port instead, https://github.com/mycrl/turn-rs/blob/main/turn/src/processor/allocate.rs#L88-L91.

Thus, bind_port also couldn't find anything and ended with another 403 Forbidden.

[mycrl]

Thank you, I am happy with how my program has helped you.

First of all, I'm not complaining that your proposed problem doesn't exist or is boring, and I hope my tone doesn't bother you. This project has been pretty much tried and tested for standard stun and turn session processes, including having been used in many business systems for a long time, so there shouldn't be too many problems with basic business turn-rs.
Next, I'll walk you through troubleshooting the problem step-by-step.

First of all, the ip_is_local part that you brought up in your question, the meaning of this check is that if an illegitimate turn request is received, such as requesting an address that doesn't belong to the current turn server, it should be rejected, which really doesn't belong to the RFC, but this is a defensive process, and turn servers are supposed to be rejecting illegitimate requests, since It is not possible to create permissions or bind peers for requests that do not belong to the current turn server.

First, do you have your external address set to an external address, but your connectivity test is using a local address such as 127.0.0.1, which would cause turn-rs to reject your request due to an external mismatch. Because in this case, the turn client will most likely also set the external address to 127.0.0.1 when constructing the turn request, so ip_is_local won't get through here, I'm not sure if other implementations of turn servers allow for this, but it seems to be reasonable for me to do it this way. I'm guessing that's most likely the case with your problem.

Secondly, you can try to run turn-rs in a local environment with both bind and external configured to 127.0.0.1 and test if it passes, if it passes, then the problem is the situation I have already mentioned above.

For example, when your external address is set to 192.168.1.100, then the turn client should need to use turn:192.168.1.100:3478 when communicating with the turn server.

[lherman-cs]

Thanks for the response. Knowing the project has been tried and tested in production is reassuring. I've probably made a mistake with my configuration or environment.

The reject logic reasoning makes sense to me. Yes, I set the external address to the public IP. This is the result from running both client and servers locally with externalIP set to 127.0.0.1. The request still fails on Forbidden 403.

[turn]
realm = "turn.lukas-coding.us"

[[turn.interfaces]]
transport = "udp"
bind = "127.0.0.1:3478"
external = "127.0.0.1:3478" # with external exposure, I replaced "127.0.0.1" with my "<externalIP>"

[log]
level = "info"

[api]
bind = "127.0.0.1:3001"

[auth]
[auth.static_credentials]
demo = "demo"

The TURN ping test consists of the following:

1. TURN allocates for the client to learn about its relay address:port
2. Send a binding request to get the client's externalIP:port.
3. Use the allocated relay address:port (from step 1) to send a ping to the client's externalIP:port (from step 2).

On a side note, running https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/ results in no candidates. But, I see allocations happening on the server side without errors.

I have also replicated a similar setup with Coturn. Both client tests against coturn resulted in candidates.

What tools would you recommend in testing the turn server?

[mycrl]

I tested it locally and didn't check that there was a problem.

I tested using trickle ice as well, with the turn server running locally and the config file using the example config file from the project, and didn't change anything else except uncommenting user1=test.

This is my test result, which has successfully assigned the address and created the permissions

[mycrl]

I'll test it again with https://github.com/firefart/stunner, and let me know if you have more info on it as well, because I recently posted a new version that was not in the production environment with a few changes, which theoretically shouldn't be a problem, but maybe I changed something to cause a problem with the new version .

[mycrl]

@lherman-cs I simply wrote a webrtc test page to force the use of turn forwards to create rtc, I tested it and found no problems.
Can you provide your wireshark dump file if it's convenient? If you don't want your net information to be compromised, you can send it to my private email: [email protected]

[lherman-cs]

Interesting. I found that the Webrtc trickle ice test page works with Chrome, not Firefox. I'll check stunner and get the wireshark dump too.

Chrome

Firefox

[mycrl]

Thanks, it's ok, I'm ok with go language, the go language is fairly simple...

[mycrl]

@lherman-cs I released a beta version of 3.2 for you to try.

[lherman-cs]

The Forbidden error still exists. ip_is_local (https://github.com/mycrl/turn-rs/pull/106/files#diff-244d1430cca1ece39f98009197f22cc02164be5ac028b963dbe58038f1e48b61R93) still rejected. Since external_ip must be set to the turn server's IP, the peer's IP doesn't match.

Even after removing the check, the turn test passed the CreatePermission request but no data came through. Unfortunately, I don't have much bandwidth to dig deeper into this issue.

This problem is not as important as the Firefox issue for me. So, please feel free to put this issue as a backlog.

[mycrl]

external must be the external ip of the turn server this is required, even in the RFC this is explicitly checked.

In the RFC, the port of the peer address can be ignored, but the ip part must be an address that the turn server can be assigned.

As a typical example, if in the configuration file, your external is 192.168.1.100, which is the external ip address of the turn server, and you make a turn request directly from this machine, such as turn:127.0.0.1:3478, then this case will be rejected for sure. Because of the mismatch, normally turn servers are not allowed to be used in this way. If you use turn:192.168.1.100:3478 to initiate the request there is no problem.

I tested the beta version using both pion and webrtc-rs examples and found no problems.
And I also tested real webrtc scenarios using chrome and firefox, creating 4 rtc peer connectin to each other at the same time for half an hour to an hour of real world testing, and no problems.

I think your problem may be due to your complex network topology or problems with the turn client implementation.

You can provide more information if you are interested.

[mycrl]

Also, are you using v3.2.0-beta or v3.2.0-beta.1 for testing? Theoretically, these two versions don't do much of anything to limit permission creation requests anymore, they normally create permissions successfully and don't make any state changes to the permission creation requests, so if you can't successfully do the subsequent steps even after commenting out the ip check, that's not the problem.

I'll keep that principle, I'm happy to help you out, and if you can provide a wireshark dump all the better.

ps: https://github.com/mycrl/turn-rs/blob/main/turn/src/operation/create_permission.rs#L78 Now this part doesn't actually change any state, so the fact that you can't continue the process after commenting it out has nothing to do with this part really. It has nothing to do with this section.

[mycrl]

After such a long time, I bring rather depressing news.

Having recently gone through the details of the RFC and the coturn implementation from scratch and re-implemented the turn-rs session handling, I've found that coturn is more forgiving and implements the details in the RFC perfectly.

I tried to get turn-rs to implement behaviour consistent with coturn, especially when creating permissions and channel bindings, but I found it very difficult to implement.

For example, coturn correctly handles the fact that the xor peer address is an address known to a peer, not an address known to the turn server, such as the typical IP of an intranet device, but I've been working for a long time on how to create associations where the two parties aren't related, because for turn-rs I don't know the association between the two parties! It's a challenge to forward the message correctly.

I didn't really look at the coturn implementation, as there are advantages to only allowing both parties to communicate using the address assigned by the turn server, which allows both parties to hide their identities and actual addresses.

However, turn-rs currently works in the vast majority of cases, at least the common clients work as expected, and basically the common implementations work as expected in non-intranet environments.

This is undeniably an issue, but for now it can be tentatively classified as some limitation of turn-rs rather than a bug.