SonicWALL VPN Tunnel Configuration Best Practice for Remote Desktop Services
6 min read
Here at Cantarus, our multi-purpose kalaniCloud hosting is used for a variety of different hosting requirements, from websites and backup data to email and Windows Remote Desktop Services (RDS), formerly Terminal Services (TS). This article focuses on the latter, and specifically on providing such Remote Desktop Protocol (RDP) services via a site-to-site VPN tunnel using Dell SonicWALL firewalls at each end, because we experienced some issues with intermittent, recurring dropped connections and a web search showed that (a) we were far from the only ones and (b) no single website had provided a comprehensive solution.
RDP is a streaming protocol and is very sensitive to interruption in the connection. A momentary drop in connection can cause the RDS client to disconnect, freezing the screen for the end user until the RDS client automatically attempts to reconnect. This reconnection process can take anywhere between a few tens of seconds and a minute or more and is very disruptive for the end user.
The sections below describe how to achieve best RDS performance over SonicWALL site-to-site VPN tunnels and many of the settings will also apply to connections using the software SonicWALL Global VPN Client (GVPNC), particularly PMTU since this can vary between different client Internet connections.
Configuration Items to Consider
In my experience, the single biggest cause of dropped RDS connections over VPN tunnels is due to TCP timeout settings that are too low.
When creating a firewall rule in a SonicWALL firewalls, the TCP Connection Inactivity Timeout is set to 15 minutes by default. Although one might consider that an active RDS session should not be considered inactive by the SonicWALL, in practice this value can indeed cause the RDS connections to be dropped. Based on experience, I recommend this is changed to at least 120 minutes.
Changing the TCP Connection Inactivity Timeout value is straightforward; simply edit the appropriate firewall rule, navigate to the Advanced tab and change the setting there. However, it is important to do this only for firewall rules covering just RDS traffic as otherwise the timeout for all traffic is changed which can result in excessive numbers of inactive connections accumulating on the firewall and consuming resources.
A common issue with implementing the above for VPN tunnel firewall rules is that SonicWALLs, by default, automatically create the firewall rules associated with the VPN tunnels and these auto-generated rules cover all traffic types between the end-points. To address this, I recommend creating your own custom firewall rules and preventing the automatic creation of rules (which is more secure as not all services must be opened) which is achieved as follows:
- For Policy-based VPN tunnels: Edit the VPN tunnel, navigate to the Advanced tab and check the Suppress automatic Access Rules creation for VPN Policy checkbox. Note that if other traffic types are traversing the VPN tunnel, you will need to manually create rules for those, as well as the new RDS-specific rule.
- For Route-based VPN tunnels: Edit the custom route for the VPN tunnel and uncheck the Auto-add Access Rules checkbox.
Route-based VPN tunnels are my preference when working with SonicWALL firewalls at both ends of a VPN tunnel as they are more flexible in that the end-point subnets do not need to be specified (custom routes are created instead) meaning clashes between end-point subnets can be avoided.
As RDS is a streaming protocol, packet fragmentation should be avoided. SonicWALL firewalls do provide fragmented packet handling functionality and this is controlled via the VPN > Advanced page.
I recommend checking both the Enabled Fragmented Packet Handling and Ignore DF (Don't Fragment) Bit. This may seem contrary to the paragraph above which states that RDS traffic shouldn't be fragmented; the ideal approach is to ensure that packet fragmentation does not occur by using correct PMTU settings (see below) but if it does, then these settings prevent the packets being dropped which would likely cause issues with the RDS session. Further, failing to fragment packets can interfere with Path MTU Discovery traffic.
Whichever approach you choose, ensure it is identical on the SonicWALL firewalls at both ends of the VPN tunnel and be aware that it is a global setting applying to all VPN tunnels terminating at each appliance.
Path Maximum Transmission Unit (Path MTU or PMTU)
As described above, fragmentation of the RDP streaming protocol is undesirable and should be avoided. The most common cause of such fragmentation is incorrect Maximum Transmission Unit (MTU) values for the traffic's path.
The PMTU (Path Maximum Transmission Unit) is the largest packet that can traverse a given connection (path) without fragmentation. The most common MTU value for UK Internet connections is 1,500 bytes and this should be set appropriately in the WAN interface configuration on the SonicWALL firewalls.
Path Maximum Transmission Unit Discovery (PMTUD) is a technique in computer networking for determining the maximum transmission unit (MTU) size on the network path between two Internet Protocol (IP) hosts, usually with the goal of avoiding IP fragmentation. More information can be found here. Note that SonicWALL firewalls do not honour or pass to the LAN MTU Path Discovery messages as they are unauthenticated and can used as a denial of service attack.
Whilst later versions of SonicWALL firmware provide a PMTUD under the Diagnostics tab, an alternative and very simple manual method of determining the PMTU is via ping from the command line. A ping command of the format ping <target FQDN or IP> -f -l <send buffer size in bytes> can be used with the last parameter being varied until the ping response is no longer fragmented, allowing for the fact that the IP + ICMP header size is itself 28 bytes so a value of 1472 would be returned by this test on a line with an MTU of 1,500. More information can be found here.
When deploying a site-to-site VPN tunnel between two SonicWALL (or other) devices the PMTU is reduced by 56 bytes due to the cryptographic overhead associated with an IPSEC VPN tunnel. Therefore, when protocols sensitive to fragmentation - for example RDP for RDS - are traversing a VPN tunnel over Internet connections with MTUs of 1,500, it is important that the connection endpoints (e.g. the RDS server and client machine) have their MTU set to no more than 1416 bytes (1,500 - 28 - 56). This is not necessary when using Windows Point-to-Point Tunnelling Protocol (PPTP) VPN since Windows automatically adjusts the MTU to account for the cryptographic overhead.
One easy way of making this MTU change is to configure the Windows network adpator on both ends as per Option 2 in this article.
There is a good technical article for addressing the same issue on Cisco devices here.
Limitations on Internet connection bandwidth (often referred to as 'contention') is a common cause of RDS performance problems, both in terms of poor responsiveness and drop-outs.
Organisations experiencing bandwidth problems often wrongly assume that they simply need more bandwidth. However, Dell SonicWALL firewalls provide advanced bandwidth management capabilities to ensure that traffic which is sensitive to latency and connection speed is prioritised over other traffic, making much better use of existing bandwidth and avoiding potentially substantial costs associated with upgrading an Internet connection.
I recommend that RDS traffic is given the highest (real time) bandwidth management priority and that an appropriate amount of bandwidth is reserved for it. This can be done at a firewall rule level or via the SonicWALL's Application Intelligence and Control (AIC) feature for correctly-licensed appliances. It is amazing how much of a difference to RDS usability this one simple change can deliver!
Licensed Dell SonicWALL firewalls provide a comprehensive set of on-appliance security services including Gateway Anti-Virus (GAV), Anti-Spyware (AS) and Intrusion Prevention Service (IPS). These services can scan specific traffic types (e.g. SMTP, FTP, etc.) or the whole TCP stream for threats. Whilst they are very efficient in terms of scanning speed, they do introduce some additional latency (typically at least 1ms) and I have also seen them cause dropped RDS connections when applied to the whole TCP stream.
Assuming that the end-points are sufficiently trusted, it is worth considering disabling the firewall's scanning services (at least AS and IPS) to obtain best RDS performance and connection reliability. I would recommend that both the RDS server and connecting clients are secured with suitable anti-virus software using the latest definition database.
A combination of optimum TCP timeout, packet fragmentation, PMTU, bandwidth management and security services settings can ensure outstanding performance and reliability of RDS over SonicWALL site-to-site VPN tunnels. My experience tells me that providing the same user experience over GVPNC is more difficult so we exclusively use SonicWALL NetExtender or Mobile Connect for remote/mobile users.
Thanks for reading and I hope this information helps someone somewhere! If you have any additions or corrections, please let us know via the Comments section below.