RDP is one of those terms everyone in hosting uses constantly without necessarily knowing what it actually does under the hood. This covers what RDP actually is, the port and encryption it runs on, what happens in the seconds between clicking “Connect” and seeing a desktop, and the specific, official steps for fixing a connection that won’t establish.
What RDP Actually Is
Remote Desktop Protocol is “a proprietary protocol developed by Microsoft which provides a user with a graphical interface to connect to another computer over a network connection” [1] — built on the ITU-T T.128 application-sharing protocol Microsoft acquired from Data Connection Limited [1]. It’s a transport, not a platform: RDP itself doesn’t manage users, licensing, or multi-session behavior — that’s Remote Desktop Services sitting on top of it. RDP just carries the screen, keyboard, and mouse traffic between client and host.
The Default Port, and What Runs On It
By default, an RDP server listens on both TCP port 3389 and UDP port 3389 [1]. That’s also exactly why it’s worth not leaving exposed to the open internet unmodified — it’s one of the most consistently scanned ports on the internet, precisely because it’s so predictable.
Encryption: Three Different Layers Over the Protocol’s History
RDP’s security has evolved across three distinct approaches, and knowing which one is active matters:
- RDP Security Layer (legacy): built-in encryption using 56-bit or 128-bit RC4, available since early versions [1][2]. Weak by modern standards — don’t rely on this alone.
- TLS (RDP 5.2+): modern transport-layer encryption, with current implementations supporting TLS 1.0 through 1.3 [1]. This is what a properly configured server should be using for session encryption.
- NLA / CredSSP (RDP 6.0+): authenticates the user before the full session is established, rather than after [2] — this is the setting that matters most for security and should be on by default on any server you’re using.
RDP’s security history isn’t spotless — CredSSP itself had a serious remote-code-execution flaw (CVE-2018-0886) [2], and the wormable BlueKeep vulnerability (CVE-2019-0708) affected unpatched systems broadly enough that Microsoft patched even end-of-life Windows XP to contain it [2]. Neither is a reason to avoid RDP — both are a reason to keep it patched and to make sure NLA and TLS are actually active, not just available.
How a Connection Actually Establishes
At a high level: the client sends a connection request to the server’s listening port, the server responds and the two negotiate which security protocol to use (RDP Security Layer, TLS, or CredSSP/NLA), authentication happens according to whichever was negotiated, and only after that succeeds does the server begin streaming the actual desktop session [1]. With NLA active, that authentication step happens before any session resources are allocated at all — which is also why NLA meaningfully reduces the attack surface exposed to an unauthenticated connection attempt.
The Most Common Connection Problems (and the Real Fix)
Microsoft’s own general troubleshooting guide walks through these in order of likelihood, and it’s worth following that order rather than guessing [3]:
| Symptom | Check | Fix |
|---|---|---|
| Can’t enable RDP, or it re-disables itself | Registry key fDenyTSConnections at HKLM\SYSTEM\CurrentControlSet\Control\Terminal Server [3] | Set to 0; if it reverts, a Group Policy is overriding it — check with gpresult /H [3] |
| Connection refused entirely | Are the TermService and UmRdpService services running? [3] | Start both via Services or PowerShell |
| Listener not responding | Run qwinsta — look for rdp-tcp with status Listen [3] | If missing, the listener config may need re-importing from a working machine [3] |
| Right port, still nothing | Run netstat -ano | find "3389" — is another process holding the port? [3] | Reconfigure the conflicting app, or (last resort) move RDP to a different port [3] |
| Everything local looks fine | Test reachability from outside with psping <IP>:3389 [3] | If it fails, a network or Windows Firewall rule is blocking the port [3] |
What’s Actually Going Wrong?
What are you seeing?
Pick the closest match.
Frequently Asked Questions
What port does RDP use?
TCP and UDP port 3389 by default [1]. It can be changed, but Microsoft’s own troubleshooting documentation doesn’t recommend it for standard deployments [3].
Is RDP encrypted by default?
Yes, but the strength varies by configuration — legacy RC4-based encryption is weak, while a properly configured modern server uses TLS 1.2/1.3 alongside NLA [1][2]. Don’t assume “encrypted” automatically means “strongly encrypted” without checking which layer is actually active.
What’s the difference between the RDP Security Layer and NLA?
The RDP Security Layer just encrypts traffic. NLA (via CredSSP) requires authentication to succeed before a full session is even allocated, which is a meaningfully stronger security posture [2].
My RDP connection just times out with no error — where do I start?
Follow Microsoft’s diagnostic order: confirm the service is enabled (fDenyTSConnections), confirm the RDP services are running, confirm the listener is active (qwinsta), confirm nothing else is on port 3389 (netstat), then test reachability from outside (psping) [3].
Conclusion
RDP is the transport, not the whole system — it runs on port 3389, has gone through three generations of security (legacy RC4, TLS, and NLA/CredSSP authentication), and establishes a session through a negotiate-then-authenticate sequence before any desktop actually streams. When it breaks, it’s almost always one of five specific things: the service is disabled, the services aren’t running, the listener isn’t active, something else is on the port, or a firewall is blocking it — check them in that order instead of guessing.
References
- Wikipedia — “Remote Desktop Protocol” — en.wikipedia.org
- Wikipedia — “Remote Desktop Protocol” (Security section) — en.wikipedia.org
- Microsoft Learn — “General Remote Desktop connection troubleshooting” (official) — learn.microsoft.com