DigitalJoel

2010/10/12

Windows 7 and the Mac Mini Server VPN

Filed under: troubleshooting — Tags: , — digitaljoel @ 12:54 pm

I spent a few hours in the last week and a half attempting to connect my Windows 7 computer to a VPN provided by a new Mac Mini Server.  The win 7 computer is at my house, and the mac mini is at my friend’s house.  The wrinkle here is that we both have routers that provide NAT services.  Little did I know, that would be the only real problem.

In order to configure the connection, you want to set it as an L2TP connection, enable the CHAP and MS-CHAPS authentication scheme, and in the advanced properties for the L2TP connection make sure you put in the plain text shared secret that the VPN server admin should provide to you.

Finally, in order to connect through the NAT devices, you will want to have a look at this link.

http://support.microsoft.com/kb/926179

It doesn’t list Windows 7 in the “Applies to” section, but it worked for me on Windows 7.  The gist is, you want to create this new registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PolicyAgent\AssumeUDPEncapsulationContextOnSendRule.

The path up to PolicyAgent should already exist, so you only have to create the AssumeUDPEncapsulationContextOnSendRule DWORD(32) key.  If your computer AND the VPN server are both behind a NAT device (like a router) then you will want to set the value to 2.  This is likely what you will need.  If only the server is behind a NAT device, then set the value to 1.  If you feel like you need to set the value to 0, then this blog post isn’t for you in the first place.

After changing the registry, you’ll likely have to reboot.

Hopefully this’ll save someone the several hours myself (and my buddy!) had to spend figuring this out.

Blog at WordPress.com.