wsl2_ciscoanyconnect_workaround.md

WSL 2 Cisco AnyConnect Networking Workaround

Overview

WSL 2 uses a Hyper-V Virtual Network adapter. Network connectivity works without any issue when a VPN is not in use. However when a Cisco AnyConnect VPN session is established Firewall Rules and Routes are added which breaks connectivity within the WSL 2 VM. This issue is tracked WSL/issues/4277

Below outline steps to automatically configure the Interface metric on VPN connect and update DNS settings (/etc/resolv.conf) on connect/disconnect.

Manual Configuration

Set Interface Metrics

After connecting to the VPN, you'll want to modify the Interface Metric of the Cisco VPN Adapter

PS C:\Users\gyurgyik> Get-NetAdapter | Where-Object {$_.InterfaceDescription -Match "Cisco AnyConnect"} | Set-NetIPInterface -InterfaceMetric 6000

Run the following command in Powershell with Administrative permission.

At this point you should have connectivity in your container (but without name resolution). You can test this by running ping 8.8.8.8.

Set DNS servers in Linux VM

Once connected the VPN determine the DNS servers that are configured:

PS C:\Users\gyurgyik> (Get-NetAdapter | Where-Object InterfaceDescription -like "Cisco AnyConnect*" | Get-DnsClientServerAddress).ServerAddresses
10.10.0.124
10.10.0.132

Update /etc/resolv.conf

N-20S5PF20MB4R:~$ cat /etc/resolv.conf
nameserver 10.10.0.124
nameserver 10.10.0.132

Verify Connectivity

ping google.com -c 4

Automatic Configuration

Create Scripts

Save the following scripts to %homepath%\wsl\scripts

setCiscoVpnMetric.ps1

Get-NetAdapter | Where-Object {$_.InterfaceDescription -Match "Cisco AnyConnect"} | Set-NetIPInterface -InterfaceMetric 6000

setDns.ps1

$dnsServers = (Get-NetAdapter | Where-Object InterfaceDescription -like "Cisco AnyConnect*" | Get-DnsClientServerAddress).ServerAddresses -join ','
$searchSuffix = (Get-DnsClientGlobalSetting).SuffixSearchList -join ','

function set-DnsWsl($distro) {
    if ( $dnsServers ) {    
        wsl.exe -d $distro -u root /opt/wsl_dns.py --servers $dnsServers --search $searchSuffix
    }
    else {
        wsl.exe -d $distro -u root /opt/wsl_dns.py
    }
}

set-DnsWsl fedora

wsl_dns.py

Download from: https://gist.githubusercontent.com/pyther/20bba4aee1a7e1485dd065adbf736891/raw/bb00f46d30da1b1eb7ba0fec4e6946a654c7b186/wsl_dns.py

Disable WSL Resolv Update

For each Linux instance:

1. Disable automatic updating of resolv.conf by WSL

   $ cat <<EOF > /etc/wsl.conf
   [network]
   generateResolvConf = false
   EOF
   

2. Restart/Shutdown WSL: wsl --shutdown (WARNING: this will kill all current sessions!)

Copy wsl_dns.py to Linux VMs

For each VM, run:

$ cp /mnt/c/Users/$username/wsl/scripts/wsl_dns.py /opt/wsl_dns.py
$ chmod +x /opt/wsl_dns.py

Create Scheduled Tasks

Windows Scheduled Tasks allows you to trigger an action when a certain log event comes in. The Cisco AnyConnect VPN client generates a number of log events.

We will create two tasks. The first task, will configure the interface metric when the VPN connects. The second task, will execute the dns update script inside of your Linux VM when the VPN Connects and Disconnects.

Cisco AnyConnect Events

• 2039: VPN Established and Passing Data
• 2061: Network Interface for the VPN has gone down
• 2010: VPN Termination
• 2041: The entire VPN connection has been re-established.

Procedure

1. Open Task Scheduler
2. Create a Folder called WSL (Optional, but easier to find rules later)
3. Create Rules
   1. Update AnyConnect Adapter Interface Metric for WSL2
      • General: Check: Run with highest privileges
      • Triggers:
        • On an Event, Log: Cisco AnyConnect Secure Mobility Client, Source: acvpnagent, Event ID: 2039
        • On an Event, Log: Cisco AnyConnect Secure Mobility Client, Source: acvpnagent, Event ID: 2041
      • Action: Start a program, Program: Powershell.exe, Add arguments: -WindowStyle Hidden -NonInteractive -ExecutionPolicy Bypass -File %HOMEPATH%\wsl\scripts\setCiscoVpnMetric.ps1
      • Condition: Uncheck: Start the task only if the computer is on AC power
   2. Update DNS in WSL2 Linux VMs
      • Triggers:
        • On an Event, Log: Cisco AnyConnect Secure Mobility Client, Source: acvpnagent, Event ID: 2039
        • On an Event, Log: Cisco AnyConnect Secure Mobility Client, Source: acvpnagent, Event ID: 2010
        • On an Event, Log: Cisco AnyConnect Secure Mobility Client, Source: acvpnagent, Event ID: 2061
        • On an Event, Log: Cisco AnyConnect Secure Mobility Client, Source: acvpnagent, Event ID: 2041
        • At log on: At log on of $USER
      • Action: Start a program, Program: Powershell.exe, Add arguments: -WindowStyle Hidden -NonInteractive -ExecutionPolicy Bypass -File %HOMEPATH%\wsl\scripts\setDns.ps1
      • Condition: Uncheck: Start the task only if the computer is on AC power
4. Test: Connect to the VPN, a powershell window should pop-up briefly

FAQ

Q: Does traffic orginating from the Linux VM still route through the VPN?
A: Yes, I believe so. I did not see any leaked traffic when running a tcpdump on my router.

Q: Are VPN resources accessible from the Linux VM?
A: Yes

Q: Can the Linux VM communicate with Windows?
A: No, it appears a firewall rule is preventing traffic between Windows and the Linux VM.

Q: Can I still run WSL1 instances?
A: Yes, you can run WSL1 and WSL2 insatnces simutaneously

Q: How do I revert/disable these changes?
A: Disable scheduled Tasks, remove/modify /etc/wsl.conf from each WSL Instance, Reboot

How is setting the metric for cisco vpn higher to 6000 (not preferred) make wsl work? Doesn't WSL need to pass through the VPN adapter?

@c094728 traffic is not NAT-ed by vEthernet (WSL). Hence you have private network traffic entering into your VPN, which will be blackholed.
By routing it through your router, it will get NATed, hence Internet

@arussu So are you saying that wsl traffic does not use vpn adapter? Using these settings along with proxy settings I can get to our corporate network from inside wsl and that is only accessible over vpn.

@c094728 for accessing corporate resources there are specific routes that point to the VPN tunnel. When sending a packet, a route table lookup will be performed, if there is a specific route for the destination IP that point to the VPN then that will be used.
But in case WSL, you want it to access the internet through you local router, but there are 2 default routes in the routing table, one pointing to your router and the other to the VPN.
The tiebreaker is the metric. Raising the metric for VPN routes will make them less preferred, including the default route, as such, all your traffic not destined to VPN will go through your local router.

@arussu isn't what you are describing called split tunneling. That is not allowed with our vpn configuration. When connected to the VPN, all traffic must go through the VPN. I can access both internal vpn only corporate addresses as well as outside web sites from within WSL so I don't think that is what is happening.

Stopped working today. Any ideas ? There was a windows update on my pc and it rebooted overnight.

so the resolv.conf that is generated from wsl_dns.py when the vpn is not connected did not work for me. I added a function to read the dns nameservers from the windows host and add them at the front.

I added this function:

def get_dns_client_server_addresses():
    p = subprocess.run(
        [
            PSCMD,
            "Get-DnsClientServerAddress -AddressFamily ipv4 | Select InterfaceAlias,InterfaceIndex,ServerAddresses | ConvertTo-Json -Compress",
        ],
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
    )

    if not p.stdout:
        return None

    data = json.loads(p.stdout.decode("utf-8"))
    result = []
    for item in sorted(data, key=lambda i: i["InterfaceIndex"]):
        if "Cisco" in item["InterfaceAlias"]:
            continue
        server_addresses = item["ServerAddresses"]
        for server_address in server_addresses:
            if server_address not in result:
                result.append(server_address)
    return result

And added it to main like this:

def main():
    # Assumes we only have one IP configured on the Linux eth0 interface
    eth0_network = get_iface_networks("eth0")[0]
    eth0_gateway = str(get_first_network_address(eth0_network))
    default_gateway = get_default_gateway()
    windows_dns_servers = get_dns_client_server_addresses() # <--- added
    # Sanity Checks
    if eth0_gateway != default_gateway:
        print("Warning: default gateway does not match guessed gateway of eth0")
        print("Warning: script makes assumptions that may not be true on your system!")

    # Calling powershell from Linux seems weird, so... yeah...
    vpn_dns_servers = get_cisco_vpn_dns_servers()
    dns_search = get_dns_search_list()

    # Always include the hyper-v resolver, as if the VPN disconnects, linux will try all resolvers (albit slower)
    if vpn_dns_servers:
        dns_servers = vpn_dns_servers + windows_dns_servers + [eth0_gateway]  # <-- modified
    else:
        dns_servers = windows_dns_servers + [eth0_gateway] # <-- modified

    config = generate_resolv(dns_servers, dns_search)
    print("Generated resolv.conf")
    print(config)
    write_resolv(config)

Had to use dos2unix /opt/wsl_dns.py to run the script successfully on Debian, otherwise I get the error: /usr/bin/env: ‘python3\r’: No such file or directory
Instead I got the following syntax error:

File "/opt/wsl_dns.py", line 24
    return re.search(re.compile(r'(?<=inet )(.*)(?=\ brd)', re.M), os.popen(f'ip addr show {dev}').read()).groups()

                ^
SyntaxError: invalid syntax

It works on Ubuntu and Ubuntu 18.04 following the steps, altoughdos2unix had to be used on Ubuntu (same problem as on Debian).

setDns.ps1 also throws the following warning:

Warning: default gateway does not match guessed gateway of eth0
Warning: script makes assumptions that may not be true on your system!

Any idea?

Your python version is likely too old for f strings.

…

On Tue, Nov 23, 2021, 3:29 AM Viktor Nagy ***@***.***> wrote: ***@***.**** commented on this gist. ------------------------------ Syntax error on Debian File "/opt/wsl_dns.py", line 24 return re.search(re.compile(r'(?<=inet )(.*)(?=\ brd)', re.M), os.popen(f'ip addr show {dev}').read()).groups() ^ SyntaxError: invalid syntax — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://gist.github.com/b7c03579a5ea55fe431561b502ec1ba8#gistcomment-3971501>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABY4EE62QG2BSKXOF7RFAN3UNNGFTANCNFSM4VTBWOUQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

the windows script is not functioning anymore:

setDns.ps1:1 char:95
+ ... iption -like "Cisco AnyConnect*" | Get-DnsClientServerAddress).Server ...
+                                        ~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (13:UInt32) [Get-DnsClientServerAddress], CimJobException
    + FullyQualifiedErrorId : CmdletizationQuery_NotFound_InterfaceIndex,Get-DnsClientServerAddress

You should connect to the vpn(cisco any connect), then the function will be work

works great but i added an additional trigger with Event ID 2040. That should re-run the script if AnyConnect decides to reconnect the VPN Tunnel. In my case this caused the Interface Metrics to be wrong again

Thanks for this @pyther

It has inspired me to create a slightly easier to use / more reliable version of the wsl_dns.py utility: https://github.com/jacob-pro/wsl2-dns-agent

One of the main things I found was this bug: microsoft/WSL#6977
It causes /etc/resolv.conf to get overwritten when a WSL2 distribution shuts down, this is especially a problem in Windows 11, because they get shutdown automatically due to vmIdleTimeout. I was able to work around it by setting (and unsetting) the read only flag on the file.

Another problem I noticed is that you have to be very careful with modifying the metric on the anyconnect adapter, because it will then have a lower internet metric than your Ethernet/WiFi - for me this caused DNS in Windows to take up to 10s as it tried those DNS servers first - the solution was to increase those metrics to then be higher than the anyconnect adapter!

I think it's really awesome you posted this to help people!

Thanks @pyther. Want to add something in case someone needs it. Have been using this for a while, but recently stopped working. I figured out that in my case the name of the application and source should be changed to "Cisco Secure Client - AnyConnect VPN" and csc_vpnagent. Not sure if it's general since some cisco update. These can be double-checked using Event Viewer.

Thanks @PrinceS17,
This was exactly my case. It is clearly connected with the recent Cisco VPN client update

I have disabled ipv6 on the adapters to get around this issue.

I have disabled ipv6 on the adapters to get around this issue.

Are you saying disabling IPv6 on your interfaces complete fixed the entire AnyConnect WSL2 issue or something more specific?

Yep, fixed everything.

This worked for me. Thank you so much