Making per-link DNS servers pushed from OpenVPN to systemd-resolved survive NetworkManager connection changes

Update: the problem described in this article seems to be resolved in Fedora 36 (NetworkManager 1.38.0-2.fc36). It is still present in Fedora 35 (NetworkManager 1.32.12-2.fc35).

Update: there a companion article about solving this problem for Wireguard in here.

In an the Manage Linux client DNS settings in OpenVPN and Wireguard article we described how to use the update-systemd-resolved script to update systemd-resolved per-link DNS servers automatically based on DNS servers pushed from an OpenVPN server. This is very convenient if you have to be connected to multiple remote locations each with their own DNS servers and zones.

However, it is likely that you're also using NetworkManager which complicates things somewhat. First you need to ensure that NetworkManager is actually pushing the DNS settings it receives from DHCP (e.g. your WIFI router) to systemd-resolved instead of managing /etc/resolv.conf by itself. This may be the case already, but if it is not, you can add a simple configuration fragment (e.g. /etc/NetworkManager/conf.d/10-dns-systemd-resolved.conf) to force it:

[main]
dns=systemd-resolved
systemd-resolved=false

Then restart NetworkManager to make the changes active. You can then verify that DNS servers are pushed by using the systemd-resolve command:

$ systemd-resolve --status
Global
       LLMNR setting: yes                 
MulticastDNS setting: yes                 
  DNSOverTLS setting: no                  
      DNSSEC setting: allow-downgrade     
    DNSSEC supported: no                  
  Current DNS Server: 192.168.10.1        
         DNS Servers: 192.168.10.1        
--- snip ---
          DNS Domain: example.org          

If you connect to or disconnect from a WIFI access point the DNS servers should appear or disappear, respectively. This means that NetworkManager is correctly updating global systemd-resolved DNS serttings.

Now comes the gotcha. This "push DNS settings to systemd-resolved" feature in NetworkManager has a nasty side-effect: it wipes out per-link DNS settings whenever you change or lose your connection. Here's an example before a connection interruption...

$ systemd-resolve --status|grep -A 12 tun0
Link 191 (tun0)
      Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
DefaultRoute setting: yes
       LLMNR setting: yes
MulticastDNS setting: no
  DNSOverTLS setting: no
      DNSSEC setting: allow-downgrade
    DNSSEC supported: yes
  Current DNS Server: 10.201.12.9
         DNS Servers: 10.201.12.9
                      10.201.12.10
          DNS Domain: example.org

...and after NetworkManager reconnected and then pushed DNS settings to systemd-resolved:

$ systemd-resolve --status|grep -A 8 tun0
Link 190 (tun0)
      Current Scopes: LLMNR/IPv4 LLMNR/IPv6
DefaultRoute setting: no
       LLMNR setting: yes
MulticastDNS setting: no
  DNSOverTLS setting: no
      DNSSEC setting: allow-downgrade
    DNSSEC supported: yes

The DNS servers are gone and will only come back if the OpenVPN connection is restarted. This typically only happens if your OpenVPN connection times out, for example if you suspended your laptop for a sufficiently long period.

I suppose the proper fix would be to fix NetworkManager or systemd-resolved so that per-link DNS server settings are not wiped out like this. While waiting for the fix what you can do is add a dispatcher script (e.g. /etc/NetworkManager/dispatcher.d/10-restart-openvpn-clients.sh) to NetworkManager to ensure that all running OpenVPN connections are restarted whenever NetworkManager has brought a connection up:

#!/bin/sh
# Restart running openvpn clients
PATH=/bin:/sbin:/usr/bin:/usr/sbin

IFACE="$1"
ACTION="$2"

if echo $IFACE|grep -qvE '^tun[[:digit:]]+$'; then
    if [ "$ACTION" = "up" ]; then
        pkill --signal SIGHUP openvpn
    fi
fi

The script ensures that the interface that has been brought up is not an OpenVPN tun interface. This is necessary because NetworkManager considers all interfaces, including tun interfaces, connections:

$ nmcli conn show|grep tun0
NAME  UUID                                 TYPE  DEVICE
tun0  810235c2-59f4-4ab4-b37e-bf9c098dc594 tun   tun0

This means that NetworkManager will trigger "up" action even for the tun interfaces when they are brought up. In other words, without filtering out tun interfaces OpenVPN will go into a restart loop.

An alternative is to only restart OpenVPN when a certain interface, e.g. your WIFI inteface, is brought up. Just change the interface check line above to look something like this:

if echo $IFACE|grep -qE '^wlp2s0$'; then

If you have selinux disabled also make sure that your NetworkManager configuration fragments and dispatcher scripts have the correct labels, e.g.

$ ls -Z /etc/NetworkManager/dispatcher.d/
system_u:object_r:NetworkManager_initrc_exec_t:s0 10-restart-openvpn-clients.sh
system_u:object_r:NetworkManager_initrc_exec_t:s0 no-wait.d
system_u:object_r:NetworkManager_initrc_exec_t:s0 pre-down.d
system_u:object_r:NetworkManager_initrc_exec_t:s0 pre-up.d

If there is discrepancy you have to relabel the files.

To debug any issues use a combination of these commands:

$ systemd-resolve --status
$ journalctl -f --unit=openvpn-client@myvpn
$ journalctl -f --unit=NetworkManager