Disable WAN interfaces when CARP is down

10-wancarp

#!/usr/local/bin/php

<?php

/*
    This script can be used with OPNsense when using CARP in certain circumstances where CARP is desired on the LAN side
    but where CARP cannot run on the WAN side. This script runs each time an event by CARP is generated. In the event that 
    there are multiple LAN interfaces where CARP is enabled, this script will check that all interfaces where CARP is 
    enabled are in a particular state (MASTER/BACKUP) before enabling or disabling its WAN interfaces.
*/

// this file should be saved to /usr/local/etc/rc.syshook.d/carp/10-wancarp
// some ideas taken from https://gist.github.com/taxilian/eecdc1fb17cf70e8080118cf6d8af412

/* --- EDIT AS NEEDED --- */
// my primary WAN is on the opt2 interface and secondary WAN is on the wan interface
$interfaces = array('opt2', 'wan');

require_once("config.inc");
require_once("interfaces.inc");
require_once("util.inc");

$subsystem = !empty($argv[1]) ? $argv[1] : '';
$type = !empty($argv[2]) ? $argv[2] : '';

if ($type != 'MASTER' && $type != 'BACKUP' && $type != 'INIT' && $type != 'DISABLED') {
    log_error("CARP '$type' event unknown from source '{$subsystem}'");
    exit(1);
}

if (!strstr($subsystem, '@')) {
    log_error("CARP '$type' event triggered from wrong source '{$subsystem}'");
    exit(1);
}

// Get a list of CARP statuses across interfaces that have it
$cmd = "/sbin/ifconfig -m -v | grep 'carp:' | awk '{print $2}'";
$cmd2 = "/sbin/ifconfig -m -v | grep 'carp:' | awk '{print $2}' | wc -l | tr -d ' '";
exec($cmd, $ifconfig_data, $ret);
exec($cmd2, $ifconfig_num, $ret);
$activeCARPInterfaces = $ifconfig_num[0];

// Log what the CARP subsystem is doing
// After INIT, the subsystem will become BACKUP, then MASTER if needed
if($type == 'INIT' && $activeCARPInterfaces > 0) {
  log_error("CARP on '{$subsystem}' is now INIT, but other subsystems are still active");
} else if($type == 'INIT' && $activeCARPInterfaces == 0) {
  log_error("CARP on '{$subsystem}' is now INIT, and appears to be the last active");
}
else if($type == 'MASTER' && $activeCARPInterfaces > 0) {
  log_error("CARP on '{$subsystem}' is now MASTER, but other subsystems are still active");
} else if($type == 'MASTER' && $activeCARPInterfaces == 0) {
  log_error("CARP on '{$subsystem}' is now MASTER, and appears to be the last active");
}
else if($type == 'BACKUP' && $activeCARPInterfaces > 0) {
  log_error("CARP on '{$subsystem}' is now BACKUP, other subsystems are still active");
} else if($type == 'BACKUP' && $activeCARPInterfaces == 0) {
  log_error("CARP on '{$subsystem}' is now BACKUP, and appears to be the last active");
}

// If no CARP subsystems/interfaces are active, then disable WAN interfaces
// This will generally apply when CARP is disabled and reports no statuses
if($activeCARPInterfaces == 0) {
  log_error("No CARP subsystems are active, deactivating WAN interfaces");

  foreach ($interfaces as $ifkey) {
      unset($config['interfaces'][$ifkey]['enable']);
      interface_configure(false, $ifkey, false, false);
      write_config("disable interface '$ifkey' due CARP event '$type'", false);
  }

  exit(0);
}

// Keep track of active MASTER and BACKUP instances
$masterCount = 0;
$backupCount = 0;

// Loop over $ifconfig_data and count how many are "MASTER" and how many are "BACKUP"
foreach ($ifconfig_data as $line) {
  if (strpos($line, 'MASTER') !== false) {
    $masterCount++;
  } else if (strpos($line, 'BACKUP') !== false) {
    $backupCount++;
  }
}

// Toggle WAN interfaces depending if all interfaces are MASTER or BACKUP
if ($masterCount == $activeCARPInterfaces && $masterCount > 0) {
  // The current node is all MASTER
  log_error("All CARP subsystems are MASTER, activating WAN interfaces");

  foreach ($interfaces as $ifkey) {
      $config['interfaces'][$ifkey]['enable'] = '1';
      interface_configure(false, $ifkey, false, false);
      write_config("enable interface '$ifkey' due CARP event '$type'", false);
  }
} else if ($backupCount == $activeCARPInterfaces && $backupCount > 0) {
  // The current node is all BACKUP
  log_error("Not all CARP subsystems are MASTER, deactivating WAN interfaces");

  foreach ($interfaces as $ifkey) {
      unset($config['interfaces'][$ifkey]['enable']);
      interface_configure(false, $ifkey, false, false);
      write_config("disable interface '$ifkey' due CARP event '$type'", false);
  }
}

?>

@skl283 I'm having a similar issue; IPv4 works instantly but my backup router won't pick up IPv6 information even if I reload using the command or gui. I have a very limited understanding of IPv6, not really sure how Router Advertisements work/why my backup router isn't working when my primary works each time. I have my IPv6 DUID set the same on each machine and I use the same MAC on my single WAN on both machines. Is it wise to use the same DUID for IPv6 in this scenario? I have Interfaces > Settings > IPv6 DHCP > Prevent Release enabled on both machines.

I also notice when I enter persistent CARP maintenance mode, I get a warning from OPNsense that an issue was detected with the machine and it has been demoted to backup, which usually doesn't show up, the machine would normally just go into backup. I have to enter persistent CARP maintenance mode again for it to properly engage, then press the button again to leave it. Not sure what's causing that warning, though a require_once("system.inc"); helped me fix some errors I was seeing reported by OPNsense, though not all.

Hey there everyone looking at this thread - I hope this script has helped out in some way!

I am softly abandoning my further development however. My setup where this was used was running two OPNsense instances virtually acting as my primary home firewall/router, but I have since replaced it with a Ubiquiti firewall - the two main reasons for myself being that it would be less time developing and troubleshooting this script as well as I could foreshadow an edge case where my Proxmox cluster might not gain quorum and be able to boot up virtual machines (including the OPNsenses) and thus would be a catch 22 and very bad. I still could have a use case for this within my dedicated cloud server, but one OPNsense there is enough and two aren't necessary.

Thank you, namaste, and good luck!