Skip to content

Instantly share code, notes, and snippets.

@Nullcaller
Last active September 1, 2024 01:26
Show Gist options
  • Save Nullcaller/ec00480aa8ef9cf20d0f8835eff52896 to your computer and use it in GitHub Desktop.
Save Nullcaller/ec00480aa8ef9cf20d0f8835eff52896 to your computer and use it in GitHub Desktop.
A RouterOS v7.8-compatible script for automatic generation of an address list containing all Facebook CDN addresses required for maintaining a normal connection from the DNS cache entries. Designed for decent stability and low CPU usage spikes. This is a combination of scripts by @jgrossiord and @ZloyXEP with some additions.
:delay delay-time=#REPLACE_THIS_WITH_SOME_ARBITRARY_RESULT_OF_HITTING_YOUR_NUMPAD_WITH_A_FIST#ms;
:foreach i in=[/ip dns cache find] do={
# Turn this on while turning off other delays for ease of manual use. Don't forget to reenable later.
# :delay delay-time=5ms;
:delay delay-time=50ms;
:local cacheName [/ip dns cache all get $i name];
:local cacheType [/ip dns cache all get $i type];
:if (($cacheType="A" or $cacheType="CNAME") and (($cacheName~"^([0-9A-Za-z\\-]*\\.)*facebook(\\.[0-9A-Za-z\\-]*)*\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*fbcdn(\\.[0-9A-Za-z\\-]*)*\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*fbstatic(\\.[0-9A-Za-z\\-]*)*\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*fbexternal(\\.[0-9A-Za-z\\-]*)*\\.?\$"))) do={
# IP-based rule
:if ($cacheType="A") do={
:local cacheData [/ip dns cache all get $i data];
:local record [/ip firewall address-list find where (address=$cacheData) and (list=vpn_loopback_sites_dynamic)];
# One day I just randomly found 0.0.0.0 in the list. I'm not sure how it got there, so I introduced this safety measure.
:if (($cacheData~"^0\\.") or ($cacheData~"^10\\.") or ($cacheData~"^127\\.") or ($cacheData~"^192\\.168\\.")) do={
:put ("script tried putting $cacheData into the address list, was prevented");
} else={
:if ($record="") do={
:put ("add: $cacheName $cacheType $cacheData");
/ip firewall address-list add address=$cacheData comment=$cacheName timeout=14d list=vpn_loopback_sites_dynamic
} else={
:put ("renew: $cacheName $cacheType $cacheData");
/ip firewall address-list set $record timeout=14d;
}
}
}
# Domain-based rule
:if ($cacheType="A") do={
:local record [/ip firewall address-list find where (address=$cacheName) and (list=vpn_loopback_sites_dynamic)];
:if ($record="") do={
:put ("add: $cacheName $cacheType");
/ip firewall address-list add address=$cacheName timeout=14d list=vpn_loopback_sites_dynamic
} else={
:put ("renew: $cacheName $cacheType");
/ip firewall address-list set $record timeout=14d;
}
}
:if ($cacheType="CNAME") do={
:local cacheData [/ip dns cache all get $i data];
:local record [/ip firewall address-list find where (address=$cacheData) and (list=vpn_loopback_sites_dynamic)];
:if ($record="") do={
:put ("add: $cacheName $cacheType $cacheData");
/ip firewall address-list add address=$cacheData comment=$cacheName timeout=14d list=vpn_loopback_sites_dynamic
} else={
:put ("renew: $cacheName $cacheType $cacheData");
/ip firewall address-list set $record timeout=14d;
}
:local record [/ip firewall address-list find where (address=$cacheName) and (list=vpn_loopback_sites_dynamic)];
:if ($record="") do={
:put ("add: $cacheName $cacheType");
/ip firewall address-list add address=$cacheName timeout=14d list=vpn_loopback_sites_dynamic
} else={
:put ("renew: $cacheName $cacheType");
/ip firewall address-list set $record timeout=14d;
}
}
}
}
@Nullcaller
Copy link
Author

Nullcaller commented Mar 31, 2023

My current line 6 that is supposed to hopefully maybe with some luck and chance and if the stars align add Microsoft's CDN to the list, except for OneDrive, Windows Update and Office:

:if (($cacheType="A" or $cacheType="CNAME") and (($cacheName~"^([0-9A-Za-z\\-]*\\.)*microsoft(\\.[0-9A-Za-z\\-]*)*\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*bing(\\.[0-9A-Za-z\\-]*)*\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*msn(\\.[0-9A-Za-z\\-]*)*\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*msftauth(\\.[0-9A-Za-z\\-]*)*\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*virtualearth(\\.[0-9A-Za-z\\-]*)*\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*bingtravel(\\.[0-9A-Za-z\\-]*)*\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*msidentity(\\.[0-9A-Za-z\\-]*)*\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*[0-9A-Za-z\\-]*msn[0-9A-Za-z\\-]*\\.([0-9A-Za-z\\-]*\\.)*akamaized\\.net\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*[0-9A-Za-z\\-]*microsoft[0-9A-Za-z\\-]*\\.([0-9A-Za-z\\-]*\\.)*akamaized\\.net\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*live.com\\.?\$") or ($cacheName~"^([0-9A-Za-z\\-]*\\.)*takelessons.com\\.?\$")) and !(($cacheName~"storage") or ($cacheName~"1drv") or ($cacheName~"onedrive") or ($cacheName~"office") or ($cacheName~"sharepoint") or ($cacheName~"windows") or ($cacheName~"windowsupdate") or ($cacheName~"ntservicepack") or ($cacheName~"update\\.microsoft\\.com\\.?\$") or ($cacheName~"download\\.microsoft\\.com\\.?\$"))) do={

I don't know if it'll work correctly though. Currently testing it. Microsoft seems to have a very disorganized decentralized CDN compared to Facebook and Twitter. Facebook and Twitter settled neatly into a few domains, while Microsoft just can't get enough of them.

@klugev303
Copy link

Please help me to create a script to determine YouTube IP addresses using a script in Mikrotik (RouterOS7.15.3)
Doesn't work with the code below, thanks in advance!

:delay delay-time=246ms;
:foreach i in=[/ip dns cache find] do={
:delay delay-time=50ms;
:local cacheName [/ip dns cache all get $i name];
:local cacheType [/ip dns cache all get $i type];
:if (($cacheType="A" or $cacheType="CNAME") and (($cacheName~"^([0-9A-Za-z\-]\.)googlevideo(\.[0-9A-Za-z\-])\.?$") or ($cacheName~"^([0-9A-Za-z\-]\.)youtube(\.[0-9A-Za-z\-])\.?$") or ($cacheName~"^([0-9A-Za-z\-]\.)ytimg(\.[0-9A-Za-z\-])\.?$") or ($cacheName~"^([0-9A-Za-z\-]\.)youtu(\.[0-9A-Za-z\-])\.?$"))) do={
# IP-based rule
:if ($cacheType="A") do={
:local cacheData [/ip dns cache all get $i data];
:local record [/ip firewall address-list find where (address=$cacheData) and (list=Youtube_IP)];
# One day I just randomly found 0.0.0.0 in the list. I'm not sure how it got there, so I introduced this safety measure.
:if (($cacheData~"^0\.") or ($cacheData~"^10\.") or ($cacheData~"^127\.") or ($cacheData~"^192\.168\.")) do={
:put ("script tried putting $cacheData into the address list, was prevented");
} else={
:if ($record="") do={
:put ("add: $cacheName $cacheType $cacheData");
/ip firewall address-list add address=$cacheData comment=$cacheName timeout=14d list=Youtube_IP
} else={
:put ("renew: $cacheName $cacheType $cacheData");
/ip firewall address-list set $record timeout=14d;
}
}
}

	# Domain-based rule
	:if ($cacheType="A") do={
		:local record [/ip firewall address-list find where (address=$cacheName) and (list=Youtube_IP)];
		:if ($record="") do={
			:put ("add: $cacheName $cacheType");
			/ip firewall address-list add address=$cacheName timeout=14d list=Youtube_IP
		} else={
			:put ("renew: $cacheName $cacheType");
			/ip firewall address-list set $record timeout=14d;
		}
	}
	:if ($cacheType="CNAME") do={
		:local cacheData [/ip dns cache all get $i data];
		:local record [/ip firewall address-list find where (address=$cacheData) and (list=Youtube_IP)];
		:if ($record="") do={
			:put ("add: $cacheName $cacheType $cacheData");
			/ip firewall address-list add address=$cacheData comment=$cacheName timeout=14d list=Youtube_IP
		} else={
			:put ("renew: $cacheName $cacheType $cacheData");
			/ip firewall address-list set $record timeout=14d;
		}
		:local record [/ip firewall address-list find where (address=$cacheName) and (list=Youtube_IP)];
		:if ($record="") do={
			:put ("add: $cacheName $cacheType");
			/ip firewall address-list add address=$cacheName timeout=14d list=Youtube_IP
		} else={
			:put ("renew: $cacheName $cacheType");
			/ip firewall address-list set $record timeout=14d;
		}
	}
}

}

@Nullcaller
Copy link
Author

The idea is that you use this line of code:

:if (($cacheType="A" or $cacheType="CNAME") and (#matchers#)) do={

where #matchers# is replaced with a string that looks like

(#matcher_1#) or (#matcher_2#) or ... or (#matcher_n#)

where #matcher_i# for i=1..n is replaced with

$cacheName~"#regexp_i#"

and #regexp_i# is a properly-escaped regex pattern that the domain names you'd like to put into the list vpn_loopback_sites_dynamic should match. (well, one of the patterns)

Regex101 is a great resource to test your regular expressions, and Regex Learn is a great resource if you don't even know how to make them.

Otherwise, look at my code to understand how to properly escape them for MikroTik. Good luck!

@Nullcaller
Copy link
Author

I may or may not make a similar script for YouTube myself in the future, though.

@klugev303
Copy link

I may or may not make a similar script for YouTube myself in the future, though.

I'm afraid I won't figure it out myself(
I ask for help, in the current situation of YouTube blocking, I'm looking for the best solution to this problem through Mikrotik, Thanks in advance, I'll be waiting impatiently!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment