Each year as Apple's World Wide Developer Conference (WWDC) approaches, I start to think about upgrading macOS to the version that was released at WWDC the previous year. I reckon that's a good way of minimising your pain and suffering, not to mention giving developers a chance to patch things that got broken when the API goal-posts moved.
The end of May 2025 was when I decided to upgrade both a 2019-era Intel iMac and an Apple M2 MacBook from Sonoma 14.7.1 to Sequoia 15.5.
Aside from the seven fractions of eternity that macOS updates seem to demand these days, it all seemed to go swimmingly. The only explicit grizzle was from Carbon Copy Cloner, which wanted a later version (on my to-do list).
I'm an old guy. I've used a bunch of source code utilities in my life, including CVS, Subversion and Git (plus tools most people have never heard of like UPDATE and sccs).
Right now I have a mixture of Subversion and Git. The Subversion stuff is in a mental bucket marked "should be migrated sometime."
Now, without boring anyone with all the gory details, I have a crontab entry on each Mac which fires every couple of hours to keep some subversion trees in sync:
20 */2 * * * subversion_autosyncBasically, if I commit something on one machine, it'll be on the other machine within an hour or so without me needing to remember to do anything. Sort of like a poor man's Dropbox that doesn't involve the cloud.
This scheme has been working away, silently, for almost 20 years. I kid you not. The initial log entry on the subversion_autosync script is 05 Jul 2005.
Long story short, I noticed some oddities on the iMac which led me to suspect that the synchronisation process had stopped working.
I'm a firm believer in logging. The only log that is useful is the one that's already captured the problem. Whenever I set up scripts that are intended to run in the background, I instrument them like this:
# set up logging if launched by cron or as a launch agent
if [ "$(tty)" = "not a tty" ] ; then
LOGFILE="$HOME/Library/Logs/$(basename "$0").log"
touch "$LOGFILE"
exec >> "$LOGFILE"
exec 2>> "$LOGFILE"
fiThat way, any logging messages will get preserved in ~/Library/Logs.
The log was full of this:
svn: E170013: Unable to connect to a repository at URL 'svn://svnserver.your.domain.com/trunk/Documents/Synchronised'
svn: E000065: Can't connect to host 'svnserver.your.domain.com': No route to host
Meanwhile, over on the M2 laptop, the script was working as usual.
Triggering the command by hand worked as expected on both machines so it seemed to be the combination of:
- launched by
cron; and - running on
x86_64(Intel).
I wondered if this was Apple's way of telling me that I should've stopped using cron in favour of a Launch Agent so I gave that a whirl:
{
"Debug" => 1
"Label" => "arpa.home.your.svn.autosync"
"ProgramArguments" => [
0 => "/Users/moi/.local/bin/subversion_autosync"
]
"StandardErrorPath" => "/Users/moi/Library/Logs/subversion_autosync.log"
"StandardOutPath" => "/Users/moi/Library/Logs/subversion_autosync.log"
"StartInterval" => 30
}The start interval of 30 seconds was chosen so I could keep trying things in conjunction with a tail -f watching the log.
Same problem. So, not cron but a problem of background tasks more generally, but only on Intel.
I also have another (unrelated) crontab entry:
*/5 * * * * publish_mac_temperatureThis only runs on the Intel iMac because the utility it relies on to fetch the CPU temperature doesn't work on Apple Silicon. The publish_mac_temperature script packages-up an MQTT message and transmits it using:
mosquitto_pub -h mosquitto.your.domain.comThat script was working.
So, we have the situation where:
-
cronand/orlaunchdis initiating two scripts:subversion_autosyncpublish_mac_temperature
-
Both of those scripts invoke a utility which involves network access:
script utility domain name subversion_autosyncsvnsvnserver.your.domain.com publish_mac_temperaturemosquitto_pubmosquitto.your.domain.com -
In terms of the data communications:
- The domain names referenced by the utilities are local DNS CNAMEs which resolve to two different Raspberry Pis.
- The DNS server is the same host as
svnserver. - All devices (Macs, Raspberry Pis) are on the same /24 subnet.
-
In terms of the utilities:
- both were installed using HomeBrew.
- One utility is failing while the other isn't.
It looks like svn has resolved the alias to the IP address and that connectivity has failed after that. Because it's working, it's kinda self-evident that mosquitto_pub has resolved its alias so that implies that, despite Jeff Geerling's T-shirts, it ain't a DNS issue. All other things being equal, whatever is stopping svn from working should also be stopping mosquito_pub. Why are they different?
Googling the topic finds things like:
-
Reddit:
-
GitHub:
-
Apple discussions:
In the past, not granting background-process tools full disk access has caused all manner of grief so, while I can't see how that's related, I make sure none of that got lost on the upgrade. There's also advice about turning off the Firewall. I've never turned it on but I double-check anyway.
While I'm there, I wade through System Preferences » Privacy and Security in case anything else jumps out. That's when I notice a brand new category: "Local Network".
This was hinted at in dbeaver, 765513 and 778457 but the penny didn't really drop.
On the Intel Mac, "Local Network" has the Arduino IDE. On the M2 laptop, it's got the Arduino IDE and svn.
W.T.A.F.?
778457 also mentions code-signing.
The vital clue!
Compare/contrast:
-
Intel Mac:
$ codesign -d -v /usr/local/Cellar/subversion/1.14.5_2/bin/svn /usr/local/Cellar/subversion/1.14.5_2/bin/svn: code object is not signed at all
-
M2 Laptop:
$ codesign -d -v /opt/homebrew/Cellar/subversion/1.14.5_2/bin/svn 2>&1 | grep -e "^Signature" Signature=adhoc
why
codesignwrites its output tostderris a mystery.
Back to the Intel Mac where the Launch Agent is still triggering the script every 30 seconds and the watch on the log is showing constant failure:
$ codesign --sign «MyIdentityHere» /usr/local/Cellar/subversion/1.14.5_2/bin/svnThe next time the Launch Agent fires, I get a prompt to allow svn which takes up residence in "Local Network", so now the Intel and M2 Macs are the same.
More importantly, the subversion_autosync script starts working again.
Cookin' with photons!
A slightly deeper dive into this code-signing malarky.
First, the Intel Mac:
$ cd /usr/local/Cellar/subversion/1.14.5_2/bin
$ ls -1 | xargs file
svn: Mach-O 64-bit executable x86_64
svnadmin: Mach-O 64-bit executable x86_64
svnbench: Mach-O 64-bit executable x86_64
svndumpfilter: Mach-O 64-bit executable x86_64
svnfsfs: Mach-O 64-bit executable x86_64
svnlook: Mach-O 64-bit executable x86_64
svnmucc: Mach-O 64-bit executable x86_64
svnrdump: Mach-O 64-bit executable x86_64
svnserve: Mach-O 64-bit executable x86_64
svnsync: Mach-O 64-bit executable x86_64
svnversion: Mach-O 64-bit executable x86_64
$ ls -1 | xargs -I % codesign -d -vv %
svn: code object is not signed at all
svnadmin: code object is not signed at all
svnbench: code object is not signed at all
svndumpfilter: code object is not signed at all
svnfsfs: code object is not signed at all
svnlook: code object is not signed at all
svnmucc: code object is not signed at all
svnrdump: code object is not signed at all
svnserve: code object is not signed at all
svnsync: code object is not signed at all
svnversion: code object is not signed at all
$ cd /usr/local/Cellar/mosquitto/2.0.21/bin
$ ls -1 | xargs file
mosquitto_ctrl: Mach-O 64-bit executable x86_64
mosquitto_passwd: Mach-O 64-bit executable x86_64
mosquitto_pub: Mach-O 64-bit executable x86_64
mosquitto_rr: Mach-O 64-bit executable x86_64
mosquitto_sub: Mach-O 64-bit executable x86_64
$ ls -1 | xargs -I % codesign -d -vv %
mosquitto_ctrl: code object is not signed at all
mosquitto_passwd: code object is not signed at all
mosquitto_pub: code object is not signed at all
mosquitto_rr: code object is not signed at all
mosquitto_sub: code object is not signed at allInterpretation:
- All the Subversion and Mosquitto binaries are pure
x86_64(not "universal"). - None is code-signed.
- The lack of code-signing for
mosquitto_pubmeans this doesn't explain why thepublish_mac_temperaturescript kept working when thesubversion_autosyncscript went nuts.
A slightly wider search of the HomeBrew Cellar is yet to discover any x86_64 binary that is code-signed. I'm not saying HomeBrew packages deployed on Intel Macs are never signed. I just couldn't find any examples.
Now let's take a look at the M2 laptop (I'll cut down the lines a bit but you can take it as read there are no exceptions buried in the ...):
$ cd /opt/homebrew/Cellar/subversion/1.14.5_2/bin
$ ls -1 | xargs file
svn: Mach-O 64-bit executable arm64
svnadmin: Mach-O 64-bit executable arm64
...
$ ls -1 | xargs -I % codesign -d -vv % 2>&1 | grep -e "^Signature" -e "^Executable"
Executable=/opt/homebrew/Cellar/subversion/1.14.5_2/bin/svn
Signature=adhoc
Executable=/opt/homebrew/Cellar/subversion/1.14.5_2/bin/svnadmin
Signature=adhoc
...
$ cd /opt/homebrew/Cellar/mosquitto/2.0.21/bin
$ ls -1 | xargs file
mosquitto_pub: Mach-O 64-bit executable arm64
mosquitto_sub: Mach-O 64-bit executable arm64
...
$ ls -1 | xargs -I % codesign -d -vv % 2>&1 | grep -e "^Signature" -e "^Executable"
Executable=/opt/homebrew/Cellar/mosquitto/2.0.21/bin/mosquitto_pub
Signature=adhoc
Executable=/opt/homebrew/Cellar/mosquitto/2.0.21/bin/mosquitto_sub
Signature=adhoc
...Interpretation:
- All the Subversion and Mosquitto binaries are pure
arm64(not "universal"). - All are code-signed.
This time a slightly wider search of the Cellar didn't chuck up any examples of arm64 binaries that were not code-signed.
All of which leads to the key question:
Why does HomeBrew appear to be signing
arm64binaries but not signingx86_64binaries?
Our journey begins with the HomeBrew v4.3.0 release notes 14 May 2024 which includes:
Homebrew/homebrew-cask requires code signing of all casks. Expect removal of casks that are not code signed from Homebrew/homebrew-cask in future. This is because code signing is required on Apple Silicon which is used by a growing majority of all Homebrew users.
Subversion isn't a cask (at least not as I understand it) so this begs the question of when code-signing arm64 binaries expanded to non-casks. I couldn't find anything to explain that history so we'll put that to one side.
The above quote includes a link to HomeBrew Pull Request 17002 which might have been the end of the journey but for Pull Request 17009 which seemed to revert it.
In short, stuffed if I know!
If you've reached this point in the discussion, well done you! No doubt you're asking how to sign any binaries of your own because the «MyIdentityHere» token used above doesn't actually help anyone.
There are a few ways. If you're compiling your own code with Xcode you'll probably have a developer certificate so you can use that:
$ codesign --sign "Apple Development" path/to/binaryIf not, you can download Apple Configurator 2 from the App Store. When you launch that and sign in with your AppleID you get a certificate that seems to be able to be used for code-signing:
$ codesign --sign "Apple Configurator 2" path/to/binaryI'm a long-time user of Apple Configurator so I can't actually say exactly which steps cause that certificate to be added to your keychain. I think it's when you use the Account menu to sign in with your AppleID but I can't be certain. You might have to experiment a bit. Sorry.
Or you can use the Certificate Assistant in the Keychain Access tool to generate self-signed certs (I know those work).
Another possibility is to Google the construction of certs using openssl. They may work but my initial attempts were not successful and I haven't gone far enough down that rabbit hole to figure out why.
Why svn and not mosquitto_pub? 🤷
My guess is TN3151 Choosing the right networking API might have something to do with it.