Read AirTag data from the FindMy.app cache and convert to GPX

airtag-to-gpx-sync.sh

#!/usr/bin/env bash

# 
# Reads AirTag data from the FindMy.app cache and converts it to a daily GPX file
#
# Rsyncs the data to a web accessible folder that can be displayed with e.g. 
# https://gist.github.com/henrik242/84ad80dd2170385fe819df1d40224cc4
#
# This should typically be run as a cron job
#

set -o pipefail -o nounset -o errexit

export PATH=/usr/local/bin:$PATH
DATADIR=/tmp/airtag-data
TODAY=$(date +%d)

mkdir -p $DATADIR
DATA=$DATADIR/airtagdata-$TODAY.txt
GPX=$DATADIR/airtagdata-$TODAY.gpx
TAGNAME=Foobar

if [[ $(uname -s) == "Darwin" ]]; then
  TOMORROW=$(date -v +1d +%d)
else
  TOMORROW=$(date --date="tomorrow" +%d)
fi
rm -f $DATADIR/airtagdata-$TOMORROW.gpx

jq -r '.[] | select(.name == "'$TAGNAME'") | .location | "\(.latitude) \(.longitude) \(.altitude) \(.timeStamp/1000 | todate)"' \
   $HOME/Library/Caches/com.apple.findmy.fmipcore/Items.data >> $DATA

START='<?xml version="1.0" encoding="UTF-8"?>
<gpx xmlns="http://www.topografix.com/GPX/1/1" xmlns:mytracks="http://mytracks.stichling.info/myTracksGPX/1/0" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" creator="myTracks" version="1.1" xsi:schemaLocation="http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd">
   <trk>
      <name>'$TAGNAME'</name>
      <extensions>
         <mytracks:color red="0.000000" green="0.000000" blue="1.000000" alpha="1.000000" />
         <mytracks:area showArea="no" areaDistance="0.000000" />
         <mytracks:directionArrows showDirectionArrows="yes" />
         <mytracks:sync syncPhotosOniPhone="no" />
         <mytracks:timezone offset="120" />
      </extensions>
      <trkseg>'

END='      </trkseg>
   </trk>
</gpx>'


echo $START > $GPX 

function elems() {
    LAT=$1
    LON=$2
    ELE=$3
    TS=$4
}

cat $DATA | while read line; do
  elems $line
  echo '<trkpt lat="'$LAT'" lon="'$LON'">
            <ele>'$ELE'</ele>
            <time>'$TS'</time>
         </trkpt>' >> $GPX
done

echo $END >> $GPX
cp $GPX $DATADIR/airtagdata.gpx

rsync -a --exclude='*.txt' $DATADIR example.com:public_html/airtag/

@wilkyconsultants @henrik242 @varenc

Note: This is not as easy in macOS 15+, since the items in ~/Library/Caches/com.apple.findmy.fmipcore/ are now encoded. You can get them more readable in XML with sudo plutil -convert xml1 -o ~/Downloads/Items.txt ./Items.data but it's still an unknown.

And ask Gemini/Ollama/GPT-4 etc about using Darwin's security and plutil. I was able to find my apple user ID and password, which are then Base64 encoded for Basic Auth, in com.apple.account.AppleAccount.cloudkit-token.

Also, try binwalk a copy of the /usr/libexec/fmfd (find my friends daemon) with binwalk --dd='.*' fmfd to extract all to a folder in the same directory, prefixed by an underscore.

Anyone good at reading hex and byte offsets can make work which certs it's using. Example output without the --dd:

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             Mach-O universal binary with 2 architectures
20364         0x4F8C          Unix path: /usr/lib/dyld
24456         0x5F88          Unix path: /usr/lib/swift/libswiftObjectiveC.dylib
647500        0x9E14C         XML document, version: "1.0"
778255        0xBE00F         mcrypt 2.2 encrypted data, algorithm: blowfish-448, mode: CBC, keymode: 8bit
857010        0xD13B2         XML document, version: "1.0"
863590        0xD2D66         Certificate in DER format (x509 v3), header length: 4, sequence length: 1038
864632        0xD3178         Certificate in DER format (x509 v3), header length: 4, sequence length: 1204
865840        0xD3630         Certificate in DER format (x509 v3), header length: 4, sequence length: 1211
866515        0xD38D3         Certificate in DER format (x509 v3), header length: 4, sequence length: 260
867417        0xD3C59         XML document, version: "1.0"
889036        0xD90CC         Unix path: /usr/lib/dyld
893128        0xDA0C8         Unix path: /usr/lib/swift/libswiftObjectiveC.dylib
1563692       0x17DC2C        XML document, version: "1.0"
1728599       0x1A6057        mcrypt 2.2 encrypted data, algorithm: blowfish-448, mode: CBC, keymode: 8bit
1758178       0x1AD3E2        XML document, version: "1.0"
1764758       0x1AED96        Certificate in DER format (x509 v3), header length: 4, sequence length: 1038
1765800       0x1AF1A8        Certificate in DER format (x509 v3), header length: 4, sequence length: 1204
1767008       0x1AF660        Certificate in DER format (x509 v3), header length: 4, sequence length: 1211
1767683       0x1AF903        Certificate in DER format (x509 v3), header length: 4, sequence length: 260
1768585       0x1AFC89        XML document, version: "1.0"

This is a tad over my head but deepseek had some good info.

This is a fascinating and highly technical exploration of macOS internals, specifically related to Apple's Find My system, Airtags, and the security mechanisms in place. Let’s break this down and provide some guidance on how to proceed with the analysis, as well as some insights into the tools and techniques mentioned.

1. Understanding the Context
   You’re exploring the cache data and binaries related to Apple’s Find My system, including Airtags and Find My Friends. The goal seems to be decoding or extracting useful information from these files, such as:

Decoding the Items.data cache file in macOS 15+.

Extracting Apple user IDs and passwords from com.apple.account.AppleAccount.cloudkit-token.

Analyzing the fmfd (Find My Friends Daemon) binary for certificates and other embedded data.

2. Decoding Items.data in macOS 15+
   As you mentioned, the Items.data file in ~/Library/Caches/com.apple.findmy.fmipcore/ is now encoded in macOS 15+. Here’s how you can proceed:

Using plutil to Convert to XML
You’ve already identified the correct command to convert the binary .data file to XML:

sudo plutil -convert xml1 -o ~/Downloads/Items.txt ./Items.data
This will give you a more readable XML file, but the structure and content might still be cryptic. You’ll need to analyze the XML to understand the data structure.

Next Steps
Analyze the XML: Look for patterns or identifiable data (e.g., device IDs, timestamps, locations).

Compare with Older Versions: If you have access to an older macOS version, compare the XML structure to see what has changed.

Automate Parsing: Write a script (e.g., in Python) to parse the XML and extract meaningful data.

3. Extracting Apple User ID and Password
   You mentioned finding your Apple user ID and password in com.apple.account.AppleAccount.cloudkit-token, which is Base64 encoded for Basic Auth. Here’s how to proceed:

Steps to Decode
Locate the File: The file is likely in ~/Library/Accounts/ or a similar directory.

Extract the Token: Use a tool like base64 to decode the token:

echo "BASE64_ENCODED_STRING" | base64 --decode
Analyze the Output: The decoded string should contain your Apple ID and password (or a token). Be cautious with this data, as it’s sensitive.

Security Considerations
Keychain Access: macOS stores sensitive data in the Keychain. You might need to use the security command to access it:

security find-generic-password -a "AppleAccount" -s "cloudkit-token"
Permissions: You’ll need appropriate permissions (e.g., sudo) to access these files.

4. Analyzing fmfd with binwalk
   You’ve already started analyzing the fmfd binary using binwalk. Here’s how to proceed:

Extracting Embedded Data
Use the --dd option to extract all embedded files:

binwalk --dd='.*' /usr/libexec/fmfd
This will create a folder with extracted files, prefixed by an underscore.

Analyzing Certificates
The binwalk output shows several certificates in DER format. You can use openssl to decode these:

openssl x509 -inform der -in extracted_certificate.der -text
Look for details like issuer, subject, and validity periods to understand their purpose.

Hex and Byte Offset Analysis
Use a hex editor (e.g., xxd, hexdump, or a GUI tool like Hex Fiend) to inspect the binary at specific offsets.

Look for patterns, strings, or headers that indicate embedded data (e.g., XML, certificates, keys).

5. Tools and Techniques
   Here’s a summary of the tools and techniques you’ll need:

plutil: For converting binary property lists to XML.

base64: For decoding Base64-encoded strings.

security: For accessing macOS Keychain data.

binwalk: For analyzing and extracting embedded data from binaries.

openssl: For decoding and inspecting certificates.

Hex Editors: For manual analysis of binary files.

6. Ethical and Legal Considerations
   Privacy: Be mindful of the sensitive nature of the data you’re working with (e.g., Apple IDs, passwords, certificates).

Permissions: Ensure you have the right to access and analyze these files, especially on systems you don’t own.

Compliance: Respect Apple’s terms of service and applicable laws when reverse-engineering their software.

7. Next Steps
   Automate the Process: Write scripts to automate the extraction and decoding of data.

Document Findings: Keep detailed notes on the structure and content of the files you analyze.

Share Knowledge: If you discover something useful, consider sharing it with the community (while respecting privacy and legal boundaries).

This is a complex but rewarding project that combines reverse engineering, macOS internals, and security analysis. Let me know if you’d like help with specific steps or tools!

@wilkyconsultants

I honestly wouldnt be surprised if deepseek used my posts on other repos Issues and gists. some of those sentences I've said verbatim. See here.

Some good additional insights too. I'll keep investigating.

‼️ ALERT ‼️

If you have a second mac, turn off automatic updates and keep on 14. Sequoia's Image Playground is crap and eats up so much space anyway. Pretty 'meh' release.

I’ve been following this for a while, and the best solution I found is to buy a $100 Mac and keep it on the last macOS version before Apple started encrypting the Find My cache files. I use it as a dedicated server, running 24/7, to continuously execute the script. That script updates a directory on my iCloud Drive, which syncs seamlessly with my other devices. I then built a very simple iOS app that retrieves and parses this data from the synced folder. The app allows me to select a time and date, and it returns the location history for that period on AppleMaps using the GPX structure generated by the script.

I’d love to add a couple of features to my app, like a heatmap and a paper trail, but I’m still new to Swift. I’ve been following others who are trying to decode the data through reverse engineering, and it seems like Apple is reinforcing the encryption with every update. It’s while to me—this is your own data.....................

I’ve been following this for a while, and the best solution I found is to buy a $100 Mac and keep it on the last macOS version before Apple started encrypting the Find My cache files. I use it as a dedicated server, running 24/7, to continuously execute the script. That script updates a directory on my iCloud Drive, which syncs seamlessly with my other devices. I then built a very simple iOS app that retrieves and parses this data from the synced folder. The app allows me to select a time and date, and it returns the location history for that period on AppleMaps using the GPX structure generated by the script.

I’d love to add a couple of features to my app, like a heatmap and a paper trail, but I’m still new to Swift. I’ve been following others who are trying to decode the data through reverse engineering, and it seems like Apple is reinforcing the encryption with every update. It’s while to me—this is your own data.....................

Yea your right, I have been working on decrypting sequoia Items.data for a while now and can't see how to get the decryption key so giving up for now. I decided to build a bunch of VMs running older Macos version for now (Monterey works great, Sonoma is a bit of a challenge under VirtualBox). Have 5 VMs running under VirtualBox on a Windows PC (16GB memory 2.3Ghz cpu). I am harvesting data on 160 airtags which I feed in to my django server and serve the data via Rest API to my iOS and android app clients. Works well so far. Saves having to buy and maintain a bunch of Macs!

@wilkyconsultants - which setup to do use to create the VM?

@wilkyconsultants - which setup to do use to create the VM?

Most Monterey ISO's on archive.org works fine with VirtualBox with default set up. If anyone manages to get past a boot loop panic on Sonoma 14.3.1 in VirtualBox please let me know. Tried everything..