-
-
Save henrik242/1da3a252ca66fb7d17bca5509a67937f to your computer and use it in GitHub Desktop.
| #!/usr/bin/env bash | |
| # | |
| # Reads AirTag data from the FindMy.app cache and converts it to a daily GPX file | |
| # | |
| # Rsyncs the data to a web accessible folder that can be displayed with e.g. | |
| # https://gist.github.com/henrik242/84ad80dd2170385fe819df1d40224cc4 | |
| # | |
| # This should typically be run as a cron job | |
| # | |
| set -o pipefail -o nounset -o errexit | |
| export PATH=/usr/local/bin:$PATH | |
| DATADIR=/tmp/airtag-data | |
| TODAY=$(date +%d) | |
| mkdir -p $DATADIR | |
| DATA=$DATADIR/airtagdata-$TODAY.txt | |
| GPX=$DATADIR/airtagdata-$TODAY.gpx | |
| TAGNAME=Foobar | |
| if [[ $(uname -s) == "Darwin" ]]; then | |
| TOMORROW=$(date -v +1d +%d) | |
| else | |
| TOMORROW=$(date --date="tomorrow" +%d) | |
| fi | |
| rm -f $DATADIR/airtagdata-$TOMORROW.gpx | |
| jq -r '.[] | select(.name == "'$TAGNAME'") | .location | "\(.latitude) \(.longitude) \(.altitude) \(.timeStamp/1000 | todate)"' \ | |
| $HOME/Library/Caches/com.apple.findmy.fmipcore/Items.data >> $DATA | |
| START='<?xml version="1.0" encoding="UTF-8"?> | |
| <gpx xmlns="http://www.topografix.com/GPX/1/1" xmlns:mytracks="http://mytracks.stichling.info/myTracksGPX/1/0" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" creator="myTracks" version="1.1" xsi:schemaLocation="http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd"> | |
| <trk> | |
| <name>'$TAGNAME'</name> | |
| <extensions> | |
| <mytracks:color red="0.000000" green="0.000000" blue="1.000000" alpha="1.000000" /> | |
| <mytracks:area showArea="no" areaDistance="0.000000" /> | |
| <mytracks:directionArrows showDirectionArrows="yes" /> | |
| <mytracks:sync syncPhotosOniPhone="no" /> | |
| <mytracks:timezone offset="120" /> | |
| </extensions> | |
| <trkseg>' | |
| END=' </trkseg> | |
| </trk> | |
| </gpx>' | |
| echo $START > $GPX | |
| function elems() { | |
| LAT=$1 | |
| LON=$2 | |
| ELE=$3 | |
| TS=$4 | |
| } | |
| cat $DATA | while read line; do | |
| elems $line | |
| echo '<trkpt lat="'$LAT'" lon="'$LON'"> | |
| <ele>'$ELE'</ele> | |
| <time>'$TS'</time> | |
| </trkpt>' >> $GPX | |
| done | |
| echo $END >> $GPX | |
| cp $GPX $DATADIR/airtagdata.gpx | |
| rsync -a --exclude='*.txt' $DATADIR example.com:public_html/airtag/ |
@tmozes make sure your terminal has full disk access - This article can help
Just in case it is helpful, I wrote a script that logs more information (history location, battery level, etc.) and supports more devices (AirTag, iPhone, MacBook, etc.).
https://github.com/fjxmlzn/FindMyHistory
Suggestions are welcome!
@fjxmlzn Cool!
Hello @henrik242 , I was looking at your great code but I still have a question.. Could be possible to get from findmy cache not only lat, long and the actual timestamp also the timestamp of the last update of the airtag?
It could happen that I read the cache right now but the last update of the Airtag was many days ago.. Thank you very much :-)
@sarto89 Just look at your $HOME/Library/Caches/com.apple.findmy.fmipcore/Items.data file and you might find what you're looking for.
Looks like with the MacOS Sonoma 14.4 update the $HOME/Library/Caches/com.apple.findmy.fmipcore/ files became encrypted. Any idea how to decrypt these to get the file in plain text again like it was is 14.3.1 and before? I was expecting apple to chop our legs off, it has happened.
Are you sure they are encrypted? What does the command file $HOME/Library/Caches/com.apple.findmy.fmipcore/* say?
Bummer. I'm still on Ventura, so I'm not affected (yet).
Bummer. I'm still on Ventura, so I'm not affected (yet).
I am on Sonoma 14.3.1 but I built an airtag harvesting app that I was planning to go to production with for users to track the history on a map of their FindMy tags. Putting that on hold until I find out if I can decryot these files. This was expected as Apple frowns on tracking the history..
Maybe try putting that string in a file by itself and try to figure out what kind of encryption it is. Is it base64 encoded?
Also hitting this on 14.4.1
% plutil -p /Users/xxx/Library/Caches/com.apple.findmy.fmipcore/Items.data
{
"encryptedData" => {length = 40501, bytes = 0x82f73cb2 91a2aa9f 867bd9c0 30c79f5e ... d491f5f6 52b03543 }
"signature" => {length = 64, bytes = 0xcffcffcc d3befa46 13c3dd0b 0166762d ... eed8541f b9afbce1 }
}
yes, the encryption started in 14.4 so we would expect subsequent releases 14.4.x+ to now have it permanently. Stay with macOS 10.5 to 14.3.1 and your safe. It would be nice if we could decrypt the files but we need the key to decrypt them and I can't see any way to get that as Apple does not want you to look at your own files.
The key has to be somewhere (keychain?), it should be possible to reverse engineer and re-implement the decryption
Not very familiar with GitHub, hoping that commenting will allow me to get notified about any further comments here. I've backed up my cached FindMy files just in case the encryption is cracked in the future. If anyone needs a machine running 14.4.1 for testing, let me know!
Anyone have any updates? Hoping to get to figure out how to decrypt this. If it helps I found that it seems to look up these keychain items:
com.apple.account.AppleAccount.cloudkit-token
com.apple.account.AppleAccount.find-my-friends-app-token
com.apple.account.AppleAccount.find-my-iphone-app-token
com.apple.account.AppleAccount.find-my-iphone-siri-token
com.apple.account.AppleAccount.find-my-iphone-token
com.apple.account.AppleAccount.key-transparency-token
com.apple.account.AppleAccount.maps-token
com.apple.account.AppleAccount.search-party-token
com.apple.account.AppleAccount.token
com.apple.account.DeviceLocator.find-my-iphone-app-token
com.apple.account.DeviceLocator.find-my-iphone-siri-token
com.apple.account.DeviceLocator.token
com.apple.account.FindMyFriends.find-my-friends-app-token
But that might be a total red herring since thats are associated with accountsd. The Find My app logs also mention Initializing SharingCircleKeychain with 6 secretValues but unclear what Keychain items that maps to. If any. I suspect Apple might hide some Keychain items from the GUI completely.
Anyone have any updates? Hoping to get to figure out how to decrypt this. If it helps I found that it seems to look up these keychain items:
com.apple.account.AppleAccount.cloudkit-token com.apple.account.AppleAccount.find-my-friends-app-token com.apple.account.AppleAccount.find-my-iphone-app-token com.apple.account.AppleAccount.find-my-iphone-siri-token com.apple.account.AppleAccount.find-my-iphone-token com.apple.account.AppleAccount.key-transparency-token com.apple.account.AppleAccount.maps-token com.apple.account.AppleAccount.search-party-token com.apple.account.AppleAccount.token com.apple.account.DeviceLocator.find-my-iphone-app-token com.apple.account.DeviceLocator.find-my-iphone-siri-token com.apple.account.DeviceLocator.token com.apple.account.FindMyFriends.find-my-friends-app-tokenBut that might be a total red herring since thats are associated with
accountsd. The Find My app logs also mentionInitializing SharingCircleKeychain with 6 secretValuesbut unclear what Keychain items that maps to. If any. I suspect Apple might hide some Keychain items from the GUI completely.
It seems no way to get by this, I tried many avenues. I have become aware of an Apple FindMy API that seems to be available to OEM tag makers but Apple will not give me access to it since I am a software engineer not a device maker. If anyone has used the API I would love a peek at it!
https://gist.github.com/airy10/5205dc851fbd0715fcd7a5cdde25e7c8 should work on MacOs 14.xx, but it doesn't seem to work on 15.xx
@wilkyconsultants @henrik242 @varenc
Note: This is not as easy in macOS 15+, since the items in ~/Library/Caches/com.apple.findmy.fmipcore/ are now encoded.
You can get them more readable in XML with sudo plutil -convert xml1 -o ~/Downloads/Items.txt ./Items.data but it's still an unknown.
And ask Gemini/Ollama/GPT-4 etc about using Darwin's security and plutil. I was able to find my apple user ID and password, which are then Base64 encoded for Basic Auth, in com.apple.account.AppleAccount.cloudkit-token.
Also, try binwalk a copy of the /usr/libexec/fmfd (find my friends daemon) with binwalk --dd='.*' fmfd to extract all to a folder in the same directory, prefixed by an underscore.
Anyone good at reading hex and byte offsets can make work which certs it's using. Example output without the --dd:
DECIMAL HEXADECIMAL DESCRIPTION
--------------------------------------------------------------------------------
0 0x0 Mach-O universal binary with 2 architectures
20364 0x4F8C Unix path: /usr/lib/dyld
24456 0x5F88 Unix path: /usr/lib/swift/libswiftObjectiveC.dylib
647500 0x9E14C XML document, version: "1.0"
778255 0xBE00F mcrypt 2.2 encrypted data, algorithm: blowfish-448, mode: CBC, keymode: 8bit
857010 0xD13B2 XML document, version: "1.0"
863590 0xD2D66 Certificate in DER format (x509 v3), header length: 4, sequence length: 1038
864632 0xD3178 Certificate in DER format (x509 v3), header length: 4, sequence length: 1204
865840 0xD3630 Certificate in DER format (x509 v3), header length: 4, sequence length: 1211
866515 0xD38D3 Certificate in DER format (x509 v3), header length: 4, sequence length: 260
867417 0xD3C59 XML document, version: "1.0"
889036 0xD90CC Unix path: /usr/lib/dyld
893128 0xDA0C8 Unix path: /usr/lib/swift/libswiftObjectiveC.dylib
1563692 0x17DC2C XML document, version: "1.0"
1728599 0x1A6057 mcrypt 2.2 encrypted data, algorithm: blowfish-448, mode: CBC, keymode: 8bit
1758178 0x1AD3E2 XML document, version: "1.0"
1764758 0x1AED96 Certificate in DER format (x509 v3), header length: 4, sequence length: 1038
1765800 0x1AF1A8 Certificate in DER format (x509 v3), header length: 4, sequence length: 1204
1767008 0x1AF660 Certificate in DER format (x509 v3), header length: 4, sequence length: 1211
1767683 0x1AF903 Certificate in DER format (x509 v3), header length: 4, sequence length: 260
1768585 0x1AFC89 XML document, version: "1.0"
@wilkyconsultants @henrik242 @varenc
Note: This is not as easy in macOS 15+, since the items in
~/Library/Caches/com.apple.findmy.fmipcore/are now encoded. You can get them more readable in XML withsudo plutil -convert xml1 -o ~/Downloads/Items.txt ./Items.databut it's still an unknown.And ask Gemini/Ollama/GPT-4 etc about using Darwin's
securityandplutil. I was able to find my apple user ID and password, which are then Base64 encoded for Basic Auth, incom.apple.account.AppleAccount.cloudkit-token.Also, try
binwalka copy of the/usr/libexec/fmfd(find my friends daemon) withbinwalk --dd='.*' fmfdto extract all to a folder in the same directory, prefixed by an underscore.Anyone good at reading hex and byte offsets can make work which certs it's using. Example output without the --dd:
DECIMAL HEXADECIMAL DESCRIPTION -------------------------------------------------------------------------------- 0 0x0 Mach-O universal binary with 2 architectures 20364 0x4F8C Unix path: /usr/lib/dyld 24456 0x5F88 Unix path: /usr/lib/swift/libswiftObjectiveC.dylib 647500 0x9E14C XML document, version: "1.0" 778255 0xBE00F mcrypt 2.2 encrypted data, algorithm: blowfish-448, mode: CBC, keymode: 8bit 857010 0xD13B2 XML document, version: "1.0" 863590 0xD2D66 Certificate in DER format (x509 v3), header length: 4, sequence length: 1038 864632 0xD3178 Certificate in DER format (x509 v3), header length: 4, sequence length: 1204 865840 0xD3630 Certificate in DER format (x509 v3), header length: 4, sequence length: 1211 866515 0xD38D3 Certificate in DER format (x509 v3), header length: 4, sequence length: 260 867417 0xD3C59 XML document, version: "1.0" 889036 0xD90CC Unix path: /usr/lib/dyld 893128 0xDA0C8 Unix path: /usr/lib/swift/libswiftObjectiveC.dylib 1563692 0x17DC2C XML document, version: "1.0" 1728599 0x1A6057 mcrypt 2.2 encrypted data, algorithm: blowfish-448, mode: CBC, keymode: 8bit 1758178 0x1AD3E2 XML document, version: "1.0" 1764758 0x1AED96 Certificate in DER format (x509 v3), header length: 4, sequence length: 1038 1765800 0x1AF1A8 Certificate in DER format (x509 v3), header length: 4, sequence length: 1204 1767008 0x1AF660 Certificate in DER format (x509 v3), header length: 4, sequence length: 1211 1767683 0x1AF903 Certificate in DER format (x509 v3), header length: 4, sequence length: 260 1768585 0x1AFC89 XML document, version: "1.0"
This is a tad over my head but deepseek had some good info.
This is a fascinating and highly technical exploration of macOS internals, specifically related to Apple's Find My system, Airtags, and the security mechanisms in place. Let’s break this down and provide some guidance on how to proceed with the analysis, as well as some insights into the tools and techniques mentioned.
- Understanding the Context
You’re exploring the cache data and binaries related to Apple’s Find My system, including Airtags and Find My Friends. The goal seems to be decoding or extracting useful information from these files, such as:
Decoding the Items.data cache file in macOS 15+.
Extracting Apple user IDs and passwords from com.apple.account.AppleAccount.cloudkit-token.
Analyzing the fmfd (Find My Friends Daemon) binary for certificates and other embedded data.
- Decoding Items.data in macOS 15+
As you mentioned, the Items.data file in ~/Library/Caches/com.apple.findmy.fmipcore/ is now encoded in macOS 15+. Here’s how you can proceed:
Using plutil to Convert to XML
You’ve already identified the correct command to convert the binary .data file to XML:
sudo plutil -convert xml1 -o ~/Downloads/Items.txt ./Items.data
This will give you a more readable XML file, but the structure and content might still be cryptic. You’ll need to analyze the XML to understand the data structure.
Next Steps
Analyze the XML: Look for patterns or identifiable data (e.g., device IDs, timestamps, locations).
Compare with Older Versions: If you have access to an older macOS version, compare the XML structure to see what has changed.
Automate Parsing: Write a script (e.g., in Python) to parse the XML and extract meaningful data.
- Extracting Apple User ID and Password
You mentioned finding your Apple user ID and password in com.apple.account.AppleAccount.cloudkit-token, which is Base64 encoded for Basic Auth. Here’s how to proceed:
Steps to Decode
Locate the File: The file is likely in ~/Library/Accounts/ or a similar directory.
Extract the Token: Use a tool like base64 to decode the token:
echo "BASE64_ENCODED_STRING" | base64 --decode
Analyze the Output: The decoded string should contain your Apple ID and password (or a token). Be cautious with this data, as it’s sensitive.
Security Considerations
Keychain Access: macOS stores sensitive data in the Keychain. You might need to use the security command to access it:
security find-generic-password -a "AppleAccount" -s "cloudkit-token"
Permissions: You’ll need appropriate permissions (e.g., sudo) to access these files.
- Analyzing fmfd with binwalk
You’ve already started analyzing the fmfd binary using binwalk. Here’s how to proceed:
Extracting Embedded Data
Use the --dd option to extract all embedded files:
binwalk --dd='.*' /usr/libexec/fmfd
This will create a folder with extracted files, prefixed by an underscore.
Analyzing Certificates
The binwalk output shows several certificates in DER format. You can use openssl to decode these:
openssl x509 -inform der -in extracted_certificate.der -text
Look for details like issuer, subject, and validity periods to understand their purpose.
Hex and Byte Offset Analysis
Use a hex editor (e.g., xxd, hexdump, or a GUI tool like Hex Fiend) to inspect the binary at specific offsets.
Look for patterns, strings, or headers that indicate embedded data (e.g., XML, certificates, keys).
- Tools and Techniques
Here’s a summary of the tools and techniques you’ll need:
plutil: For converting binary property lists to XML.
base64: For decoding Base64-encoded strings.
security: For accessing macOS Keychain data.
binwalk: For analyzing and extracting embedded data from binaries.
openssl: For decoding and inspecting certificates.
Hex Editors: For manual analysis of binary files.
- Ethical and Legal Considerations
Privacy: Be mindful of the sensitive nature of the data you’re working with (e.g., Apple IDs, passwords, certificates).
Permissions: Ensure you have the right to access and analyze these files, especially on systems you don’t own.
Compliance: Respect Apple’s terms of service and applicable laws when reverse-engineering their software.
- Next Steps
Automate the Process: Write scripts to automate the extraction and decoding of data.
Document Findings: Keep detailed notes on the structure and content of the files you analyze.
Share Knowledge: If you discover something useful, consider sharing it with the community (while respecting privacy and legal boundaries).
This is a complex but rewarding project that combines reverse engineering, macOS internals, and security analysis. Let me know if you’d like help with specific steps or tools!
I honestly wouldnt be surprised if deepseek used my posts on other repos Issues and gists. some of those sentences I've said verbatim. See here.
Some good additional insights too. I'll keep investigating.
If you have a second mac, turn off automatic updates and keep on 14. Sequoia's Image Playground is crap and eats up so much space anyway. Pretty 'meh' release.
I’ve been following this for a while, and the best solution I found is to buy a $100 Mac and keep it on the last macOS version before Apple started encrypting the Find My cache files. I use it as a dedicated server, running 24/7, to continuously execute the script. That script updates a directory on my iCloud Drive, which syncs seamlessly with my other devices. I then built a very simple iOS app that retrieves and parses this data from the synced folder. The app allows me to select a time and date, and it returns the location history for that period on AppleMaps using the GPX structure generated by the script.
I’d love to add a couple of features to my app, like a heatmap and a paper trail, but I’m still new to Swift. I’ve been following others who are trying to decode the data through reverse engineering, and it seems like Apple is reinforcing the encryption with every update. It’s while to me—this is your own data.....................
I’ve been following this for a while, and the best solution I found is to buy a $100 Mac and keep it on the last macOS version before Apple started encrypting the Find My cache files. I use it as a dedicated server, running 24/7, to continuously execute the script. That script updates a directory on my iCloud Drive, which syncs seamlessly with my other devices. I then built a very simple iOS app that retrieves and parses this data from the synced folder. The app allows me to select a time and date, and it returns the location history for that period on AppleMaps using the GPX structure generated by the script.
I’d love to add a couple of features to my app, like a heatmap and a paper trail, but I’m still new to Swift. I’ve been following others who are trying to decode the data through reverse engineering, and it seems like Apple is reinforcing the encryption with every update. It’s while to me—this is your own data.....................
Yea your right, I have been working on decrypting sequoia Items.data for a while now and can't see how to get the decryption key so giving up for now. I decided to build a bunch of VMs running older Macos version for now (Monterey works great, Sonoma is a bit of a challenge under VirtualBox). Have 5 VMs running under VirtualBox on a Windows PC (16GB memory 2.3Ghz cpu). I am harvesting data on 160 airtags which I feed in to my django server and serve the data via Rest API to my iOS and android app clients. Works well so far. Saves having to buy and maintain a bunch of Macs!
@wilkyconsultants - which setup to do use to create the VM?
@wilkyconsultants - which setup to do use to create the VM?
Most Monterey ISO's on archive.org works fine with VirtualBox with default set up. If anyone manages to get past a boot loop panic on Sonoma 14.3.1 in VirtualBox please let me know. Tried everything..
The best recommendation is to buy a cheap Mac that supports macOS Sonoma 14.3.1 (minimum models: MacBook Air 2018, MacBook Pro 2018, Mac mini 2018, iMac 2019, or newer).
Install macOS Sonoma 14.3.1, the last version before Apple started encrypting Find My cache files.
This allows you to read AirTag and shared item locations directly from:
~/Library/Caches/com.apple.findmy.fmipcore/Items.data
~/Library/Caches/com.apple.findmy.fmipcore/Devices.data
With this setup, you can still extract the data in plain format using available scripts.
In newer macOS versions, reading this data is practically impossible due to the encryption implemented by Apple.
The best recommendation is to buy a cheap Mac that supports macOS Sonoma 14.3.1 (minimum models: MacBook Air 2018, MacBook Pro 2018, Mac mini 2018, iMac 2019, or newer). Install macOS Sonoma 14.3.1, the last version before Apple started encrypting Find My cache files. This allows you to read AirTag and shared item locations directly from:
~/Library/Caches/com.apple.findmy.fmipcore/Items.data ~/Library/Caches/com.apple.findmy.fmipcore/Devices.dataWith this setup, you can still extract the data in plain format using available scripts. In newer macOS versions, reading this data is practically impossible due to the encryption implemented by Apple.
I am using a Mac Mini 2012 with Sonoma 14.3.1 for my data collection along with a bunch of VMs running Monterey. Sonoma is better as it can access shared tags as well but Monterey is a good fall back since it can run in a small Macos VM under windows. With a 16 GB memory on a windows box you can run 5 VMs with no issues getting you 5 x 32 tags = 160 on 1 box!
Hi Henrik, please see below: