Skip to content

Instantly share code, notes, and snippets.

@mrvdb
Last active February 5, 2020 07:38
Show Gist options
  • Save mrvdb/c517f216b54facfc6ce9685720d331ba to your computer and use it in GitHub Desktop.
Save mrvdb/c517f216b54facfc6ce9685720d331ba to your computer and use it in GitHub Desktop.
#!/usr/bin/env python
# -*- mode:python -*-
#
# Converts suunto zip files (which contain json files) to a gpx file
# until Suunto gets their act together and let me have my own data in
# a normal way.
# Needs: python 3.7
#
# These zip files are producted by the Suunto app and typically on
# android are located at:
# <Internal Storage>/Android/data/com.stt.suunto/files/smlzip/
#
import sys
import gpxpy
import gpxpy.gpx
import json
import datetime
import zipfile
import math
# The relevant json snippet which is in a json array 'Samples'
# looks like this:
# {
# "Attributes": {
# "suunto/sml": {
# "Sample": {
# "GPSAltitude": 94, <-- assuming meters
# "Latitude": 0.88924328690504395, <-- in radians
# "Longitude": 0.10261437316962527, <-- in radians
# "UTC": "2019-04-18T07:25:29.000+00:00" <-- assumed what gps thinks of time?
# }
# }
# },
# "Source": "suunto-123456789", <-- number is the id of the device
# "TimeISO8601": "2019-04-18T09:25:29.000+02:00" <-- assumed what watch thinks of time?
# },
# Suunto specific stuff
SUUNTO_SAMPLES = 'samples.json'
def gpx_track(zip):
# Three keys make a trackpoint
TRKPT = ('Latitude', 'Longitude', 'UTC')
# Create the main gpx object
gpx = gpxpy.gpx.GPX()
# We are getting one track presumably from the points we recorded
track = gpxpy.gpx.GPXTrack()
gpx.tracks.append(track)
# Points are added to segments (take breaks into account later?)
segment = gpxpy.gpx.GPXTrackSegment()
track.segments.append(segment)
# Read in the json file
with zip.open(SUUNTO_SAMPLES, 'r') as json_data:
suunto_json = json.load(json_data)
for s in suunto_json["Samples"]:
if "Sample" in s["Attributes"]["suunto/sml"]:
sample = s["Attributes"]["suunto/sml"]["Sample"]
# See if we have enough for a trackpoint
if all(key in sample for key in TRKPT):
# Lat and long are radians
# To decimal degrees = * 180 / PI
lat = (sample["Latitude"] * 180) / math.pi
lon = (sample["Longitude"] * 180) / math.pi
# I'm assuming this is in meters, and we can use it as is
ele = sample['GPSAltitude'] if 'GPSAltitude' in sample else 0
time = datetime.datetime.fromisoformat(sample['UTC'])
# Create the gpx point
point = gpxpy.gpx.GPXTrackPoint(latitude=lat, longitude=lon,
time=time, elevation=ele)
segment.points.append(point)
# Write out the gpx file
with open(zip.filename.replace(".zip", ".gpx"), "w") as out:
out.write(gpx.to_xml())
if __name__ == "__main__":
# Argument is expected to be a zip file
with zipfile.ZipFile(sys.argv[1]) as zip:
# There should be a samples.json in the zip file
if SUUNTO_SAMPLES in zip.namelist():
gpx_track(zip)
else:
print("No track samples found in zip file!")
@mrvdb
Copy link
Author

mrvdb commented Oct 15, 2019

@acdcrix I sent you an email, did you receive that? The issue is that the content of the zip file seems to be different from the time I created this script. If you sent me a copy of the zip, I can take a look.

@acdcrix
Copy link

acdcrix commented Oct 15, 2019 via email

@mrvdb
Copy link
Author

mrvdb commented Oct 15, 2019

Ah, I see there are new records at the end of type 'R-R' Not sure what that is, but the check in the python script is then needed indeed.

I'll add it to the script so others can find it.

@wiktorn
Copy link

wiktorn commented Dec 31, 2019

R-R data is raw data about heart rate. IBI is Inter-Beat-Interval, I guess that is in milliseconds. Based on that, it's possible to calculate BPM, though it's not that easy, because of the way it's represented in file and probably it needs some smoothing to account for missing/spurious beats.

There are also additional data available, that could fit into gpx file.

Add namespace declaration for track point extensions:

gpx.nsmap['gpxtpx'] = 'http://www.garmin.com/xmlschemas/TrackPointExtension/v2'

then after opening zip file, prepare extra attributes:

    with zip.open(SUUNTO_SAMPLES, 'r') as json_data:
        suunto_json = json.load(json_data)

        attributes_data = {}
        for s in suunto_json["Samples"]:
            ex_attributes = s.get("Attributes", {}).get("suunto/sml", {}).get("Sample", {})
            if 'Altitude' in ex_attributes:
                key = datetime.datetime.fromisoformat(s["TimeISO8601"]).replace(microsecond=0)
                attributes_data[key] = ex_attributes

And later, when processing samples:

                    # Create the gpx point
                    point = gpxpy.gpx.GPXTrackPoint(latitude=lat, longitude=lon,
                                                    time=time, elevation=ele)
                    # handle extension data
                    tpe = ET.Element('gpxtpx:TrackPointExtension')
                    ex_data = attributes_data.get(datetime.datetime.fromisoformat(s["TimeISO8601"]).replace(microsecond=0), {})
                    def add_ext(key, tag_name, value_conv=lambda x: x):
                        if key in ex_data and ex_data[key]:
                            tag = ET.SubElement(tpe, tag_name)
                            tag.text = value_conv(ex_data[key])
                            return True
                        return False

                    add_ext('Temperature', 'gpxtpx:atemp', str)
                    add_ext('Cadence', 'gpxtpx:cad', lambda x: str(x*60))
                    add_ext('Speed', 'gpxtpx:speed', str)
                    point.extensions.append(tpe)

                    segment.points.append(point)

Haven't found yet, what extensions cover speed, distance and power, that are also present.

Also, it might be better to index all by dates, join data, and later on, calculate necessary attributes.

@mrvdb
Copy link
Author

mrvdb commented Jan 6, 2020

Interesting additions. Perhaps I should turn this gist into a repository so we can have issues and pull requests. Am I correct in that the trackpointextension schema is Garmin specific or is that more widely supported?

Schema Doc is here: https://www8.garmin.com/xmlschemas/TrackPointExtensionv2.xsd

@mrvdb
Copy link
Author

mrvdb commented Jan 6, 2020

Script is now maintained at normal github repository at https://github.com/mrvdb/suuntoapp2gpx

@wiktorn
Copy link

wiktorn commented Jan 6, 2020

I've found similar extensions with different namespace
Namespace: http://www.cluetrust.com/XML/GPXDATA/1/0
Schema doc: http://www.cluetrust.com/Schemas/gpxdata10.xsd

Not sure, which standard is more widespread. As far as I tested with Garmin BaseCamp, it only reads namespace: http://www.garmin.com/xmlschemas/TrackPointExtension/v1, and v2 is ignored.

Both Garmin and Cluetrust extensions are read by some popular trackers I've tested, I'm not sure if they at all verify namespaces of extensions :)

@mrvdb
Copy link
Author

mrvdb commented Jan 6, 2020

Could you create an issue in the new repository so we can continue discussing it there? I'll get the repo details like README etc. filled in later this week.

@mike-kilo
Copy link

Hi Marcel,

Great script, comes really useful for my mother's Suunto watch, the GPX generated was accepted by Strava.

I had huge problems installing Python 3.8 (I should update my system soon). I went on and modified your script a bit so it runs Python 3.5, see my fork.

Regards,

Mike

@mrvdb
Copy link
Author

mrvdb commented Feb 3, 2020

Hi @mike-kilo ,

Not sure if you missed it, but I'm keeping track of this script in a "real" repository now: https://github.com/mrvdb/suuntoapp2gpx

At the moment, the script isn't different from the gist though.

@mike-kilo
Copy link

Hi Marcel,

I did miss it indeed. Would you care for a pull request (for Python 3.4 - 3.6) or you'd rather stick to Python 3.7?

@mrvdb
Copy link
Author

mrvdb commented Feb 5, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment