icalendarExtract.ipynb

icalendarextract.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "icalendarExtract.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyN0lpMfxz/Su3/WPeciLGsB",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/lsloan/42eb5afd59b7387059448b8c6fa67500/icalendarextract.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9n1XT1lPB3EV"
      },
      "source": [
        "# iCalendar Extract\n",
        "\n",
        "Get data from a file of calendar events stored in the [iCalendar format](https://en.wikipedia.org/wiki/ICalendar)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pL51a0kFFaCU"
      },
      "source": [
        "## Documentation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sh4QBEyZ6zBg"
      },
      "source": [
        "### About\n",
        "\n",
        "I had a file of calendar events exported from Google Calendar that I wanted to analyze.  Essentially, find all events after a certain date and total the durations of all of them.  Google Calendar didn't offer an easy way to do this and I wasn't comfortable with submitting my calendar data to any of the free online tools that claimed to do this.\n",
        "\n",
        "So, I wrote my own tool using Python and I'm disseminating it via Google Colab.\n",
        "\n",
        "### To Use\n",
        "\n",
        "1. Make a copy of this notebook.\n",
        "\n",
        "    Either click the \"Copy to Drive\" button in the toolbar  at the top of the page, or under the \"File\" menu, select \"Save a copy in Drive\".\n",
        "\n",
        "    The new notebook will be owned by you.\n",
        "\n",
        "1. Upload an iCalendar data file.\n",
        "\n",
        "    iCalendar files usually have filename extensions like `.ical`, `.ics`, `.ifb`, or `.icalendar`.\n",
        "\n",
        "    Upload the iCalendar file into the new notebook's \"Files\" section in the left pane, naming it \"`calendarDump.ics`\".\n",
        "\n",
        "1. Optional: Set a begin date.\n",
        "\n",
        "    If a begin date is needed, set one by uncommenting the second definition of `beginDate` in the [main program](#scrollTo=yVq9zBYBAj4C) below, in the \"Code\" section.  Specifying a year, month, and day.\n",
        "    \n",
        "    If no begin date is needed, the line for \"earliest possible date\" will be used by default.\n",
        "\n",
        "1. Run it.\n",
        "\n",
        "    Under the \"Runtime\" menu, select \"Run all\".\n",
        "\n",
        "### More\n",
        "\n",
        "To see more technical documentation about this notebook, go to the \"[Additional Documentation](#scrollTo=7J7L924zemxa)\" section below.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d32CC0yuFMpe"
      },
      "source": [
        "## Code"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HpbfVs1j6tup",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "43c87e0a-42c4-404b-d5a6-c4746b596b56"
      },
      "source": [
        "%pip install icalendar"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting icalendar\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/c5/14/85ffbeefb591e8be0d51e4611750653549f60b27221ce6b0f9c724b3f5c0/icalendar-4.0.7-py2.py3-none-any.whl (74kB)\n",
            "\r\u001b[K     |████▍                           | 10kB 11.1MB/s eta 0:00:01\r\u001b[K     |████████▉                       | 20kB 14.5MB/s eta 0:00:01\r\u001b[K     |█████████████▎                  | 30kB 17.1MB/s eta 0:00:01\r\u001b[K     |█████████████████▋              | 40kB 17.1MB/s eta 0:00:01\r\u001b[K     |██████████████████████          | 51kB 5.7MB/s eta 0:00:01\r\u001b[K     |██████████████████████████▌     | 61kB 6.3MB/s eta 0:00:01\r\u001b[K     |██████████████████████████████▉ | 71kB 5.4MB/s eta 0:00:01\r\u001b[K     |████████████████████████████████| 81kB 3.8MB/s \n",
            "\u001b[?25hRequirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from icalendar) (2018.9)\n",
            "Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/dist-packages (from icalendar) (2.8.1)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil->icalendar) (1.15.0)\n",
            "Installing collected packages: icalendar\n",
            "Successfully installed icalendar-4.0.7\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yVq9zBYBAj4C",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "50091371-b1fa-4457-927d-91db8cd948ba"
      },
      "source": [
        "from datetime import datetime, timedelta\n",
        "\n",
        "import pytz\n",
        "from icalendar import Calendar, Event\n",
        "\n",
        "with open('calendarDump.ics', 'r') as calendarFile:\n",
        "    calendarData = calendarFile.read()\n",
        "\n",
        "calendar: Calendar = Calendar.from_ical(calendarData)\n",
        "\n",
        "# May not be the \"proper\" way to get the time zone, but works with data \n",
        "# from Google Calendar.  Running this in Google Colab, we're unable to\n",
        "# get the user's time zone from the browser, so this is the alternative.\n",
        "timezone = pytz.timezone(calendar.get('X-WR-TIMEZONE', 'US/Eastern'))\n",
        "print(f'Using time zone \"{timezone.zone}\"...\\n')\n",
        "\n",
        "# earliest possible date\n",
        "beginDate = datetime(datetime.min.year, 1, 1, tzinfo=pytz.UTC)\n",
        "# beginDate = datetime(2021, 1, 6, tzinfo=pytz.UTC)\n",
        "\n",
        "durationTotal = timedelta(0)\n",
        "\n",
        "# In this loop, get only Events, not other types which may have been read\n",
        "# from the calendar data, and sort them by their starting date and time.\n",
        "event: Event\n",
        "for event in sorted((_ for _ in calendar.subcomponents\n",
        "                     if isinstance(_, Event)\n",
        "                        and _.decoded('dtstart') >= beginDate),\n",
        "                    key=lambda _: _.decoded('dtstart')):\n",
        "    print(f'Summary: \"{event[\"summary\"]}\"')\n",
        "    print(f'Description: \"{event[\"description\"]}\"')\n",
        "    start = event.decoded('dtstart')\n",
        "    end = event.decoded('dtend')\n",
        "    duration = end - start\n",
        "    durationTotal += duration\n",
        "    print(f'{start.astimezone(timezone)}; {end.astimezone(timezone)}; {duration}')\n",
        "    print('- ' * 30)\n",
        "\n",
        "print(f'Total: {durationTotal}')\n"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using time zone \"US/Eastern\"...\n",
            "\n",
            "Summary: \"Networld+Interop Conference\"\n",
            "Description: \"Networld+Interop Conference\"\n",
            "1996-09-18 10:30:00-04:00; 1996-09-20 18:00:00-04:00; 2 days, 7:30:00\n",
            "- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \n",
            "Summary: \"Calendaring Interoperability Planning Meeting\"\n",
            "Description: \"Discuss how we can test c&s interoperability\n",
            "\"\n",
            "1997-03-24 07:30:00-05:00; 1997-03-24 16:00:00-05:00; 8:30:00\n",
            "- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \n",
            "Summary: \"XYZ Project Review\"\n",
            "Description: \"Project XYZ Review Meeting\"\n",
            "1998-03-12 08:30:00-05:00; 1998-03-12 09:30:00-05:00; 1:00:00\n",
            "- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - \n",
            "Total: 2 days, 17:00:00\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7J7L924zemxa"
      },
      "source": [
        "## Additional documentation\n",
        "\n",
        "For people interested in slightly more technical details."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_YTW6gfzeZWp"
      },
      "source": [
        "### Possible Development\n",
        "\n",
        "Some possible improvements:\n",
        "\n",
        "1. Better documentation.\n",
        "1. Consider using `icalevents` Python module to make reading files easier and add support for reading via URL.\n",
        "1. User interface elements (GUI) for picking a calendar data file, specifying a start date, and any other inputs.\n",
        "1. Filter calendar data by the events' \"`summary`\" (name) or the \"`description`\".\n",
        "1. Compute a bill for work done at an hourly pay rate.\n",
        "\n",
        "### About the `icalendar` Module\n",
        "\n",
        "This notebook uses Python's package manager to install the `icalendar` module.  I'm not excited about that module, but it's been maintained relatively recently and it does the job.  I can think of several things I would do differently.  If I had the time to work on it, I'd propose to the author some changes and offer to implement them.  If the author weren't interested, I might write a new module that subclasses it.\n",
        "\n",
        "Some ideas:\n",
        "\n",
        "1. Data types have `from_ical()` and `to_ical()` methods, but they should also have `decoded()` methods.\n",
        "1. Subcomponents should have properties or methods for accessing their common properties.  It's a good practive to avoid using strings in code where they're not necessary.  For example:\n",
        "    ```python\n",
        "    event.summary # not: event['summary']\n",
        "    event.description # not: event['description']\n",
        "    event.dtstart.decoded() # not: event.decoded('dtstart')\n",
        "    event.dtend.decoded() # not: event.decoded('dtend')\n",
        "    ```\n",
        "\n",
        "    The last two example lines actually combine the first idea from above.\n",
        "1. Support reading from a file by passing a file object to the `Calendar.from_ical()` method.\n",
        "1. Include filtering and sorting methods for convenience, but don't interfere with existing DIY techniques.\n",
        "\n",
        "### View in GitHub\n",
        "\n",
        "This notebook is also available as a GitHub gist.\n",
        "See it at:\n",
        "\n",
        "* https://gist.github.com/lsloan/42eb5afd59b7387059448b8c6fa67500"
      ]
    }
  ]
}

I just added support for using the time zone specified in the calendar data. This should make it easier to understand the dates and times displayed.