poor man's webcomponents, circa May 2010

genhtml.rb

# genhtml.rb

require 'rubygems'
require 'xml/libxml'
require 'libxslt'

xml_file = ARGV[0]

stylesheet_doc = XML::Document.file('homework.xsi')
stylesheet = LibXSLT::XSLT::Stylesheet.new(stylesheet_doc)
xml_doc = XML::Document.file(xml_file)
result = stylesheet.apply(xml_doc)
File.open("writeup.html",'w') {|f| f.puts result}

homework.css

body { font-size: 10pt; font-family: Verdana, sans-serif; }

div#center { background-color: #eee; padding: 1em; margin: 10px auto; 
width: 960px; border: 1px solid #999; }

h2 { color: #333; padding: .25em; font-size: 12pt; border-top: 1px solid #666; }

h3 { color: #333; border-top: 1px dotted #999; }

div.stanza { white-space: pre; text-align: right;}
div.part { padding-bottom: .5em; }
div.part div.part-content { margin: 0% 5%; }

.answer { padding: .5em 0; font-weight: bold; }

div.part li { margin-bottom: 0.5em; }
div.toc li { margin-bottom: 0.5em; }
div.toc a:link, div.toc a:visited, div.toc a:active { color: #3333aa; text-decoration: none; }
div.toc a:hover { color: #aa6666; text-decoration: underline; }

.tests { white-space: pre; #display: none; padding: 1em; background-color: #999; }
.tests .pass { background-color: #99ff99; }
.tests .fail { background-color: #ff9999; }

.prompt { white-space: pre; color: #ddd; background-color: #555; margin: 0.5em 0; padding: 0.5em; font-family: monospace; }

.figure { text-align: center; font-style: italic; font-size: 9pt; padding: 1em; font-weight: normal; }
.figure.right { float: right;  }
.figure.center { margin: 0 auto; }
.figure.left { float: left; }
.figure.thumb img { max-height: 200px; max-width: 200px; }
.figure img { max-width: 100%; border: 1px solid #999; }
.figure a:link, .figure a:visited, .figure a:active { color: #333; text-decoration: none; }
.figure a:hover { color: #aa6666; }

homework.xml

<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet type="text/xsl" href="homework.xsi"?>
<homework>
<stanza>
  <name>Larry Kyrala</name>
  <assignment>Homework 5</assignment>
  <date>March 6, 2010</date>
  <professor>Professor Pfister</professor>
  <class>CS 171 Visualization</class>
</stanza>
<part id="part1" name="Part 1: Exploration with Tableau (40%)">
Using the data file <link>2009-10-CS171-Survey.xls</link> and 
<a href="http://www.tableausoftware.com/public/training">Tableau public</a>, 
answer the following questions:
<ol>
  <li><question>What kind of computers do younger programmers use?</question>
    <answer>
    <p>It appears that younger people love Mac OS X.</p>
    <figure image="answer1.jpg" style="center"/>
    However, as I pointed out on the forums, it seems that multiple choices in the "operating system(s)" category
    on the survey were extracted in field order, hence for students entering multiple operating systems *including*
    Windows XP, they actually would have been counted as only Windows XP.  This may actually have introduced a 
    substantial bias in the results as charted.
    </answer></li>
  <li><question>What operating systems do more comfortable programmers use?</question>
    <answer>
    <p>To make this more interesting I'll interpret "comfortable" programmers as years of experience. (snark! ;)
    But hmmm, this is interesting too:</p>
    <figure image="answer2.jpg" style="center"/>
    <p>What's going on here?  Here's my take: this shows that as programmers become more experienced
    they seem to have a greater breadth of experience with a variety of operating systems.  However, we
    also see that in older programmers who are inexperienced, they tend to favor the older operating 
    systems, unless they have been coding for less than 6 months.</p>
    <p>This is where it would be useful to get all the OSes that people selected instead of just the 
    first selected field from the original form.  ;)</p>
    </answer></li>
  <li><question>What is the relationship between how long one has been programming 
and the frequency of coding?</question>
    <answer>
    It seems that if you have been programming for a long time, you likely do it more frequently than others.
    <figure image="answer3.jpg" style="center"/>
    </answer></li>
  <li><question>What is the relationship between concentration and primary programming 
language, and how does it change over time (2009 / 2010)?</question>
    <answer>
    I had to group some of the fields together to be fair (there were near dups like "x &amp; y" vs. "x and y", etc.)
    After I got done with that, I sorted them by counts decending in two directions to get this yearly trend:
    <figure image="answer4.jpg" style="center"/>
    My interpretation of this is that Comp Sci dominates in the number of languages (with C showing a definate uptick),
    while the C programming language dominates across concentrations.  However, it is important to remember that
    CS 171 is a "self-qualifying" population -- i.e. we expect to see a greater proportion of CS-savy students than 
    may be represented within the general populations for each concentration. The concentration categories could also
    be more consistent in the source data if they were constrained instead of being arbitrary text.
    </answer></li>
  <li><question>What is the relationship between age, programming language, and 
programming experience, and how does it change over time (2009 / 2010)?</question>
    <answer>
    This is interesting.  I mapped the top 5 languages by count to colors in descending order and laid them out thus:
    <figure image="answer5.jpg" style="center"/>
    So, this shows a pretty solid trend in CS 171 among the young coders who are just starting out and (interestingly)
    those who have been coding for a while) in the C language.  It's also fairly popular among them in the intermediate
    experience categories, however they are holding steady.  Older experienced coders are showing a preference for
    Java with a slight uptick.  It's also interesting although harder to see in this view that all ranges of
    older coders are using HTML/CSS, while only the more experienced younger coders appear to be using it.
    </answer></li>
</ol>
</part>
<part id="part2" name="Part 2: Designing an Interaction Using Classmate Data (60%)">
<p>I chose to model the second question.</p>
<h4>The Application:</h4>
<p>The sketch can be found here: <link>survey/applet/index.html</link></p>
<figure image="screenshot.jpg" style="center" link="survey/applet/index.html">Survey Sketch</figure>
<h4>Instructions:</h4>
<ul>
  <li>MOUSEWHEEL UP/DOWN or keyboard LEFT/RIGHT - cycle through experience levels.</li>
  <li>tooltips are supported to show counts and OS... simply mouse over the circle of interest.</li>
</ul>
<h4>Parameters:</h4>
<ul>
  <li>dimension (ordinal) <b>Age</b> -- it's ordinal because of the bins.</li>
  <li>dimension (nominal) <b>What operating system(s) do you run on your computer(s)?</b></li>
  <li>dimension (ordinal) <b>How long have you been programming?</b></li>
  <li>measure (quantitative) count of the slice</li>
</ul>
<h4>Principles:</h4>
<ul>
  <li>size is best mapped to a qualitative value, hence I assigned it to the count of the slice -- "slice" 
  in this case is the unique combination of age, os and experience at any given point on the plot.</li>
  <li>animation is a good way to correlate change over time.  By introducing a linear interpolation between
  experience levels as the user cycles through them, it is much easier for the user to see how values have
  changed (inspiration: Hans Rosling's TED talk).</li>
  <li>color is used categorically which is ideal for Operating System since there are only 5 values.
  The specific colors were matched to Tableau's palette so that the comparison between my Tableau screenshots
  and this applet wouldn't be jarring.  i.e. the viewer can take the color mapping they learned from 
  part 1 and reuse it in part 2.</li>
  <li>count is also mapped to the vertical axis because size isn't great for communicating 
  small quantitative changes by itself. This is a reinforcing channel.</li>
  <li>highlighting moused over points provides useful feedback so the user knows which object
  is displaying the tooltip in tight clusters.</li>
</ul>
<h4>Discussion:</h4>
<p>I added a slight variance to the os plot so that data points wouldn't obscure each other.</p>
<p>It's interesting that Windows XP rises in significance among the population as experience increases. 
While Mac seems to be the favorite of inexperienced younger people.</p>
<p>However, I think this is a naive interpretation... there are some interesting things going on:</p>
<ol>
  <li>Because only the first os in the form was considered (i.e. if students select more than one) there is
  a substantial ordering bias on the data collected.  Indeed, "Windows XP" is the first OS on the form.  I think
  that explains the staggering bias towards XP, but why then are the young relatively unaffected?</li>
  <li>Well, here's the interesting part. It's fairly safe to say that the younger and less experienced you
  are, you are more likely to have used only one computer system, while more experienced programmers may use several.
  Hence, among the younger less experienced population, I believe we are seeing a much more 
  representative sampling of OSes used because they are likely demographic to use only one machine.  As we increase the
  experience, we can see that the Windows XP bias is overwhelming -- this seems to affect the older programmers
  the most (25-44).</li>
</ol>
</part>
</homework>

homework.xslt

<?xml version="1.0" encoding="ISO-8859-1"?>
<xsl:stylesheet version="1.0" 
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:xhtml="http://www.w3.org/1999/xhtml">
<xsl:output method="html" indent="yes" version="4.0"/>

<xsl:template match="/">
  <html>
    <title><xsl:value-of select="/homework/stanza/assignment"/></title>
    <link href="homework.css" rel="stylesheet" type="text/css"/>
  <body>
  <div class="alpha-shadow">
  <div id="center">
  <xsl:for-each select="/homework/stanza">
    <div class="stanza"><xsl:value-of select="."/></div>
  </xsl:for-each>
    <div class="toc">
      <h2>Table of Contents</h2>
      <ul>
      <xsl:for-each select="/homework/part">
        <li><a href="#{@id}"><xsl:value-of select="@name"/></a></li>
        <xsl:if test="subpart"><ul>
        <xsl:for-each select="subpart">
          <li><a href="#{@id}"><xsl:value-of select="@name"/></a></li>
        </xsl:for-each>
        </ul></xsl:if>
      </xsl:for-each>
      </ul>
    </div>
  <xsl:for-each select="/homework/part">
    <div class="part">
      <a name="{@id}"/>
      <h2><xsl:value-of select="@name"/></h2>
      <div class="part-content"><xsl:apply-templates/></div>
    </div>
  </xsl:for-each>
  </div>
  </div>
  </body>
  </html>
</xsl:template>

<xsl:template match="subpart">
    <a name="{@id}"/>
    <h3><xsl:value-of select="@name"/></h3>
    <div class="subpart-content"><xsl:apply-templates/></div>
</xsl:template>

<xsl:template match="question"><div class="question"><xsl:apply-templates/></div></xsl:template>
<xsl:template match="answer"><div class="answer"><xsl:apply-templates/></div></xsl:template>
<xsl:template match="tests"><div class="tests">Tests:<xsl:apply-templates/></div></xsl:template>
<xsl:template match="pass"><span class="pass"><xsl:apply-templates/></span></xsl:template>
<xsl:template match="fail"><span class="fail"><xsl:apply-templates/></span></xsl:template>

<xsl:template match="prompt"><div class="prompt"><xsl:apply-templates/></div></xsl:template>

<xsl:template match="link"><a href="{.}"><xsl:value-of select="."/></a></xsl:template>
<xsl:template match="figure"><div class="figure {@style}">
  <xsl:choose>
    <xsl:when test="@link"><a href="{@link}" target="_blank"><img src="images/{@image}"/><br/><xsl:value-of select="."/></a></xsl:when>
    <xsl:otherwise><a href="images/{@image}" target="_blank"><img src="images/{@image}"/><br/><xsl:value-of select="."/></a></xsl:otherwise>
  </xsl:choose>
</div></xsl:template>



<!-- identity template -->
<xsl:template match="@*|node()">
    <xsl:copy>
        <xsl:apply-templates select="@*|node()"/>
    </xsl:copy>
</xsl:template>

</xsl:stylesheet>

writeup.html

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html xmlns:xhtml="http://www.w3.org/1999/xhtml">
  <title>Homework 5</title>
  <link href="homework.css" rel="stylesheet" type="text/css"/>
  <body>
    <div class="alpha-shadow">
      <div id="center">
        <div class="stanza">
  Larry Kyrala
  Homework 5
  March 6, 2010
  Professor Pfister
  CS 171 Visualization
</div>
        <div class="toc">
          <h2>Table of Contents</h2>
          <ul>
            <li>
              <a href="#part1">Part 1: Exploration with Tableau (40%)</a>
            </li>
            <li>
              <a href="#part2">Part 2: Designing an Interaction Using Classmate Data (60%)</a>
            </li>
          </ul>
        </div>
        <div class="part">
          <a name="part1"/>
          <h2>Part 1: Exploration with Tableau (40%)</h2>
          <div class="part-content">
Using the data file <a href="2009-10-CS171-Survey.xls">2009-10-CS171-Survey.xls</a> and 
<a href="http://www.tableausoftware.com/public/training">Tableau public</a>, 
answer the following questions:
<ol>
  <li><div class="question">What kind of computers do younger programmers use?</div>
    <div class="answer">
    <p>It appears that younger people love Mac OS X.</p>
    <div class="figure center"><a href="images/answer1.jpg" target="_blank"><img src="images/answer1.jpg"/><br/></a></div>
    However, as I pointed out on the forums, it seems that multiple choices in the "operating system(s)" category
    on the survey were extracted in field order, hence for students entering multiple operating systems *including*
    Windows XP, they actually would have been counted as only Windows XP.  This may actually have introduced a 
    substantial bias in the results as charted.
    </div></li>
  <li><div class="question">What operating systems do more comfortable programmers use?</div>
    <div class="answer">
    <p>To make this more interesting I'll interpret "comfortable" programmers as years of experience. (snark! ;)
    But hmmm, this is interesting too:</p>
    <div class="figure center"><a href="images/answer2.jpg" target="_blank"><img src="images/answer2.jpg"/><br/></a></div>
    <p>What's going on here?  Here's my take: this shows that as programmers become more experienced
    they seem to have a greater breadth of experience with a variety of operating systems.  However, we
    also see that in older programmers who are inexperienced, they tend to favor the older operating 
    systems, unless they have been coding for less than 6 months.</p>
    <p>This is where it would be useful to get all the OSes that people selected instead of just the 
    first selected field from the original form.  ;)</p>
    </div></li>
  <li><div class="question">What is the relationship between how long one has been programming 
and the frequency of coding?</div>
    <div class="answer">
    It seems that if you have been programming for a long time, you likely do it more frequently than others.
    <div class="figure center"><a href="images/answer3.jpg" target="_blank"><img src="images/answer3.jpg"/><br/></a></div>
    </div></li>
  <li><div class="question">What is the relationship between concentration and primary programming 
language, and how does it change over time (2009 / 2010)?</div>
    <div class="answer">
    I had to group some of the fields together to be fair (there were near dups like "x &amp; y" vs. "x and y", etc.)
    After I got done with that, I sorted them by counts decending in two directions to get this yearly trend:
    <div class="figure center"><a href="images/answer4.jpg" target="_blank"><img src="images/answer4.jpg"/><br/></a></div>
    My interpretation of this is that Comp Sci dominates in the number of languages (with C showing a definate uptick),
    while the C programming language dominates across concentrations.  However, it is important to remember that
    CS 171 is a "self-qualifying" population -- i.e. we expect to see a greater proportion of CS-savy students than 
    may be represented within the general populations for each concentration. The concentration categories could also
    be more consistent in the source data if they were constrained instead of being arbitrary text.
    </div></li>
  <li><div class="question">What is the relationship between age, programming language, and 
programming experience, and how does it change over time (2009 / 2010)?</div>
    <div class="answer">
    This is interesting.  I mapped the top 5 languages by count to colors in descending order and laid them out thus:
    <div class="figure center"><a href="images/answer5.jpg" target="_blank"><img src="images/answer5.jpg"/><br/></a></div>
    So, this shows a pretty solid trend in CS 171 among the young coders who are just starting out and (interestingly)
    those who have been coding for a while) in the C language.  It's also fairly popular among them in the intermediate
    experience categories, however they are holding steady.  Older experienced coders are showing a preference for
    Java with a slight uptick.  It's also interesting although harder to see in this view that all ranges of
    older coders are using HTML/CSS, while only the more experienced younger coders appear to be using it.
    </div></li>
</ol>
</div>
        </div>
        <div class="part">
          <a name="part2"/>
          <h2>Part 2: Designing an Interaction Using Classmate Data (60%)</h2>
          <div class="part-content">
<p>I chose to model the second question.</p>
<h4>The Application:</h4>
<p>The sketch can be found here: <a href="survey/applet/index.html">survey/applet/index.html</a></p>
<div class="figure center"><a href="survey/applet/index.html" target="_blank"><img src="images/screenshot.jpg"/><br/>Survey Sketch</a></div>
<h4>Instructions:</h4>
<ul>
  <li>MOUSEWHEEL UP/DOWN or keyboard LEFT/RIGHT - cycle through experience levels.</li>
  <li>tooltips are supported to show counts and OS... simply mouse over the circle of interest.</li>
</ul>
<h4>Parameters:</h4>
<ul>
  <li>dimension (ordinal) <b>Age</b> -- it's ordinal because of the bins.</li>
  <li>dimension (nominal) <b>What operating system(s) do you run on your computer(s)?</b></li>
  <li>dimension (ordinal) <b>How long have you been programming?</b></li>
  <li>measure (quantitative) count of the slice</li>
</ul>
<h4>Principles:</h4>
<ul>
  <li>size is best mapped to a qualitative value, hence I assigned it to the count of the slice -- "slice" 
  in this case is the unique combination of age, os and experience at any given point on the plot.</li>
  <li>animation is a good way to correlate change over time.  By introducing a linear interpolation between
  experience levels as the user cycles through them, it is much easier for the user to see how values have
  changed (inspiration: Hans Rosling's TED talk).</li>
  <li>color is used categorically which is ideal for Operating System since there are only 5 values.
  The specific colors were matched to Tableau's palette so that the comparison between my Tableau screenshots
  and this applet wouldn't be jarring.  i.e. the viewer can take the color mapping they learned from 
  part 1 and reuse it in part 2.</li>
  <li>count is also mapped to the vertical axis because size isn't great for communicating 
  small quantitative changes by itself. This is a reinforcing channel.</li>
  <li>highlighting moused over points provides useful feedback so the user knows which object
  is displaying the tooltip in tight clusters.</li>
</ul>
<h4>Discussion:</h4>
<p>I added a slight variance to the os plot so that data points wouldn't obscure each other.</p>
<p>It's interesting that Windows XP rises in significance among the population as experience increases. 
While Mac seems to be the favorite of inexperienced younger people.</p>
<p>However, I think this is a naive interpretation... there are some interesting things going on:</p>
<ol>
  <li>Because only the first os in the form was considered (i.e. if students select more than one) there is
  a substantial ordering bias on the data collected.  Indeed, "Windows XP" is the first OS on the form.  I think
  that explains the staggering bias towards XP, but why then are the young relatively unaffected?</li>
  <li>Well, here's the interesting part. It's fairly safe to say that the younger and less experienced you
  are, you are more likely to have used only one computer system, while more experienced programmers may use several.
  Hence, among the younger less experienced population, I believe we are seeing a much more 
  representative sampling of OSes used because they are likely demographic to use only one machine.  As we increase the
  experience, we can see that the Windows XP bias is overwhelming -- this seems to affect the older programmers
  the most (25-44).</li>
</ol>
</div>
        </div>
      </div>
    </div>
  </body>
</html>

I've been digging into WebComponents custom elements and been very impressed with how it disposes with non-semantic divs and other extraneous markup that should really be part of the presentation layer (CSS, etc.). WebComponents is the future.

But it also reminds me of much earlier (and perhaps klunkier) experiments that I was doing with XSLT that were extensions of microformats to something very like custom elements. I always had an idea to gemify some kind of Ruby DSL that would make defining the transformations easier than dealing with raw XSLT, but now WebComponents pretty much does that standard! So awesome!

Anyway, since my school pages disappeared after graduation, here are the source files that I refer to in my blog post talking about 'deep-linking' and personal experiments of semantic content in the web: Ben Ward says "Link to it!!".

Here's a rough description of the files above and how they were used:

   genhtml.rb - Ruby file to run the pipeline as follows:
 homework.xml - 1) first take the semantic xml (i.e. custom elements)
homework.xslt - 2) run it through the xlst which produces...
 writeup.html - 3) the final output
 homework.css - and some css to make it look pretty

(images and other source omitted for brevity)