Skip to content

Instantly share code, notes, and snippets.

@cwant

cwant/README.md

Last active April 19, 2023 19:27
Show Gist options
  • Save cwant/6578c922a1aa82b4fab6444fc1b3884a to your computer and use it in GitHub Desktop.
Save cwant/6578c922a1aa82b4fab6444fc1b3884a to your computer and use it in GitHub Desktop.
Download ZIP
20 hour D3 course -- some notes
Raw
README.md

Motivation

A few years ago I wrote a (mostly) JS website that aims to show the impact of resource asks on scheduling resources: https://slurm-wizard.uofa.c3.ca/

This was written mostly with Plotly and jQuery, and was not easy to write, in particular it was difficult to get Plotly to do what I wanted it to with it's high-level interface.

Plotly is written with D3, so I thought it might be time to explore this lower-level library.

So I looked into a 20 hour course on YouTube ...

Course/Author

  • Author: Curran Kelleher
  • His big project is VizHub, a playground for exploring JS based web technologies.
  • Course: https://www.youtube.com/watch?v=xkBheRZTkaw
  • Part of freeCodeCamp
  • 20 hours
  • The course has a very "live coding" feel about it (so if you like the Carpentries, you might like this approach)
  • I played the course on double speed, pausing often as I needed to work on code.
  • The course is broken into several lessons with exercises, with each lesson starting with a section dedicated to looking at student submitted exercise solutions from the previous lesson.

Unfortunately, I started writing these notes only after I finished the course, so my memory might be spotty, and my notes might not match the exact order things are presented in the course.

D3

From the website:

D3 allows you to bind arbitrary data to a Document Object Model (DOM), and then apply data-driven transformations to the document. For example, you can use D3 to generate an HTML table from an array of numbers. Or, use the same data to create an interactive SVG bar chart with smooth transitions and interaction.

D3 is not a monolithic framework that seeks to provide every conceivable feature. Instead, D3 solves the crux of the problem: efficient manipulation of documents based on data. This avoids proprietary representation and affords extraordinary flexibility, exposing the full capabilities of web standards such as HTML, SVG, and CSS. With minimal overhead, D3 is extremely fast, supporting large datasets and dynamic behaviors for interaction and animation. D3’s functional style allows code reuse through a diverse collection of official and community-developed modules.

D3 is quite low level when compared with Plotly -- in fact, Plotly is built on top of D3.

Works with SVG/HTML/CSS.

Like jQuery, has an infractructure for selecting elements in the DOM

D3's primary author is Mike Bostock, who wrote an influential article about the design philosophy "Towards reusable charts"

Working with ES6

  • I decided not to use VizHub, since I want charts that will work regardless of the platform that I will be working with.

Modules

  • The charts use ES6 modules, which I only have a passing familiarity with.
  • The course had exammples of using Rollup, but I primarly used webpack to bundle my JS assets (which makes ES6 modules work through transpilation, and is more similar to Ruby on Rails workflows that I'm familiar with)
  • So I primarily write a index.js, but my HTML loads in a bundle.js that is derived from index.js (this happens automatically at save using guard with a Guardfile)

NPM packages

  • I became familiar with working with a package.json file to install dependencies with NPM (e.g., d3)
  • I alternatively also worked with the unpkg CDN to load dependencies, e.g., https://unpkg.com/d3

Function syntax

Old style anonymous function:

function (x) { return x*x;}

New style:

(x) => {x*x}

So definiing a (non-anonymous) function looks like:

const f = (x) => {x*x}

These functions are often fed into mapping operations.

Working without D3

  • There were a few sections on how HTML, SVG and CSS inter-relate.
    • A section on how to hand craft SVG elements in HTML documents
    • exploring on creating charts with Figma, and exploring the SVG generated.
  • Some sections on how to make Sol LeWitt style art using non-D3 vanilla javascript to produce the SVG
  • The workflow is to create an empty web page and add a SVG element, then populate it with circles, rectancles and paths

e.g. https://vizhub.com/wolmoe/dd21b1750adc45128cc5d30fe612580d?edit=files&file=index.html

Working with D3

Sections on creating Sol LeWitt style art with D3 Again, creating SVG elements in an empty web page

e.g. https://vizhub.com/curran/9dea26a11f5344d78b46b69ec8ca3e7b?edit=files&file=index.html

On-going Scatterplot project

The main project is working with the Iris dataset. A D3 project is created that initially just shows a couple of dimensions of the data:

https://vizhub.com/curran/9e556254c18b4266a8da5306fbee7f22?edit=files&file=index.html

This evolves and is refactored to animate the data rendered (x values rotate through the data columns):

https://vizhub.com/curran/f1b1dbcb355f4d459b86e29d4f134d7c?edit=files&file=index.js

which leads into a discussion about idempotent actions:

  • Most parts (e.g., axes, data points only need to be created once)
  • Some parts then can be updated and not re-created.
  • We'd like to be able to run a scatter plot function multiple times and not have it always appending parts of the DOM that already exist.
  • D3 has support for code that runs when you "enter" (create), update, and "exit" (destroy) elements of the visualization. This can lead to animations.

The discussion finally gets to transitions and using menus:

https://vizhub.com/curran/8406495076e94ed3a82bae65cb5a05f5?edit=files&file=index.js

Selectors

This doesn't look too different from jQuery:

const svg = select('body')
      .append('svg')
      .attr('width', width)
      .attr('height', height);

Focus on breaking problems into data processing pipelines and visualization pipelines.

A map to turn rows from a CSV file into the JS objects we might want

const parseRow = (d) => {
  d.sepal_length = +d.sepal_length;
  d.sepal_width = +d.sepal_width;
  d.petal_length = +d.petal_length;
  d.petal_width = +d.petal_width;
  return d;
}

Setting up an entire pipeline (scatterPlot is something we write to generate the plot):

  const plot = scatterPlot()
        .width(width)
        .height(height)
        .data(await csv(csvUrl, parseRow))
        .xValue((d) => d.petal_width)
        .yValue((d) => d.sepal_length)
        .margin({
          top: 20,
          right: 20,
          bottom: 50,
          left: 120
        })
        .radius(5);

The above creates a function, so we run it on the SVG element we created:

svg.call(plot);
# Same as plot(svg)

The focus with this sort of design is on chaining operations and mapping.

Thoughts

  • I enjoyed the course. it was a good intro to D3, and some of the philosophy behind it. Curran has an extordinary level of expertise, and is a great teacher. The live coding approach worked well. The review of student-submitted work also had a lot of value.
  • Good overview of modern JS programming, and a good reason to practice setting up JS projects (beyond just copy/pasting boilerplate code, which has been my approach so far to my projects)
  • I would have liked to have more of a tour of D3 features.
  • I think D3 would be a good fit if you are doing dataviz all of the time and need a high level of control. I can see myself forgetting things if I only use the library infrequently.
  • I will probably stick with Plotly for most things (especially if I am coming from Python)
  • There are some D3 Python libraries that might be worth exploring
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment