Workshop: Observable Framework

workshop.md

Getting Started

Create a project

You can set up a new Observable Framework project using the official template by running this command (you should probably cd into the directory where you want the project to live first):

npx @observablehq/framework@latest create 

All of the default options it asks about should be fine, and it's definitely a good idea to let it initialize a git repository since it will make deploying using GitHub Pages easier below.

Run locally

Look at package.json and note the scripts object. Each can be run via npm run <name>.

To run an http server on localhost that will update when you modify the files in the project, run:

npm run dev

It will automatically open a browser window, but note that it's just a normal URL (albeit one that's pointing to localhost and using a non-standard port). If you want to open it in a different browser, just copy and paste the URL.

Topics

• Markdown basics
  • Frontmatter: fenced metadata at the top of the file
  • Basic syntax: headers, paragraphs, lists
  • Inline HTML: you can interleave raw html within the markdown
  • Images use a special (odd) syntax: ![description-text](image-file-name.jpg)
• HTML classes and layout
  • Grids for complex layouts: grid, grid-cols-#, grid-rowspan-#
  • Cards for framing content: card
  • Notes: note, tip, warning, caution: callouts
• CSS styles
  • page styles can be added in a <style>...</style> block at the top of the Markdown file (right after the frontmatter)
  • individual html elements can have inline styles using the style attribute <div style="color:red; background:white;">
  • consult the MDN reference to see all the proprties you can change
• JS code blocks: the equivalent of cells in Notebook
  • create multi-line blocks of javascript code with triple-backticks (and don't forget the js suffix):

    ```js
      <code goes here>
    ```
    

  • a block with a single expression will insert its value (text or object) into the page content at that point
  • declaring something as const is a way to share it by name with other code blocks—note that a cell containing one or more declarations will not be inserted into the page content
  • calling display(...) will add its argument to the page

    A useful pattern for keeping your code clean is to declare your complex pieces of content (like D3 code) as variables in a code block at the top of your file and then just referencing it in a call to display() within the page content

• Inline expressions: Inserting calculated values into markdown
  • Use the ${expression} syntax to reference a variable declared elsewhere or even call a function and insert its return value (which should either be a string or an HTML DOM node)
• File attachments: Load data and media files from within your project directory
  • Look in example-report.md for an example of loading a JSON file
• Loading data: Write a loader script with the proper file extension to use javascript or python to do your data processing then emit just the subset of your data that the D3 code will need for rendering.
  • The advantage of this over using a FileAttachment, is that the loader script will be run once when you build the site and its output will be cached, so you can potentially load less data and avoid waiting for your slow loader script to execute on each page load.
  • Look in launches.csv.js for an example loader script (creating a CSV with JavaScript) and in example-dashboard.md to see how its output is accessed as a FileAttachment (note the lack of a .js file extension)
• User Interface: the Inputs module provides most of the UI elements you're familiar with from Notebooks
• External libraries: you can use the import statement to pull in third-party code from the vast NPM library

Exercises

• Edit the observable.config.js file and pick a title and a theme for your project
• Try adding a <style>...</style> block at the beginning (but after the frontmatter) of example-report.md and write some rules that re-style the <h2> and <h3> headings:

  h2{
    /* try changing: color, font-family, font-size, text-transform, text-shadow, etc. */
  }
  
  h3{
    /* make some additional style choices for the smaller subheads */
  }

• Try wrapping a few words in one of the page’s sentences in a <span> tag and setting an ‘inline style’ using the style attribute:

  Lorem ipsum <span style="background: linear-gradient(transparent, cyan)">dolor sit amet</span>, consectetur adipiscing elit.

• Try wrapping some other words in a span with a class name that you've chosen (it can be anything, but try to stick to lowercase characters and hyphens):

  Lorem ipsum dolor sit amet, consectetur <span class="essential">adipiscing</span> elit.

  • now add a rule to the <style> block that applies to it (in CSS, prepending a "." to a name makes it clear it's refencing a class rather than a tag name):

    .essential{
      text-decoration: underline;
    }

• Create a new page whose file name is project.md but whose page title is set to "Project Sketches" using markdown frontmatter
• Use a File Attachment (or the ![]() shorthand) to load an image then add it to the page as the main content of a <details> element (don't forget to also set a <summary> message).
• Make a 3 x 3 grid (similar to the 2 x 2 grids here) but have the middle row be a single cell that spans all 3 columns:

  A B C 
  --D--
  E F G
  

• Use the color classes to make each label in the grid a different color from its neighbors
• Use a slider input to display the first n words from a long piece of text:
  • Declare a const variable in a code block that pointing to an array containing single-word strings
    • You can create one by splitting a long text string like: "foo bar baz".split(" ")
    • Consider using a FileAttachment to load a text file rather than adding the string inline
  • Add a code block that contains an Inputs.range control—make sure its maximum is set to be the total number of elements in your array of words.
  • Add another code block that uses the current range value to select the number of words from your array to display, then display a single string with the first n words from your array on the page (take a look at Array.slice(), .join(), and possibly display() for this)
• Try reading individual CSV files from a Zip archive
  • Download this Zip of gapminder data and make sure it remains a Zip file (i.e., if your browser automatically decompresses it, make a new archive or try running this from the command line):

    curl -O https://datavis.cs.columbia.edu/files/data/gapminder.zip

  • Move gapminder.zip into your project’s src/data subdirectory and load it from project.md using the special .zip() method on the FileAttachment object
  • Use .filenames to find out what its contents are called, then use the .file() method to extract the "Continents" file and the .csv() method to unpack its data
  • Display the rows using an Inputs.table
  • Add a list of all the country names to the page (using each row’s Entity attribute)
• Write a loader that fetches just the life expectancy data and filters it:
  • Note that while you're debugging your script you can run it directly from the command line to see its output. For instance if your loader file is called life-2010.csv.js you can run it with:

    node src/data/life-2010.csv.js

  • Look at the launches.csv.js to see how it uses its text() helper method to make an asynchronous HTTP request and then parse the result
  • Import csvParse from the d3-dsv module and use it to parse the gapminder data once it has arrived
  • Filter the rows and discard all of them except the data for 2010 string)
  • Use csvFormat to write the 2010 data to stdout
  • Load your 2010 data from project.md and display the rows using an Inputs.table
• Copy your now-working loader to a file called gdp-2010.csv.js and modify its source URL to fetch GDP data
  • Load the GDP data in project.md the same way you loaded the life expectancy data
• Create a scatter plot drawing each country as a dot whose x-position is GDP and y-position is life expectancy (you might want to use a scaleLinear for life expectancy and a scaleLog for GDP). Make sure you're using the actual extent of the data to set the domains properly
  • Add a radio button input that lets you choose between three colors for the dots
  • Use the selected color for drawing the dots and have the chart re-render itself each time it's changed
  • If you feel like going further:
    • add another loader that extracts the 2010 population data
    • Use population to set the radius of the dots
    • Have the radio buttons choose between 'gdp', 'population', and 'life expectancy'. Use whichever is selected to color the dots using a sequential palette.

Deploying to the web

Using GitHub pages

If you have a GitHub account, you can create a world-accessible version of your project. First, add your repository to your account, then follow these instructions for creating a .github/workflows/deploy.yml file.

Once this file has been added and you've enabled Pages on the repo, you'll be able to update the live version of the site just by pushing code changes to GitHub.

Building a static copy for hosting

Alternatively, if you have access to a web server that allows you to upload files manually, you can create a version that can be SFTP/rsync'd to it by running:

npm run build

Look in the build/ subdirectory to see the files it generated—note that index.html is the landing page and any other pages you created are the other *.html files in the directory.

Finally

Once your demo framework project is living somewhere on the web, share the link on #participation