Skip to content

Instantly share code, notes, and snippets.

@fitzgen

fitzgen/cargo-build-hooks-for-wasm.md

Last active November 3, 2019 10:47
Show Gist options
  • Save fitzgen/23a62ebbd67574b9f6f72e5ac8eaeb67 to your computer and use it in GitHub Desktop.
Save fitzgen/23a62ebbd67574b9f6f72e5ac8eaeb67 to your computer and use it in GitHub Desktop.
Download ZIP
Rust and WebAssembly Working Group meeting notes from the 2019 Rust All Hands
Raw
cargo-build-hooks-for-wasm.md
Audience WG-wasm + cargo team (+ maybe WG-embedded?)
When TBD (45 minutes)
Where Snow
Meeting Style Discussion and brainstorming

Deliverables

  • Sketch of an eventual ideal we’d like to end up at
  • Sketch of a minimal RFC that is a first step towards that eventual ideal

Agenda

  • Intro to the kind of thing we are talking about here (5 minutes)
    • What sorts of “post-build”/etc tasks do we do for wasm, that we would like to hook into cargo
  • Define a long-term ideal state we would like to eventually reach (20 minutes)
  • Map out a series of smaller incremental steps we can take to reach that (20 minutes)
    • And these steps could be a succession of RFCs

+Embedded use cases for Cargo build hooks

Notes

  • doesn’t make sense for libraries on crates.io
    • only for binaries and dylibs
    • we don’t want to run dependencies’ post-build hooks
  • how do we make incremental work?
    • how does cargo know when to re-execute this?
  • [post-build-dependencies] in Cargo.toml?
  • alternatively, should we have build tasks instead of post-build scripts?
    • depend on a package, and we just run whatever is in the package
    • takes a generic build script of some kind from that package and runs it
  • how does it communicate its dependencies and how do we give it its inputs?
  • how do different targets work?
  • maybe reuse build.rs for post-builds
    • then we can skip post-build-dependencies in Cargo.toml?
    • would still have to have some sort of flag turned on in Cargo.toml
  • no implicit rerun-if for everything like build.rs has
    • you have to explicitly say the things you depend on (other than the artifact you’re being run on)
  • incremental steps we can take to move this forward:
    • create RFC for minimal post-build.rs
      • write minimal rust code
      • ability to depend on crates
      • needs to know:
        • output artifact(s) location in target dir
        • metadata on artifact config (opt-level, target, etc)
        • can read Cargo.toml for extra metadata
        • location of lockfile
        • am I running for a cdylib or a bin or a …
      • cargo:output-artifact=…
Raw
modular-toolkit.md
Audience WG-wasm and anyone else interested!
When TBD (115 minutes)
Where TBD
Meeting Style Discussion and brainstorming

Deliverables

  • Plan for how to collect the libraries into toolkit’s umbrella crate
  • Prioritized list of libraries we would like to have in the toolkit
  • Guidelines and standards for toolkit’s libraries
  • Name for toolkit

Agenda

  • What do we mean “Tide-like toolkit” (5 minutes)
  • What standards do we want to have for these libraries? (20 minutes)
    • Ideally something we could give to contributor Alice who wants to lead dev of the X library
  • What libraries should we build? (60 minutes)
    • Review rust+wasm and JS frameworks for components that can be pulled apart
    • Which do we want to commit to doing ourselves, and which would we accept for inclusion if outside contribs impl’d to our standards?
  • How to combine pieces into a whole? (20 minutes)
  • Naming… (10 minutes)

Notes

  • list of libraries

  • 3 things being built, 3 step migration story:

    • tiny wasm module- fits into existing js app
    • frankenapp?
    • pure rust+wasm frontend

  • types of things:

    • potentially opinionated bits: consistent interfaces
      • redux- state mgmt
      • vdom
      • routing (react-router, choo)
    • utilities that give “nice interfaces” over things everyone wants

Possible Crates (from Choo)

Opinionated

The “web” to the “web-sys” libraries

  • timers
  • indexed db
  • local storage
  • web gl
  • 2d canvas
  • svg
  • fetch / http requests
  • web sockets
  • Listen for Server Sent Events https://github.com/yerkopalma/choo-sse (probably best expressed as a futures stream)
  • webaudio
  • DOMContentLoaded

Unopinionated

Undefined

Guidelines and Standards for Toolkit Libraries

[ ] API Guidelines

  • See also: https://github.com/rust-lang-nursery/api-guidelines

  • unwrap_throw internally instead of unwrap

  • APIs take functions instead of wasm-bindgen closures

  • JsValue should generally never show up in APIs

  • into_raw() escape hatch for getting raw web_sys thing being wrapped

    • should it be a trait? [ ] tests with wasm-bindgen-test and wasm-pack test [ ] cargo fmt [ ] cargo clippy [ ] docs with examples

  • deny(missing_docs, missing_debug_implementations, missing_examples) [ ] dependabot [ ] if unopinionated or an interface, should be in rustwasm github org [ ] contributing.md [ ] owner(s) [ ] readme.md

  • cargo generate

    • umbrella-app
    • umbrella-lib

this work may extend influence past wasm work

How to put pieces together into holistic package

  • choose defaults for interfaces
  • put them together
  • prelude with re-exports of little lib/utility crates?
    • experience of quicli users
      • where does this come from? where are its docs?
    • can we just use cargo generate?
      • this avoids re-exports
      • but has boilerplate
  • guiding principles:
    • Try and minimize boilerplate
    • Allow “opinionated” parts to be swapped out
    • We should never re-export macros
  • The web crates to the web-sys should live together in a repository under the rustwasm organization.

Names

  • swell (as in good and as in a rising wave; has a w)
  • ella (as in umbrella)
  • gloo
  • wacko
  • waddle
  • watermelon
  • wax
  • wander
  • wonder
  • Adam Levine
  • wave
  • wayward
  • wintermute
  • wasome
Raw
road-to-wasm-pack-1.0.md
Audience WG-wasm (+ WG-cli for UX philosophy/guidelines input?)
When TBD (60 minutes)
Where TBD
Meeting Style Discussion and work

Deliverables

  • Timeline / prioritization of wasm-pack work from WG-wasm 2019 roadmap
  • Philosophy of and guidelines for wasm-pack UX

Agenda

  • Discuss and create general philosophy and guidelines for wasm-pack UX (30 minutes)
    • Does it makes sense to make this an RFC?
  • Create timeline / prioritization of wasm-pack work from our roadmap (30 minutes)

Terminal Support and Interface

MiniRFC: rustwasm/wasm-pack#298

  • What should we print and when?

    • teaching users what the tool is doing for them
      • currently shows what it is going to do (and sometimes the results of what it did)
      • can this teaching mode be a verbose flag?
        • eg cargo does this; doesn’t show linker flags by default
      • can also switch to —quiet mode
        • since advanced users know flags, but beginners don’t know flags
      • how important is it that we teach users the steps wasm-pack does vs its overall job?
    • what do users need to know?
      • what module system was used
      • where the package is
      • progress bar
      • this seems like results of things we do, rather than we are going to do X now
      • if w32-u-u is already there, we shouldn’t print that we are checking for it
      • however if it is not there, and we have to download it, then we print what we are doing
    • we should copy cargo’s timeout for printing long running processes
    • we always want to be able to answer “why is it hanging right now?”
      • example - short running tasks don’t print
      • example - long running tasks will print what’s happening if necessary (maybe after a timeout)
    • but we don’t want to print out “we are going to do this 1ms task right now”
    • we always want to print the end results of a command

  • terminal support

    • piping to file with no text should always work well
    • extra features must be detected
      • if we can’t reliably detect, we must pessimistically assume they don’t exist
    • we should aim to give a good minimal experience to as many terminals as we can
    • feature detection
      • emoji
      • colors
      • line clearing (progress bars)
      • terminal width
    • - 3 outputs - text file, terminal, machine readable - https://github.com/killercup/convey/
  • The other type of UX
    • menu prompt
      • survey?
      • doesn’t even make sense to publish something you’ve never built?
    • how cargo like do we want to be?
      • release vs debug builds by default?

Work to Do

  • MVP convey
    • Define a philosophy for wasm-pack's user experience, interaction, and display. Once defined and agreed upon, we should triage each wasm-pack subcommand and ensure that it is consistent with our philosophy.
  • Generate JavaScript API documentation from the Rust doc comments on #[wasm_bindgen] exports.
  • Finish and implement the RFC for library crates depending on external NPM packages.
  • Finish and implement the RFC for local JavaScript snippets.
  • Support running Binaryen's [wasm-opt](https://github.com/rustwasm/wasm-pack/issues/159) on Rust-generated Wasm.
  • Integrate [cargo generate](https://github.com/rustwasm/wasm-pack/issues/373) into [wasm-pack](https://github.com/rustwasm/wasm-pack/issues/373) for new project scaffolding. This would smooth the developer on ramp, by making one less tool required to get up and running.
  • RFC and implementation for generating portable, universal NPM packages that work on the Web, with Node.js, and in any minimal JavaScript environment.

    Timeline

  • Monthly minor releases (with occasional patch releases)

March April May June July August September October November December

Raw
wasm-multithreading-and-rayon.md
Audience WG-wasm + rayon folks, and anyone else interested!
When Thursday at 10am
Where Snow
Meeting Style Discussion and brainstorming

Deliverables

  • Sketch of a design for foundational wasm multithreading library
  • Plan for implementing rayon on top of it

Agenda

  • Review of wasm-/web-specific constraints to multithreading (10 minutes)
  • Create sketch of library’s design (30 minutes)
    • What does supporting JS code look like, how is it distributed
    • What does public API look like?
  • Does it make sense, and if so how to get rayon working on top of this library when targeting wasm32? (10 minutes)
  • Can we come up with another threading-related pun as crate name? (0 minutes)
    • weft?
    • wayon

Notes

  • SharedArrayBuffer is coming
  • one linear memory shared across many wasm modules
  • each wasm module in another web worker, on another thread
  • wasm itself doesn’t have thread spawning instructions (yet)
    • just atomic waits/reads/notifies for now
  • main thread can never block
    • attempting to wait on the main thread will throw an exception
  • why can’t we get std::thread::spawn to work?
    • needs wasm-bindgen to do thread spawning stuff, std doesn’t depend on wasm-bindgen
  • hooking up imports object for instantiating the wasm module on a web worker is the hard part
  • give rayon a closure that spawns threads instead of std::thread::spawn
    • an argument to the thread pool builder
  • par_iter won’t work on the main thread
    • need a variation on par_iter that returns a future and doesn’t block
    • must be implemented with atomics, comparexchg, and JS promises
  • rayon threads wait on a condvar for work
    • that won’t work on the Web
    • can we get them to do thread park/unpark or futexes or something?
  • Goal: get the raytracing demo working with a git branch of rayon with the spawn fun
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment