Notes from a talk by Mark Hayter
Mark Hayter is Senior Engineering Director in the Chrome OS Hardware team at Google, and currently a visitor from industry with the Computer Architecture team in the Computer Lab.
- 1990s: Apple Newton, Palm Pilot used graffiti input (one letter at a time, stylised), Lectrice
- 2000s: Microsoft Tablet PC
- 2010s: iPad Pro, MS Surface, Chromebook
Comparison: Lectrice (1996) and Acer Chromebook Tab (2018). Both have slot for stylus in the case ("garaged stylus"). Moved from hardware buttons to capacitive touchscreen. 2.6 kg → 0.55 kg. Lots more CPU and RAM. Otherwise surprisingly similar.
In Chromebooks, stylus initially introduced in educational context: stylus is better than keyboard/trackpad for younger kids. Requires stylus to be cheap and robust since kids chew on the stylus, drop it, etc.
Then expanded to consumer context: requires low latency ink <10ms, notetaking, document markup, UI interactions (screenshots, assistant interaction, laser pointer). User interaction on taking pen out of the slot in the case: instead of lock screen, go directly to handwriting note input.
Low latency ink (announced at Google I/O '18): use a machine learning model to predict where the stroke is going to go. Makes it look as if the latency is zero! Looks a bit weird when the model mispredicts and has to correct, but generally good.
Trade-offs of garaged stylus vs fancy stylus. Google's stats show that when stylus slots into the case, people are much more likely to use it; if you have to remember to put it in your bag, people forget. However, slot-in stylus feels much less nice to use.
Staedtler (pencil company) makes a stylus that really looks and feels like a pencil!
History of stylus technology:
- In 1990s, used resistive touch (two layers of film, stylus/finger pushes make contact between layers). Poor optical performance (film messes up picture due to reflections and light absorption).
- Wacom EMR: stylus contains L-C (solenoid and capacitor) circuit that resonates on a particular RF frequency. Passive, no battery needed. Press button on stylus → change capacitance. Pressure → detune capacitor. Sensor is a thin PCB glued onto the back of the display.
- Active stylus (Pixelbook stylus is in this category): injects RF signal into capacitive sensor. Advantage that no separate EMR sensor is needed, simplifying the manufacturing process. Advantage that you can distinguish multiple styluses. Accuracy of active styluses is still not quite as good as Wacom (but gap is closing). Battery size (AAAA battery) determines thickness of pen. Apple pencil uses supercapacitor or rechargeable battery? Cost model: can charge an enormous mark-up for active stylus since it's considered an "optional accessory" rather than part of the core product.
Google's vision: want all touchscreen devices to be stylus-enabled, so that user does not have to make choice. Want a stylus for one Chromebook to work with all Chromebooks. However, at the moment there are several competing active stylus systems. Will a pen for a Microsoft Surface work with a HP whatnot tablet? Who knows.
Universal Stylus Initiative (USI) aims to standardise protocol between sensor and stylus, frequency band used, etc. Current status: specification version 1.0 released; have test rig, multiple manufacturers, interop tests. Several big manufacturers are participating, but Apple are not.
I asked about problems with simultaneous finger and stylus input: apparently there is no hardware reason why there would be such a limitation. Mark's guess is that it's overzealous palm rejection, probably happening in software, although it's possible that events are getting blocked at the controller level.
