While mouse event tracking is already possible in various modes and encodings, this feature seems most used by alt-screen applications however.
The reason why primary screen applications (such as your shell) usually don't make use of the mouse, is the design problem, that whenever the application is requesting mouse tracking events, the user cannot use the mouse to select text on the screen anymore.
With this VT extension, a new mode is introduced to inform the terminal that the user may still use the mouse to select text, but still have mouse events being reported to the application to allow passive tracking of mouse events.
This can then be used by shells to still act on mouse click events to reposition the cursor or mouse-hover events over commands to enable the possibility for the shell to create tooltips.
This spec is NOT trying to revolutionize mouse protocol support in the terminal nor trying to handle every little corner case that may or may not arise in the usage scenarios.
We however attempt to address a current need in a way that this specification is easily understood, easily implementable on both sides, and most importantly, helpful to users.
The semantics of mouse reporting are not altered.
The application only receives mouse tracking events for the main page area.
Passive mouse tracking mode will ONLY work in conjunction with SGR transport mode (?1006).
Using it with any other transport mode (e.g. X10, UTF-8, or URXVT) will be ignored.
The SGR mouse events received by the application will contain an additional parameter to the end indicating if this event has also been handled by the user interface (terminal frontend) or not.
While mode ?2029 is active mode ?1007 has no effect (i.e. wheel events are not translated to arrow keys).
Use DECRQM to detect if the mode is supported and if so, its current state. Passive mouse tracking is considered supported if and only if the reply contains state value 1 (set) or value 2 (reset).
Sending CSI ? 2029 h to the terminal will enable passive mouse tracking.
This will implicitly activate DEC mode ?1006 (SGR) as well as ?1002 (Button click event tracking).
In case mouse move events are desired as well, the application is free to send CSI ? 2029 ; 1003 h in one go.
Since only SGR encoding is supported for passive mouse tracking, the syntax looks equivalent to this, except that an additional parameter is added to indicate whether or not the user interface (terminal) has handled this event as well.
Mouse Press (and move) Syntax: CSI ? <ButtonAndModifierMask> ; <Column> ; <Line> ; <HandledFlag> M
Mouse Release Syntax: CSI ? <ButtonAndModifierMask> ; <Column> ; <Line> ; <HandledFlag> m
Whereas <ButtonAndModifierMask, Column, <Line> are equivalent to how it is handled already.
<HandledFlag> is either 1 or greater to indicate that the user interface has handled this event as well,
or value 0 to indicate that the user interface has not handled this event.
Sending CSI ? 2029 l will disable passive mouse tracking and also have any mouse event mode as well as mouse event encoding mode disabled. It is therefore sufficient to simply send this single VT sequence to have it all cleared.
When disabling other mouse modes (events as well as encoding) should alo disable passive mouse tracking implicitly in order to best avoid leaking passive mouse tracking into another application by accident.
While alt-screen application do not have the problem, it must be ensured for primary screen applications with scrollback lines available, that the mouse coordinates are mapped to the correct cell position. With top-left being (1, 1), when the viewport of the user interface is scrolled up by 5 lines, that grid cell must still be reported as (1, 1) as the connected terminal application does not have the notion of viewport or would be even able to reference these.
Applications like the shell (e.g. Bash, ZSH, Fish) are likely to frequently execute child processes, with having the PTY handles inherited. In case of fear that some mouse events might leak into the next application, the application may do the following:
- Disable mouse protocol (
CSI ?2029 l) - Discard any pending input from stdin that may have be read without waiting.
- Execute the child process.
Alternatively a very conservative way:
- Disable mouse protocol (
CSI ?2929 l) - Request
DA1 - Consume and discard any input until
DA1's response has been received. - Execute child process.
| Support | Terminal/Tookit/App | Notes |
|---|---|---|
| n/a | xterm.js | |
| n/a | Windows Terminal | |
| ✅ | Contour | see: contour-terminal/contour#882 |
| n/a | mintty | |
| n/a | notcurses | |
| n/a | foot | |
| n/a | Wezterm | |
| n/a | VTE / gnome-terminal | |
| n/a | iTerm2 | |
| n/a | Kitty | |
| n/a | Alacritty | |
| n/a | Konsole | |
| n/a | st |
@christianparpart I'd have a few questions regard its technical implications here, maybe you answer that with some hints on how you plan to implement this.
First mouse tracking works on the normal buffer as fine as on the alternate buffer (take the the "fine" with a grain of salt - it is the same sh*t as over there due to awkward protocols/encodings). Imho the fact that mostly alt buffer apps use it comes from the idea to fully control that screen state, while that assumption normally cannot be made on the normal buffer with scrollback (as there is other output as well). So here I cannot quite follow your reasoning with the text selection & clipboard semantics.
On the tech level - we have several mouse actions that can be tracked:
Now the question is - how to make mouse reports working on scrollbuffer while still maintaining its other mouse sensitive semantics? To list these other mouse sensitive semantics on the normal buffer:
On a first glance it should be possible to just keep everything working as it currently is, but also send the requested tracking info to app side on top. This would be the simplest approach and prolly easiest to implement on TE side (and appside has to be aware of incoming mouse reports anyway).
But on a second look I am not sure anymore, if for example a scroll action in the scroll buffer should be reported to app side (prolly a shell) - it has no means to act upon this. So I am not sure whether reporting wheel actions makes any sense in a "passive tracking mode". But the same argument can be applied to the DRAG event - if the user points & highlights anything far up in the scrollbuffer - whats the deal here for app side, beside "sniffing" on what the user does with its mousey (might that even raise security concerns)?
At this point imho a passive mouse tracking mode has very limited use cases for the scrollbuffer, if any at all. I still like the idea, but for a very different purpose - on the alternate buffer. Here a passive mode would enable to use all the normal mouse semantics along with the overloads from app side, e.g. a user can still point & select text while the app underneath keeps working as before.
These are just some early thoughts from my side on this, most likely I've overlooked edge cases, where it will show benefits, or would make mouse interaction with the terminal alot easier. Glad to hear about those.