BiDirectional Text

BiDiSupport.md

This gist will show the support of BiDirectional text in the terminal emulators and console programs. You can read more about the standardization efforts at the dedicated page of FreeDesktop Terminal BiDi working group.

How to test

Logical Order	◀ ◀ ◀ RTL	LTR ▶ ▶ ▶
WHAT IS UNICODE؟ in arabic	in arabic ؟EDOCINU SI TAHW	؟EDOCINU SI TAHW in arabic
	ما هو الترميز الموحد يونيكود؟ in Arabic	ما هو الترميز الموحد يونيكود؟ in Arabic

Supporting BiDirectional text

According to a new standard

Please note, that specification is preliminary yet, and VTE implementation is still work in progress:

• all libvte based terminals - https://gitlab.gnome.org/GNOME/vte/issues/53
  • libvte-based Gnome Terminal - https://bugs.launchpad.net/ubuntu/+source/vte/+bug/263822
  • libvte-based sakura
  • libvte-based xfce4-terminal
  • libvte-based Terminator
  • libvte-based Tilix - written in D. Similar user interface as for Terminator.
  • libvte-based Lilyterm - https://github.com/Tetralet/LilyTerm/
  • libvte-based ROXTerm
  • libvte-based evilvte https://github.com/caleb-/evilvte
  • libvte-based Termit
  • libvte-based Tilda
  • libvte-based tinyterm
  • libvte-based Pantheon Terminal
  • libvte-based lxterminal

Incompatible and incomplete implementations

• mlterm
• konsole
• qterminal
• mintty Cygwin and MSYS/MSYS2 - Windows platform

Supporting BiDirectional text programs, but incompatible

• neatvi - Vi-like editor aligrudi/neatvi#8

Not supporting BiDirectional text

• another libvte-based Termite
• st (from suckless)
• xst fork of st gnotclub/xst#55
• iterm2
• alacritty - written in Rust alacritty/alacritty#663
• kitty - uses OpenGL kovidgoyal/kitty#2109
• cool-retro-term
• Termux - Android platform
• Black Screen - crossplatform, HTML/CSS/JS-based
• Hyper.app - crossplatform, HTML/CSS/JS-based (Electron)
• hterm - HTML/CSS/JS-based (ChromeOS)
• Windows Terminal - Windows platform, standard terminal microsoft/terminal#538
• Tera Term - Windows platform
• ConEmu - Windows platform Maximus5/ConEmu#1665
• FinalTerm - abandoned, iTerm2 borrowing it's ideas and features.
• MacTerm - Mac OS X platform
• Windows 10 bash and cmd.exe console - https://wpdev.uservoice.com/forums/266908-command-prompt-console-windows-subsystem-for-l/suggestions/34937857-bidirectional-text-support

Terminal multiplexers

• tmux - not yet tmux/tmux#1936
• screen - not yet
• pymux - not yet prompt-toolkit/pymux#77
• dvtm - not yet martanne/dvtm#99

Not supporting BiDirectional text programs

• neovim - neovim/neovim#553
• vim - vim/vim#204
• radare2 - https://github.com/radare/radare2/issues/2032
• irssi - irssi/irssi#1133

There's finally some movement on this; Unicode Technical Committee №175 approved a Working Group to be called the Terminal Complex Script Support Working Group (TCSS WG) after Microsoft asked for it in L2/23-107.¹

Microsoft wants to do it in a libvte-incompatible way so I plan to attend every meeting of the WG I can just to discuss…why, it seems they may not have even known about prior art. My document is L2/23-153.²

Footnotes

• ¹ L2/23-107
  Li, Renzhi et al. Proper Complex Script Support in Text Terminals. 2023-04-24.
• ² L2/23-153
  Brennan, Fredrick R. Opposition to and Comment on L2/23-107. 2023-06-30.

@ctrlcctrlv : Re: "Opposition to and Comment on L2/23-107", the BiDi in Terminal Emulators draft proposal is by Egmont Koblinger.

Yes, sorry, my mistake; @XVilka told me this already via email; but the Unicode office is closed on weekends :-) I'll submit a revised (“R”) version to L2 via docsubmit@ next week.

what about wezterm ?

What about it?

This output is wrong:

For «Wikipedia», ויקיפדיה is the correct writing order, not הידפיקיו.

It fails the two major tests of the debu.tanuki.agency test:

It accepts the fribidi'd Hebrew as-is meaning it's on par with kitty (i.e. broken) and mangles the Devanagari worse than Konsole or GNOME Terminal but slightly better than Kitty yet far behind mlterm.

@ctrlcctrlv could you share with me how to run the tests you ran with wezterm?

@wez Sure!

Do:

ssh [email protected]

No password. That's not how I ssh to デブ.狸.agency and is there as a joke against would-be crackers and skiddies, but it's useful for this too! (As that message is just my MOTD.)

So is there a terminal that supports bidi yet? Gnome terminal isn't that good. I've tried to run Neovim inside it, and got the wrong behavior. mlterm seems to be the one with the best behavior out of all the ones that I've tried, but its documentation is lacking and it doesn't look very nice out of the box.

that is the state of the art. mlterm does the best job.

PuTTY (which is more famous in Windows but also runs on linux, known as `pterm`) also has usefully good bidi 👍 - [x] Aligns RTL lines to the right side, processing them as RTL base-direction text. It seems to be using first-strong heuristic to decide line direction. - [x] Treats each line as "paragraph" for purpose of bidi algorithm. - [ ] Long line wrapping creates separate "paragraphs". This is sub-optimal as a long logical line may get broken into paragraphs of different base direction [depending on break points](https://terminal-wg.pages.freedesktop.org/bidi/prior-work/terminals.html#common-problems). AFAICT wrapping is permanent — resizing window to increase width doesn't reflow to join them, they remain broken at original wrap point; resizing to reduce width _hides_ text without reflowing — but it's not lost, resizing back reveals it. This mixed model is a bit weird. - [x] Supports "paired bracket" addition of newer bidi algorithm, so that e.g. `HEBREW f(x) TEXT` looks like `TXET f(x) WERBEH` not `TXET (f(x WERBEH` as it tended to look once upon the time.

perhaps bidi better on win32 but i see dotted circles

fails both deva and hebr

cosmic-term supports bidirectional text and ligatures via cosmic-text: https://fosstodon.org/@soller/111602927867586005

Konsole is actually doing a great job at bidi, in fact it is the only terminal that renders the RTL text from the right edge of the terminal window.

See this screenshot from Konsole displaying English and Persian bible text in one window.

I'm not sure if others can render LTR and RTL text in such fashion.

The settings window

I'd say all terminals I've seen to do bidi do it that way - how else? To check, however, please upload your test file.

The konsole feature "Table characters BiDi mode override" might be interesting but unfortunately it is not really specified (which characters? examples?); maybe I'll submit them an issue.

@wolandark

In any VTE-based terminal, echo '\e[?2501h' to enable auto-direction, and you should get the same behavior.

This comes from terminal-wg's BiDI draft proposal.

@MoSal Neat! I didn't know that. After running the escape sequence, XFCE terminal also behaves the same as Konsole does.

@mintty here is a sample of the text (though what I shared in the screenshot was being fetched from an API).

As mentioned above, doing as MoSal suggested, makes XFCE terminal (which is VTE) behave the same way as Konsole, so I guess my defense of Konsole is relevant, though in my defense there is no need to run any escape sequence in Konsole to get this behavior.

though in my defense there is no need to run any escape sequence in Konsole to get this behavior.

FWIW, LTR instead of AUTO as a default paragraph direction was a design choice which I, as a native speaker of an RTL language, tend to agree with.

The konsole feature "Table characters BiDi mode override" might be interesting

It appears that mintty already has a mode that seems to render table layout like konsole: echo -e "\e[?2501l".