unicode-segmentation v1.12.0 indic script segmentation test

README.md

grapheme count test

The crate unicode-segmentation has fixed the segmentation logic for indic scritps in v1.12.0. ref. unicode-rs/unicode-segmentation#125

unicode-segmentation v1.11.0

$ cargo update -p unicode-segmentation --precise 1.11.0
$ cargo run

नमस्ते (4 graphemes): ["न", "म", "स\u{94d}", "त\u{947}"]
नमस्ते (4 graphemes): ["न", "म", "स\u{94d}", "त\u{947}"]
नमस्ते (4 graphemes): ["न", "म", "स\u{94d}", "त\u{947}"]
র্যর্য (4 graphemes): ["র\u{9cd}", "য", "র\u{9cd}", "য"]
র্যর্য (4 graphemes): ["র\u{9cd}", "য", "র\u{9cd}", "য"]

unicode-segmentation v1.12.0

$ cargo update -p unicode-segmentation --precise 1.12.0
$ cargo run

नमस्ते (3 graphemes): ["न", "म", "स\u{94d}त\u{947}"]
नमस्ते (3 graphemes): ["न", "म", "स\u{94d}त\u{947}"]
नमस्ते (3 graphemes): ["न", "म", "स\u{94d}त\u{947}"]
র্যর্য (2 graphemes): ["র\u{9cd}য", "র\u{9cd}য"]
র্যর্য (2 graphemes): ["র\u{9cd}য", "র\u{9cd}য"]

Cargo.toml

[package]
name = "rust-grapheme-count-test"
version = "0.1.0"
edition = "2024"

[dependencies]
unicode-segmentation = "1.11.0"

main.rs

use unicode_segmentation::UnicodeSegmentation;

fn main() {
    let g = |s: &'static str| {
        let graphemes = s.graphemes(true).collect::<Vec<_>>();
        println!("{s} ({} graphemes): {:?}", graphemes.len(), graphemes);
    };

    g("नमस्ते");
    g("नमस्ते");
    g("नमस्ते");

    g("র্যর্য");
    g("র্যর্য");
}