Last active
May 16, 2016 23:13
-
-
Save riddochc/854b232f1a2f8a84e50fc185bb649608 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (defn truncate-safely-to [s bytelimit] | |
| "Truncate a UTF-8 string to the longest valid UTF-8 string less than bytelimit bytes" | |
| (if (< bytelimit 1) "" ; empty string | |
| (let [utf8 (.getBytes s "UTF-8") | |
| eos (- bytelimit 1) | |
| idx (if (<= (alength utf8) bytelimit) | |
| bytelimit | |
| (if (= 0 (bit-and (aget utf8 eos) 0x80)) | |
| eos | |
| (first ;; get the character | |
| (drop-while ;; before any at the end | |
| #(let [c (aget utf8 %)] ;; that have 0x80 set and 0x40 clear | |
| (and (> (bit-and 0x80 c) 0) ;; which indicates a non-initial byte | |
| (= (bit-and 0x40 c) 0))) ;; of a multibyte utf-8 character | |
| (range eos 0 -1))) ;; iterate back from the end | |
| ))] | |
| (java.util.Arrays/copyOfRange utf8 0 idx)))) | |
| (let [ustr "This is ⟑ ┳∈Ⓢⓣ"] | |
| (map #(prn (truncate-safely-to ustr %)) (range 1 27))) |
Author
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Some places I want to put a string have a byte-length limit. It's good to avoid cutting off UTF-8 strings mid-codepoint.