Skip to content

Instantly share code, notes, and snippets.

@riddochc
Last active May 16, 2016 23:13
Show Gist options
  • Select an option

  • Save riddochc/854b232f1a2f8a84e50fc185bb649608 to your computer and use it in GitHub Desktop.

Select an option

Save riddochc/854b232f1a2f8a84e50fc185bb649608 to your computer and use it in GitHub Desktop.
(defn truncate-safely-to [s bytelimit]
"Truncate a UTF-8 string to the longest valid UTF-8 string less than bytelimit bytes"
(if (< bytelimit 1) "" ; empty string
(let [utf8 (.getBytes s "UTF-8")
eos (- bytelimit 1)
idx (if (<= (alength utf8) bytelimit)
bytelimit
(if (= 0 (bit-and (aget utf8 eos) 0x80))
eos
(first ;; get the character
(drop-while ;; before any at the end
#(let [c (aget utf8 %)] ;; that have 0x80 set and 0x40 clear
(and (> (bit-and 0x80 c) 0) ;; which indicates a non-initial byte
(= (bit-and 0x40 c) 0))) ;; of a multibyte utf-8 character
(range eos 0 -1))) ;; iterate back from the end
))]
(java.util.Arrays/copyOfRange utf8 0 idx))))
(let [ustr "This is ⟑ ┳∈Ⓢⓣ"]
(map #(prn (truncate-safely-to ustr %)) (range 1 27)))
@riddochc

Copy link
Copy Markdown
Author

Some places I want to put a string have a byte-length limit. It's good to avoid cutting off UTF-8 strings mid-codepoint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment