hanxiao
/ testRegex.js
Last active
April 22, 2025 02:43
Regex for chunking by using all semantic cues
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // Updated: Aug. 20, 2024 | |
| // Run: node testRegex.js whatever.txt | |
| // Live demo: https://jina.ai/tokenizer | |
| // LICENSE: Apache-2.0 (https://www.apache.org/licenses/LICENSE-2.0) | |
| // COPYRIGHT: Jina AI | |
| const fs = require('fs'); | |
| const util = require('util'); | |
| // Define variables for magic numbers | |
| const MAX_HEADING_LENGTH = 7; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 很实用的编程英语词库，共收录一千五百余条词汇。 | |
| 第一部分： | |
| application 应用程式 应用、应用程序 | |
| application framework 应用程式框架、应用框架 应用程序框架 | |
| architecture 架构、系统架构 体系结构 | |
| argument 引数（传给函式的值）。叁见 parameter 叁数、实质叁数、实叁、自变量 | |
| array 阵列 数组 | |
| arrow operator arrow（箭头）运算子 箭头操作符 |