Adam Zhou amazingchow

🎯

Focusing

Hype starts to get real here and strong narrative, time to pay attention for me.

amazingchow / testRegex.js

Created October 26, 2024 07:25 — forked from hanxiao/testRegex.js

Regex for chunking by using all semantic cues

	// Updated: Aug. 20, 2024
	// Run: node testRegex.js whatever.txt
	// Live demo: https://jina.ai/tokenizer
	// LICENSE: Apache-2.0 (https://www.apache.org/licenses/LICENSE-2.0)
	// COPYRIGHT: Jina AI
	const fs = require('fs');
	const util = require('util');

	// Define variables for magic numbers
	const MAX_HEADING_LENGTH = 7;