Magnus Nissel magnusnissel
magnusnissel
/ lexical_diversity_yule.py
Last active
October 1, 2024 08:38
Yule's K and Yule's I for lexical diversity in Python 3 (quick & dirty)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import collections | |
| import re | |
| def tokenize(s): | |
| tokens = re.split(r"[^0-9A-Za-z\-'_]+", s) | |
| return tokens | |
| def get_yules(s): | |
| """ | |
| Returns a tuple with Yule's K and Yule's I. |
NewerOlder