JeremiahKamama/max_tokens.py

Created July 9, 2021 06:52

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Select an option

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/JeremiahKamama/b062a17dac985dfa56dcec6cfdb57888.js"></script>
Save JeremiahKamama/b062a17dac985dfa56dcec6cfdb57888 to your computer and use it in GitHub Desktop.

Download ZIP

Raw

max_tokens.py

	max_tokens = np.mean(num_tokens) + 2 * np.std(num_tokens)
	max_tokens = int(max_tokens)
	max_tokens

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment