hanneshapke/before_tokenization_with_berttokenizer.py

Created March 9, 2020 16:47

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/hanneshapke/7a8c6fe147d9bc0c71c93c646622504d.js"></script>
Save hanneshapke/7a8c6fe147d9bc0c71c93c646622504d to your computer and use it in GitHub Desktop.

Download ZIP

Raw

before_tokenization_with_berttokenizer.py

	[
	"Clara is playing the piano."
	"Maria likes to play soccer.",
	"Hi Tom!"
	]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment