Skip to content

Instantly share code, notes, and snippets.

@pythonlessons
Created August 24, 2023 10:12
Show Gist options
  • Save pythonlessons/a59a021dd82f22243c9c1ab05b56c928 to your computer and use it in GitHub Desktop.
Save pythonlessons/a59a021dd82f22243c9c1ab05b56c928 to your computer and use it in GitHub Desktop.
transformers_nlp_data
['fueron los asbestos aquí. ¡eso es lo que ocurrió!', 'me voy de aquí.', 'una vez, juro que cagué una barra de tiza.', 'y prefiero mudarme, ¿entiendes?']
["<start>it was the asbestos in here, that's what did it!", "<start>i'm out of here.", '<start>one time, i swear i pooped out a stick of chalk.', '<start>and i will move, do you understand me?']
["it was the asbestos in here, that's what did it!<eos>", "i'm out of here.<eos>", 'one time, i swear i pooped out a stick of chalk.<eos>', 'and i will move, do you understand me?<eos>']
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment