Skip to content

Instantly share code, notes, and snippets.

@ashleyha

ashleyha/hf_helsinki-nlp_translation.py

Created December 16, 2020 17:57
Show Gist options
  • Save ashleyha/04cb3879b1d869adfce9e0cd0794f094 to your computer and use it in GitHub Desktop.
Save ashleyha/04cb3879b1d869adfce9e0cd0794f094 to your computer and use it in GitHub Desktop.
Download ZIP
example using the huggingface transformers library with the Helsinki-NLP/opus-mt-ru-en model
Raw
hf_helsinki-nlp_translation.py
from transformers import MarianTokenizer, AutoModelForSeq2SeqLM
text = 'Рада познакомиться'
mname = 'Helsinki-NLP/opus-mt-ru-en'
tokenizer = MarianTokenizer.from_pretrained(mname)
model = AutoModelForSeq2SeqLM.from_pretrained(mname)
input_ids = tokenizer.encode(text, return_tensors="pt")
outputs = model.generate(input_ids)
decoded = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(decoded) #Nice to meet you
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment