Skip to content

Instantly share code, notes, and snippets.

@ShairozS
Created August 4, 2020 00:47
Show Gist options
  • Select an option

  • Save ShairozS/acef10a355321fe6b14845a3bc2c1ab8 to your computer and use it in GitHub Desktop.

Select an option

Save ShairozS/acef10a355321fe6b14845a3bc2c1ab8 to your computer and use it in GitHub Desktop.
Use a trained Gensim LDA model to classify the topics in a list of text
def find_topic(textlist, dictionary, lda):
'''
https://stackoverflow.com/questions/16262016/how-to-predict-the-topic-of-a-new-query-using-a-trained-lda-model-using-gensim
For each query ( document in the test file) , tokenize the
query, create a feature vector just like how it was done while training
and create text_corpus
'''
text_corpus = []
for query in textlist:
temp_doc = tokenize(query.strip())
current_doc = []
temp_doc = list(temp_doc)
for word in range(len(temp_doc)):
current_doc.append(temp_doc[word])
text_corpus.append(current_doc)
'''
For each feature vector text, lda[doc_bow] gives the topic
distribution, which can be sorted in descending order to print the
very first topic
'''
tops = []
for text in text_corpus:
doc_bow = dictionary.doc2bow(text)
topics = sorted(lda[doc_bow],key=lambda x:x[1],reverse=True)[0]
tops.append(topics)
return(tops)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment