Created
August 4, 2020 00:47
-
-
Save ShairozS/acef10a355321fe6b14845a3bc2c1ab8 to your computer and use it in GitHub Desktop.
Use a trained Gensim LDA model to classify the topics in a list of text
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def find_topic(textlist, dictionary, lda): | |
| ''' | |
| https://stackoverflow.com/questions/16262016/how-to-predict-the-topic-of-a-new-query-using-a-trained-lda-model-using-gensim | |
| For each query ( document in the test file) , tokenize the | |
| query, create a feature vector just like how it was done while training | |
| and create text_corpus | |
| ''' | |
| text_corpus = [] | |
| for query in textlist: | |
| temp_doc = tokenize(query.strip()) | |
| current_doc = [] | |
| temp_doc = list(temp_doc) | |
| for word in range(len(temp_doc)): | |
| current_doc.append(temp_doc[word]) | |
| text_corpus.append(current_doc) | |
| ''' | |
| For each feature vector text, lda[doc_bow] gives the topic | |
| distribution, which can be sorted in descending order to print the | |
| very first topic | |
| ''' | |
| tops = [] | |
| for text in text_corpus: | |
| doc_bow = dictionary.doc2bow(text) | |
| topics = sorted(lda[doc_bow],key=lambda x:x[1],reverse=True)[0] | |
| tops.append(topics) | |
| return(tops) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment