Abdullah Al Nahas abdullah-alnahas

GenAI, RAG, Prompt Engineering, NLP, DL, ML, Python

Bergvca / Pyspark_LDA_Example.py

Created February 3, 2016 13:59

Example on how to do LDA in Spark ML and MLLib with python

	import findspark
	findspark.init("[spark install location]")

	import pyspark
	import string
	from pyspark import SparkContext
	from pyspark.sql import SQLContext
	from pyspark.mllib.util import MLUtils
	from pyspark.sql.types import *
	from pyspark.ml.feature import CountVectorizer, CountVectorizerModel, Tokenizer, RegexTokenizer, StopWordsRemover