Skip to content

Instantly share code, notes, and snippets.

@dipanjanS

dipanjanS/spark_log_analytics_1.py

Created April 6, 2019 20:34
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save dipanjanS/41c099d008d3e5148c8a841bdb0326d6 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save dipanjanS/41c099d008d3e5148c8a841bdb0326d6 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
spark_log_analytics_1.py
# configure spark variables
from pyspark.context import SparkContext
from pyspark.sql.context import SQLContext
from pyspark.sql.session import SparkSession
sc = SparkContext()
sqlContext = SQLContext(sc)
spark = SparkSession(sc)
# load up other dependencies
import re
import pandas as pd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment