Skip to content

Instantly share code, notes, and snippets.

@dipanjanS

dipanjanS/spark_log_analytics_32.py

Created April 9, 2019 17:41
Show Gist options
  • Save dipanjanS/97fdc5b8b04ddc418376d7249c35df42 to your computer and use it in GitHub Desktop.
Save dipanjanS/97fdc5b8b04ddc418376d7249c35df42 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
spark_log_analytics_32.py
not200_df = (logs_df
.filter(logs_df['status'] != 200))
error_endpoints_freq_df = (not200_df
.groupBy('endpoint')
.count()
.sort('count', ascending=False)
.limit(10)
)
error_endpoints_freq_df.show(truncate=False)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment