Skip to content

Instantly share code, notes, and snippets.

@dennyglee

dennyglee/define_window_specification.py

Created March 24, 2016 18:55
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save dennyglee/c2bad25837c5d4ad0193 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save dennyglee/c2bad25837c5d4ad0193 to your computer and use it in GitHub Desktop.
Download ZIP
Introducing Window Functions in Spark SQL: Define window specification
Raw
define_window_specification.py
from pyspark.sql.window import Window
# Defines partitioning specification and ordering specification.
windowSpec = \
Window \
.partitionBy(...) \
.orderBy(...)
# Defines a Window Specification with a ROW frame.
windowSpec.rowsBetween(start, end)
# Defines a Window Specification with a RANGE frame.
windowSpec.rangeBetween(start, end)
from pyspark.sql.window import Window
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment