Skip to content

Instantly share code, notes, and snippets.

@jtrive84

jtrive84/Python_Higher_Order_Functions.md

Created November 28, 2016 02:21
Show Gist options
  • Save jtrive84/29ab2961ae4b52f2006e21edacdab2e8 to your computer and use it in GitHub Desktop.
Save jtrive84/29ab2961ae4b52f2006e21edacdab2e8 to your computer and use it in GitHub Desktop.
Download ZIP
Introduction to Python's higher order functions and typical usage scenarios.
Raw
Python_Higher_Order_Functions.md

Higher Order Functions

Relevant Links:

Python Functional Programming HOWTO
Python's itertools module
Python Sorting Wiki

From the builtin functions section of the Python docs, the call signatures for a few of the higher-order functions we'll be covering are:

max(iterable, [, key, default])
min(iterable, [, key, default])
sort(key=None, reverse=None)
sorted(iterable[, key][, reverse])

Anonymous Functions: lambda

Besides the def statement, Python provides an expression form that generates function objects. Like def, the expression creates a function to be called later, but it returns the function instead of assigning it to a name. This is why lambdas are sometimes known as anonymous functions. In practice, they are often used as a way to inline a function definition, or to defer execution of a piece of code.

The lambda's general form is the keyword lambda, followed by one or more arguments (exactly like with def), followed by an expression after a colon:

lambda arg1, arg2, ... argN: expression using arg1, arg2, ... , argN

Function objects returned by running lambda work exactly the same as those created and assigned by def's, but there are a few differences that makelambdas useful in specialized roles:

  • lambda is an expression, not a statement. Because of this, a lambda can appear in places a def is not permitted by Python's syntax. As an expression, lambda returns a value (a new function) that can optionally be assigned a name. In contrast, the def statement always assigns the new function to the name in the header, instead of returning it as a result.

  • The body of a lambda is limited to a single expression. The lambda's body is similiar to what you'd put in a def body's return statement: you type the result as an expression, instead of explicitly returning it. Because it is limited to an expression, a lambda is less general than a def, since you can only squeeze only so much logic into a lambda. This is by design, to limit program nesting: lambda is designed for coding simple functions, and def handles larger tasks.

What follows are examples using the lambda expression:

# lambda function to add 2 numbers:
>>> adder = lambda x, y: x + y
>>> adder(5, 8)
13

# binding lambda to another variable:
>>> adder = lambda x, y: x + y
>>> f = adder
>>> f(5,8)
13

# using lambda to implement ternary operator:
>>> t = lambda n: True if n%2==0 else False
>>> t(7)
False

# using lambda to extract item at index 3 in seq:
>>> l = [1,2,3,4,5]
>>> extract = lambda seq: seq[3]
>>> extract(l)
4

# using lambda to extract a dict element:
>>> records = {'FIELD_1': 300,'FIELD_2': 600,'FIELD_3': 900}
>>> extract = lambda d: d['FIELD_1']
>>> extract(records)
300

lambda can also be used along with Python's Higher Order Functions.

The call signatures for min and max:

max(iterable, [, key])
min(iterable, [, key])

The key argument specifies a one-argument ordering function.

For non-nested sequences, the behavior of min and max is predictible and returns values as expected:

>>> vals = [4, 6, 8, 10, 12, 14]
>>> print("max of `vals`: {}".format(max(vals)))
max of `vals`: 14

>>> print("min of `vals`: {}".format(min(vals)))
min of `vals`: 4

However, the behavior isn't as clearly defined when dealing with nested iterables:

>>> vals = [(.24, 7), (.12, 14), (.06, 28), (.03, 56)]
>>> print("max of `vals`: {}".format(max(vals)))
max of `vals`: (.24, 7)

>>> print("min of `vals`: {}".format(min(vals)))
min of `vals`: (.03, 56)

Or when dealing with nested iterables containing strings:

>>> vals = [("Monday"   , .24,  7), 
            ("Tuesday"  , .12, 14), 
            ("Wednesday", .06, 28), 
            ("Thursday" , .03, 56)]
            
>>> print("max of `vals`: {}".format(max(vals)))
max of `vals`: ('Wednesday', 0.06, 28)

>>> print("min of `vals`: {}".format(min(vals)))
min of `vals`: ('Monday', 0.24, 7)

The default behavior of min and max is to compare items at the lowest level of nesting's 0th-index if no further instruction is given.

min and max are Higher Order Functions, in that they both can take another function as an argument. Recall the lambda form that returned an element from a given sequence:

>>> l = [1,2,3,4,5]
>>> extract = lambda seq: seq[3]
>>> extract(l)
4

extract returns the 4th element (index 3 with a 0-based index) of seq. We can use an anonymous lambda expression, instructing min and max to perform the test based on an element other than the 0th index of the nested iterable. In vals below, here's an example of varying the index with which to test:

>>> vals = [("Monday"   , .24,  7, "A"), 
            ("Tuesday"  , .12, 14, "R"), 
            ("Wednesday", .06, 28, "X"), 
            ("Thursday" , .03, 56, "U")]

# min and max of vals based on 0th index (default behavior):
>>> min_vals = min(vals)
>>> max_vals = max(vals)
>>> print("min of `vals` based on index 0: {}".format(min_vals))
min of `vals` based on index 0: ('Monday', 0.24, 7, 'A')

>>> print("max of `vals` based on index 0: {}".format(max_vals))
max of `vals` based on index 0: ('Wednesday', 0.06, 28, 'X')

We can modify the key with which to compare by passing a lambda expression to the key argument of min and max:

# min and max of vals based on 1st index:
>>> vals = [("Monday"   , .24,  7, "A"), 
            ("Tuesday"  , .12, 14, "R"), 
            ("Wednesday", .06, 28, "X"), 
            ("Thursday" , .03, 56, "U")]
>>> min_vals = min(vals, key=lambda x: x[1])
>>> max_vals = max(vals, key=lambda x: x[1])

>>> print("min of `vals` based on index 1: {}".format(min_vals))
min of `vals` based on index 1: ('Thursday', 0.03, 56, 'U')

>>> print("max of `vals` based on index 1: {}".format(max_vals))
max of `vals` based on index 1: ('Monday', 0.24, 7, 'A')


# min and max of vals based on 2nd index:
>>> vals = [("Monday"   , .24,  7, "A"), 
            ("Tuesday"  , .12, 14, "R"), 
            ("Wednesday", .06, 28, "X"), 
            ("Thursday" , .03, 56, "U")]
>>> min_vals = min(vals, key=lambda x: x[2])
>>> max_vals = max(vals, key=lambda x: x[2])

>>> print("min of `vals` based on index 2: {}".format(min_vals))
min of `vals` based on index 2: ('Monday', 0.24, 7, 'A')

>>> print("max of `vals` based on index 2: {}".format(max_vals))
max of `vals` based on index 2: ('Thursday', 0.03, 56, 'U')


# min and max of vals based on 3rd index:
>>> vals = [("Monday"   , .24,  7, "A"), 
            ("Tuesday"  , .12, 14, "R"), 
            ("Wednesday", .06, 28, "X"), 
            ("Thursday" , .03, 56, "U")]
>>> min_vals = min(vals, key=lambda x: x[3])
>>> max_vals = max(vals, key=lambda x: x[3])

>>> print("min of `vals` based on index 3: {}".format(min_vals))
min of `vals` based on index 3: ('Monday', 0.24, 7, 'A')

>>> print("max of `vals` based on index 3: {}".format(max_vals))
max of `vals` based on index 3: ('Wednesday', 0.06, 28, 'X')

Sorting

Python's sort and sorted functions are also Higher Ordered Functions. A similiar construct to using lambda's can be used to sort nested sequences. There are two common approaches:

sort(key=None, reverse=None)       # Strictly a list method: sorts list in place
sorted(iterable[, key][, reverse]) # Avaiable for any iterable. Return a new sorted list

Typical usage:

# calling `sort` on list:
>>> lst = [33, 23, 67, 14]
>>> print("Unsorted `lst`: {}".format(lst))
Unsorted `lst`: [33, 23, 67, 14]

# call lst.sort():
>>> lst.sort()
>>> print("Sorted `lst`  : {}".format(lst))
Sorted `lst`  : [14, 23, 33, 67]

Note that sort will not work on any data structure other than lists:

# calling `sort` on a tuple:
>>> tpl = (33, 23, 67, 14)
>>> print("Unsorted tpl: {}".format(tpl))
Unsorted tpl: (33, 23, 67, 14)

# call lst.sort():
>>> tpl.sort()
>>> print("Sorted lst  : {}".format(tpl))
AttributeError: 'tuple' object has no attribute 'sort'

sorted will work with any iterable, nested or otherwise:

>>> lst = [33, 23, 67, 14]
>>> tpl = (33, 23, 67, 14)
>>> strs = "Andromeda"

>>> print("sorted(lst) : {}".format(sorted(lst)))
sorted(lst) : [14, 23, 33, 67]

>>> print("sorted(tpl) : {}".format(sorted(tpl)))
sorted(tpl) : [14, 23, 33, 67]

>>> print("sorted(strs): {}".format(sorted(strs)))
sorted(strs): ['A', 'a', 'd', 'd', 'e', 'm', 'n', 'o', 'r']

And, as before, if we have a nested sequence, we can provide a key indicating which index to use for comparison:

# (ticker, close price, volume)
>>> closing = [("PGR", 31.51 , 3400000), 
               ("TRV", 117.18, 1506000), 
               ("CNA", 32.09 ,  152000)]

# sorting by various tuple elements:
>>> print("Sorting by ticker: {}".format(sorted(closing,key=lambda x: x[0])))
Sorting by ticker: [('CNA', 32.09, 152000), ('PGR', 31.51, 3400000), ('TRV', 117.18, 1506000)]

>>> print("Sorting by close : {}".format(sorted(closing,key=lambda x: x[1])))
Sorting by close : [('PGR', 31.51, 3400000), ('CNA', 32.09, 152000), ('TRV', 117.18, 1506000)]

>>> print("Sorting by volume: {}".format(sorted(closing,key=lambda x: x[2])))
Sorting by volume : [('CNA', 32.09, 152000), ('TRV', 117.18, 1506000), ('PGR', 31.51, 3400000)]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment