Chapter 8 Functional Programming
This chapter introduces techniques from Functional Programming, which is a programming paradigm centered on functions rather than on variables and statements as you’ve been doing so far (known as imperative programming). Functional programming offers another way to think about giving “instructions” to a computer, which can make it easier to think about and implement some algorithms. While not a completely functional language, Python does contain a number of “functional-programming-like” features that can be mixed with the imperative strategies we’re used to, allowing for more compact, readable, and “Pythonic” code in some cases.
8.1 Functions ARE Variables
Previously we’ve described functions as “named sequences of instructions”, or groupings of lines of code that are given a name. But in a functional programming paradigm, functions are first-class objects—that is, they are “things” (values) that can be organized and manipulated just like variables.
In Python, functions ARE variables:
# create a function called `say_hello` def say_hello(name): print("Hello, "+name) # what kind of thing is `say_hello` ? type(say_hello) # <class 'function'>
x = 3 defines a variable for a value of type
msg = "hello" defines a variable for a value of type
string, the above
say_hello function is actually a variable for a value of type
Note that we refer to the function by its name without the parentheses!
This is why it is possible to accidentally “overwrite” built-in functions by assigning values to variables like
The fact that functions are variables is the core realization to make when programming in a functional style. You need to be able to think about functions as things (nouns), rather than as behaviors (verbs). If you imagine that functions are “recipes”, then you need to think about them as pages from the cookbook (that can be bound together or handed to a friend), rather than just the sequence of actions that they tell you to perform.
And because functions are just another type of variable, they can be used anywhere that a “regular” variable can be used. For example, functions are values, so they can be assigned to other variables!
# Create a function `say_hello` def say_hello(name): print("Hello, "+name) # Assign the `say_hello` value to a new variable `greet` greet = say_hello # Call the function assigned to the `greet` variable greet("world") # prints "Hello world"
It helps to think of functions as just a special kind of list. Just as lists have a special syntax
 (bracket notation) that can be used to “get” a value from the list, functions have a special syntax
() (parentheses) that can be used to “call” and run the function.
Moreover, functions are values, so they can be passed as parameters to other functions!
# Create a function `say_hello` def say_hello(name): print("Hello, "+name) # A function that takes ANOTHER FUNCTION as an argument # This function will call the argument function, passing it "world" def do_with_world(func_to_call): # call the given function with an argument of "world" func_to_call("world") # Call `do_with_world`, saying the "thing to do" is `say_hello` do_with_world(say_hello) # prints "Hello world"
In this case, the
do_with_world function will call whatever function it is given, passing in a value of
"world". (You can think of this as similar to having a function that accesses the
'world' key of a given dictionary).
Important note: when you pass
say_hello as an argument, you don’t put any parentheses after it! Putting the parentheses after the function name calls the function, causing it to perform the lines of code it defines. This will cause the expression containing the function to resolve to its returned value, rather than being the function value itself. It’s like passing in the baked cake rather than the recipe page.
def greet(): # version with no args return "Hello" # Print out the function print(greet) # prints <function greet>, the function # Resolve the expression, then print that out print(greet()) # prints "Hello", which is what `greet()` resolves to.
A function that is passed into another is commonly referred to as a callback function: it is an argument that the other function will “call back to” and execute when needed.
Functions can take more than one callback function as arguments, which can be a useful way of composing behaviors.
def do_at_once(first_callback, second_callback): first_callback() # execute the first function second_callback() # execute the second function print("at the same time! ") def pat_head(): print("pat your head", end=" ") def rub_belly(): print("rub your belly", end=" ") # Pass in the callbacks to "do at once" do_at_once(pat_head, rub_belly)
This idea of passing functions are arguments to other functions is at the heart of functional programming, and is what gives it expressive power: you can define program behavior primarily in terms of the behaviors that are run, and less in terms of the data variables used.
8.1.1 lambdas: Anonymous Functions
You have previously used anonymous variables in your programs, or values which are not assigned a variable name (so remain anonymous). These values were defined as literals or expressions and passed directly into functions, rather than assigning them to variables:
my_list = [1,2,3] # a named variable (not anonymous) print(my_list) # pass in non-anonymous variable print([1,2,3]) # pass in anonymous value
Because functions are variables, it is also possible to define anonymous functions: functions that are not given a name, but instead are passed directly into other functions. In Python, these anonymous functions are referred to as lambdas (named after lambda calculus, which is a way of defining algorithms in terms of functions).
Lambdas are written using the following general syntax:
lambda arg1, arg2: expression_to_return
You indicate that you are defining a lambda function with the keyword
lambda (rather than the keyword
def used for named functions). This is followed by a list of arguments separated by commas (what normally goes inside the
() parentheses in a named function definition), then a colon
:, then the expression that will be returned by the anonymous function.
For example, compare the following named and anonymous function definitions:
# Named function to square a value def square(x): return x**2 # Anonymous function to square a value lambda x: x**2 # Named function to combine first and last name def make_full_name(first, last): return first + " " + last # Anonymous function to combine first and last name lambda first, last: first + " " + last
You’re basically replacing
def and the function name with the word
lambda, removing the parentheses around the arguments, and removing the
Just as other expressions can be assigned to variables, lambda functions can be assigned to variables in order to give them a name. This is the equivalent of having defined them as named functions in the first place:
square = lambda x: x**2 make_full_name = lambda first, last: first + " " + last
There is one major restriction on what kind of functions can be defined as anonymous lambdas: they must be functions that consist of only a single returned expression. That is, they need to be a function that contains exactly one line of code, which is a
return statement (as in the above examples). This means that lambdas are short functions that usually perform very simple transformations to the arguments… exactly what you want to do with functional programming!
8.2 Functional Looping
Why should you care about treating functions as variables, or defining anonymous lambda functions? Because doing so allows you to replace loops with function calls in some situations. For particular kinds of loops, this can make the code more expressive (more clearly indicative of what it is doing).
For example, consider the following loop:
def square(n): # a function that squares a number return n**2 numbers = [1,2,3,4,5] # an initial list squares =  # the transformed list for number in numbers: transformed = square(number) squares.append(transformed) print(squares) # [1, 4, 9, 16, 25]
This loop represents a mapping operation: it takes an original list (e.g., of numbers 1 to 5) and produces a new list with each of the original elements transformed in a certain way (e.g., squared). This is a common operation to apply: maybe you want to “transform” a list so that all the values are rounded or lowercase, or you want to map a list of words to a list of their lengths. It is possible to make these changes uses the same pattern as above: create an empty list, then loop through the original list and
append the transformed values to the new list.
However, Python also provides a built-in function called
map() that directly performs this kind of mapping operation on a list without needing to use a loop:
def square(n): # a function that squares a number return n**2 numbers = [1,2,3,4,5] # an initial list squares = list(map(square, numbers)) print(squares) # [1, 4, 9, 16, 25]
map() function takes a list and produces a new list with each of the elements transformed. The
map() function should be passed two arguments: the second is the list to transform, and the first is the name of a callback function that will do the transformation. This callback function must take in a single argument (an element to transform) and return a value (the transformed element).
Note that in Python 3, the
map() function returns an iterator, which is a list-like sequence similar to that returned by a dictionary’s
items() methods. Thus in order to interact with it as a list, it needs to be converted using the
map() callback function (e.g.,
square() in the above example) can also be specified using an anonymous lambda, which allows for concisely written code (but often at the expense of readability—see List Comprehensions below for a more elegant, Pythonic solution).
numbers = [1,2,3,4,5] # an initial list squares = list(map(lambda n:n**2, numbers))
A second common operation is to filter a list of elements, removing elements that you don’t want (or more accurately: only keeping elements that you DO want). For example, consider the following loop:
def is_even(n): # a function that determines if a number is even remainder = n % 2 # get remainder when dividing by 2 (modulo operator) return remainder == 0 # True if no remainder, False otherwise numbers = [2,7,1,8,3] # an initial list evens =  # the filtered list for number in numbers: if is_even(number): evens.append(number) print(evens) # [2, 8]
With this filtering loop, we are keeping the values for which the
is_even() function returns true (the function determines “what to let in” not “what to keep out”), which we do by appending the “good” values to a new list.
map(), Python provides a built-in function called
filter() that will directly perform this filtering:
def is_even(n): # a function that determines if a number is even return (n % 2) == 0 # True if no remainder, False otherwise numbers = [2,7,1,8,3] # an initial list evens = list(filter(is_even, numbers)) print(evens) # [2, 8]
filter() function takes a list and produces a new list that contains only the elements that do match a specific criteria. The
filter() function takes in two arguments: the second is the list to filter, and the first is the name of a callback function that will do the filtering. This callback function must take in a single argument (an element to consider) and return
True if the element should be included in the filtered list (or
False if it should not be included).
filter() both produce list-like sequences, it is possible to take the returned value from one function and pass it in as the argument to the next. For example:
numbers = [1,2,3,4,5,6] # Get the squares of EVEN numbers only filtered = filter(is_even, numbers) # filter the numbers squares = map(square, filtered) # map the filtered values print(list(squares)) # [4, 16, 36] # Or in one statement, passing results anonymously squares = map(square, filter(is_even, numbers)) # watch out for the parentheses! print(list(squares)) # [4, 16, 36]
This structure can potentially make it easier to understand the code’s intent: it is “
The third important operation in functional programming (besides mapping and filtering) is reducing a list. Reducing a list means to aggregate that lists values together, transforming the list into a single value. For example, the built-in
sum() function is a reducing operation (and in fact, the most common one!): it reduces a list of numbers to a single summed value. Thus you can think of
reduce() as a generalization of the
sum() function—but rather than just adding (
+) the values together,
reduce() allows you to specify what operation to perform when aggregating (e.g., multiplication).
reduce() function can be complex to interpret, it was actually removed from the set of “core” built-in functions in Python 3 and relegated to the
functools module. Thus you need to import the function in order to use it:
from functools import reduce
To understand how a reduce operation works, consider the following basic loop:
def add_twice(initial, new_amount): # a function that adds a second number twice return initial + new_amount + new_amount numbers = [1,2,3,4,5] # an initial list running_total = 0 # an accumulated aggregate for number in numbers: running_total = add_twice(running_total, number) print(running_total) # 30 (0+2*1+2*2+2*3+2*4+2*5)
This loop reduces the list into an “accumulated” product (factorial) of all the numbers in the list. Inside the loop, the
multiply() function is called and passed the “current total” and the “new value” to be combined into the aggregate (in that order). The resulting total is then reassigned as the “current total” for the next iteration.
reduce() function does exactly this work: it takes as arguments a callback function used to combine the current running total with the new value, and a list of values to combine. Whereas the
filter() callback functions each took 1 argument, the
reduce() callback function requires 2 arguments: the first will be the “running total”, and the second will be the “new value” to mix into the aggregate. (While this ordering doesn’t influence the factorial example, it is relevant for other operations):
def add_twice(initial, new_amount): # a function that adds a second number twice return initial + new_amount + new_amount numbers = [1,2,3,4,5] # an initial list product = reduce(add_twice, numbers, 0) # start aggregating with 0 print(running_total) # 30 (0+2*1+2*2+2*3+2*4+2*5)
Note that the
reduce() function aggregates into a single value, so the result doesn’t need to be converted from an iterator to a list!
To summarize, the
reduce() operations work as follows:
All together, the map, filter, and reduce operations form the basis for a functional consideration of a program. Indeed, these kinds of operations are very common when discussing data manipulations: for example, the famous MapReduce model involves “mapping” each element through a complex function (on a different computer no less!), and then “reducing” the results into a single answer.
- Functional Programming in Python (IBM) (note: Python 2)
- Map, Filter, Lambda, and List Comprehensions in Python (note: Python 2)
- Functional Programming in Python (O’Reilly) (short eBook)
- Functional Programming HOWTO (advanced, not recommended)