List comprehensions are great at what they do. But maps are something rather different, and they have their place too.
A list comprehension is a clever little Python construct that can build a list from another list (or other iterable). For example:
k = [x + 3 for x in s]
This code takes an iterable
sand adds 3 to each element, returning the result as a list. So if
[1, 2, 3]the result would be
[4, 5, 6].
We could do this with a the map function:
k = list(map(lambda x: x + 3, s))
This is obviously longer and uglier than the list comprehension. That is hardly surprising because this example is exactly what list comprehensions are for.
The map function has a different purpose, although there is some overlap. It is a functional programming construct that applies a function to one or more iterables, and returns a lazy iterator as a result.
The list comprehension example presupposes that we want a list as a result. But we don’t always need a list. For example, if we were intending to loop over
kwe wouldn’t really care if it was a list, an iterator would do just as well. So our example would become:
k = map(lambda x: x + 3, s)
And if we are taking a functional approach, we might already have a curried add function,
addc(3)returns a new function that adds 3 to its argument. In other words, it returns the equivalent of the previous lambda function.
Currying is covered in more detail in my book Functional Programming in Python.
Our example now becomes:
k = map(addc(3), s)
Now this is already starting to look more readable than the list comprehension. It is certainly more declarative — it says what it does (mapping a function onto an iterable), rather than how it does it (the irrelevant variable and for statement in the list comprehension). But there is more.
Maps can take multiple arguments
Suppose we wanted to compare the corresponding elements of two iterables, and retain the largest value of each pair. To do this with a list comprehension would most likely need zip:
k = [max(x, y) for x, y in zip(a, b)]
This really contains far too much implementation detail, compared to the map equivalent:
k = map(max, a, b)
This simply applies the max function successive pairs of elements drawn from
Map is lazy
Map uses lazy iteration. What does this mean? Well in the following example:
a = map(fn_1, s)
r = sum(a)
The initial call to map doesn’t call the function
fn_1at all. In fact it doesn’t even read values from
s. The map functions just returns a map object (which is a type of iterator). When the sum function starts to consume the values from the map object, the map object starts requesting values from
s. This applies with chained map functions like this:
a = map(fn_1, s)
b = map(fn_2, a)
c = map(fn_3, b)
r = sum(c)
This code sets up a processing pipeline that reads values from
sand passes them through the three functions in turn. But none of that happens until
sumstarts consuming values.
This has two useful consequences:
- You can split complex calculations into stages without requiring excessive amounts of memory, as shown above. If we split a list comprehension into several stages, each stage would create a complete, in-memory list of the intermediate results.
- Elements are passed through the pipeline one by one. So the first element is processed by each function, then the second element is processed by each function, and so on. This means that there is minimal latency in getting the first result. With list comprehensions, every element has to be processed by the first function, then every element is processed by the second function, etc so it can take a long time to see any results.
In some applications, this can be critical. For example if you are processing a stream of messages coming over a network, the initial iterable
sis potentially infinite — you can’t wait for it to end before you process the first message.
In other situations, it might be infeasible to store all the data at once. For example, if
sis a stream of video frames, you will probably want to process each one individually. Maps are naturally suited to this.
List comprehensions are great. But so are maps. It doesn’t always matter which one you use, but there are situations when maps can make a significant improvement to the readability and efficiency of your code.