Lists

Because they are sequences, lists support all the sequence operations we discussed for strings; the only difference is that the results are usually lists instead of strings. For instance, given a three-item list:

>>> L = [123, 'spam', 1.23]            # A list of three different-type objects
>>> len(L)                             # Number of items in the list
3

we can index, slice, and so on, just as for strings:

>>> L[0]                               # Indexing by position
123
>>> L[:-1]                             # Slicing a list returns a new list
[123, 'spam']

>>> L + [4, 5, 6]                      # Concat/repeat make new lists too
[123, 'spam', 1.23, 4, 5, 6]
>>> L * 2
[123, 'spam', 1.23, 123, 'spam', 1.23]

>>> L                                  # We're not changing the original list
[123, 'spam', 1.23]

Python’s lists may be reminiscent of arrays in other languages, but they tend to be more powerful. For one thing, they have no fixed type constraint—the list we just looked at, for example, contains three objects of completely different types (an integer, a string, and a floating-point number). Further, lists have no fixed size. That is, they can grow and shrink on demand, in response to list-specific operations:

>>> L.append('NI')                     # Growing: add object at end of list
>>> L
[123, 'spam', 1.23, 'NI']

>>> L.pop(2)                           # Shrinking: delete an item in the middle
1.23
>>> L                                  # "del L[2]" deletes from a list too
[123, 'spam', 'NI']

Here, the list append method expands the list’s size and inserts an item at the end; the pop method (or an equivalent del statement) then removes an item at a given offset, causing the list to shrink. Other list methods insert an item at an arbitrary position (insert), remove a given item by value (remove), add multiple items at the end (extend), and so on. Because lists are mutable, most list methods also change the list object in place, instead of creating a new one:

>>> M = ['bb', 'aa', 'cc']
>>> M.sort()
>>> M
['aa', 'bb', 'cc']
>>> M.reverse()
>>> M
['cc', 'bb', 'aa']

The list sort method here, for example, orders the list in ascending fashion by default, and reverse reverses it—in both cases, the methods modify the list directly.

Although lists have no fixed size, Python still doesn’t allow us to reference items that are not present. Indexing off the end of a list is always a mistake, but so is assigning off the end:

>>> L
[123, 'spam', 'NI']

>>> L[99]
...error text omitted...
IndexError: list index out of range

>>> L[99] = 1
...error text omitted...
IndexError: list assignment index out of range

This is intentional, as it’s usually an error to try to assign off the end of a list (and a particularly nasty one in the C language, which doesn’t do as much error checking as Python). Rather than silently growing the list in response, Python reports an error. To grow a list, we call list methods such as append instead.

One nice feature of Python’s core data types is that they support arbitrary nesting—we can nest them in any combination, and as deeply as we like. For example, we can have a list that contains a dictionary, which contains another list, and so on. One immediate application of this feature is to represent matrixes, or “multidimensional arrays” in Python. A list with nested lists will do the job for basic applications (you’ll get “...” continuation-line prompts on lines 2 and 3 of the following in some interfaces, but not in IDLE):

>>> M = [[1, 2, 3],               # A 3 × 3 matrix, as nested lists
         [4, 5, 6],               # Code can span lines if bracketed
         [7, 8, 9]]
>>> M
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]

Here, we’ve coded a list that contains three other lists. The effect is to represent a 3 × 3 matrix of numbers. Such a structure can be accessed in a variety of ways:

>>> M[1]                          # Get row 2
[4, 5, 6]

>>> M[1][2]                       # Get row 2, then get item 3 within the row
6

The first operation here fetches the entire second row, and the second grabs the third item within that row (it runs left to right, like the earlier string strip and split). Stringing together index operations takes us deeper and deeper into our nested-object structure.^[13]

In addition to sequence operations and list methods, Python includes a more advanced operation known as a list comprehension expression, which turns out to be a powerful way to process structures like our matrix. Suppose, for instance, that we need to extract the second column of our sample matrix. It’s easy to grab rows by simple indexing because the matrix is stored by rows, but it’s almost as easy to get a column with a list comprehension:

>>> col2 = [row[1] for row in M]             # Collect the items in column 2
>>> col2
[2, 5, 8]

>>> M                                        # The matrix is unchanged
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]

List comprehensions derive from set notation; they are a way to build a new list by running an expression on each item in a sequence, one at a time, from left to right. List comprehensions are coded in square brackets (to tip you off to the fact that they make a list) and are composed of an expression and a looping construct that share a variable name (row, here). The preceding list comprehension means basically what it says: “Give me row[1] for each row in matrix M, in a new list.” The result is a new list containing column 2 of the matrix.

List comprehensions can be more complex in practice:

>>> [row[1] + 1 for row in M]                 # Add 1 to each item in column 2
[3, 6, 9]

>>> [row[1] for row in M if row[1] % 2 == 0]  # Filter out odd items
[2, 8]

The first operation here, for instance, adds 1 to each item as it is collected, and the second uses an if clause to filter odd numbers out of the result using the % modulus expression (remainder of division). List comprehensions make new lists of results, but they can be used to iterate over any iterable object—a term we’ll flesh out later in this preview. Here, for instance, we use list comprehensions to step over a hardcoded list of coordinates and a string:

>>> diag = [M[i][i] for i in [0, 1, 2]]      # Collect a diagonal from matrix
>>> diag
[1, 5, 9]

>>> doubles = [c * 2 for c in 'spam']        # Repeat characters in a string
>>> doubles
['ss', 'pp', 'aa', 'mm']

These expressions can also be used to collect multiple values, as long as we wrap those values in a nested collection. The following illustrates using range—a built-in that generates successive integers, and requires a surrounding list to display all its values in 3.X only (2.X makes a physical list all at once):

>>> list(range(4))                           # 0..3 (list() required in 3.X)
[0, 1, 2, 3]
>>> list(range(−6, 7, 2))                    # −6 to +6 by 2 (need list() in 3.X)
[−6, −4, −2, 0, 2, 4, 6]

>>> [[x ** 2, x ** 3] for x in range(4)]     # Multiple values, "if" filters
[[0, 0], [1, 1], [4, 8], [9, 27]]
>>> [[x, x / 2, x * 2] for x in range(−6, 7, 2) if x > 0]
[[2, 1, 4], [4, 2, 8], [6, 3, 12]]

As you can probably tell, list comprehensions, and relatives like the map and filter built-in functions, are too involved to cover more formally in this preview chapter. The main point of this brief introduction is to illustrate that Python includes both simple and advanced tools in its arsenal. List comprehensions are an optional feature, but they tend to be very useful in practice and often provide a substantial processing speed advantage. They also work on any type that is a sequence in Python, as well as some types that are not. You’ll hear much more about them later in this book.

As a preview, though, you’ll find that in recent Pythons, comprehension syntax has been generalized for other roles: it’s not just for making lists today. For example, enclosing a comprehension in parentheses can also be used to create generators that produce results on demand. To illustrate, the sum built-in sums items in a sequence—in this example, summing all items in our matrix’s rows on request:

>>> G = (sum(row) for row in M)              # Create a generator of row sums
>>> next(G)                                  # iter(G) not required here
6
>>> next(G)                                  # Run the iteration protocol next()
15
>>> next(G)
24

The map built-in can do similar work, by generating the results of running items through a function, one at a time and on request. Like range, wrapping it in list forces it to return all its values in Python 3.X; this isn’t needed in 2.X where map makes a list of results all at once instead, and is not needed in other contexts that iterate automatically, unless multiple scans or list-like behavior is also required:

>>> list(map(sum, M))                        # Map sum over items in M
[6, 15, 24]

In Python 2.7 and 3.X, comprehension syntax can also be used to create sets and dictionaries:

>>> {sum(row) for row in M}                  # Create a set of row sums
{24, 6, 15}

>>> {i : sum(M[i]) for i in range(3)}        # Creates key/value table of row sums
{0: 6, 1: 15, 2: 24}

In fact, lists, sets, dictionaries, and generators can all be built with comprehensions in 3.X and 2.7:

>>> [ord(x) for x in 'spaam']                # List of character ordinals
[115, 112, 97, 97, 109]
>>> {ord(x) for x in 'spaam'}                # Sets remove duplicates
{112, 97, 115, 109}
>>> {x: ord(x) for x in 'spaam'}             # Dictionary keys are unique
{'p': 112, 'a': 97, 's': 115, 'm': 109}
>>> (ord(x) for x in 'spaam')                # Generator of values
<generator object <genexpr> at 0x000000000254DAB0>

To understand objects like generators, sets, and dictionaries, though, we must move ahead.