String Methods

As introduced in Chapter 4, methods are simply functions that are associated with and act upon particular objects. Technically, they are attributes attached to objects that happen to reference callable functions which always have an implied subject. In finer-grained detail, functions are packages of code, and method calls combine two operations at once—an attribute fetch and a call:

Putting these two together allows us to call a method of an object. The method call expression:

object.method(arguments)

is evaluated from left to right—Python will first fetch the method of the object and then call it, passing in both object and the arguments. Or, in plain words, the method call expression means this:

Call method to process object with arguments.

If the method computes a result, it will also come back as the result of the entire method-call expression. As a more tangible example:

>>> S = 'spam'
>>> result = S.find('pa')     # Call the find method to look for 'pa' in string S

This mapping holds true for methods of both built-in types, as well as user-defined classes we’ll study later. As you’ll see throughout this part of the book, most objects have callable methods, and all are accessed using this same method-call syntax. To call an object method, as you’ll see in the following sections, you have to go through an existing object; methods cannot be run (and make little sense) without a subject.

Table 7-3 summarizes the methods and call patterns for built-in string objects in Python 3.3; these change frequently, so be sure to check Python’s standard library manual for the most up-to-date list, or run a dir or help call on any string (or the str type name) interactively. Python 2.X’s string methods vary slightly; it includes a decode, for example, because of its different handling of Unicode data (something we’ll discuss in Chapter 37). In this table, S is a string object, and optional arguments are enclosed in square brackets. String methods in this table implement higher-level operations such as splitting and joining, case conversions, content tests, and substring searches and replacements.

As you can see, there are quite a few string methods, and we don’t have space to cover them all; see Python’s library manual or reference texts for all the fine points. To help you get started, though, let’s work through some code that demonstrates some of the most commonly used methods in action, and illustrates Python text-processing basics along the way.

As we’ve seen, because strings are immutable, they cannot be changed in place directly. The bytearray supports in-place text changes in 2.6, 3.0, and later, but only for simple 8-bit types. We explored changes to text strings earlier, but let’s take a quick second look here in the context of string methods.

In general, to make a new text value from an existing string, you construct a new string with operations such as slicing and concatenation. For example, to replace two characters in the middle of a string, you can use code like this:

>>> S = 'spammy'
>>> S = S[:3] + 'xx' + S[5:]          # Slice sections from S
>>> S
'spaxxy'

But, if you’re really just out to replace a substring, you can use the string replace method instead:

>>> S = 'spammy'
>>> S = S.replace('mm', 'xx')         # Replace all mm with xx in S
>>> S
'spaxxy'

The replace method is more general than this code implies. It takes as arguments the original substring (of any length) and the string (of any length) to replace it with, and performs a global search and replace:

>>> 'aa$bb$cc$dd'.replace('$', 'SPAM')
'aaSPAMbbSPAMccSPAMdd'

In such a role, replace can be used as a tool to implement template replacements (e.g., in form letters). Notice that this time we simply printed the result, instead of assigning it to a name—you need to assign results to names only if you want to retain them for later use.

If you need to replace one fixed-size string that can occur at any offset, you can do a replacement again, or search for the substring with the string find method and then slice:

>>> S = 'xxxxSPAMxxxxSPAMxxxx'
>>> where = S.find('SPAM')            # Search for position
>>> where                             # Occurs at offset 4
4
>>> S = S[:where] + 'EGGS' + S[(where+4):]
>>> S
'xxxxEGGSxxxxSPAMxxxx'

The find method returns the offset where the substring appears (by default, searching from the front), or −1 if it is not found. As we saw earlier, it’s a substring search operation just like the in expression, but find returns the position of a located substring.

Another option is to use replace with a third argument to limit it to a single substitution:

>>> S = 'xxxxSPAMxxxxSPAMxxxx'
>>> S.replace('SPAM', 'EGGS')         # Replace all
'xxxxEGGSxxxxEGGSxxxx'

>>> S.replace('SPAM', 'EGGS', 1)      # Replace one
'xxxxEGGSxxxxSPAMxxxx'

Notice that replace returns a new string object each time. Because strings are immutable, methods never really change the subject strings in place, even if they are called “replace”!

The fact that concatenation operations and the replace method generate new string objects each time they are run is actually a potential downside of using them to change strings. If you have to apply many changes to a very large string, you might be able to improve your script’s performance by converting the string to an object that does support in-place changes:

>>> S = 'spammy'
>>> L = list(S)
>>> L
['s', 'p', 'a', 'm', 'm', 'y']

The built-in list function (an object construction call) builds a new list out of the items in any sequence—in this case, “exploding” the characters of a string into a list. Once the string is in this form, you can make multiple changes to it without generating a new copy for each change:

>>> L[3] = 'x'                        # Works for lists, not strings
>>> L[4] = 'x'
>>> L
['s', 'p', 'a', 'x', 'x', 'y']

If, after your changes, you need to convert back to a string (e.g., to write to a file), use the string join method to “implode” the list back into a string:

>>> S = ''.join(L)
>>> S
'spaxxy'

The join method may look a bit backward at first sight. Because it is a method of strings (not of lists), it is called through the desired delimiter. join puts the strings in a list (or other iterable) together, with the delimiter between list items; in this case, it uses an empty string delimiter to convert from a list back to a string. More generally, any string delimiter and iterable of strings will do:

>>> 'SPAM'.join(['eggs', 'sausage', 'ham', 'toast'])
'eggsSPAMsausageSPAMhamSPAMtoast'

In fact, joining substrings all at once might often run faster than concatenating them individually. Be sure to also see the earlier note about the mutable bytearray string available as of Python 3.0 and 2.6, described fully in Chapter 37; because it may be changed in place, it offers an alternative to this list/join combination for some kinds of 8-bit text that must be changed often.

Another common role for string methods is as a simple form of text parsing—that is, analyzing structure and extracting substrings. To extract substrings at fixed offsets, we can employ slicing techniques:

>>> line = 'aaa bbb ccc'
>>> col1 = line[0:3]
>>> col3 = line[8:]
>>> col1
'aaa'
>>> col3
'ccc'

Here, the columns of data appear at fixed offsets and so may be sliced out of the original string. This technique passes for parsing, as long as the components of your data have fixed positions. If instead some sort of delimiter separates the data, you can pull out its components by splitting. This will work even if the data may show up at arbitrary positions within the string:

>>> line = 'aaa bbb  ccc'
>>> cols = line.split()
>>> cols
['aaa', 'bbb', 'ccc']

The string split method chops up a string into a list of substrings, around a delimiter string. We didn’t pass a delimiter in the prior example, so it defaults to whitespace—the string is split at groups of one or more spaces, tabs, and newlines, and we get back a list of the resulting substrings. In other applications, more tangible delimiters may separate the data. This example splits (and hence parses) the string at commas, a separator common in data returned by some database tools:

>>> line = 'bob,hacker,40'
>>> line.split(',')
['bob', 'hacker', '40']

Delimiters can be longer than a single character, too:

>>> line = "i'mSPAMaSPAMlumberjack"
>>> line.split("SPAM")
["i'm", 'a', 'lumberjack']

Although there are limits to the parsing potential of slicing and splitting, both run very fast and can handle basic text-extraction chores. Comma-separated text data is part of the CSV file format; for more advanced tools on this front, see also the csv module in Python’s standard library.

Other string methods have more focused roles—for example, to strip off whitespace at the end of a line of text, perform case conversions, test content, and test for a substring at the end or front:

>>> line = "The knights who say Ni!\n"
>>> line.rstrip()
'The knights who say Ni!'
>>> line.upper()
'THE KNIGHTS WHO SAY NI!\n'
>>> line.isalpha()
False
>>> line.endswith('Ni!\n')
True
>>> line.startswith('The')
True

Alternative techniques can also sometimes be used to achieve the same results as string methods—the in membership operator can be used to test for the presence of a substring, for instance, and length and slicing operations can be used to mimic endswith:

>>> line
'The knights who say Ni!\n'

>>> line.find('Ni') != −1       # Search via method call or expression
True
>>> 'Ni' in line
True

>>> sub = 'Ni!\n'
>>> line.endswith(sub)          # End test via method call or slice
True
>>> line[-len(sub):] == sub
True

See also the format string formatting method described later in this chapter; it provides more advanced substitution tools that combine many operations in a single step.

Again, because there are so many methods available for strings, we won’t look at every one here. You’ll see some additional string examples later in this book, but for more details you can also turn to the Python library manual and other documentation sources, or simply experiment interactively on your own. You can also check the help(S.method) results for a method of any string object S for more hints; as we saw in Chapter 4, running help on str.method likely gives the same details.

Note that none of the string methods accepts patterns—for pattern-based text processing, you must use the Python re standard library module, an advanced tool that was introduced in Chapter 4 but is mostly outside the scope of this text (one further brief example appears at the end of Chapter 37). Because of this limitation, though, string methods may sometimes run more quickly than the re module’s tools.

The history of Python’s string methods is somewhat convoluted. For roughly the first decade of its existence, Python provided a standard library module called string that contained functions that largely mirrored the current set of string object methods. By popular demand, in Python 2.0 these functions were made available as methods of string objects. Because so many people had written so much code that relied on the original string module, however, it was retained for backward compatibility.

Today, you should use only string methods, not the original string module. In fact, the original module call forms of today’s string methods have been removed completely from Python 3.X, and you should not use them in new code in either 2.X or 3.X. However, because you may still see the module in use in older Python 2.X code, and this text covers both Pythons 2.X and 3.X, a brief look is in order here.

The upshot of this legacy is that in Python 2.X, there technically are still two ways to invoke advanced string operations: by calling object methods, or by calling string module functions and passing in the objects as arguments. For instance, given a variable X assigned to a string object, calling an object method:

X.method(arguments)

is usually equivalent to calling the same operation through the string module (provided that you have already imported the module):

string.method(X, arguments)

Here’s an example of the method scheme in action:

>>> S = 'a+b+c+'
>>> x = S.replace('+', 'spam')
>>> x
'aspambspamcspam'

To access the same operation through the string module in Python 2.X, you need to import the module (at least once in your process) and pass in the object:

>>> import string
>>> y = string.replace(S, '+', 'spam')
>>> y
'aspambspamcspam'

Because the module approach was the standard for so long, and because strings are such a central component of most programs, you might see both call patterns in Python 2.X code you come across.

Again, though, today you should always use method calls instead of the older module calls. There are good reasons for this, besides the fact that the module calls have gone away in 3.X. For one thing, the module call scheme requires you to import the string module (methods do not require imports). For another, the module makes calls a few characters longer to type (when you load the module with import, that is, not using from). And, finally, the module runs more slowly than methods (the module maps most calls back to the methods and so incurs an extra call along the way).

The original string module itself, without its string method equivalents, is retained in Python 3.X because it contains additional tools, including predefined string constants (e.g., string.digits) and a Template object system—a relatively obscure formatting tool that predates the string format method and is largely omitted here (for details, see the brief note comparing it to other formatting tools ahead, as well as Python’s library manual). Unless you really want to have to change your 2.X code to use 3.X, though, you should consider any basic string operation calls in it to be just ghosts of Python past.