Clients can use the simple module file we just wrote by running an import or from statement. Both statements find, compile, and run a module file’s code, if it hasn’t yet been loaded. The chief difference is that import fetches the module as a whole, so you must qualify to fetch its names; in contrast, from fetches (or copies) specific names out of the module.
Let’s see what this means in terms of code. All of the following examples wind up calling the printer function defined in the prior section’s module1.py module file, but in different ways.
In the first example, the name module1 serves two different purposes—it identifies an external file to be loaded, and it becomes a variable in the script, which references the module object after the file is loaded:
>>>import module1# Get module as a whole (one or more) >>>module1.printer('Hello world!')# Qualify to get names Hello world!
The import statement simply lists one or more names of modules to load, separated by commas. Because it gives a name that refers to the whole module object, we must go through the module name to fetch its attributes (e.g., module1.printer).
By contrast, because from copies specific names from one file over to another scope, it allows us to use the copied names directly in the script without going through the module (e.g., printer):
>>>from module1 import printer# Copy out a variable (one or more) >>>printer('Hello world!')# No need to qualify name Hello world!
This form of from allows us to list one or more names to be copied out, separated by commas. Here, it has the same effect as the prior example, but because the imported name is copied into the scope where the from statement appears, using that name in the script requires less typing—we can use it directly instead of naming the enclosing module. In fact, we must; from doesn’t assign the name of the module itself.
As you’ll see in more detail later, the from statement is really just a minor extension to the import statement—it imports the module file as usual (running the full three-step procedure of the preceding chapter), but adds an extra step that copies one or more names (not objects) out of the file. The entire file is loaded, but you’re given names for more direct access to its parts.
Finally, the next example uses a special form of from: when we use a * instead of specific names, we get copies of all names assigned at the top level of the referenced module. Here again, we can then use the copied name printer in our script without going through the module name:
>>>from module1 import *# Copy out _all_ variables >>>printer('Hello world!')Hello world!
Technically, both import and from statements invoke the same import operation; the from * form simply adds an extra step that copies all the names in the module into the importing scope. It essentially collapses one module’s namespace into another; again, the net effect is less typing for us. Note that only * works in this context; you can’t use pattern matching to select a subset of names (though you could with more work and a loop through a module’s __dict__, discussed ahead).
And that’s it—modules really are simple to use. To give you a better understanding of what really happens when you define and use modules, though, let’s move on to look at some of their properties in more detail.
In Python 3.X, the from ...* statement form described here can be used only at the top level of a module file, not within a function. Python 2.X allows it to be used within a function, but issues a warning anyhow. It’s rare to see this statement used inside a function in practice; when present, it makes it impossible for Python to detect variables statically, before the function runs. Best practice in all Pythons recommends listing all your imports at the top of a module file; it’s not required, but makes them easier to spot.
One of the most common questions people seem to ask when they start using modules is, “Why won’t my imports keep working?” They often report that the first import works fine, but later imports during an interactive session (or program run) seem to have no effect. In fact, they’re not supposed to. This section explains why.
Modules are loaded and run on the first import or from, and only the first. This is on purpose—because importing is an expensive operation, by default Python does it just once per file, per process. Later import operations simply fetch the already loaded module object.
As one consequence, because top-level code in a module file is usually executed only once, you can use it to initialize variables. Consider the file simple.py, for example:
print('hello')
spam = 1 # Initialize variable
In this example, the print and = statements run the first time the module is imported, and the variable spam is initialized at import time:
%python>>>import simple# First import: loads and runs file's code hello >>>simple.spam# Assignment makes an attribute 1
Second and later imports don’t rerun the module’s code; they just fetch the already created module object from Python’s internal modules table. Thus, the variable spam is not reinitialized:
>>>simple.spam = 2# Change attribute in module >>>import simple# Just fetches already loaded module >>>simple.spam# Code wasn't rerun: attribute unchanged 2
Of course, sometimes you really want a module’s code to be rerun on a subsequent import. We’ll see how to do this with Python’s reload function later in this chapter.
Just like def, import and from are executable statements, not compile-time declarations. They may be nested in if tests, to select among options; appear in function defs, to be loaded only on calls (subject to the preceding note); be used in try statements, to provide defaults; and so on. They are not resolved or run until Python reaches them while executing your program. In other words, imported modules and names are not available until their associated import or from statements run.
Also, like def, the import and from are implicit assignments:
import assigns an entire module object to a single name.
from assigns one or more names to objects of the same names in another module.
All the things we’ve already discussed about assignment apply to module access, too. For instance, names copied with a from become references to shared objects; as with function arguments, reassigning a copied name has no effect on the module from which it was copied, but changing a shared mutable object through a copied name can also change it in the module from which it was imported. To illustrate, consider the following file, small.py:
x = 1 y = [1, 2]
When importing with from, we copy names to the importer’s scope that initially share objects referenced by the module’s names:
%python>>>from small import x, y# Copy two names out >>>x = 42# Changes local x only >>>y[0] = 42# Changes shared mutable in place
Here, x is not a shared mutable object, but y is. The names y in the importer and the importee both reference the same list object, so changing it from one place changes it in the other:
>>>import small# Get module name (from doesn't) >>>small.x# Small's x is not my x 1 >>>small.y# But we share a changed mutable [42, 2]
For more background on this, see Chapter 6. And for a graphical picture of what from assignments do with references, flip back to Figure 18-1 (function argument passing), and mentally replace “caller” and “function” with “imported” and “importer.” The effect is the same, except that here we’re dealing with names in modules, not functions. Assignment works the same everywhere in Python.
Recall from the preceding example that the assignment to x in the interactive session changed the name x in that scope only, not the x in the file—there is no link from a name copied with from back to the file it came from. To really change a global name in another file, you must use import:
%python>>>from small import x, y# Copy two names out >>>x = 42# Changes my x only >>>import small# Get module name >>>small.x = 42# Changes x in other module
This phenomenon was introduced in Chapter 17. Because changing variables in other modules like this is a common source of confusion (and often a bad design choice), we’ll revisit this technique again later in this part of the book. Note that the change to y[0] in the prior session is different; it changes an object, not a name, and the name in both modules references the same, changed object.
Notice in the prior example that we have to execute an import statement after the from to access the small module name at all. from only copies names from one module to another; it does not assign the module name itself. At least conceptually, a from statement like this one:
from module import name1, name2 # Copy these two names out (only)
is equivalent to this statement sequence:
import module # Fetch the module object name1 = module.name1 # Copy names out by assignment name2 = module.name2 del module # Get rid of the module name
Like all assignments, the from statement creates new variables in the importer, which initially refer to objects of the same names in the imported file. Only the names are copied out, though, not the objects they reference, and not the name of the module itself. When we use the from * form of this statement (from module import *), the equivalence is the same, but all the top-level names in the module are copied over to the importing scope this way.
Notice that the first step of the from runs a normal import operation, with all the semantics outlined in the preceding chapter. Because of this, the from always imports the entire module into memory if it has not yet been imported, regardless of how many names it copies out of the file. There is no way to load just part of a module file (e.g., just one function), but because modules are byte code in Python instead of machine code, the performance implications are generally negligible.
Because the from statement makes the location of a variable more implicit and obscure (name is less meaningful to the reader than module.name), some Python users recommend using import instead of from most of the time. I’m not sure this advice is warranted, though; from is commonly and widely used, without too many dire consequences. In practice, in realistic programs, it’s often convenient not to have to type a module’s name every time you wish to use one of its tools. This is especially true for large modules that provide many attributes—the standard library’s tkinter GUI module, for example.
It is true that the from statement has the potential to corrupt namespaces, at least in principle—if you use it to import variables that happen to have the same names as existing variables in your scope, your variables will be silently overwritten. This problem doesn’t occur with the simple import statement because you must always go through a module’s name to get to its contents (module.attr will not clash with a variable named attr in your scope). As long as you understand and expect that this can happen when using from, though, this isn’t a major concern in practice, especially if you list the imported names explicitly (e.g., from module import x, y, z).
On the other hand, the from statement has more serious issues when used in conjunction with the reload call, as imported names might reference prior versions of objects. Moreover, the from module import * form really can corrupt namespaces and make names difficult to understand, especially when applied to more than one file—in this case, there is no way to tell which module a name came from, short of searching the external source files. In effect, the from * form collapses one namespace into another, and so defeats the namespace partitioning feature of modules. We will explore these issues in more detail in the section Module Gotchas (see Chapter 25).
Probably the best real-world advice here is to generally prefer import to from for simple modules, to explicitly list the variables you want in most from statements, and to limit the from * form to just one import per file. That way, any undefined names can be assumed to live in the module referenced with the from *. Some care is required when using the from statement, but armed with a little knowledge, most programmers find it to be a convenient way to access modules.
The only time you really must use import instead of from is when you must use the same name defined in two different modules. For example, if two files define the same name differently:
# M.py def func():...do something...# N.py def func():...do something else...
and you must use both versions of the name in your program, the from statement will fail—you can have only one assignment to the name in your scope:
# O.py from M import func from N import func # This overwrites the one we fetched from M func() # Calls N.func only!
An import will work here, though, because including the name of the enclosing module makes the two names unique:
# O.py import M, N # Get the whole modules, not their names M.func() # We can call both names now N.func() # The module names make them unique
This case is unusual enough that you’re unlikely to encounter it very often in practice. If you do, though, import allows you to avoid the name collision. Another way out of this dilemma is using the as extension, which we’ll cover in Chapter 25 but is simple enough to introduce here:
# O.py from M import func as mfunc # Rename uniquely with "as" from N import func as nfunc mfunc(); nfunc() # Calls one or the other
The as extension works in both import and from as a simple renaming tool (it can also be used to give a shorter synonym for a long module name in import); more on this form in Chapter 25.