Module Namespaces

I’ve mentioned that files morph into namespaces, but how does this actually happen? The short answer is that every name that is assigned a value at the top level of a module file (i.e., not nested in a function or class body) becomes an attribute of that module.

For instance, given an assignment statement such as X = 1 at the top level of a module file M.py, the name X becomes an attribute of M, which we can refer to from outside the module as M.X. The name X also becomes a global variable to other code inside M.py, but we need to consider the notion of module loading and scopes a bit more formally to understand why:

Here’s a demonstration of these ideas. Suppose we create the following module file in a text editor and call it module2.py:

print('starting to load...')
import sys
name = 42

def func(): pass

class klass: pass

print('done loading.')

The first time this module is imported (or run as a program), Python executes its statements from top to bottom. Some statements create names in the module’s namespace as a side effect, but others do actual work while the import is going on. For instance, the two print statements in this file execute at import time:

>>> import module2
starting to load...
done loading.

Once the module is loaded, its scope becomes an attribute namespace in the module object we get back from import. We can then access attributes in this namespace by qualifying them with the name of the enclosing module:

>>> module2.sys
<module 'sys' (built-in)>

>>> module2.name
42

>>> module2.func
<function func at 0x000000000222E7B8>

>>> module2.klass
<class 'module2.klass'>

Here, sys, name, func, and klass were all assigned while the module’s statements were being run, so they are attributes after the import. We’ll talk about classes in Part VI, but notice the sys attribute—import statements really assign module objects to names, and any type of assignment to a name at the top level of a file generates a module attribute.

In fact, internally, module namespaces are stored as dictionary objects. These are just normal dictionaries with all the usual methods. When needed—for instance, to write tools that list module content generically as we will in Chapter 25—we can access a module’s namespace dictionary through the module’s __dict__ attribute. Continuing the prior section’s example (remember to wrap this in a list call in Python 3.X—it’s a view object there, and contents may vary outside 3.3 used here):

>>> list(module2.__dict__.keys())
['__loader__', 'func', 'klass', '__builtins__', '__doc__', '__file__', '__name__',
'name', '__package__', 'sys', '__initializing__', '__cached__']

The names we assigned in the module file become dictionary keys internally, so some of the names here reflect top-level assignments in our file. However, Python also adds some names in the module’s namespace for us; for instance, __file__ gives the name of the file the module was loaded from, and __name__ gives its name as known to importers (without the .py extension and directory path). To see just the names your code assigns, filter out the double-underscore names as we’ve done before, in Chapter 15’s dir coverage and Chapter 17’s built-in scope coverage:

>>> list(name for name in module2.__dict__.keys() if not name.startswith('__'))
['func', 'klass', 'name', 'sys']
>>> list(name for name in module2.__dict__ if not name.startswith('__'))
['func', 'sys', 'name', 'klass']

This time we’re filtering with a generator instead of a list comprehension, and can omit the .keys() because dictionaries generate their keys automatically though implicitly; the effect is the same. We’ll see similar __dict__ dictionaries on class-related objects in Part VI too. In both cases, attribute fetch is similar to dictionary indexing, though only the former kicks off inheritance in classes:

>>> module2.name, module2.__dict__['name']
(42, 42)

Speaking of attribute fetch, now that you’re becoming more familiar with modules, we should firm up the notion of name qualification more formally too. In Python, you can access the attributes of any object that has attributes using the qualification (a.k.a. attribute fetch) syntax object.attribute.

Qualification is really an expression that returns the value assigned to an attribute name associated with an object. For example, the expression module2.sys in the previous example fetches the value assigned to sys in module2. Similarly, if we have a built-in list object L, L.append returns the append method object associated with that list.

It’s important to keep in mind that attribute qualification has nothing to do with the scope rules we studied in Chapter 17; it’s an independent concept. When you use qualification to access names, you give Python an explicit object from which to fetch the specified names. The LEGB scope rule applies only to bare, unqualified names—it may be used for the leftmost name in a name path, but later names after dots search specific objects instead. Here are the rules:

In Part VI, we’ll see that attribute qualification means a bit more for classes—it’s also the place where something called inheritance happens—but in general, the rules outlined here apply to all names in Python.

As we’ve learned, it is never possible to access names defined in another module file without first importing that file. That is, you never automatically get to see names in another file, regardless of the structure of imports or function calls in your program. A variable’s meaning is always determined by the locations of assignments in your source code, and attributes are always requested of an object explicitly.

For example, consider the following two simple modules. The first, moda.py, defines a variable X global to code in its file only, along with a function that changes the global X in this file:

X = 88                        # My X: global to this file only
def f():
    global X                  # Change this file's X
    X = 99                    # Cannot see names in other modules

The second module, modb.py, defines its own global variable X and imports and calls the function in the first module:

X = 11                        # My X: global to this file only

import moda                   # Gain access to names in moda
moda.f()                      # Sets moda.X, not this file's X
print(X, moda.X)

When run, moda.f changes the X in moda, not the X in modb. The global scope for moda.f is always the file enclosing it, regardless of which module it is ultimately called from:

% python modb.py
11 99

In other words, import operations never give upward visibility to code in imported files—an imported file cannot see names in the importing file. More formally:

Such behavior is part of the lexical scoping notion—in Python, the scopes surrounding a piece of code are completely determined by the code’s physical position in your file. Scopes are never influenced by function calls or module imports.^[46]

In some sense, although imports do not nest namespaces upward, they do nest downward. That is, although an imported module never has direct access to names in a file that imports it, using attribute qualification paths it is possible to descend into arbitrarily nested modules and access their attributes. For example, consider the next three files. mod3.py defines a single global name and attribute by assignment:

X = 3

mod2.py in turn defines its own X, then imports mod3 and uses qualification to access the imported module’s attribute:

X = 2
import mod3

print(X, end=' ')             # My global X
print(mod3.X)                 # mod3's X

mod1.py also defines its own X, then imports mod2, and fetches attributes in both the first and second files:

X = 1
import mod2

print(X, end=' ')             # My global X
print(mod2.X, end=' ')        # mod2's X
print(mod2.mod3.X)            # Nested mod3's X

Really, when mod1 imports mod2 here, it sets up a two-level namespace nesting. By using the path of names mod2.mod3.X, it can descend into mod3, which is nested in the imported mod2. The net effect is that mod1 can see the Xs in all three files, and hence has access to all three global scopes:

% python mod1.py
2 3
1 2 3

The reverse, however, is not true: mod3 cannot see names in mod2, and mod2 cannot see names in mod1. This example may be easier to grasp if you don’t think in terms of namespaces and scopes, but instead focus on the objects involved. Within mod1, mod2 is just a name that refers to an object with attributes, some of which may refer to other objects with attributes (import is an assignment). For paths like mod2.mod3.X, Python simply evaluates from left to right, fetching attributes from objects along the way.

Note that mod1 can say import mod2, and then mod2.mod3.X, but it cannot say import mod2.mod3—this syntax invokes something called package (directory) imports, described in the next chapter. Package imports also create module namespace nesting, but their import statements are taken to reflect directory trees, not simple file import chains.