At a base level, package imports are straightforward—in the place where you have been naming a simple file in your import statements, you can instead list a path of names separated by periods:
import dir1.dir2.mod
The same goes for from statements:
from dir1.dir2.mod import x
The “dotted” path in these statements is assumed to correspond to a path through the directory hierarchy on your computer, leading to the file mod.py (or similar; the extension may vary). That is, the preceding statements indicate that on your machine there is a directory dir1, which has a subdirectory dir2, which contains a module file mod.py (or similar).
Furthermore, these imports imply that dir1 resides within some container directory dir0, which is a component of the normal Python module search path. In other words, these two import statements imply a directory structure that looks something like this (shown with Windows backslash separators):
dir0\dir1\dir2\mod.py # Or mod.pyc, mod.so, etc.
The container directory dir0 needs to be added to your module search path unless it’s the home directory of the top-level file, exactly as if dir1 were a simple module file.
More formally, the leftmost component in a package import path is still relative to a directory included in the sys.path module search path list we explored in Chapter 22. From there down, though, the import statements in your script explicitly give the directory paths leading to modules in packages.
If you use this feature, keep in mind that the directory paths in your import statements can be only variables separated by periods. You cannot use any platform-specific path syntax in your import statements, such as C:\dir1, My Documents.dir2, or ../dir1—these do not work syntactically. Instead, use any such platform-specific syntax in your module search path settings to name the container directories.
For instance, in the prior example, dir0—the directory name you add to your module search path—can be an arbitrarily long and platform-specific directory path leading up to dir1. You cannot use an invalid statement like this:
import C:\mycode\dir1\dir2\mod # Error: illegal syntax
But you can add C:\mycode to your PYTHONPATH variable or a .pth file, and say this in your script:
import dir1.dir2.mod
In effect, entries on the module search path provide platform-specific directory path prefixes, which lead to the leftmost names in import and from statements. These import statements themselves provide the remainder of the directory path in a platform-neutral fashion.[47]
As for simple file imports, you don’t need to add the container directory dir0 to your module search path if it’s already there—per Chapter 22, it will be if it’s the home directory of the top-level file, the directory you’re working in interactively, a standard library directory, or the site-packages third-party install root. One way or another, though, your module search path must include all the directories containing leftmost components in your code’s package import statements.
If you choose to use package imports, there is one more constraint you must follow: at least until Python 3.3, each directory named within the path of a package import statement must contain a file named __init__.py, or your package imports will fail. That is, in the example we’ve been using, both dir1 and dir2 must contain a file called __init__.py; the container directory dir0 does not require such a file because it’s not listed in the import statement itself.
More formally, for a directory structure such as this:
dir0\dir1\dir2\mod.py
and an import statement of the form:
import dir1.dir2.mod
the following rules apply:
dir1 and dir2 both must contain an __init__.py file.
dir0, the container, does not require an __init__.py file; this file will simply be ignored if present.
dir0, not dir0\dir1, must be listed on the module search path sys.path.
To satisfy the first two of these rules, package creators must create files of the sort we’ll explore here. To satisfy the latter of these, dir0 must be an automatic path component (the home, libraries, or site-packages directories), or be given in PYTHONPATH or .pth file settings or manual sys.path changes.
The net effect is that this example’s directory structure should be as follows, with indentation designating directory nesting:
dir0\ # Container on module search path
dir1\
__init__.py
dir2\
__init__.py
mod.py
The __init__.py files can contain Python code, just like normal module files. Their names are special because their code is run automatically the first time a Python program imports a directory, and thus serves primarily as a hook for performing initialization steps required by the package. These files can also be completely empty, though, and sometimes have additional roles—as the next section explains.
As we’ll see near the end of this chapter, the requirement of packages to have a file named __init__.py has been lifted as of Python 3.3. In that release and later, directories of modules with no such file may be imported as single-directory namespace packages, which work the same but run no initialization-time code file. Prior to Python 3.3, though, and in all of Python 2.X, packages still require __init__.py files. As described ahead, in 3.3 and later these files also provide a performance advantage when used.
In more detail, the __init__.py file serves as a hook for package initialization-time actions, declares a directory as a Python package, generates a module namespace for a directory, and implements the behavior of from * (i.e., from .. import *) statements when used with directory imports:
The first time a Python program imports through a directory, it automatically runs all the code in the directory’s __init__.py file. Because of that, these files are a natural place to put code to initialize the state required by files in a package. For instance, a package might use its initialization file to create required data files, open connections to databases, and so on. Typically, __init__.py files are not meant to be useful if executed directly; they are run automatically when a package is first accessed.
Package __init__.py files are also partly present to declare that a directory is a Python package. In this role, these files serve to prevent directories with common names from unintentionally hiding true modules that appear later on the module search path. Without this safeguard, Python might pick a directory that has nothing to do with your code, just because it appears nested in an earlier directory on the search path. As we’ll see later, Python 3.3’s namespace packages obviate much of this role, but achieve a similar effect algorithmically by scanning ahead on the path to find later files.
In the package import model, the directory paths in your script become real nested object paths after an import. For instance, in the preceding example, after the import the expression dir1.dir2 works and returns a module object whose namespace contains all the names assigned by dir2’s __init__.py initialization file. Such files provide a namespace for module objects created for directories, which would otherwise have no real associated module file.
from * statement behaviorAs an advanced feature, you can use __all__ lists in __init__.py files to define what is exported when a directory is imported with the from * statement form. In an __init__.py file, the __all__ list is taken to be the list of submodule names that should be automatically imported when from * is used on the package (directory) name. If __all__ is not set, the from * statement does not automatically load submodules nested in the directory; instead, it loads just names defined by assignments in the directory’s __init__.py file, including any submodules explicitly imported by code in this file. For instance, the statement from submodule import X in a directory’s __init__.py makes the name X available in that directory’s namespace. (We’ll see additional roles for __all__ in Chapter 25: it serves to declare from * exports of simple files as well.)
You can also simply leave these files empty, if their roles are beyond your needs (and frankly, they are often empty in practice). They must exist, though, for your directory imports to work at all.
Don’t confuse package __init__.py files with the class __init__ constructor methods we’ll meet in the next part of the book. The former are files of code run when imports first step through a package directory in a program run, while the latter are called when an instance is created. Both have initialization roles, but they are otherwise very different.
[47] The dot path syntax was chosen partly for platform neutrality, but also because paths in import statements become real nested object paths. This syntax also means that you may get odd error messages if you forget to omit the .py in your import statements. For example, import mod.py is assumed to be a directory path import—it loads mod.py, then tries to load a mod\py.py, and ultimately issues a potentially confusing “No module named py” error message. As of Python 3.3 this error message has been improved to say “No module named ‘m.py’; m is not a package.”