The Basics

A function decorator is a kind of runtime declaration about the function whose definition follows. The decorator is coded on a line just before the def statement that defines a function or method, and it consists of the @ symbol followed by a reference to a metafunction—a function (or other callable object) that manages another function.

In terms of code, function decorators automatically map the following syntax:

@decorator              # Decorate function
def F(arg):
    ...

F(99)                   # Call function

into this equivalent form, where decorator is a one-argument callable object that returns a callable object with the same number of arguments as F (in not F itself):

def F(arg):
    ...
F = decorator(F)        # Rebind function name to decorator result

F(99)                   # Essentially calls decorator(F)(99)

This automatic name rebinding works on any def statement, whether it’s for a simple function or a method within a class. When the function F is later called, it’s actually calling the object returned by the decorator, which may be either another object that implements required wrapping logic, or the original function itself.

In other words, decoration essentially maps the first of the following into the second—though the decorator is really run only once, at decoration time:

func(6, 7)
decorator(func)(6, 7)

This automatic name rebinding accounts for the static method and property decoration syntax we met earlier in the book:

class C:
    @staticmethod
    def meth(...): ...            # meth = staticmethod(meth)

class C:
    @property
    def name(self): ...           # name = property(name)

In both cases, the method name is rebound to the result of a built-in function decorator, at the end of the def statement. Calling the original name later invokes whatever object the decorator returns. In these specific cases, the original names are rebound to a static method router and property descriptor, but the process is much more general than this—as the next section explains.

A decorator itself is a callable that returns a callable. That is, it returns the object to be called later when the decorated function is invoked through its original name—either a wrapper object to intercept later calls, or the original function augmented in some way. In fact, decorators can be any type of callable and return any type of callable: any combination of functions and classes may be used, though some are better suited to certain contexts.

For example, to tap into the decoration protocol in order to manage a function just after it is created, we might code a decorator of this form:

def decorator(F):
    # Process function F
    return F

@decorator
def func(): ...                  # func = decorator(func)

Because the original decorated function is assigned back to its name, this simply adds a post-creation step to function definition. Such a structure might be used to register a function to an API, assign function attributes, and so on.

In more typical use, to insert logic that intercepts later calls to a function, we might code a decorator to return a different object than the original function—a proxy for later calls:

def decorator(F):
    # Save or use function F
    # Return a different callable: nested def, class with __call__, etc.

@decorator
def func(): ...                  # func = decorator(func)

This decorator is invoked at decoration time, and the callable it returns is invoked when the original function name is later called. The decorator itself receives the decorated function; the callable returned receives whatever arguments are later passed to the decorated function’s name. When coded properly, this works the same for class-level methods: the implied instance object simply shows up in the first argument of the returned callable.

In skeleton terms, here’s one common coding pattern that captures this idea—the decorator returns a wrapper that retains the original function in an enclosing scope:

def decorator(F):                     # On @ decoration
    def wrapper(*args):               # On wrapped function call
        # Use F and args
        # F(*args) calls original function
    return wrapper

@decorator                            # func = decorator(func)
def func(x, y):                       # func is passed to decorator's F
    ...

func(6, 7)                            # 6, 7 are passed to wrapper's *args

When the name func is later called, it really invokes the wrapper function returned by decorator; the wrapper function can then run the original func because it is still available in an enclosing scope. When coded this way, each decorated function produces a new scope to retain state.

To do the same with classes, we can overload the call operation and use instance attributes instead of enclosing scopes:

class decorator:
    def __init__(self, func):         # On @ decoration
        self.func = func
    def __call__(self, *args):        # On wrapped function call
        # Use self.func and args
        # self.func(*args) calls original function

@decorator
def func(x, y):                       # func = decorator(func)
    ...                               # func is passed to __init__

func(6, 7)                            # 6, 7 are passed to __call__'s *args

When the name func is later called now, it really invokes the __call__ operator overloading method of the instance created by decorator; the __call__ method can then run the original func because it is still available in an instance attribute. When coded this way, each decorated function produces a new instance to retain state.

One subtle point about the prior class-based coding is that while it works to intercept simple function calls, it does not quite work when applied to class-level method functions:

class decorator:
    def __init__(self, func):           # func is method without instance
        self.func = func
    def __call__(self, *args):          # self is decorator instance
        # self.func(*args) fails!       # C instance not in args!

class C:
    @decorator
    def method(self, x, y):             # method = decorator(method)
        ...                             # Rebound to decorator instance

When coded this way, the decorated method is rebound to an instance of the decorator class, instead of a simple function.

The problem with this is that the self in the decorator’s __call__ receives the decorator class instance when the method is later run, and the instance of class C is never included in *args. This makes it impossible to dispatch the call to the original method—the decorator object retains the original method function, but it has no instance to pass to it.

To support both functions and methods, the nested function alternative works better:

def decorator(F):                       # F is func or method without instance
    def wrapper(*args):                 # class instance in args[0] for method
                                        # F(*args) runs func or method
    return wrapper

@decorator
def func(x, y):                         # func = decorator(func)
    ...
func(6, 7)                              # Really calls wrapper(6, 7)

class C:
    @decorator
    def method(self, x, y):             # method = decorator(method)
        ...                             # Rebound to simple function

X = C()
X.method(6, 7)                          # Really calls wrapper(X, 6, 7)

When coded this way wrapper receives the C class instance in its first argument, so it can dispatch to the original method and access state information.

Technically, this nested-function version works because Python creates a bound method object and thus passes the subject class instance to the self argument only when a method attribute references a simple function; when it references an instance of a callable class instead, the callable class’s instance is passed to self to give the callable class access to its own state information. We’ll see how this subtle difference can matter in more realistic examples later in this chapter.

Also note that nested functions are perhaps the most straightforward way to support decoration of both functions and methods, but not necessarily the only way. The prior chapter’s descriptors, for example, receive both the descriptor and subject class instance when called. Though more complex, later in this chapter we’ll see how this tool can be leveraged in this context as well.

Syntactically, class decorators appear just before class statements, in the same way that function decorators appear just before def statements. In symbolic terms, for a decorator that must be a one-argument callable that returns a callable, the class decorator syntax:

@decorator                 # Decorate class
class C:
    ...

x = C(99)                  # Make an instance

is equivalent to the following—the class is automatically passed to the decorator function, and the decorator’s result is assigned back to the class name:

class C:
    ...
C = decorator(C)           # Rebind class name to decorator result

x = C(99)                  # Essentially calls decorator(C)(99)

The net effect is that calling the class name later to create an instance winds up triggering the callable returned by the decorator, which may or may not call the original class itself.

New class decorators are coded with many of the same techniques used for function decorators, though some may involve two levels of augmentation—to manage both instance construction calls, as well as instance interface access. Because a class decorator is also a callable that returns a callable, most combinations of functions and classes suffice.

However it’s coded, the decorator’s result is what runs when an instance is later created. For example, to simply manage a class just after it is created, return the original class itself:

def decorator(C):
    # Process class C
    return C

@decorator
class C: ...                                    # C = decorator(C)

To instead insert a wrapper layer that intercepts later instance creation calls, return a different callable object:

def decorator(C):
    # Save or use class C
    # Return a different callable: nested def, class with __call__, etc.

@decorator
class C: ...                                    # C = decorator(C)

The callable returned by such a class decorator typically creates and returns a new instance of the original class, augmented in some way to manage its interface. For example, the following inserts an object that intercepts undefined attributes of a class instance:

def decorator(cls):                             # On @ decoration
    class Wrapper:
        def __init__(self, *args):              # On instance creation
            self.wrapped = cls(*args)
        def __getattr__(self, name):            # On attribute fetch
            return getattr(self.wrapped, name)
    return Wrapper

@decorator
class C:                             # C = decorator(C)
    def __init__(self, x, y):        # Run by Wrapper.__init__
        self.attr = 'spam'

x = C(6, 7)                          # Really calls Wrapper(6, 7)
print(x.attr)                        # Runs Wrapper.__getattr__, prints "spam"

In this example, the decorator rebinds the class name to another class, which retains the original class in an enclosing scope and creates and embeds an instance of the original class when it’s called. When an attribute is later fetched from the instance, it is intercepted by the wrapper’s __getattr__ and delegated to the embedded instance of the original class. Moreover, each decorated class creates a new scope, which remembers the original class. We’ll flesh out this example into some more useful code later in this chapter.

Like function decorators, class decorators are commonly coded as either “factory” functions that create and return callables, classes that use __init__ or __call__ methods to intercept call operations, or some combination thereof. Factory functions typically retain state in enclosing scope references, and classes in attributes.

As for function decorators, some callable type combinations work better for class decorators than others. Consider the following invalid alternative to the class decorator of the prior example:

class Decorator:
    def __init__(self, C):                    # On @ decoration
        self.C = C
    def __call__(self, *args):                # On instance creation
        self.wrapped = self.C(*args)
        return self
    def __getattr__(self, attrname):          # On atrribute fetch
        return getattr(self.wrapped, attrname)

@Decorator
class C: ...                                  # C = Decorator(C)

x = C()
y = C()                                       # Overwrites x!

This code handles multiple decorated classes (each makes a new Decorator instance) and will intercept instance creation calls (each runs __call__). Unlike the prior version, however, this version fails to handle multiple instances of a given class—each instance creation call overwrites the prior saved instance. The original version does support multiple instances, because each instance creation call makes a new independent wrapper object. More generally, either of the following patterns supports multiple wrapped instances:

def decorator(C):                             # On @ decoration
    class Wrapper:
        def __init__(self, *args):            # On instance creation: new Wrapper
            self.wrapped = C(*args)           # Embed instance in instance
    return Wrapper

class Wrapper: ...
def decorator(C):                             # On @ decoration
    def onCall(*args):                        # On instance creation: new Wrapper
        return Wrapper(C(*args))              # Embed instance in instance
    return onCall

We’ll study this phenomenon in a more realistic context later in the chapter too; in practice, though, we must be careful to combine callable types properly to support our intent, and choose state policies wisely.