Have you ever seen those “@” tags on top of Python functions and classes? Those are decorators – functions that wrap around other functions. Confusing? At first, but they’re easy with practice. Useful? Very!

The Talk

I delivered a talk on decorators at PyGotham TV 2020, PyCon India 2020, PyTexas 2020, and DevNation Day 2021. Here’s the recording from PyTexas:

And here’s the recording from PyCon India:

Of course, I mentioned testing in this talk, too:

And here’s the Conf42: Python 2022 recording:

The Transcript

[Camera]

Hello, PyTexas 2020! It’s Pandy Knight here. I’m the Automation Panda, and I’m a big Python fan, just like y’all.

Have you ever seen those “@” tags on top of Python functions? Maybe you’ve seen them on top of methods and classes, too. Those are decorators, one of Python’s niftiest language features. Decorators are essentially wrappers – they wrap additional code around existing definitions. When used right, they can clean up your code better than OxiClean! Let’s learn how to use them.

[Slide]

So, here’s a regular old “hello world” function. When we run it, …

[Slide]

…It prints “Hello World!” Nothing fancy here.

[Slide]

Now, let’s take that function…

[Slide]

…And BAM! Add a decorator. Using this “@” sign, we just added a decorator named “tracer” to “hello_world”. So, what is this decorator?

[Slide]

“Tracer” is just another function. But, it’s special because it takes in another function as an argument!

[Slide]

Since “tracer” decorates “hello_world”, the “hello_world” function is passed into “tracer” as an argument. Wow!

So, what’s inside “tracer”?

[Slide]

This decorator has an inner function named “wrapper”. Can you even do that? With Python, yes, you can! The “wrapper” function prints “Entering”, calls the function originally passed into the decorator, and then prints “Exiting”.

[Slide]

When “tracer” decorates “hello_world”, that means “hello_world” will be wrapped by “Entering” and “Exiting” print statements.

[Slides]

Finally, the decorator returns the new “wrapper” function. Any time the decorated function is called, it will effectively be replaced by this new wrapper function. 

[Slides]

So, when we call “hello_world”, the trace statements are now printed, too. Wow! That’s amazing. That’s how decorators work.

[Slide] Decorators [Slide] wrap [Slide] functions [Slide] around [Slide] functions!

[Slide]

Think about them like candy bars. The decorator is like the foil wrapper, and the decorated function is like the chocolate inside.

[Slide]

But how is this even possible? That decorator code looks confusing!

[Slide]

Decorators are possible because, in Python, functions are objects. In fancy language, we say functions are “first-order” values. Since functions are just objects, …

[Slide]

…We can pass them into other functions as arguments, …

[Slide]

…define new functions inside existing functions, …

[Slide]

…and return a function from a function.

[Slide]

This is all part of a paradigm called “Functional Programming.” Python supports functional programming because functions can be treated like objects. That’s awesome!

[Slide]

So, using functions as objects, decorators change how functions are called.

[Slide]

Decorators create an “outer” decorator function around an “inner” decorated function. Remember, the outer function is like the foil wrapper, and the inner function is like the chocolate.

[Slide]

Creating an outer function lets you add new code around the inner function. Some people call this “advice.” You can add advice before or after the inner function. You could even skip the inner function!

[Slide]

The best part is, decorators can be applied to any function. They make sharing code easy so you don’t repeat yourself!

[Slide]

Decorators are reminiscent of a paradigm called “Aspect-Oriented Programming,” in which code can be cleverly inserted before and after points of execution. Neat!

[Slide]

So remember, decorators wrap functions around functions, like candy bars!

[Slide]

Hold on, now! We have a problem in that Python code!

[Slide]

If the “wrapper” function effectively replaces “hello_world”, then what identity does “hello_world” report?

[Slide]

Its name is “wrapper”…

[Slide]

And its help is also “wrapper”! That’s not right!

[Slide]

Never fear! There’s an easy solution. The “functools” module provides a decorator named “wraps”. Put “functools.wraps” on the “wrapper” function and pass in the inner function object, and decorated functions once again show the right identity. That’s awesome.

[Slide]

But wait, there’s another problem!

[Slide]

How do decorators work with inputs and outputs? What if we decorate a function with parameters and a return value?

[Slide]

If we try to use the current “tracer”, …

[Slide]

…We get an error! Arguments broke it!

[Slide]

We can fix it! First, add “star args” and “star-star k-w-args” to the “wrapper” function’s parameters, and then pass them through to the inner function. This will make sure all arguments go through the decorator into the decorated function.

[Slide]

Then, capture the inner function’s return value and return it from the “wrapper” function. This makes sure return values also pass through. If the inner function has no return value, don’t worry – the decorator will pass through a “None” value.

[Slide]

When we call the function with the updated “tracer”, …

[Slide]

…we see tracing is now successful again!

[Slide]

When we check the return value, …

[Slide]

…it’s exactly what we expect. It works!

[Slide]

Wow, that’s awesome!

[Slide]

But wait, there’s more!

[Slide]

You can write a decorator to call a function twice!

[Slide]

Start with the decorator template…

[Slide]

…and call the inner function twice! Return the final return value for continuity.

[Slide]

BAM! It works!

[Slide]

But wait, there’s more!

[Slide]

You can write a timer decorator!

[Slide]

Start with the template, …

[Slide]

…call the inner function, …

[Slide]

…and surround it with timestamps using the “time” module!

[Slide]

BAM! Now you can time any function!

[Slide]

But wait, there’s more!

[Slide]

You can also add more than one decorator to a function! This is called “nesting”. Order matters. Decorators are executed in order of closeness to the inner function. So, in this case, …

[Slide]

…”call_twice” is applied first, and then “timer” is applied.

[Slide]

If these decorators are reversed, …

[Slide]

…then each inner function call is timed. Cool!

[Slide]

But wait, there’s more!

[Slide]

You can scrub and validate function arguments! Check out these two simple math functions.

[Slide]

Create a decorator to scrub and validate inputs as integers.

[Slide]

Add the wrapper function, and make sure it has positional args.

[Slide]

Then, cast all args as ints before passing them into the inner function.

[Slide]

Now, when calling those math functions, all numbers are integers! Using non-numeric inputs also raises a ValueError!

[Slide]

But wait, there’s more!

[Slide]

You can create decorators with parameters! Here’s a decorator that will repeat a function 5 times.

[Slide]

The “repeat” function is a little different. Instead of taking in the inner function object, it takes in the parameter, which is the number of times to repeat the inner function.

[Slide]

Inside, there’s a “repeat_decorator” function that has a parameter for the inner function. The “repeat” function returns the “repeat_decorator” function.

[Slide]

Inside “repeat_decorator” is the “wrapper” function. It uses “functools.wraps” and passes through all arguments. “repeat_decorator” returns “wrapper”.

[Slide]

Finally, “wrapper” contains the logic for calling the inner function multiple times, according to the “repeat” decorator’s parameter value.

[Slide]

Now, “hello_world” runs 5 times. Nifty!

[Slide]

But wait, there’s more!

[Slide]

Decorators can be used to save state! Here’s a decorator that will count the number of times a function is called.

[Slide]

“count_calls” has the standard decorator structure.

[Slide]

Outside the wrapper, a “count” attribute is initialized to 0. This attribute is added to the wrapper function object.

[Slide]

Inside the wrapper, the count is incremented before calling the inner function. The “count” value will persist across multiple calls.

[Slide]

Initially, the “hello_world” count value is 0.

[Slide]

After two calls, the count value goes up! Awesome!

[Slide]

But wait, there’s more!

[Slide]

Decorators can be used in classes! Here, the “timer” decorator is applied to this “hello” method.

[Slider]

As long as parameters and return values are set up correctly, decorators can be applied equally to functions and methods.

[Slide]

Decorators can also be applied directly to classes!

[Slide]

When a decorator is applied to a class, it wraps the constructor.

[Slide]

Note that it does not wrap each method in the class.

[Slide]

Since decorators can wrap classes and methods in addition to functions, it would technically be more correct to say that decorators wrap callables around callables!

[Slide]

So all that’s great, but can decorators be tested? Good code must arguably be testable code. Well, today’s your lucky day, because yes, you can test decorators!

[Slide]

Testing decorators can be a challenge. We should always try to test the code we write, but decorators can be tricky. Here’s some advice:

[Slide]

First, separate tests for decorator functions from decorated functions. For decorator functions, focus on intended outcomes. Try to focus on the “wrapper” instead of the “inner” function. Remember, decorators can be applied to any callable, so cover the parts that make decorators unique. Decorated functions should have their own separate unit tests.

[Slide]

Second, apply decorators to “fake” functions used only for testing. These functions can be simple or mocked. That way, unit tests won’t have dependencies on existing functions that could change. Tests will also be simpler if they use slimmed-down decorated functions.

[Slide]

Third, make sure decorators have test coverage for every possible way it could be used. Cover decorator parameters, decorated function arguments, and return values. Make sure the “name” and “help” are correct. Check any side effects like saved state. Try it on methods and classes as well as functions. With decorators, most failures happen due to overlooked edge cases.

[Slide]

Let’s look at a few short decorator tests. We’ll use the “count_calls” decorator from earlier.

There are two decorated functions to use for testing. The first one is a “no operation” function that does nothing. It has no parameters or returns. The second one is a function that takes in one argument and returns it. Both are very simple, but they represent two equivalences classes of decoratable functions.

[Slide]

The test cases will verify outcomes of using the decorator. For “count_calls”, that means tests will focus on the “count” attribute added to decorated functions.

The first test case verifies that the initial count value for any function is zero.

[Slide]

The second test calls a function three times and verifies that count is three.

[Slide]

The third test exercises the “same” function to make sure arguments and return values work correctly. It calls the “same” function, asserts the return value, and asserts the count value.

This collection of tests is by no means complete. It simply shows how to start writing tests for decorators. It also shows that you don’t need to overthink unit tests for decorators. Simple is better than complex!

[Slide]

Up to this point, we’ve covered how to write your own decorators. However, Python has several decorators available in the language and in various modules that you can use, absolutely free!

[Slide]

Decorators like “classmethod”, “staticmethod”, and “property” can apply to methods in a class. Frameworks like Flask and pytest have even more decorators. Let’s take a closer look.

[Slide]

Let’s start by comparing the “classmethod” and “staticmethod” decorators. We’ll revisit the “Greeter” class we saw before.

[Slide]

The “classmethod” decorator will turn any method into a “class” method instead of an “instance” method. That means this “hello” method here can be called directly from the class itself instead of from an object of the class. This decorator will pass a reference to the class into the method so the method has some context of the class. Here, the reference is named “c-l-s”, and the method uses it to get the class’s name. The method can be called using “Greeter.hello”. Wow!

[Slide]

The “staticmethod” decorator works almost the same as the “classmethod” decorator, except that it does not pass a reference to the class into the method.

[Slide]

Notice how the method parameters are empty – no “class” and no “self”. Methods are still called from the class, like here with “Greeter.goodbye”. You could say that “staticmethod” is just a simpler version of “classmethod”.

[Slide]

Next, let’s take a look at the “property” decorator. To show how to use it, we’ll create a class called “Accumulator” to keep count of a tally.

[Slide]

Accumulator’s “init” method initializes a “count” attribute to 0.

[Slide]

An “add” method adds an amount to the count. So far, nothing unusual.

[Slide]

Now, let’s add a property. This “count” method has the “property” decorator on it. This means that “count” will be callable as an attribute instead of a method, meaning that it won’t need parentheses. It is effectively a “getter”. The calls to “count” in the “init” and “add” methods will actually call this property instead of a raw variable.

Inside the “count” property, the method returns an attribute named “underscore-count”. The underscore means that this variable should be private. However, this class hasn’t set that variable yet.

[Slide]

That variable is set in the “setter” method. Setters are optional for properties. Here, the setter validates that the value to set is not negative. If the value is good, then it sets “underscore-count”. If the value is negative, then it raises a ValueError.

“underscore-count” is handled internally, while “count” is handled publicly as the property. The getter and setter controls added by the “property” decorator let you control how the property is handled. In this class, the setter protects the property against bad values!

[Slide]

So, let’s use this class. When an Accumulator object is constructed, its initial count is 0.

[Slide]

After adding an amount to the object, its count goes up.

[Slide]

Its count can be directly set to non-negative values. Attempting to set the count directly to a negative value raises an exception, as expected. Protection like that is great!

[Slide]

Python packages also frequently contain decorators. For example, Flask is a very popular Web “micro-framework” that enables you to write Web APIs with very little Python code.

[Slide]

Here’s an example “Hello World” Flask app taken directly from the Flask docs online. It imports the “flask” module, creates the app, and defines a single endpoint at the root path that returns the string, “Hello, World!” Flask’s “app.route” decorator can turn any function into a Web API endpoint. That’s awesome!

[Slide]

Another popular Python package with decorators is pytest, Python’s most popular test framework.

[Slide]

One of pytest’s best features is the ability to parametrize test functions to run for multiple input combinations. Test parameters empower data driven testing for wider test coverage!

[Slide]

To show how this works, we’ll use a simple test for basic arithmetic: “test addition”. It asserts that a plus b equals c.

[Slide]

The values for a, b, and c must come from a list of tuples. For example, 1 plus 2 equals 3, and so forth.

[Slide]

The “pytest.mark.parametrize” decorator connects the list of test values to the test function. It runs the test once for each tuple in the list, and it injects the tuple values into the test case as function arguments. This test case would run four times. Test parameters are a great way to rerun test logic without repeating test code.

[Slide]

So, act now, before it’s too late!

[Slide]

When should you use decorators in your Python code?

[Slide]

Use decorators for aspects.

[Slide]

An aspect is a special cross-cutting concern. They are things that happen in many parts of the code, and they frequently require repetitive calls.

Think about something like logging. If you want to add logging statements to different parts of the code, then you need to write multiple logging calls in all those places. Logging itself is one concern, but it cross-cuts the whole code base. One solution for logging could be to use decorators, much like we saw earlier with the “tracer” decorator.

[Slide]

Good use cases for decorators include logging, profiling, input validation, retries, and registries. These are things that typically require lots of extra calls inserted in duplicative ways. Ask yourself this:

[Slide]

Should the code wrap something else? If yes, then you have a good candidate for a decorator.

[Slide]

However, decorators aren’t good for all circumstances. You should avoid decorators for “main” behaviors, because those should probably be put directly in the body of the decorated function. Avoid logic that’s complicated or has heavy conditionals, too, because simple is better than complex. You should also try to avoid completely side-stepping the decorated function – that could confuse people!

[Slide]

Ask yourself this: is the code you want to write the wrapper or the candy bar itself? Wrappers make good decorators, but candy bars do not.

[Slide]

I hope you’ve found this infomercial about decorators useful! If you want to learn more, …

[Slide]

…check out this Real Python tutorial by Geir Arne Hjelle named “Primer on Python Decorators”. It covers everything I showed here, plus more.

[Slide]

Thank you very much for listening! Again, my name is Pandy Knight – the Automation Panda and a bona fide Pythonista. Please read my blog at AutomationPanda.com, and follow me on Twitter @AutomationPanda. I’d love to hear what y’all end up doing with decorators!