Python

Book Review: Python Testing with pytest

tl;dr

Title Python Testing with pytest
Author Brian Okken (@brianokken)
Publication 2017 (The Pragmatic Programmers)
Summary How to use all the features of pytest for Python test automation – “simple, rapid, effective, and scalable.”
Prerequisites Intermediate-level Python programming.

Summary

Python Testing with pytest is the book on pytest. Brian Okken covers all the ins and outs of the framework. The book is useful both as tutorial for learning pytest as well as a reference for specific framework features. It covers:

  • Getting started with pytest
  • Writing simple tests as functions
  • Writing more interesting tests with assertions, exceptions, and parameters
  • Using all the different execution options
  • Writing fixtures to flexibly separate concerns and reuse code
  • Using built-in fixtures like tmpdir, pytestconfig, and monkeypatch
  • Using configuration files to control execution
  • Integrating pytest with other tools like pdb, tox, and Jenkins

Appendices also cover:

  • Using Python virtual environments
  • Installing packages with pip
  • An overview of popular plugins like pytest-xdist and pytest-cov
  • Packaging and distributing Python packages

Praises

This book is a comprehensive guide to pytest. It thoroughly covers the framework’s features and gives pointers to more info elsewhere. Even though pytest has excellent online documentation, I still recommend this book to anyone who wants to become a pytest master. Online docs tend to be fragmented with each piece limited in scope, whereas books like this one are designed to be read progressively and orderly for maximal understanding of the material.

I love how this book is example-driven. Each section follows a simple yet powerful outline: idea → code → output → explanation. Having real code with real output truly cements the point of each mini-lesson. New topics are carefully unfolded so that they build upon previous topics, making the book read like a collection of tutorials. Examples at the end of every chapter challenge the readers to practice what they learn. The formatting of each section also looks great.

The extra info on related topics like pip and virtualenv is also a nice touch. Python pros probably don’t need it, but beginners might get stuck without it.

The rocket ship logo on the cover is also really cool!

Takeaways

pytest is one of the best functional test frameworks currently available in any language. It breaks the clunky xUnit mold, in which class structures were awkwardly superimposed over test cases as if one size fits all. Instead, pytest is lightweight and minimalist because it relies on functions and fixtures. Scope is much easier to manage, code is more reusable, and side effects can more easily be avoided. pytest has taken over Python testing because it is so Pythonic.

Brian’s concise writing style has also inspired me to be more direct in my own writing. I tend to be rather verbose out of my desire to be descriptive. However, fewer words often leave a more powerful impression. They also make the message easier to comprehend. Python is beloved for its concise expressiveness, and as a Pythonista, it would be fitting for me to adopt that trait into my English.

If I had a wish list for a second edition, I’d love to see more info about assertions and other plugins (namely pytest-bdd). I think an appendix with comparisons to other Python test frameworks would also be valuable.

A Warning

I ordered a physical copy of this book directly from Amazon (not a third-party seller). Unfortunately, that copy was missing all the introductory content: the table of contents, the acknowledgements, and the preface. The first page after the front cover was Chapter 1. Befuddled, I reached out to Brian Okken (who I personally met at PyCon 2018). We suspected that it was either a misprint or a bootleg copy. Either way, we sent the evidence to the publisher, and Amazon graciously exchanged my defective copy for the real deal. Please look out for this sort of problem if you purchase a printed copy of this book for yourself!

 

If you want to learn more about pytest, please read my article Python Testing 101: pytest.

Book Review: Hands-On Enterprise Automation with Python

tl;dr

Title Hands-On Enterprise Automation with Python (available from Amazon and Packt)
Author Bassem Aly
Publication 2018 (Packt Publishing)
Summary Using Python packages for network, system, and infrastructure automation.
Prerequisites Intermediate Python programming. Intermediate administrative skills.

Summary

Hands-On Enterprise Automation with Python is an excellent resource for learning how to automate common administrative tasks like running commands, scraping network config, and setting up systems. Python is a natural fit for such tasks with its impressive package library, its easy learning curve, and its concise syntax.

The number of Python tools and modules this book covers is stunning:

  • Developing Python code with PyCharm
  • Managing network devices with paramiko, netmiko, and telnetlib
  • Using regular expressions with re
  • Charting data with matplotlib
  • Templating YAML files with Jinja2
  • Multiprocessing with multiprocessing
  • Running local system commands with subprocess
  • Running remote system commands with fabric
  • Getting system info with platform
  • Sending emails with smtplib
  • System administration with Ansible
  • Interacting with a MySQL database using MySQLdb
  • Storing files in Amazon S3 using boto3
  • Packet sniffing and manipulating with scapy

Each new topic is introduced with background information, setup steps, and example code. Instructions are given for the reader to set up their own test environment and try things out. Later chapters also show how to use modules together to build more powerful automation. Most examples favor Python 2 but can be made compatible with Python 3.

Praises

The best thing about this book is how it covers an incredible breadth of topics in such a readable way. Rather than being dry manual pages pulled from a cryptic doc site, each chapter is a tutorial with explanations and real-world code examples. Readers can easily read through the book cover-to-cover or seek topics directly as a reference.

Another great thing is that the author always introduces new concepts before applying them. While intermediate skills with Python and administration are presumed as a prerequisite, the introductory chapters nevertheless show how to set up a full Python development environment with a network lab for testing. Before showing how to use any particular module, the author explains what the problem is and why the module should be used. This makes the material very accessible, especially for non-sysadmins.

Comments

  • The tasks showcased are network-heavy.
  • The setup primarily relies on Linux.
  • Many pages are dedicated to workbench setup.
  • Modules are covered at an introductory level and not in depth.

Takeaways

I thoroughly enjoyed reading Hands-On Enterprise Automation with Python. As a Software Engineer in Test, the word “automation” almost always means “test automation.” Reading this book was a healthy glimpse at other no-less-important types of automation oriented more toward admin tools and scripts. I could definitely leverage many of the modules covered in this book for my own work, too – several of them cross domains.

I really liked the three main reasons the author gave for using Python for automation: it is readable, it has so many libraries, and it has power in its conciseness. These reasons ring true for many applications, especially the point about modules. Since there’s a module for nearly everything, Python programming often simplifies to recipes for using those modules. Arguably, this book is a cookbook full of sysadmin recipes.

Reading this book also made me reminiscent of my days working at MaxPoint, where I first learned Python to build a test automation framework. I used many of the same modules shared in this book for the same tasks. I felt comfort in my familiarity and validation in my past efforts.

PyOhio 2018 Reflections

PyOhio 2018 was a free Python conference hosted at Ohio State University in Columbus, OH from July 28-29. I had the pleasure of not only attending but also speaking at PyOhio, and my company, PrecisionLender, graciously covered my travel expenses. I had a great time. Here’s my retrospective on the conference.

My Talk

The main reason I went to PyOhio was because I was honored to be a speaker. When I was at an Instagram dinner at PyCon 2018, I met a few conference organizers who encouraged me to propose talks at other Python conferences. On a whim the next morning, I spitballed an idea for a talk about building a test automation solution from the ground up in Python. After talking with a number of people, I realized how test automation is such a struggle everywhere. I took inspiration from Ying Li’s keynote and crafted a story about how Amanda the Panda, a Bamboozle employee, becomes a test automation champion. And, BOOM! My talk proposal was accepted for PyOhio and PyGotham! The video recording for my talk, “Egad! How Do We Start Writing (Better) Tests?”, is below:

Arrival

Good news: Raleigh and Columbus have direct flights. Bad news: they are either early-morning or late-night direct flights. So, I left Raleigh on Friday morning before the conference and spent the day in Columbus. Surprisingly, the security line at RDU wrapped around 2/3 of the Terminal 2 perimeter, but I still boarded the flight on time. Once I landed in Columbus, I took the COTA AirConnect bus downtown for the low price of $2.75.

My goal for Friday was personal development. I rarely get a chance to escape the rigors of everyday life to focus on myself. Personal retreats let me clear my mind, dream big, and begin taking action. And on this day, I started writing my first test automation book – a dream I’ve held for over a year now. I spent a few hours at Wolf’s Ridge Brewery, sampling beers with lunch as I developed a rough outline for my project.

My evening was low-key. I took a nap at my hotel, the Blackwell Inn and Pfahl Conference Center. For dinner, I ate at White Castle for the first time – and it was pretty darn good. After practicing my talk, I got a tiramisu bubble tea from Vivi as a night cap.

The Conference

PyOhio was a much smaller conference than PyCon. There were fewer vendor tables but nevertheless a wide selection of stellar talks. As a result, the conference felt more intimate and more focused. Perhaps that feeling was due also to the venue: the third floor of the Ohio Union had full rooms with “cozy” hallways. Hats off to the organizers, too – everything ran smoothly and professionally.

As soon as I arrived, I scored my name badge, my swag bag, and my official PyOhio 2018 t-shirt. The opening keynote from Adrienne Lowe, “From Support to Engineering and Beyond: What to Take with You, and What to Leave Behind,” about the highs and lows of trying to make it as a developer was exceptionally inspiring. Engineers often don’t talk about how hard the job is, especially for newcomers to the industry. Everybody suffers from imposter syndrome. Everybody feels inadequate. Everybody is tempted to quit, even to the point of tears. The vulnerability in hearing others say, “Me, too,” is so relatable and so relieving.

The first talk-talk I attended was Trey Hunner’s “Easier Classes: Python Classes Without All the Cruft.” Trey gave an excellent overview of writing more sophisticated Python classes. TL;DR: upgrade to 3.7 and use dataclasses.

The next talk I attended was Leo Guinan’s “Go with the Flow: Automating Your Workflows with Airflow.” Apache Airflow is a platform for automating workflows. As an automationeer, it struck me as being like a continuous integration system generalized for non-build purposes. The Q&A portion of the talk was lit.

After finding an authentic Chinese restaurant for lunch, my friend Matt arrived! I worked with Matt in the testing space at LexisNexis. He drove all the way from Dayton to see my talk and hang out. We spent the early afternoon catching up, and we went to Hook Hearted Brewing for dinner after the conference because we’re beer buddies. I was so thankful he came to support me – it meant a lot!

My talk was at 3:45pm. Other than discovering my Thunderbolt-to-HDMI adapter was a dud, the talk went very well. I decided to stick to a script for this talk because most of it followed a story, and I’m glad I did. (For my PyCon talk, I chose instead to speak without a script and rely instead on the slides alone.) There were about 30 people in the audience. Many expressed appreciation for my presentation!

The last talk of the day for me was Jace Browning’s “Automated Regression Testing with Splinter and Jupyter.” It was the perfect follow-up to my talk. Whereas mine was mostly high-level, Jace showed implementation and execution. I loved how he compared raw Selenium WebDriver calls to splinter calls, and I was thrilled to see hands-on test execution using Jupyter. One of the things that makes Python so great for automation is that modules can be called from the interpreter – and Jupyter notebooks make that so easy.

The Second Day

Sunday was a shorter conference day. The opening keynote, Lorena Mesa’s “Now is better than Never: What the Zen of Python can teach us about Data Ethics,” didn’t start until 11:40am. Lorena showed us what the Zen of Python can teach us about data ethics in a scary, modern world.

I got lunch at Chatime: dan dan noodles (or rather, an imitation thereof) and a matcha latte with grass jelly. Yum! After lunch, I attended Daniel Lindeman’s “Python in Serverless Architectures.” Now I know what the buzzword “serverless” means! I even found out that I had already developed a serverless app using Django and Heroku. There are some really cool ways test automation could take advantage of serverless architectures.

Another one of my favorite talks of the afternoon was Vince Salvino’s “Containers Without the Magic.” Vince broke down how easy containers are to use. It was a great refresher for me.

Open Spaces

At 3:15 on Sunday, I tried something new: I hosted an open space for test automation. “Open spaces” are rooms that can be reserved for a time slot to meet up informally about a common interest. (For example, PyCon had a juggling open space!) At first, nobody showed up to my open space, but after a few minutes, one lady walked in. She had been a software tester for years and wanted to start doing automation. I walked her through as much info as I could before time was up. She was very grateful for the guidance I offered. It worked out nicely that she was the only person to come to my open space so that she could really get value out of it. (My friend Jason also popped in and helped out; more on him below.)

The After-Party

At conferences, my biggest fear is being awkwardly alone. I want to spend time with good people, both new and familiar. Thankfully, PyOhio didn’t disappoint.

Backstory: At PyCon 2018, I met a guy named Julian who runs PyBites (together with his buddy Bob). We really hit it off, and he invited me to join the PyBites community. They offer great code challenges and a “100 Days of Code” challenge course, as well as a blog about all things Python. Through the PyBites community, I met another guy named Jason who would be at PyOhio 2018 with me. We agreed to meet up for dinner and drinks after the Sunday talks.

(On a side note, I recommend PyBites as a great place to learn new things, hone skills, and meet great people!)

That Sunday night, it just so happened that Adrienne and Trey, two of the other speakers, intersected Jason and me as we were deciding where to go for dinner. The next thing we know (after a hotel pitstop), we’re all walking off together to Eden Burger, a local vegan burger joint. I had a vegan “cheeseburger” with fries and a “milkshake” – and they were genuinely delicious! More than the food, I enjoyed my time with new friends. I was really inspired by the cool things each of them is doing. I guess that’s Python conference magic!

Jason and I hit World of Beer after dinner. After Slack-ing for weeks, it was so good to spend time with this fine gent. We discussed Python, software, our careers, our families, and our dreams. What a perfect way to conclude PyOhio 2018!

Takeaways

There were so many takeaways from PyOhio 2018 for me:

  1. Conferences are phenomenal for professional development. The pulse I get from conferences is electrifying. I walked away from PyOhio galvanized to be an even better software engineer. The talks opened up exciting new ideas. Inspiration for several blog posts sprang forward. The people I met motivated me to try new things. I got so much vigor out of such a short time.
  2. My friends around the globe are awesome. Matt, Jason, Adrienne, Trey, Julian (vicariously), and all the other great people I met at PyOhio made my conference experience so rewarding.
  3. Good values foster wonderful communities. My company, PrecisionLender, has four major values: Be helpful, humble, honest, and human. Those values make my company such a great place to work. I see those same values in the Python community, too. People at PyOhio even asked about these values when they saw them on my PL shirt and my business card. I think that’s partially why Python conferences are always so welcoming and inspiring.
  4. Bigger conferences have more pizzazz, while smaller conferences are more intimate. PyCon 2018 was big, flashy, and awesome. I scored so much swag that I nearly couldn’t fit it all in my suitcase to carry home. PyOhio 2018, on the other hand, focused much more intently on the talks and the people. A perfect example of this was Leo Guinan’s monologue-turned-dialogue on Airflow: it was natural for people to just ask questions. Both types of conferences are good in their own ways.
  5. PyCon 2018 was likely a watershed moment for my career. I cannot reflect on PyOhio 2018 without seeing it as an extension of my PyCon 2018 experience. The only reason I attended PyOhio was because someone at PyCon encouraged me to propose a talk. The reason I met Jason is because I first met Julian. The reason I want to keep speaking is because PyCon went so well for me. The fact that both conferences were hosted in Ohio only two months apart is also rather serendipitous. Like my first trip to China, I think PyCon 2018 will have a lasting impact on my career.

EGAD! How Do We Start Writing (Better) Tests?

Some have never automated tests and can’t check themselves before they wreck themselves. Others have 1000s of tests that are flaky, duplicative, and slow. Wa-do-we-do? Well, I gave a talk about this problem at PyOhio 2018. The language used for example code was Python, but the principles apply to any language. Please watch it below!

Why Python is Great for Test Automation

Python is an incredible programming language. As Dan Callahan said in his PyCon 2018 keynote, “Python is the second best language for anything, and that’s an amazing aspiration.” For test automation, however, I believe it is one of the best choices. Here are ten reasons why:

#1: The Zen of Python

The Zen of Python, as codified in PEP 20, is an ideal guideline for test automation. Test code should be a natural bridge between plain-language test steps and the programming calls to automate them. Tests should be readable and descriptive because they describe the features under test. They should be explicit in what they cover. Simple steps are better than complicated ones. Test code should add minimal extra verbiage to the tests themselves. Python, in its concise elegance, is a powerful bridge from test case to test code.

(Want a shortcut to the Zen of Python? Run “import this” at the Python interpreter.)

#2: pytest

pytest is one of the best test frameworks currently available in any language, not just for Python. It can handle any functional tests: unit, integration, and end-to-end. Test cases are written simply as functions (meaning no side effects as long as globals are avoided) and can take parametrized inputs. Fixtures are a generic, reusable way to handle setup and cleanup operations. Basic “assert” statements have automatic introspection so failure messages print meaningful values. Tests can be filtered when executed. Plugins extent pytest to do code coverage, run tests in parallel, use Gherkin scenarios, and integrate with other frameworks like Django and Flask. Other Python test frameworks are great, but pytest is by far the best-in-show. (Pythonic frameworks always win in Python.)

#3: Packages

For all the woes about the CheeseShop, Python has a rich library of useful packages for testing: pytest, unittest, doctest, tox, logging, paramiko, requests, Selenium WebDriver, Splinter, Hypothesis, and others are available as off-the-shelf ingredients for custom automation recipes. They’re just a “pip install” away. No reinventing wheels here!

#4: Multi-Paradigm

Python is object-oriented and functional. It lets programmers decide if functions or classes are better for the needs at hand. This is a major boon for test automation because (a) stateless functions avoid side effects and (b) simple syntax for those functions make them readable. pytest itself uses functions for test cases instead of shoehorning them into classes (à la JUnit).

#5: Typing Your Way

Python’s out-of-the-box dynamic duck typing is great for test automation because most feature tests (“above unit”) don’t need to be picky about types. However, when static types are needed, projects like mypy, Pyre, and MonkeyType come to the rescue. Python provides typing both ways!

#6: IDEs

Good IDE support goes a long way to make a language and its frameworks easy to use. For Python testing, JetBrains PyCharm supports visual testing with pytest, unittest, and doctest out of the box, and its Professional Edition includes support for BDD frameworks (like pytest-bdd, behave, and lettuce) and Web development. For a lighter offering, Visual Studio Code is taking the world by storm. Its Python extensions support all the good stuff: snippets, linting, environments, debugging, testing, and a command line terminal right in the window. Atom, Sublime, PyDev, and Notepad++ also get the job done.

#7: Command Line Workflow

Python and the command line are like peanut butter and jelly – a match made in heaven. The entire test automation workflow can be driven from the command line. Pipenv can manage packages and environments. Every test framework has a console runner to discover and launch tests. There’s no need to “build” test code first because Python is an interpreted language, further simplifying execution. Rich command line support makes testing easy to manage manually, with tools, or as part of build scripts / CI pipelines.

As a bonus, automation modules can be called from the Python REPL interpreter or, even better, a Jupyter notebook. What does this mean? Automation-assisted exploratory testing! Imagine using Python calls to automatically steer a Web app to a point that requires a manual check. Calls can be swapped out, rerun, skipped, or changed on the fly. Python makes it possible.

#8: Ease of Entry

Python has always been friendly to beginners thanks to its Zen, whether those beginners are programming newbies or expert engineers. This gives Python a big advantage as an automation language choice because tests need to be done quickly and easily. Nobody wants to waste time when the features are in hand and just need to be verified. Plus, many manual software testers (often without programming experience) are now starting to do automation work (by choice or by force) and benefit from Python’s low learning curve.

#9: Strength for Scalability

Even though Python is great for beginners, it’s also no toy language. Python has industrial-grade strength because its design always favors one right way to get a job done. Development can scale thanks to meaningful syntax, good structure, modularity, and a rich ecosystem of tools and packages. Command line versatility enables it to fit into any tool or workflow. The fact that Python may be slower than other languages is not an issue for feature tests because system delays (such as response times for Web pages and REST calls) are orders of magnitude slower than language-level performance hits.

#10: Popularity

Python is one of the most popular programming languages in the world today. It is consistently ranked near the top on TIOBE, Stack Overflow, and GitHub (as well as GitHut). It is a beloved choice for Web developers, infrastructure engineers, data scientists, and test automationeers alike. The Python community also powers it forward. There is no shortage of Python developers, nor is there any dearth of support online. Python is not going away anytime soon. (Python 3, that is.)

Other Languages?

The purpose of this article is to highlight what makes Python great for test automation based on its own merits. Although I strongly believe that Python is one of the best automation languages, other choices like Java, C#, and Ruby are also viable. Check out my article The Best Programming Language for Test Automation for a comparison.

 

This article was posted with the author’s permission on both Automation Panda and PyBites.

The Panda’s Dozen: Top PyCon 2018 Talks

There were tons of great talks at PyCon 2018 – more than I could attend in person – that are now available on the PyCon 2018 YouTube channel. This post has links to my favorites. Enjoy!

Check out PyCon 2018 Reflections to read my personal reflections, too. Watch my talk, Behavior-Driven Python, too!

By the Numbers: Python Community Trends in 2017/2018 (Dmitry Filippov, Ewa Jodlowska) – At the end of 2017, the Python Software Foundation teamed up with JetBrains to conduct an official Python Developers Survey. Data science is taking Python by storm, and Python 3 now has majority adoption. There are tons of other cool statistics, too!

How Netflix does failovers in 7 minutes flat (Amjith Ramanujam) – That speed at that scale is mind-blowing. This is a fascinating talk, even for non-engineers!

Solve Your Problem With Sloppy Python (Larry Hastings) – “If you ever start writing a shell script, delete it and write a Python script instead.” This talk is a jovial reminder that Python is a powerful tool, even for hack-n-slash jobs.

The AST and Me (Emily Morehouse-Valcarcel) – Emily gives a great overview of the inner workings of the Python language. This talk is a must-see for anyone into compiler theory.

Dataclasses: The code generator to end all code generators (Raymond Hettinger) – Dataclasses are new data structures to Python to generate classes based on specs.

Pipenv: the Future of Python Dependency Management (Kenneth Reitz) – Pipenv is a new tool that makes pip, Pipfile, and virtualenv easier to use together. Kenneth gives a good overview of Python packaging and why pipenv is awesome.

Type-checked Python in the real world (Carl Meyer) – Sometimes, I wish Python had static typing. Now, it can! Facebook has done some innovative things to make it possible.

Beyond Unit Tests: Taking Your Testing to the Next Level (Hillel Wayne) – Property tests + contracts = integration tests. Hillel gives a fantastic strategy for making tests smarter.

“WHAT IS THIS MESS?” – Writing tests for pre-existing code bases (Justin Crown) – This is a pragmatic guide to adding new tests to old code. Now, you’ll never procrastinate on your tests again!

Demystifying the Patch Function (Lisa Roach) – Mocking is a great practice for limiting scope in unit tests. Whether using unittest.mock or other patching/mocking packages, this is a great talk for learning why and how to do mocking in unit tests!

Automating Code Quality (Kyle Knapp) – One of Python’s most beloved traits is its elegance. Maintaining high standards of code quality can be challenging for large projects, though. Kyle shows how to use existing tools to drive higher quality.

Keynote (Ying Li) – [35:07] – Ying is a software security engineer at Docker. In her keynote, she urged all people involved in technology to learn the basics of security. Definitely watch the video recording – she used a fun story to illustrate her points.

Keynote: The People and Python (Qumisha Goss) – [1:07:35] – “Q” is a librarian at the Detroit Public Library who taught herself Python so she could teach coding classes to kids. She shared the highs and lows of her experiences, especially in light of many disadvantages her students had. My favorite takeaway was to “cultivate greatness in others.”