books

I’m Writing a Software Testing Book!

That’s right! You read the title. I’m writing a book about software testing!

One of the most common questions people ask me is, “What books can you recommend on software testing and automation?” Unfortunately, I don’t have many that I can recommend. There are plenty of great books, but most of them focus on a particular tool, framework, or process. I haven’t found a modern book that covers software testing as a whole. Trust me, I looked – when I taught my college course on software testing at Wake Tech, the textbook’s copyright date was 2002. Its content felt just as antiquated.

I want to write a book worthy of answering that question. I want to write a treatise on software testing for our current generation of software professionals. My goal is ambitious, but I think I can do it. It will probably take a year to write. I hope to find deep joy in this endeavor.

Manning Publications will be the publisher. They accepted my proposal, and we signed a contract. The working title of the book is The Way to Test Software. The title pays homage to Julia Child’s classic, The Way to Cook. Like Julia Child, I want to teach “master recipes” that can be applied to any testing situations.

I don’t want to share too many details this early in the process, but the tentative table of contents has the following parts:

  1. Orientation
  2. Testing Code
  3. Testing Features
  4. Testing Performance
  5. Running Tests
  6. Development Practices

Python will be the language of demonstration. This should be no surprise to anyone. I chose Python because I love the language. I also think it’s a great language for test automation. Python will be easy for both beginners and experts to learn. Besides, the book is about testing, not programming – Python will be just the linguistic tool for automation.

If you’re as excited about this book as I am, please let me know! I need all the encouragement I can get. This book probably won’t enter print until 2022, given the breadth of its scope. I’ll work to get it done as soon as I can.

Book Review: pytest Quick Start Guide

tl;dr

Title pytest Quick Start Guide
(available from Packt and Amazon)
Author Bruno Oliveira (@nicoddemus)
Publication 2018 Packt Publishing
Summary “Write better Python code with simple and maintainable tests” – a very readable guide for pytest’s main features.
Prerequisites Intermediate-level Python programming.

Summary

pytest Quick Start Guide is a new book about using pytest for Python test automation. Bruno Oliveira explains not only how to use pytest but also why its features are useful. Even though this book is written as a “quick start” introduction, it nevertheless dives deep into pytest’s major features. It covers:

  1. An introduction to pytest
  2. Why pytest is superior to unittest
  3. Setting up pytest with pip and virtualenv
  4. Writing basic tests
  5. How assertions really work
  6. Common command line arguments
  7. Marks and parametrization
  8. Using and writing fixtures
  9. Popular plugins
  10. Converting unittest suites to pytest

Example code is hosted on GitHub at PacktPublishing/pytest-Quick-Start-Guide.

Praises

This book is an easy introduction to test automation in Python with pytest. Readers should have intermediate-level Python skills, but they do not need previous testing or automation skills. The progression of chapters makes it easy to start quick and then go deeper. Oliveira has a very accessible writing style, too.

The unittest refactoring guide is a hidden gem. The unittest module is great because it comes bundled with Python, but it is also clunky and not very Pythonic. Not all teams know the best way to modernize their test suites. Oliveira provides many pieces of practical advice for making the change, at varying degrees of conversion. The big changes use pytest’s fixtures and assertions.

A Comparison to Python Testing with pytest

Python Testing with pytest by Brian Okken is another popular pytest book. Both books are great resources for learning pytest, but each approaches the framework from a unique perspective. How do they compare? Here’s what I saw:

Oliveira’s book Okken’s book
Oliveira’s book is a great introductory guide for pytest. Oliveira’s writing style makes the reader feel like the author is almost teaching in person. The book’s main theme is getting the reader to use the main features of pytest pragmatically for general testing needs. The main differences in content are the unittest refactoring guide and some of the plugins and fixtures covered. This book is probably the best choice for beginners who want to learn the basics of pytest quickly. Okken’s book is introductory but also a great manual for future reference. Okken’s writing style is direct and concise, which covers more material in fewer pages. The format for each chapter is consistent: for each idea: idea → code → output → explanation; exercises at the end. Okken also covers how to create and share custom pytest plugins. This book is probably the best choice for people who want to master the ins and outs of pytest.

Ultimately, I recommend both books because they are both excellent.

Takeaways

Reading books on the same subject by different authors helps the reader learn the subject better. I’ve used pytest quite a lot myself, but I was able to learn new things from both pytest Quick Start Guide and Python Testing with pytest. Reading how experts use and think about the framework makes me a better engineer. Different writing styles and different opinions also challenge my own understandings. (It’s also funny that the authors of both pytest books have the same initials – “B.O.”)

pytest is really popular. There are now multiple good books on the subject. It’s becoming the de facto test automation framework for Python, outpacing unittest, nose, and others. These days, it seems more popular to write a pytest plugin than to create a new framework.

Book Review: Python Testing with pytest

tl;dr

Title Python Testing with pytest
Author Brian Okken (@brianokken)
Publication 2017 (The Pragmatic Programmers)
Summary How to use all the features of pytest for Python test automation – “simple, rapid, effective, and scalable.”
Prerequisites Intermediate-level Python programming.

Summary

Python Testing with pytest is the book on pytest*. Brian Okken covers all the ins and outs of the framework. The book is useful both as tutorial for learning pytest as well as a reference for specific framework features. It covers:

  • Getting started with pytest
  • Writing simple tests as functions
  • Writing more interesting tests with assertions, exceptions, and parameters
  • Using all the different execution options
  • Writing fixtures to flexibly separate concerns and reuse code
  • Using built-in fixtures like tmpdir, pytestconfig, and monkeypatch
  • Using configuration files to control execution
  • Integrating pytest with other tools like pdb, tox, and Jenkins

Appendices also cover:

  • Using Python virtual environments
  • Installing packages with pip
  • An overview of popular plugins like pytest-xdist and pytest-cov
  • Packaging and distributing Python packages

[* Update on 9/27/2018: Check out my review on another excellent pytest book, pytest Quick Start Guide by Bruno Oliveira.]

Praises

This book is a comprehensive guide to pytest. It thoroughly covers the framework’s features and gives pointers to more info elsewhere. Even though pytest has excellent online documentation, I still recommend this book to anyone who wants to become a pytest master. Online docs tend to be fragmented with each piece limited in scope, whereas books like this one are designed to be read progressively and orderly for maximal understanding of the material.

I love how this book is example-driven. Each section follows a simple yet powerful outline: idea → code → output → explanation. Having real code with real output truly cements the point of each mini-lesson. New topics are carefully unfolded so that they build upon previous topics, making the book read like a collection of tutorials. Examples at the end of every chapter challenge the readers to practice what they learn. The formatting of each section also looks great.

The extra info on related topics like pip and virtualenv is also a nice touch. Python pros probably don’t need it, but beginners might get stuck without it.

The rocket ship logo on the cover is also really cool!

Takeaways

pytest is one of the best functional test frameworks currently available in any language. It breaks the clunky xUnit mold, in which class structures were awkwardly superimposed over test cases as if one size fits all. Instead, pytest is lightweight and minimalist because it relies on functions and fixtures. Scope is much easier to manage, code is more reusable, and side effects can more easily be avoided. pytest has taken over Python testing because it is so Pythonic.

Brian’s concise writing style has also inspired me to be more direct in my own writing. I tend to be rather verbose out of my desire to be descriptive. However, fewer words often leave a more powerful impression. They also make the message easier to comprehend. Python is beloved for its concise expressiveness, and as a Pythonista, it would be fitting for me to adopt that trait into my English.

If I had a wish list for a second edition, I’d love to see more info about assertions and other plugins (namely pytest-bdd). I think an appendix with comparisons to other Python test frameworks would also be valuable.

A Warning

I ordered a physical copy of this book directly from Amazon (not a third-party seller). Unfortunately, that copy was missing all the introductory content: the table of contents, the acknowledgements, and the preface. The first page after the front cover was Chapter 1. Befuddled, I reached out to Brian Okken (who I personally met at PyCon 2018). We suspected that it was either a misprint or a bootleg copy. Either way, we sent the evidence to the publisher, and Amazon graciously exchanged my defective copy for the real deal. Please look out for this sort of problem if you purchase a printed copy of this book for yourself!

If you want to learn more about pytest, please read my article Python Testing 101: pytest.

Book Review: Hands-On Enterprise Automation with Python

tl;dr

Title Hands-On Enterprise Automation with Python (available from Amazon and Packt)
Author Bassem Aly
Publication 2018 (Packt Publishing)
Summary Using Python packages for network, system, and infrastructure automation.
Prerequisites Intermediate Python programming. Intermediate administrative skills.

Summary

Hands-On Enterprise Automation with Python is an excellent resource for learning how to automate common administrative tasks like running commands, scraping network config, and setting up systems. Python is a natural fit for such tasks with its impressive package library, its easy learning curve, and its concise syntax.

The number of Python tools and modules this book covers is stunning:

  • Developing Python code with PyCharm
  • Managing network devices with paramiko, netmiko, and telnetlib
  • Using regular expressions with re
  • Charting data with matplotlib
  • Templating YAML files with Jinja2
  • Multiprocessing with multiprocessing
  • Running local system commands with subprocess
  • Running remote system commands with fabric
  • Getting system info with platform
  • Sending emails with smtplib
  • System administration with Ansible
  • Interacting with a MySQL database using MySQLdb
  • Storing files in Amazon S3 using boto3
  • Packet sniffing and manipulating with scapy

Each new topic is introduced with background information, setup steps, and example code. Instructions are given for the reader to set up their own test environment and try things out. Later chapters also show how to use modules together to build more powerful automation. Most examples favor Python 2 but can be made compatible with Python 3.

Praises

The best thing about this book is how it covers an incredible breadth of topics in such a readable way. Rather than being dry manual pages pulled from a cryptic doc site, each chapter is a tutorial with explanations and real-world code examples. Readers can easily read through the book cover-to-cover or seek topics directly as a reference.

Another great thing is that the author always introduces new concepts before applying them. While intermediate skills with Python and administration are presumed as a prerequisite, the introductory chapters nevertheless show how to set up a full Python development environment with a network lab for testing. Before showing how to use any particular module, the author explains what the problem is and why the module should be used. This makes the material very accessible, especially for non-sysadmins.

Comments

  • The tasks showcased are network-heavy.
  • The setup primarily relies on Linux.
  • Many pages are dedicated to workbench setup.
  • Modules are covered at an introductory level and not in depth.

Takeaways

I thoroughly enjoyed reading Hands-On Enterprise Automation with Python. As a Software Engineer in Test, the word “automation” almost always means “test automation.” Reading this book was a healthy glimpse at other no-less-important types of automation oriented more toward admin tools and scripts. I could definitely leverage many of the modules covered in this book for my own work, too – several of them cross domains.

I really liked the three main reasons the author gave for using Python for automation: it is readable, it has so many libraries, and it has power in its conciseness. These reasons ring true for many applications, especially the point about modules. Since there’s a module for nearly everything, Python programming often simplifies to recipes for using those modules. Arguably, this book is a cookbook full of sysadmin recipes.

Reading this book also made me reminiscent of my days working at MaxPoint, where I first learned Python to build a test automation framework. I used many of the same modules shared in this book for the same tasks. I felt comfort in my familiarity and validation in my past efforts.