Python Program Main Function

This article will show you the best way to handle “main” functions in Python.

Python is like a scripting language in that all lines in a Python “module” (a .py file) are executed whenever the file is run. Modules don’t need a “main” function. Consider a module named stuff.py with the following code:

def print_stuff():
  print("stuff happened!")

print_stuff()

This is the output when it is run:

$ python stuff.py
stuff happened!

The print_stuff function was called as a regular line of code, not in any function. When the module ran, this line was executed.

This will cause a problem, though, if stuff is imported by another module. Consider a second module named more_stuff.py:

import stuff

stuff.print_stuff()
print("more stuff!")

At first glance, we may expect to see two lines printed. However, running more_stuff actually prints three lines:

$ python more_stuff.py
stuff happened!
stuff happened!
more stuff!

Why did “stuff happened!” get printed twice? Well, when “import stuff” was called, the stuff module was loaded. Whenever a module is loaded, all of its code is executed. The print_stuff function was called at line 4 in the stuff module. Then, it was called again at line 3 in the more_stuff module.

So, how can we avoid this problem? Simple: check the module’s __name__. The __name__ variable (pronounced “dunder name”) is dynamically set to the module’s name. If the module is the main entry point, then __name__ will be set to “__main__”. Otherwise, if the module is simply imported, then it will be set to the module’s filename without the “.py” extension.

Let’s rewrite our modules. Here’s stuff:

def print_stuff():
  print("stuff happened!")

if __name__ == '__main__':
  print_stuff()

And here’s more_stuff:

import stuff

if __name__ == '__main__':
  stuff.print_stuff()
  print("more stuff!")

If we rerun more_stuff, then the line “stuff happened!” will print only once:

$ python more_stuff.py
stuff happened!
more stuff!

As a best programming practice, Python modules should not contain any directly-called lines. They should contain only functions, classes, and variable initializations. Anything to be executed as a “main” body should be done after a check for “if __name__ == ‘__main__'”. That way, no rogue calls are made when modules are imported by other modules. The conditional check for __name__ also makes the “main” body clear to the reader.

Some people still like to have a “main” function. That’s cool. Just do it like this:

import stuff

def main():
  stuff.print_stuff()
  print("more stuff!")

if __name__ == '__main__':
  main()

For more information, read this Stack Overflow article:
What does if __name__ == “__main__”: do?

PyOhio 2019 Reflections

PyOhio 2019 was one of my favorite conferences, ever. It was my ninth Python conference and my second PyOhio conference. There were so many good things that happened in such short time that, even three weeks later, I’m still processing everything. Here are my reflections on this outstanding conference.

Roadtrip

PyOhio 2019 was held in Columbus, Ohio at the Ohio State Union. I really wanted to go because PyOhio 2018 was such a good time, and I started asking my friends in North Carolina if anyone wanted to join me. My friends Rick and Justin all emphatically replied YES! To make traveling more fun, we decided to turn it into a road trip! We dubbed ourselves the “PyCarolinas delegation”, piled into my Chrysler 300, and made the 8-hour drive in good time. Our friend Greg also joined us at PyOhio, though he traveled separately with his wife.

This was the first time I ever did a road trip to a conference. I’m so glad we did it. I felt like I got to spend great quality time with my friends. Many parts of the drive were quite scenic. We also discovered a Beef Jerky Outlet!

My Talks

At PyOhio 2019, I delivered the holy trifecta of speaking opportunities: a talk, a tutorial, and a lightning talk. I felt both honored and humbled to be chosen for all three.

My talk was entitled Surviving Without Python. I talked about how we can use Python’s principles, projects, and people to inspire us even when we don’t use Python to solve our problems:

My tutorial was entitled Hands-On Web UI Testing. Python’s popularity continues to rise, and many people use it for testing. In my tutorial, I showed how you can develop a simple yet powerful solution with Python, pytest, and Selenium WebDriver to automate Web UI tests. The tutorial project in GitHub contains the code, instructions, and slides.

My lightning talk was announcing PyCarolinas 2020. My friend Calvin and I are teaming up to bring PyCarolinas back! We are targeting June 2020 in Raleigh, NC. Please help us make it happen!

Other Talks

There were so many great talks at PyOhio. Here were a few highlights.

This was my favorite talk of the conference. Although a history lesson may seem out of place at a Python conference, Jon used it as a convicting call to action. Be sure to watch it to the end!
Dane showed how easy it is to use pytest. You don’t need to be an expert tester to write good tests!
Travis blew my mind with some of Python’s latest features. This talk was a great way to catch up on new things!
Dustin explained why it’s important to keep alive the passion we have for computing. I also loved the TI-83 Plus reference – that was in my talk, too!
Controversial? Yes. True? Yes. Aly? Preach!
Docs are vital for understanding code and features. Mason shows how to make docs just a regular part of the development process.

Sprints

Sprints are a Python conference event in which people work together on open-source projects. Conferences are the perfect time to have sprints because people are both co-located and excited. PyOhio 2019 was actually the first time I attended sprints. There were sprints on both Friday and Saturday nights. Accenture graciously hosted both sprints in their swanky office and provided refreshments.

I went to the sprints both nights with the honest intention to work on stuff. However, I spent the whole time socializing with friends. It was nevertheless time well spent! Many of us went to Jeni’s Splendid Ice Creams after the final sprint, too.

Swag

The PyOhio 2019 logo was lit! I joked that I was going to the conference just to get that logo sticker and t-shirt. Some other cool takeaways were the Numerator puzzles and the JP Morgan Chase puzzle blocks.

My sticker game is strong:

My backpack was also a hit!

Food and Drink

I had some good eats while in Columbus. My friend Mason recommended Raising Cane’s, which turned out to be awesome! So many people followed us there, too.

I also hopped an electric scooter to get bubble tea. Those scooters are so much fun. I hadn’t ridden one since PyTexas 2019! (Or maybe PyCon 2019; I don’t remember precisely.)

To relive PyOhio 2018 memories, I went to Eden Burger for a Vegan brunch on Sunday morning with friends! It was delicious and nutritious.

Our friend Greg recommended we visit Brewdog, a renowned Scottish craft brewery with a huge site on the outskirts of Columbus. We took the tour with the “beer school” course, and we stayed for a delicious dinner afterwards. So good, so very good. It was the best way to end the conference!

Friends

The best part about PyOhio 2019 was the time I spent with all my friends. I wish I could name everyone here, but there are just too many names. The Python community is the best. Conference friends are real friends. Also, for what it’s worth, they’ve given me the nickname “Pandy.”

Hope

My PyOhio 2019 experience can be summarized in one line:

Truth – I went all-in from leaving my house on Friday morning until returning Monday evening. The conference high is very real. My hope is that I’ll get to attend more great conferences like this, and also that I’ll be able to help make PyCarolinas as good as PyOhio!

Hands-On UI Testing with Python (SmartBear Webinar)

On August 14, 2019, I teamed up with SmartBear to deliver a one-hour webinar about Web UI testing with Python! It was an honor to work with Nicholas Brown, Digital Marketing Manager for CrossBrowserTesting at SmartBear Software, to make this webinar happen.

The Webinar

Source: https://crossbrowsertesting.com/resources/webinars/testing-with-python

In the webinar, I showed how to build a basic Web UI test automation solution using Python, pytest, and Selenium WebDriver. The tutorial covered automating one test, a simple DuckDuckGo search, from inception to automation. It also showed how to use CrossBrowserTesting to scale the solution so that it can run tests on any browser, any platform, and any version in the cloud as a service!

The example test project for the webinar is hosted in Github here: https://github.com/AndyLPK247/smartbear-hands-on-ui-testing-python

I encourage you to clone the Github repository and try to run the example test on your own! Make sure to get a CrossBrowserTesting trial license so you can try different browsers. You can also try to write new tests of your own. All instructions are in the README. Have fun with it!

The Q&A

After the tutorial, we took questions from the audience. Here are answers to the top questions:

How can we automate UI interactions for CAPTCHA?

CAPTCHA is a feature many websites use to determine whether or not a user is human. Most CAPTCHAs require the user to read obscured text from an image, but there are other variations. By their very nature, CAPTCHAs are designed to thwart UI automation.

When someone asked this question during the webinar, I didn’t have an answer, so I did some research afterwards. Unfortunately, it looks like there’s no easy solution. The best workarounds involve driving apps through their APIs to avoid CAPTCHAs. I also saw some services that offer to solve CAPTCHAs.

Are there any standard Page Object Pattern implementations in Python?

Not really. Mozilla maintains the PyPOM project, but I personally haven’t used it. I like to keep my page objects pretty simple, as shown in the tutorial. I also recommend the Screenplay Pattern, which handles concerns better as test automation solutions grow larger. I’m actually working on a Pythonic implementation of the Screenplay Pattern that I hope to release soon!

How can I run Python tests that use Selenium WebDriver and pytest from Jenkins?

Any major Continuous Integration tool like Jenkins can easily run Web UI tests in any major language. First, make sure the nodes are properly configured to run the tests – they’ll need Python with the appropriate packages. If you plan to use local browsers, make sure the nodes have the browsers and WebDriver executables properly installed. If you plan to use remote browsers (like with CrossBrowserTesting), make sure your CI environment can call out to the remote service. Test jobs can simply call pytest from the command line to launch the tests. I also recommend the “JUnit” pytest option to generate a JUnit-style XML test report because most CI tools require that format for displaying and tracking test results.

How can I combine API and database testing with Web UI testing?

One way to handle API and database testing is to write integration tests separate from Web UI tests. You can still use pytest, but you’d use a library like requests for APIs and SQLAlchemy for databases.

Another approach is to write “hybrid” tests that use APIs and database calls to help Web UI testing. Browsers are notoriously slow compared to direct back-end calls. For example, database calls could pre-populate data so that, upon login, the website already displays stuff to test. Hybrid tests can make tests much faster and much safer.

How can we test mobile apps and browsers using Python?

Even though our tutorial covered desktop-based browser UI interactions, the strategy for testing mobile apps and browsers is the same. Mobile tests need Appium, which is like a special version of WebDriver for mobile features. The Page Object Pattern (or Screenplay Pattern) still applies. CrossBrowserTesting provides mobile platforms, too!

Surviving Without Python


Python is such a popular language for good reason: Its principles are strong. However, if Python is “the second-best language for everything”… that means the first-best is often chosen instead. Oh no! How can Pythonistas survive a project or workplace without our favorite language?

Personally, even though I love Python, I don’t use it daily at my full time job. Nevertheless, Pythonic thinking guides my whole approach to software. I will talk about how the things that make Python great can be applied to non-Python places in three primary ways:

  1. Principles from the Zen of Python
  2. Projects that partially use Python
  3. People who build strong, healthy community

Check out my talk, Surviving Without Python, from PyOhio 2019! It was one of the most meaningful talks I’ve ever given.

Tutorial: Web Testing Made Easy with Python

Have you ever discovered a bug in a web app? Yuck! Almost everyone has. Bugs look bad, interrupt the user’s experience, and cheapen the web app’s value. Severe bugs can incur serious business costs and tarnish the provider’s reputation.

So, how can we prevent these bugs from reaching users? The best way to catch bugs is to test the web app. However, web UI testing can be difficult: it requires more effort than unit testing, and it has a bad rap for being flaky.

Never fear! Recently, I teamed up with the awesome folks at TestProject to develop a helpful tutorial that makes web UI test automation easy with the power of Python! The tutorial is named Web Testing Made Easy with Python, Pytest and Selenium WebDriver. It is available for free as a set of TestProject blog articles together with a GitHub example project.

In our tutorial, we will build a simple yet robust web UI test solution using Pythonpytest, and Selenium WebDriver. We cover strategies for good test design as well as patterns for good automation code. By the end of the tutorial, you’ll be a web test automation champ! Your Python test project can be the foundation for your own test cases, too.

How can you take the tutorial? Start reading here, and follow the instructions: https://blog.testproject.io/2019/07/16/open-source-test-automation-python-pytest-selenium-webdriver/

I personally want to thank TestProject for this collaboration. TestProject provides helpful tools that can supercharge your test automation. They offer a smart test recorder, a bunch of add-ons that act like test case building blocks, an SDK that can make test automation coding easier, and beautiful analytics to see exactly what the tests are doing. Not only is TestProject a cool platform, but the people with whom I’ve worked there are great. Be sure to check it out!

Should We Rewrite Our Test Automation in Another Language?

A Twitter friend recently asked me the following question:

I work in a Microsoft shop. We have 40 developers who use .NET (C#). We also have several manual testers and 5 automation engineers who developed our test automation solution in Python. However, our leadership wants to move everything completely to C#.

Would it be better to (a) train 40 .NET developers in Python to use the existing test solution or (b) train the testers in .NET and port the tests to C#?

This is a very tough question. It’s not as simple as asking for the best test automation language because there are people, positions, and solutions already in place. Honestly, I can’t give a conclusive answer without more context, but I can offer five points of advice.

What is the state of the Python test solution?

How big and how bad is the existing Python test automation solution? Rewriting tests that already work fine has low return-on-investment. However, rewriting tests that have problems like flakiness or false positives might be worthwhile. More tests means more time, too. Please read my article, Our Test Automation Has Problems. Should We Start Over?, to learn what problems would warrant a rewrite.

Why not have two test solutions?

If the existing Python tests are fine, then rewriting them is a huge opportunity cost. Instead of rewriting existing tests, developers and testers could spend their time writing only the new tests in a new C# solution. The Python solution would be “legacy” and would not have any new tests added to it. Old tests would disappear with deprecated features, too. Eventually, the C# tests would take over. The main drawback for this possibility is the continued maintenance of a Python stack.

Do the manual testers have any programming experience?

Many manual testers do not have strong programming skills. Some may not have any programming skills at all! They will have a big learning curve when training to do test automation. Python would be a much easier language for them to learn than C# because it is concise, readable, and friendly for beginners. Conversely, Python would be fairly easy for C# developers to learn as they go.

What advantages will conformity bring?

Retraining workers and rewriting code is no small task. From a business perspective, they are investment costs. There must be significant returns that outweigh the cost of the transition. Make sure those returns are known and real.

Will developers also automate tests?

Many teams choose to write their test automation code in the same language as the product code so that developers can more easily automate tests. However, in my experience, developers typically don’t write many tests, especially when others on the team are dedicated testers. Test automation is difficult and has unique challenges. Some developers have bad attitudes about testing, too. Changing the language probably won’t change the deeper issues.

Final Thoughts

The decision to choose between C# and Python for test automation is very personal for me. I faced this choice directly when I started working at PrecisionLender. Even though I deeply love Python, we chose to use C#. It was the right choice: we were a Microsoft shop with no test solution (yet) and no Python stack in place. My team and I have no regrets.

There is nothing with test automation that either language can’t do. Both are solid choices. The best choice for a team depends more upon the team’s situation than differences between these languages.

How Do We Write Good Gherkin as Part of BDD? (Webinar + Q&A)

On July 23, 2019, I gave a webinar entitled, “How Do We Write Good Gherkin as Part of BDD?” in collaboration with Paul Merrill and his company, Beaufort Fairmont. This webinar was the follow-up to a previous webinar, What Is BDD, and How Do We Practice It? It was an honor to partner with Paul again to go further into BDD practices. (If you want to learn more about BDD, check out Beaufort Fairmont’s two-day BDD training offering, as well as their blog and other webinars.)

To see my webinar recording, register here. Definitely watch the previous webinar first.

Just like last time, attendees asks several great questions that we simply could not answer live. I categorized all questions we received and answered them below. Please note that some questions might be rephrased or combined with others.

Questions about BDD

What is BDD?

Behavior-Driven Development! Read more here.

In a typical Agile development process, who should write feature files?

The Three Amigos! Product owners, developers, and testers should all come together to figure out behaviors. I recommend doing Example Mapping to formulate before writing Gherkin scenarios. The green example cards should be turned into feature files. The specific person who writes the feature files is up to team preference. It could be a collaborative effort, or it could be divided-and-conquered. Any one of the Three Amigos can do it.

How can we apply BDD to SAFe (Scaled Agile Framework) teams?

BDD practices like Three Amigos meetings, Example Mapping, Behavior Specification with Gherkin, and Behavior Implementation can become part of any process. All of these practices happen at the level of the development teams. Teams could even share Gherkin steps and test frameworks wherever sharing makes sense. Check out BDD 101: Behavior-Driven Agile.

What advice can you give to teams that use BDD tests frameworks solely as an automation tool and not part of a greater BDD process?

Do the best with what you’ve got. Try to show how other BDD practices can pragmatically improve your team’s development and delivery work. See also:

Questions about Gherkin Syntax

What is the difference between a scenario and a scenario outline?

A scenario is a procedure of Given-When-Then steps that covers one example for one behavior. If there are any parameters for steps, then a scenario has exactly one combination of possible inputs. A scenario outline is a Given-When-Then procedure that can have multiple examples of one behavior provided as a table of input combos. Each input row will run the same steps once, just with different parameter inputs. See BDD 101: Gherkin by Example to see examples.

What do you think about long tables in scenarios?

Long tables in Gherkin usually look terrible. They’re hard to read, and they create a wall of text. They may also include unnecessary variations. Stick to the Unique Example rule.

Are Given steps mandatory, or can scenarios start directly with When steps?

None of the step types are mandatory. It is valid to write a scenario that skips the Given and has only When-Then steps. It is also valid to write scenarios that are Given-Then or Given-When. In fact, it is syntactically valid to put steps in any order. However, I strongly recommend keeping Given-When-Then step order to properly frame behaviors.

Are quotation marks required for parameters?

No, quotation marks are not required for parameters, but they are a popular convention, and one that I recommend. Quotes make parameters easy to identify.

Questions about Gherkin Scenarios

How do we make sure each scenario focuses on an individual, independent behavior?

Do Example Mapping first as a team. Write scenarios together, or review them with others. Ask, “What makes this behavior unique?” Make sure to use strict Given-When-Then step order when defining the behavior. Rethink the scenario if it is more than 10 lines long. Look out for unnecessary complication.

What does it mean for a scenario to be “chronological”?

Scenario steps should be written as if they were on a timeline. Each step will be executed after the previous one, so its description must start where the previous one ended. Remember, steps will be automated as if they were scripts.

How do we write a very low-level scenario without having a wall of text?

Don’t write low-level scenarios! Gherkin is best for feature testing, not unit testing. Steps should focus on intention and business value. Instead of writing “type, type, click, wait,” write “log into the app.” If you absolutely must write a low-level scenario, remember that the same principles apply. Be intuitively descriptive. Focus on individual behaviors. Keep scenarios concise.

If all scenarios in a feature file have only one user, is it okay to use first-person perspective instead of third-person?

In my opinion, no. I favor third-person perspective universally. Trying to limit usage to one feature file won’t work because any step can be used by any feature file within a test project. The entire solution must be either first-person or third-person. There’s no middle ground.

Can we write Gherkin scenarios with personas?

Yes! Personas can make scenarios more meaningful and understandable. Make sure to define the personas well – they could be described under the Feature section or in a separate text file.

How do we write Gherkin scenarios that need to validate lots of information on a page?

Pick the most important pieces of information to check. You could write separate Then steps for each assertion, or you could push small-but-similar validations down to the automation level to avoid Gherkin clutter.

How do we write Gherkin scenarios for validating Web UI fields?

Typically, I treat each field validation as an independent behavior, and thus I write separate scenarios to check each field. If the scenario steps simply enter a textual value and verify a specific message, then I might make a Scenario Outline with example rows for each equivalence class of inputs.

How do we write Gherkin scenarios that have multiple inputs and setup steps? (Example: an API with ten parameters)

Gherkin allows multiple steps of the same type to be written using “And” and “But” keywords. It’s not a problem to have “Given-And-And” or “When-And-And”. If you discover that different scenarios repeat the same setup steps, then I recommend either moving those common steps to a Background section or writing a new step that covers multiple calls (for conciseness).

One example from the webinar showed searching for shoes and adding them to a shopping cart as part of one scenario. Aren’t those two different behaviors?

Here’s the scenario in question:

Scenario: Add shoes to the shopping cart
  Given the ShoeStore home page is displayed
  When the shopper searches for “red pumps”
  And the shopper adds the first result to the cart
  Then the cart has one pair of “red pumps”

We could have split this scenario into two. I just chose to define the behavior this way. This scenario is a bit more end-to-end because it covers a basic but typical workflow. It may also have leveraged existing steps, which expedites automation development. Overall, the scenario is still concise, chronological, and intuitively understandable. Remember, there is an art as well as a science to writing good Gherkin.

Questions about Automation

Do scenarios need to be independent of each other?

Yes, unequivocally. Tests that are not independent could interfere with each other and cause unexpected failures. Independence also reinforces singular behavioral focus.

How do we start a scenario “in media res” without it depending on other tests?

At the Gherkin level, write Given steps that define a new starting point for the behavior. For example, many teams develop Web apps. It’s common to think that the starting point for all tests is login. However, the starting point can be a few pages after login.

At the automation level, it may be useful to implement the Given steps by calling other steps. For example, if a Given step should start at a user’s profile page, then perhaps it could internally call the login step and the click-the-profile-link step. Test steps may repetitively do the same operations for different tests, but test case independence will be preserved, and unique failures will be reported.

What is the best way to handle preconditions like logging into a Web app?

The simplest way to handle preconditions is to write Given steps. If those Given steps are shared by all scenarios in a feature file, then move them to a Background section. Automation hooks can also perform common setup and cleanup actions, depending upon the test framework. Personally, I prefer to use hooks to do automatic login rather than repeat Given steps for many scenarios.

Is it better to set up and tear down new test objects for each test case, or is it better to use shared, pre-created objects?

That depends upon the object. Most objects like WebDrivers and page objects should have scenario scope, meaning they are created fresh for each scenario and then torn down when the scenario ends. The only time an object should be shared across scenarios is if it is immutable or very expensive to create. For example, configuration data could be read in once before all tests and then injected immutably into each scenario. The safe position is always to use fresh objects; justify why sharing is needed before trying it.

I want to use Serenity for BDD and testing. Should I use Cucumber-like Gherkin feature files, or should I use Serenity’s native methods?

That’s up to you and your team. Personally, I would still use Gherkin feature files with Serenity. I like to separate my test case from my test code. Everyone can read Gherkin feature files, but not everyone can read Java or JavaScript test methods.

If a company already has a large BDD test solution that is poorly implemented, would it be better to keep it going or try to change it?

This question can be applied to all software projects, not just BDD test solutions. The answer is situational. Personally, I favor doing things right, even if it means refactoring. Please read Our Test Automation Has Problems. Should We Start Over? for a thorough answer.

Final Questions

Why do you call yourself “Pandy” and the “Automation Panda”?

Pandas are awesome. Everybody loves them. And nobody forgets my moniker. The nickname “Pandy” came about in the Python community to distinguish me from other folks named “Andy.”

Where can I get team training in BDD?

Beaufort Fairmont provides a one- or two-day course in BDD and writing Gherkin. Sign up for more information here.