development

Python Program Main Function

This article will show you the best way to handle “main” functions in Python.

Python is like a scripting language in that all lines in a Python “module” (a .py file) are executed whenever the file is run. Modules don’t need a “main” function. Consider a module named stuff.py with the following code:

def print_stuff():
  print("stuff happened!")

print_stuff()

This is the output when it is run:

$ python stuff.py
stuff happened!

The print_stuff function was called as a regular line of code, not in any function. When the module ran, this line was executed.

This will cause a problem, though, if stuff is imported by another module. Consider a second module named more_stuff.py:

import stuff

stuff.print_stuff()
print("more stuff!")

At first glance, we may expect to see two lines printed. However, running more_stuff actually prints three lines:

$ python more_stuff.py
stuff happened!
stuff happened!
more stuff!

Why did “stuff happened!” get printed twice? Well, when “import stuff” was called, the stuff module was loaded. Whenever a module is loaded, all of its code is executed. The print_stuff function was called at line 4 in the stuff module. Then, it was called again at line 3 in the more_stuff module.

So, how can we avoid this problem? Simple: check the module’s __name__. The __name__ variable (pronounced “dunder name”) is dynamically set to the module’s name. If the module is the main entry point, then __name__ will be set to “__main__”. Otherwise, if the module is simply imported, then it will be set to the module’s filename without the “.py” extension.

Let’s rewrite our modules. Here’s stuff:

def print_stuff():
  print("stuff happened!")

if __name__ == '__main__':
  print_stuff()

And here’s more_stuff:

import stuff

if __name__ == '__main__':
  stuff.print_stuff()
  print("more stuff!")

If we rerun more_stuff, then the line “stuff happened!” will print only once:

$ python more_stuff.py
stuff happened!
more stuff!

As a best programming practice, Python modules should not contain any directly-called lines. They should contain only functions, classes, and variable initializations. Anything to be executed as a “main” body should be done after a check for “if __name__ == ‘__main__'”. That way, no rogue calls are made when modules are imported by other modules. The conditional check for __name__ also makes the “main” body clear to the reader.

Some people still like to have a “main” function. That’s cool. Just do it like this:

import stuff

def main():
  stuff.print_stuff()
  print("more stuff!")

if __name__ == '__main__':
  main()

For more information, read this Stack Overflow article:
What does if __name__ == “__main__”: do?

Surviving Without Python


Python is such a popular language for good reason: Its principles are strong. However, if Python is “the second-best language for everything”… that means the first-best is often chosen instead. Oh no! How can Pythonistas survive a project or workplace without our favorite language?

Personally, even though I love Python, I don’t use it daily at my full time job. Nevertheless, Pythonic thinking guides my whole approach to software. I will talk about how the things that make Python great can be applied to non-Python places in three primary ways:

  1. Principles from the Zen of Python
  2. Projects that partially use Python
  3. People who build strong, healthy community

Check out my talk, Surviving Without Python, from PyOhio 2019! It was one of the most meaningful talks I’ve ever given.

Sprint Planning Sucks. Can It Be Fixed?

Warning: This article contains strong opinions that might not be suitable for all audiences. Reader discretion is advised.

It’s Monday morning. After an all-too-short weekend and rush hour traffic, you finally arrive at the office. You throw your bag down at your desk, run to the break room, and queue up for coffee. As the next pot is brewing, you check your phone. It’s 8:44am… now 8:45am, and DING! A meeting reminder appears:

Sprint Planning – 9am to 3pm.

.

What’s your visceral reaction?

.

I can’t tell you mine, because I won’t put profanity on my blog.

Real Talk

In the capital-A Agile Scrum process, sprint planning is the kick-off meeting for the next iteration. The whole team comes together to talk about features, size work items with points, and commit to deliverables for the next “sprint” (typically 2 weeks long). Idealistically, team members collaborate freely as they learn about product needs and give valued input.

Let’s have some real talk, though: sprint planning sucks. Maybe that’s a harsh word, but, if you’re reading this article, then it caught your attention. Personally, my sprint planning experiences have been lousy. Why? Am I just bellyaching, or are there some serious underlying problems?

Sprint planning is a huge time commitment. 9am to 3pm is not an exaggeration. Sprint planning meetings are typically half-day to full-day affairs. Most people can’t stay focused on one thing for that long. Plus, when a sprint is only two weeks long, one hour is a big chunk of time, let alone 3, or 6, or a whole day. The longer the meeting, the higher the opportunity cost, and the deeper the boredom.

Collaboration is a farce. Planning meetings typically devolve into one “leader” (like a scrum master, product owner, or manager) pulling teeth to get info for a pre-determined list of stories. Only two people, the leader and the story-owner, end up talking, while everyone else just stares at their laptops until it’s their turn. Discussions typically don’t follow any routine beyond, “What’s the acceptance criteria?” and, “Does this look right?” with an interloper occasionally chiming in. Each team member typically gets only a few minutes of value out of an hours-long ordeal. That’s an inefficient use of everyone’s time.

No real planning actually happens. These meetings ought to be called “guessing” meetings, instead. Story point sizes are literally made up. Do they measure time or complexity? No, they really just measure groupthink. Teams even play a game called planning poker that subliminally encourages bluffing. Then, point totals are used to guess how much work can be done during the sprint. When the guess turns out to be wrong at the end of the sprint (and it always does), the team berates itself in retro for letting points slip. Every. Time.

Does It Spark Joy?

I’ve long wondered to myself if sprint planning is a good concept just implemented poorly, or if it’s conceptually flawed at its root. I’m pretty sure it’s just flawed. The meetings don’t facilitate efficient collaboration relative to their time commitments, and estimates are based on poor models. Retros can’t fix that. And gut reactions don’t lie.

So, what should we do? Should we Konmari our planning meetings to see if they spark joy? Should we get rid of our ceremonies and start over? Is this an indictment of the whole Agile Scrum process? But then, how will we know what to do, and when things can get done?

I think we can evolve our Agile process with more effective practices than sprint planning. And I don’t think that evolution would be terribly drastic.

Behavior-Driven Planning

What we really want out of a planning meeting is planning, not pulling and not predicting. Planning is the time to figure out what will be done and how it will be done. The size of the work should be based on the size of the blueprint. Enter Example Mapping.

Example Mapping is a Behavior-Driven Development practice for clarifying and confirming stories. The process is straightforward:

  1. Write the story on a yellow card.
  2. Write each rule that the story must satisfy on a blue card.
  3. Illustrate each rule with examples written on green cards.
  4. Got stuck on a question? Write it on a red card and move on.

One story should take about 20-30 minutes to map. The whole team can participate, or the team can split up into small groups to divide-and-conquer. Rules become acceptance criteria, examples become test cases, and questions become spikes.

Here’s a good walkthrough of Example Mapping.

What about story size? That’s easy – count the cards. How many cards does a story have? That’s a rough size for the work to be done based on the blueprint, not bluffing. More cards = more complexity. It’s objective. No games. Frankly, it can’t be any worse that made-up point values.

This is real planning: a blueprint with a course of action.

So, rather than doing traditional sprint planning meetings, try doing Example Mapping sessions. Actually plan the stories, and use card counts for point sizes. Decisions about priority and commitments can happen between rounds of story mapping, too. The Scrum process can otherwise remain the same.

If you want to evolve further, you could eliminate the time boxes of sprints in favor of Kanban. Two-week work item boundaries can arbitrarily fall in the middle of progress, which is not only disruptive to workflow but can also encourage bad responses (like cramming to get things done or shaming for not being complete.) Kanban treats work items as a continuous flow of prioritized work fed to a team in bite-sized pieces. When a new story comes up, it can have its own Example Mapping “planning” meeting. Now, Kanban is not for everyone, but it is popular among post-Agile practitioners. What’s important is to find what works for your team.

Rant Over

I know I expressed strong, controversial opinions in this article. And I also recognize that I’m arguing against bad examples of Agile Scrum. Nevertheless, I believe my points are fair: planning itself is not a waste of time, but the way many teams plan their sprints uses time inefficiently and sets poor expectations. There are better ways to do planning – let’s give them a try!

Frameworks: All-in-One or Piece-by-Piece?

Software frameworks are great because they apply the principle of Separation of Concerns. A framework’s tools and code handle a specific need in a standard way for developers to write other code more easily. For example:

  • Web frameworks support receiving requests and sending responses.
  • Test frameworks include test case structure, runners, and reporting mechanisms.
  • Logging frameworks control how messages are gathered and stored.
  • Dependency injection frameworks create and manage object instances.

Recently, a question hit me: How far should a framework go to separate concerns? Should a framework try to do everything all-in-one, or should it behave more like a library that focuses on doing one thing well?

Let’s look at Python Web frameworks as an example. Django, the “Web framework for perfectionists with deadlines,” provides everything a developer could want out of the box. Flask, on the other hand, is a “microframework” that prides itself on minimalism: any extras must be handled by extensions or other packages. The differences between the two become clear when comparing some of their features:

Feature Django Flask
HTTP Requests and Routing Included Werkzeug (bundled)
Templates Included Jinja2 (bundled)
Forms Included None (Flask-WTF)
Object-Relational Mapping (ORM) Included None (SQLAlchemy)
Security Included None (Flask-Security)
Localization Included None (Flask-Babel)
Admin Interface Included None

Clearly, Django is all-in-one, while Flask is piece-by-piece. To make a serious Flask app, developers must pull in many extra pieces. There are many other frameworks with similar competitions:

  • JavaScript testing: Jasmine vs. Mocha
  • JavaScript development: Angular vs. React
  • Java BDD testing: Serenity vs. Cucumber-JVM

I think each approach has its merits. All-in-one frameworks are more convenient to use, especially for beginners. For developers who are new to a domain or just need to get something up fast, all-in-ones are the better choice. They come with all units already integrated together, and they often have good documentation. However, developing an all-in-one framework takes much more work because it covers multiple concerns. Developers may also feel shoehorned into the framework’s way of doing things. All-in-ones typically dictate what they believe to be the “best” solution.

Piece-by-piece frameworks require more expertise but offer greater flexibility. Developers can pick and choose the pieces they need, and they can change the packages used by the solution more easily. Found a better ORM? Not a problem. Need to localize the site in Chinese? Add it! Solutions can avoid excess weight and stay nimble for the future. The big challenge is successful integration. Furthermore, a library or framework for a singular concern tends to solve the concern in better ways simply because project contributors give it exclusive focus. The more I learn about a space, the more I lean towards a piece-by-piece approach.

As always, pick frameworks based on the needs at hand. For example, I like to use Django to make websites for my wife’s small businesses because the admin interface is just so convenient for her, even though I could get away with Flask. However, I’ll probably pick Mocha (piece-by-piece) over Jasmine (all-in-one) whenever I return to JavaScript testing.

Speaking Pythonese

The Python community, like many groups, has its own language – and I don’t mean just Python. There are many words and phrases thrown around that may confuse people new to Python. I originally shared some terms in my article, Which Version of Python Should I Use?, but below are some more of those colloquialisms for quick reference:

Word or Phrase Meaning
Anaconda
  • Open-source implementation of Python (and R)
  • Meant for data scientists
  • Uses the conda package manager
Benevolent Dictator for Life (BDFL)
  • Guido van Rossum
  • The inventor of Python
  • Resigned in July 2018 but remains BDFL Emeritus
The CheeseShop
  • a fun code name for the Python Package Index
Class
  • A programming definition for creating objects
  • Combines attributes (variables) and behaviors (methods)
  • Useful for reusing code
Conda
  • Package manager for Python and other languages
  • Part of the Anaconda project
Core Developer
  • A developer who has commit privilege to the CPython codebase
  • Very few Pythonistas are core developers
CPython
  • The default and most widely used implementation of the Python language
  • Implemented in C
Django
  • A batteries-included Python Web framework for perfectionists with deadlines
  • Offers many features out of the box
  • The most popular Python framework in 2017
  • Size: Flask < Pyramid < Django
Flask
  • A microframework for Python Web development
  • Uses Werkzeug and Jinja2
  • Super-minimalist
  • Size: Flask < Pyramid < Django
Function
  • A definition for a callable subroutine
  • May take inputs
  • May return outputs
  • Great for code reuse
  • Functions are first-order values
The Hitchhiker’s Guide to Python
  • A popular online guide to Python
  • Opinionated
  • Shares many good best practices
Jinja2
  • A Python template engine
  • Inspired by Django’s templates
Jupyter
  • A project for interactive Python development
  • “Jupyter Notebooks” allow programmers to dynamically rewrite and rerun Python code, and then share code easily with others
  • Popular with data scientists
Module
  • A Python source code file containing definitions and statements
  • Every .py file is a module
  • May be imported by other modules to reuse code
NumPy
  • A popular Python package for scientific computing
Pandas
  • A popular Python package for data analysis
pip
  • The PyPA-recommended tool for installing Python packages
  • Command: “pip install “
  • Recursive acronym for “pip installs packages”
PyBites
  • A community of Pythoneers who improve their skills through code challenges
PyCharm
  • A popular Python IDE developed by JetBrains
  • Offers great development features
  • Has a free Community Edition and a paid Professional Edition
PyCon
  • The annual Python conference held in North America
  • GO – it will change your life!
  • Several other conferences are held worldwide
PyPy
  • An alternative Python implementation
  • Known for speed, memory usage, and compatibility
  • Good alternative to CPython for high performance workloads
Pyramid
  • A Python Web framework
  • Start small, finish big, stay finished
  • Provides many parts but not everything (such as ORM)
  • Size: Flask < Pyramid < Django
pytest
  • A lightweight-yet-powerful Python test framework (and arguably the best)
Python 2
  • The old version of Python
  • Will reach end-of-life in 2020
  • Final version will be 2.7.x
  • Please upgrade to version 3
Python 3
  • The current version of Python
  • Most packages now support Python 3
  • Has incompatibilities with Python 2
  • Please don’t use Python 2
Python Enhancement Proposal (PEP)
  • Official proposals for enhancing the Python language
Python Package Index (PyPI)
The Python Software Foundation (PSF)
  • Non-profit organization
  • Keeps Python going strong
  • Support them!
Pythoneer
  • A programmer who uses Python to solve problems
  • Styled off the word “engineer”
Pythonic
  • Describes idiomatic code for Python
  • Closely related to conciseness, readability, and elegance
  • Highly recommended
  • Follow style guidelines to learn how to be Pythonic
Pythonista
  • Someone who loves the Python language
  • Often an advanced Python programmer
Sphinx
  • A popular Python tool to generate documentation
virtualenv
  • Tool to create isolated Python environments
  • Enables programmers to use different versions of Python and packages for different projects
  • Also see venv and pipenv
Web Server Gateway Interface (WSGI)
  • A specification for how Web servers forward requests to Web applications and frameworks
  • A core piece of Python Web development
  • See PEP-333 and PEP-3333
The Zen of Python
  • The list of guiding principles for Python’s design
  • Run “import this” to see them
  • See PEP-20

Ignoring Files with Git

Git is one of the most popular version control systems (VCS) available, especially thanks to hosting vendors like GitHub. It keeps code safe and shareable. Sometimes, however, certain files should not be shared, like local settings or temporary configs. Git provides a few ways to make sure those files are ignored.

.gitignore

The easiest and most common way to ignore files is to use a gitignore file. Simply create a file named .gitignore in the repository’s root directory. Then, add names and patterns for any files and directories that should not be added to the repository. Use the asterisk (“*”) as a wildcard. For example, “*.class” will ignore all files that have the “.class” extension. Remember to add the .gitignore file to the repository so that it can be shared. As a bonus, Git hosting vendors like GitHub usually provide standard .gitignore templates for popular languages.

Any files covered by the .gitignore file will not be added to the repository. This approach is ideal for local IDE settings like .idea or .vscode, compiler output files like *.class or *.pyc, and test reports. For example, here’s GitHub’s .gitignore template for Java:

.git/info/exclude

As a best practice, .gitignore should be committed to the repository, which means all team members will share the same set of ignored files. However, some files should be ignored locally and not globally. Those files could be added to .gitignore, but large .gitignore files become cryptic and more likely to break other people’s setup. Thankfully, git provides a local-only solution: the .git/info/exclude file (under the repository’s hidden .git directory). Simply open it with a text editor and add new entries using the same file pattern format as .gitignore.

# Append a new file to ignore locally
echo "my_private_file" >> .git/info/exclude

skip-worktree

.gitignore file prevents a file from being added to a repository, but what about preventing changes from being committed to an existing file? For example, developers may want to safely override settings in a shared config file for local testing. That’s where skip-worktree comes in: it allows a developer to “skip” any local changes made to a given file. Changes will not appear under “git status” and thus will not be committed.

Use the following commands:

# Ignore local changes to an existing file
git update-index --skip-worktree path/to/file

# Stop ignoring local changes
git update-index --no-skip-worktree path/to/file

Warning: The skip-worktree setting applies only to the local repository. It is not applied globally! Each developer will need to run the skip-worktree command in their local repository.

assume-unchanged

Another option for ignoring files is assume-unchanged. Like skip-worktree, it makes Git ignore changes to files. However, whereas skip-worktree assumes that the user intends to change the file, assume-unchanged assumes that the user will not change the file. The intention is different. Large projects using slow file systems may gain significant performance optimizations by marking unused directories as assume-unchanged. This option also works with other update-index options like really-refresh.

Use the following commands:

# Assume a file will be unchanged
git update-index --assume-unchanged path/to/file

# Undo that assumption
git update-index --no-assume-unchanged path/to/file

Again, this setting applies only to the local repository – it is not applied globally.

Comparison Table

Which is the best way to ignore files?

Method Description Best Use Cases Scope
.gitignore file Prevents files from being added to the repository. Local settings, compiler output, test results, etc. Global
.git/info/exclude file Prevents local files from being added to the repository. Local settings, compiler output, test results, etc. Local
skip-worktree setting Prevents local changes from being committed to an existing file. Shared files that will have local overwrites, like config files. Local
assume-unchanged setting Allows Git to skip files that won’t be changed for performance optimization. Files and folders that a developer won’t touch. Local

Resources

NuGet Quick Reference

What is NuGet?

NuGet is a package manager for Microsoft .NET. It installs packages and manages dependencies for .NET projects. It is like Maven (Java) or pip (Python). The NuGet Gallery hosts thousands of popular packages like Json.NET, NUnit, and jQuery. If you develop .NET applications (like in C#), then you probably need to use NuGet.

Installing Packages

The easiest way to use NuGet is through Visual Studio, which includes NuGet features by default. Packages are managed per project. Right-click on a project in Solution Explorer and select “Manage NuGet Packages…” to open the project’s package manager page.

  • The Browse tab lets you search and install new packages.
  • The Installed tab shows which packages are installed and can uninstall them.
  • The Updates tab lets you update packages to their latest versions.

Nuget Package Manager Page

The NuGet Package Manager page for a project in Visual Studio

When packages are installed and updated, NuGet also pulls any dependencies they require. Visual Studio also creates a packages.config file for all dependencies. Then, just build and run!

NuGet Configuration

NuGet can be configured using a NuGet.Config file. This file can be placed under a project directory, a solution directory, or a system-wide location. One of the most common settings is the package sources: NuGet uses the public nuget.org repository by default, but others (like private company repos) can also be added. Check the nuget.config reference online for docs on all options. (Package sources can also be configured through Visual Studio under Tools > NuGet Package Manager > Package Manager Settings.)

NuGet Package Manager Console

Sometimes, it’s helpful to control NuGet directly through the Package Manager Console. From the menu bar: Tools > NuGet Package Manager > Package Manager Console. For example, when packages get messed up, I’ll run “Update-Package -Reinstall” to reinstall everything. (Right-clicking the solution and selecting “Restore NuGet Packages” never seems to work for me.) Check the help command or the official guide for more info.

Nuget Package Manager Console

The NuGet Package Manager Console in Visual Studio

NuGet CLI

The NuGet CLI nuget.exe provides the full extent of NuGet features, including the ability to make packages. It is more powerful than the Package Manager Console. It must be installed independently – it does not come with Visual Studio. Check the NuGet CLI reference online for full details. The .NET Core CLI dotnet.exe can also be used for managing packages. See the feature comparison for the differences.

nuget CLI

The NuGet CLI

Creating a NuGet Package

A NuGet package is basically a ZIP file with a .nupkg extension. It typically contains an assembly DLL and maybe other related files. Creating a NuGet package is pretty easy:

  1. Install the NuGet CLI.
  2. Create a .nuspec file for the project.
  3. Add appropriate settings to the .nuspec file.
  4. Run the “nuget pack” command to create the .nupkg file.
  5. Publish the .nupkg file to the desired destination.

The .nuspec file can be created by running the “nuget spec” command in the project’s directory. The generated <project-name>.nuspec file will contain replacement tokens that will be substituted with values from the project’s AssemblyInfo when the package is built. Make sure to set AssemblyInfo values appropriately for the substitution. The version is especially important, and the automatic version format may be useful for guaranteeing uniqueness. Be sure to add any packages upon which the project depends as dependencies, too. (The .nuspec file can also be created manually.) Refer to the .nuspec reference for full details.

The standard package creation command is “nuget pack <project-name>.nuspec”. However, if the .nuspec file contains replacement tokens, then use “nuget pack <project-name>.csproj” instead. Once the package is created, it can be published publicly to nuget.org or to a private NuGet feed.

Below is an example .nuspec file with replacement tokens:

Resources