X is unhinged. Find me on Bluesky.

X (formerly Twitter) is unhinged.

I received a DM through X today from someone claiming to be an Amazon seller asking me to review a set of “adult products” in exchange for “prepaid compensation.”

I joined Twitter back in 2017 to join the “Tech Twitter” community. It was awesome! I connected with so many colleagues, kept up with all the latest developments, and learned so much.

Ever since Elon took over, X has degraded. My feed is polluted with the man himself and all the politics that come with him. Many of my friends and followers have left. I constantly receive spammy messages from crypto bros and catfishers. Today’s DM is a new low. I can’t justify using X for professional purposes anymore.

I’m moving to Bluesky. I’ve already been there for a while. Join me and the rest of Old Tech Twitter. 🦋

Here’s my profile: https://bsky.app/profile/automationpanda.bsky.social

I Want To Be An Engineer When I Grow Up!

When I was a little boy, I wanted to be an engineer when I grew up.

That’s right, I wanted to be a locomotive engineer! Like all little boys, I absolutely loved trains. I watched all the episodes of Thomas the Tank Engine. I wore a striped blue-and-white railroad cap everywhere. And I wanted to grow up to be the guy in charge of the trains.

More than once, my parents took me to see the Chattanooga Choo Choo in Chattanooga, Tennessee. To me, the vibrantly-painted steam engine was just the coolest thing ever. I’d climb into the engine bay (which was allowed) and imagine how much fun it would be to drive it. I don’t remember much of anything from that young age, but I vividly remember pictures in my mind of the Choo Choo.

I actually did grow up to become an “engineer” – just not the kind of engineer my four-year-old self would have expected. This week, my software engineering journey brought me back to Chattanooga on a customer visit. My hotel was located right next to the old train station housing the famous Choo Choo, so, of course, I paid it a visit. It was just as I had remembered it: charming, nostalgic, and iconic. Oh, how I yearn to build things that become as cherished and enduring as the Chattanooga Choo Choo.

Our dreams may not always manifest as we expect, but I am grateful for all the opportunities I have been given and for all the incredible experiences they have brought. Even though my current job title is “Product Management,” I still identify as an engineer at heart: building solutions to tough problems. Never let go of the things that inspire you.

Bulldog-Driven Development

Today is a special day in the Knight family. On November 5, 2021, my wife and I drove down to Waxhaw, North Carolina to adopt the cutest French Bulldog puppy in the state. We named her “Suki” – a Japanese name meaning “beloved.”

Suki was the first dog my wife and I had ever owned. She was only 7.5 weeks old when we picked her up, and she weighed only 3 pounds! I could scoop her up with one hand. When she first arrived home, she was so nervous that she tried to bury her face in her new doggy bed to hide. Now, three years later, Suki is a BEAUTIFUL 20-pound beefcake who has crisscrossed the country with us. We love her dearly. She is a gift from God.

I often joke about #BDD being “Bulldog-Driven” rather than “Behavior-Driven.” Remember, though, we are all driven by something. In my software work, I absolutely take a behavior-driven mindset, but I’m also driven by my Family, my Faith, and my Frenchie. Never lose sight of what drives you.

Photo: Suki with her portrait as “AstroSuki” – my speaker gift from QA or the Highway 2024!

Running tests in a Java Maven project

Java continues to be one of the most popular languages for test automation, and Maven continues to be its most popular build tool. Adding tests in the right place to a Java project with Maven can be a bit tricky, however. Let’s briefly learn how to do it. These steps work for both JUnit and TestNG.

Test code location

Maven project follow the Standard Directory Layout. The main code should go under src/main, while all test code should go under src/test:

src/test/java should hold Java test classes
src/test/resources should hold resource files that tests use

Your project directory should look something like this:

src/
+-- main/
|   +-- java/
|   \-- resources/
|-- test/
|   +-- java/
|   \-- resources/
\-- pom.xml

Don’t put test code in the main source folder. You don’t want to include it with the final build artifact. The project might have other files as well, like a README.

Unit tests

The Maven Surefire Plugin runs unit tests during Maven’s test phase. To run unit tests:

Add maven-surefire-plugin to the plugins section of your pom.xml
Name your unit tests *Tests.java
Put them under src/test/java
Mirror the package structure from the main code always
Run tests with mvn test

There are a bunch of options for configuring the Maven Surefire Plugin. If you don’t want to configure anything special, you actually don’t need to add the plugin to the POM file. Nevertheless, it’s good practice to add it to the POM file anyway. Here’s what that would look like:

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.2.5</version>
            </plugin>
        </plugins>
    </build>

Integration tests

The Maven Failsafe Plugin runs integration tests during Maven’s integration-test phase. Integration tests are distinct from unit tests due to their external dependencies and should be treated differently. To run integration tests:

Add maven-failsafe-plugin to the plugins section of your pom.xml
Name your unit tests *IT.java
Put them under src/test/java
Mirror the package structure from the main code as appropriate
Run tests with mvn verify

Maven actually has multiple integration test phases: pre-integration-test, integration-test, and post-integration-test to handle appropriate setup, testing, and cleanup. However, none of these phases will cause the build to fail. Instead, use the verify goal to make the build fail when ITs fail.

Like the Maven Surefire Plugin, the Maven Failsafe Plugin has a bunch of options. However, to run integration tests, you must configure the Failsafe plugin in the POM file. Here’s what it looks like:

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>3.2.5</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

Maven phases

Phases in the Maven Build Lifecycle are cumulative:

Running mvn test includes the compile phase
Running mvn verify includes the compile and test phases

It is also a good practice to run mvn clean before other phases to delete the build output (target/) directory. That way, old class files and test reports are removed before generating new ones. You may also include clean with commands to run tests, like this: mvn clean test or mvn clean verify.

Customizations

You can customize how tests run. For example, you can create a separate directory for integration tests (like src/it instead of src/test). However, I recommend avoiding customizations like this. They require complicated settings in the POM file that are difficult to get right and confusing to maintain. Others who join the project later will expect Maven standards.

Test coverage and trusting your instincts

Picture this: It’s 2010, and I’m fresh out of college, eager to dive into the software industry. Little did I know, a simple interview question would challenge not only my knowledge about testing but also my instincts.

Job openings were hard to find in the wake of the Great Recession. Thankfully, I landed a few interviews with IBM, where I completed a series of internships over the summers of 2007-2009. I was willing to take any kind of job – as long as it involved coding. One of those interviews was for an entry-level position on a data warehouse team in Boston. I honestly don’t remember much from this interview, but there was one question I will never forget:

How do you know when you’ve done enough testing?

Now, remember, back in 2010, I wasn’t the Automation Panda yet. Nevertheless, since I had experience with testing during my internships, I felt prepared to give a reasonable answer. If I recall correctly, I said something about covering all paths through the code and being mindful to consider edge cases that could be overlooked. (My answer today would likely frame “completeness” around acceptable risk, but that’s not the point of the story.) I’ll never forget what the interviewer said in reply:

Well, that’s not the answer I was looking for.

Oh? What’s the “right” answer?

If you write roughly the same number of lines of test code as you write for product code, then you have enough coverage.

That answer stunned me. Despite my limited real-world experience as a recent college graduate, I knew that answer was blatantly wrong. During my internships, I wrote plenty of code with plenty of tests, and I knew from experience that there was no correlation between lines of test code and actual coverage. Even a short snippet could require multiple tests to thoroughly cover all of its variations.

For example, here’s a small Python class that keeps track of a counter:

class Counter:

  def __init__(self):
    self.count = 0

  def add(self, more=1):
    self.count += more

And here’s a set of pytest tests to cover it:

import pytest

@pytest.fixture
def counter():
  return Counter()

def test_counter_init(counter):
  assert counter.count == 0

def test_counter_add_one(counter):
  counter.add()
  assert counter.count == 1

def test_counter_add_three(counter):
  counter.add(3)
  assert counter.count == 3

def test_counter_add_twice(counter):
  counter.add()
  counter.add()
  assert counter.count == 2

There are three times as many lines of test code as product code, and I could still come up with a few more test cases.

In the moment, I didn’t know how to reply to the interviewer. He sounded very confident in his answer. All I could say was, “I don’t think I agree with that.” I didn’t have any examples or evidence to share; I just had my gut feeling.

I sensed my interviewer’s disappointment with my response. Who was I, a lowly intern, to challenge a senior engineer? Needless to say, I did not receive a job offer. I ended up taking a different job with IBM in Raleigh-Durham instead.

Nevertheless, this exchange taught me a very valuable lesson: trust your instincts. While I didn’t land the job that day, the encounter left an indelible mark on my approach to problem-solving. It instilled in me the confidence to question assumptions and trust my instincts, qualities that would shape my career trajectory in unforeseen ways. Never dismiss your instincts because you are less senior than others. You just might be right!

Judging Developers by GitHub Contributions

The main image for this article shows all my GitHub contributions for the past year (roughly April 2023 through March 2024). Check it out. ☝️🐼

Notice anything?

My contributions pretty much stopped around August 2023.

Why?

That’s when I changed jobs.

Until July 2023, all my professional coding work went into open source repositories in GitHub. After changing jobs, all my professional coding work went into closed source repositories in Azure DevOps.

Did I suddenly stop coding at that time? No. In fact, I did more coding – and arguably more serious development work – at the second job.

Tech social media periodically explodes with posts saying how developers who don’t have walls of solid green GitHub tiles aren’t “serious” about their work. How the folks writing these posts wouldn’t hire developers who don’t have enough green tiles. How any developer worth their salt should regularly contribute to open source projects outside of their 9-5 job. These posts are sometimes sarcastic, but they are, unfortunately, all too often sincere – and you can’t always tell.

These posts are rubbish. It is foolish to judge a developer by the number of GitHub contributions they make. Sure, it can be a helpful data point when reviewing someone’s body of work, but it is merely one data point. Someone’s lack of green tiles should not justify putting them down. It should not immediately disqualify them as a candidate from a job opening. Not everyone’s work involves open source contributions to a particular hosting site. Not everyone’s life permits extra work-like work outside of work, especially for zero pay.

Open source contributions are a great way to give back to the software community as well as to build up one’s skills. It’s also a great way to show one’s work publicly. But please, don’t fall for the trolling – either for the flame wars or for the insinuations of inadequacy. There are plenty of talented individuals who don’t have a wall of solid green tiles to “prove” their skills, myself included.

Software Engineering Seniority Levels

All software engineers are the same, right? Well, not exactly. There is a strata of different seniority levels, each with its own expectations for experience, responsibilities, and pay. This article is a concise collection of my observations on seniority levels for software engineers.

The levels

Every company is different, but I’ve seen most companies coalesce around the following levels:

Level	Responsibilities	In Plain Terms
Entry	Do what you are told. Learn as much as you can.	“What’s a cookie?”
Intermediate	Complete tasks independently. Get things done. Be a good team player.	“I bake chocolate chip cookies.”
Senior	Take ownership of bigger projects. Know what you’re talking about. Help other engineers get unstuck.	“I help folks mix cookie batter.”
Staff	Provide technical direction for an organization. Collaborate cross-functionally for technology alignment. Do the needful to keep projects moving. Mentor other engineers.	“I make cookie recipes.”
Principal	Provide technical direction for a company or even an industry. Collaborate with management for business alignment. Carry significant influence backed by deep experience.	“I know everything about cookies.”
Distinguished	Make exceptional technical contributions to the company. Shape the direction of the company and its technologies.	“I invented the ice cream sandwich.”
Fellow	Create and guide a crucial technology that drives a significant portion of the company’s revenue for several years.	“I am the Cookie Monster.”

These levels usually apply for all types of engineers (frontend, backend, test, data, DevOps; whatever) up to Principal. Again, specific details vary by company.

Population at each level

The breaking point tends to be between Senior and Staff. Many engineers make it to Senior. A few make it to Staff. Fewer make it to Principal. Distinguished and Fellow are very, very rare. Some companies don’t even have Distinguished or Fellow.

Earning promotions

There are two main ways to earn a promotion:

Start fulfilling responsibilities for the next seniority level above you. Then, clearly advocate for a promotion to your manager leading up to your periodic performance reviews. Finally, hope that you get selected for a promotion.
Switch to a new job that has the title you want. Usually, this needs to be at another company.

Switching companies is typically the fastest way to climb the seniority ladder as well as to increase your pay. Many companies impose quotas for annual raises and promotions.

Salary ranges

Seniority titles are important because they impact pay. Usually, a company sets a salary range for each seniority level. When you get promoted to a new level, your salary starts at the low end of the range for that level. As you progress within the level, you will (hopefully) earn raises that put you higher within the salary range. Eventually, once you hit the top of the range, you can’t earn more raises until you earn a promotion to the next seniority level. Also, good luck finding out what those salary ranges actually are. Many companies keep them secret.

Title mismatch

Someone’s seniority prefix does not always match their actual capabilities. Sometimes, their title is lower than the level at which they perform. This happened to me once when a bigger company bought the startup where I worked and reassigned everyone’s titles to fit their own career tracks. More often, though, I’ve seen “title inflation” where an engineer is given a higher title than their capabilities reflect. Companies usually do this either to hold onto important engineers or to woo prospective new hires. In those cases, titles match salary band, rather than salary matching seniority. That’s why it’s more important to look at a person’s accomplishments rather than the prefix on their title.

Becoming a manager

It is commonplace for engineers to become managers. However, engineering and management are separate tracks requiring distinct skill sets. The difference is the focus of the work: technology (engineer) or people and business (manager). If a company has a well-defined seniority ladder for engineering, then engineers don’t need to become managers to earn a “promotion.”

Other thoughts?

This information is all anecdotal based on my general experiences. What have you seen? Let me know in the comments below!

I’m Now a Software Architect!

Today, I have very exciting news to share: I have accepted a new position as Principal Architect at Cycle Labs! This is a wonderful new job opportunity for me. Watch the video below to learn more:

So, what is Cycle?

Cycle is a way of testing software from inspiration to implementation. Cycle Labs provides an end-to-end test automation platform for the Cycle Testing Process that targets enterprise software, specifically in supply chain and warehouse management. It can test everything from web UIs to APIs, desktop apps, and even green screen terminals. The cornerstone of the platform is CycleScript, a domain-specific language for automating business processes that looks a lot like Gherkin. One thing I love about Cycle is that it truly empowers behavior-focused development and testing.

Now, what is my new job?

I will be the Principal Architect at Cycle Labs! My primary responsibility is building and designing the Cycle testing platform together with the Product & Engineering team. Already, I’m leading a new product vision that will optimize how our customers use our platform for their whole test development process. We have some really cool stuff in the pipeline, and I can’t wait to share more as we build it. It hits so many of my favorite topics: quality, automation, behaviors, languages, and clean architecture.

Why Cycle Labs?

My new architect role might be a surprise for many of you, but others are probably thinking that it’s about time! I’ve known Cycle for years, and I have a long, deep, and personal history with the good folks behind it – particularly with Josh Owen, the CEO. We’ve worked together. We’ve dreamed together. And when Josh opened this new opportunity for me, I just couldn’t say no. I am thrilled to officially become part of Cycle Labs.

When do I start my new role at Cycle Labs?

Well, actually, I’ve already started! My first day was July 31, and I hit the ground running. The main office for Cycle Labs is located in downtown Raleigh, and I’ve been going there in person about once a week.

What about my speaking and community engagements?

I still plan to be active in the software developer and tester communities, but since I won’t be a Developer Advocate anymore, I will be reducing the number of engagements I accept. I’ll probably try to limit myself to only a few important in-person conferences a year and perhaps a handful of virtual events. In the future, I want to focus more on finishing the book I’m writing, and I also have a few cool ideas for other content I can produce. So, stay tuned!

Who do I want to thank?

I have so very many people to thank through my time at Applitools and into my new role at Cycle Labs.

First, I want to thank God for all the opportunities with which He has blessed me.

My wife, Lujing, thank you for always supporting me in my career.

Alex and Sam, thanks for being there when times were tough.

Medic Matt, thanks for always checking in on me and letting me come visit.

Mason Egger, thanks for giving me excellent guidance on how to run a good Developer Relations practice, and also for inviting me to give my first Python conference keynote at PyTexas 2023.

Filip Hric, thanks for being such a great content collaborator and for always being just one Zoom call away.

Sarah Watkins, thanks for the many good times shared over bubble tea, and for joining me at STAREAST and STARWEST to co-teach our BDD tutorial.

Josh Owen, thanks for hiring me to join Cycle Labs! It took a while, but it finally happened.

Everyone on the team at Cycle Labs, thanks so much for the warm welcome, and I’m excited to build great things together.

Joe Moore and Chris Harbert, y’all did right by me. Thank you. I hold much respect for you.

Angie Jones and Moshe Milman, thanks for trusting me to lead the Developer Relations practice at Applitools.

Jenny Bramble, Jenna Charlton, Erin Crise, and Tristan Lombard, y’all share the best insights and advice with deep empathy. I’m thankful that we are all good friends.

All my friends in the community – the list goes on and on – thanks for being my friends and making the past few years wonderful.

So, what’s next?

For the second half of 2023, I want to focus on making a good start at Cycle Labs. I still have a few speaking engagements lined up as well: STARWEST, DjangoCon US, Agile Testing Days, and AutomationSTAR. I have a lot of repairs to do on my classic Volkswagens. I also want to spend more time with my family and our little French Bulldog puppy.

Things change and roles change. I’ve been a Software Engineer in Test, a Developer Advocate, and now a Software Architect. It’s an exciting journey for sure. One thing that remains the same is that I’m still the Automation Panda – and nobody can change that.

Thank you all for the love and support. Let’s be excellent in all things.

The 12 Best Bubble Tea Shops in Seattle

It’s no secret that 🐼❤️🧋! Seattle has several excellent tea establishments, and each one is special. In this post, I list my top twelve favorite bubble tea shops that I enjoy whenever I visit. This is my personal list of favorites in no particular order – it’s not a comprehensive ranking because, quite frankly, there are so many tea shops I haven’t yet tried!

Updates (2/21/2026): This list needs to be updated. It does not include HEYTEA and Molly Tea, which recently opened and are excellent. Milkvue + DIY Tea Lab near the Space Needle closed.

#1. Xing Fu Tang

Xing Fu Tang is one of the best boba brands to come out of Taiwan. They pride themselves on their handmade brown sugar boba. At their locations in Taiwan, you can actually see them knead the tapioca dough and feed it through the machine to separate it into the pearl! At their Westlake Center location on 4th and Steward, they don’t have that machine on display, but you can see them cook all the boba in a big copper wok.

Pandy’s recommendation: Get the classic brown sugar boba with milk or milk tea – it’s the drink that made them famous!

#2. Rabbit Rabbit Tea Seattle

Rabbit Rabbit Tea Seattle is another bubble tea brand from Taiwan. They set themselves apart with their selection of premium teas: two black (Assam and Earl Grey), two green (Jade Green and Mountain), two oolong (Golden Jinxuan and Iron Buddha, and two kinds of Japanese buckwheat tea. They also give you little rabbit face wrappers for your straws!

Pandy’s recommendation: Get the Japanese buckwheat tea with less sugar and honey boba. I’ve never seen another boba shop sell buckwheat tea, and it’s so good!

#3. Tiger Sugar

Tiger Sugar is another famous bubble tea chain with many locations throughout the United States. Their Seattle location is at 2nd and Pike, a stone’s throw from Pike Place Market. Although this spot is little more than a pickup counter, its location makes it a convenient stop when passing through downtown. Tiger Sugar is known for bold flavors, both in sweetness and in the tea.

Pandy’s recommendation: If you want something sweet, order a drink from the Black Sugar Milk series. If you want something refreshing, order any of the teas with the mousse on top. As always, add boba!

#4. Tea King

Tea King is low-key one of my favorite boba shops ever. While well-known tea franchises are great, there’s something special about independent tea shops making their own drinks their own way. Tea King nails it. Their drinks are consistently the most excellent bubble teas I’ve tasted anywhere. Their teas are premium. Their selection is wide: milk, fruit, slush, and dessert. Their branding is on point – I put their sticker on my laptop! Their location is squeezed between Queen Anne and and Lake Union, but the trek is worth it.

Pandy’s recommendation: The TK Milk Tea with boba is an excellent rendition of the classic bubble tea. The Hojicha Latte and the Taro Coco Sago are delightful specialties you won’t readily find elsewhere.

#5. Yifang Taiwan Fruit Tea

Yifang Taiwan Fruit Tea is probably my favorite bubble tea brand. I first discovered them in San Francisco, and every time I go to a city that has a location, I try to stop by. As the name suggests, Yifang is known for their fruit teas. In fact, I didn’t really care much for fruit teas until I tried Yifang’s pineapple green tea! They also make delicious little bubble tea shaped pancakes with boba inside. Try to visit the locations in the University District or in Bellevue.

Pandy’s recommendation: Try the pineapple fruit tea with half (or less) sugar and sago. Bonus points for the Yakult version.

#6. TP Tea

TP Tea is my favorite tea shop in the International District. As yet another Taiwanese brand, they focus on classic milk teas with high-quality ingredients. Their location in the ID is always popular, and I dig the interior design. I’ve never been disappointed by their drinks, either. Look for the drinks with multiple toppings, like boba plus QQ noodles.

Pandy’s recommendation: Get the Taiwan Classic Milk Tea or the 3Q TP Milk Tea – they have multiple toppings with a variety of textures.

#7. OH! Bear Cafe & TeaHouse

OH! Bear Cafe & TeaHouse is a cute little dessert cafe in the University District. It’s an excellent place to come for a date with your significant other – and to get diabetes from all the sugar! They have pastries, ice cream, taiyaki, coffee, and bubble tea. Their most notable drink is their White Rabbit Milk Tea, which uses flavoring from China’s famous White Rabbit candy.

Pandy’s recommendation: Try the White Rabbit Milk Tea along with any of their desserts.

#8. Mochinut

Mochinut is located in South Lake Union amongst all the Google and Amazon buildings. While Mochinut is known primarily for their mochi donuts, they also serve bubble tea in distinctive clear plastic cans. All their drinks are very sweet, so be prepared. You can also order Korean-style corndogs!

Pandy’s recommendation: Come here for the mochi donuts and get a drink to pair.

#9. Happy Lemon

Happy Lemon is another well-known brand with a location in South Lake Union. Their drinks have striking presentation, and most of them are very sweet. Happy Lemon also sells bubble waffles at this location. It’s a nice stop if you’re in the neighborhood and want a sweet treat.

Pandy’s recommendation: Come here if you want something sweet.

#10. ChiCha San Chen

ChiCha San Chen is one of the most excellent tea establishments I’ve ever visited. Although their location is in Bellevue, it is so highly regarded that folks from all over the Seattle greater area go to visit. Although ChiCha San Chen did not earn any Michelin stars as widely rumored, they did earn the highest award from ITQI in 2017, which is just as good. When you walk into their shop, you smell the tea, not the sugar. They use a special “teapresso” machine named LION for brewing their tea. They offer free samples. When they serve the drinks, they put them in carriers with twine handles. My wife and I have visited only once, but we are looking forward to going back!

Pandy’s recommendation: Get anything with the green tea. I also want to try the Ding Dong Oolong Tea because I heard that it is very good.

#11. Milkvue + DIY Tea Lab

Milkvue + DIY Tea Lab is a new place that just opened next to the Space Needle. They serve coffee, boba, mochi donuts, and ice cream. The owners are super friendly, too. The bubble tea is a collab with DIY Tea Lab, which also has locations in Fremont and Ballard. Their drink sizes are all large and visually appealing. You can also order boba here before lunchtime! Don’t sleep on their ice cream, either – they have flavors like Ube + Oreo and White Rabbit!

Pandy’s recommendation: My wife and I enjoyed the Sea Salt Ube Milk Tea with a matcha mochinut. We also took a pint of the White Rabbit ice cream home!

#12. Nana’s Green Tea

Nana’s Green Tea is essentially a matcha cafe. They serve standard Japanese lunch fair like curry plates, rice bowls, and tonkatsu, but the star of the menu is the matcha. You can order matcha in pretty much any form you can imagine: lattes, frappes, ice cream, parfaits, floats, sodas, pastries, and just straight tea. They also have lattes for hojicha and red bean. It’s awesome. It’s decadent. And even though they don’t serve boba, the little mochi toppings are close enough, right?

Pandy’s recommendation: Come here for the matcha. I like the lattes and the frappes with mochi on top.

Bonus: Miro Tea

Miro Tea is the ultimate tea house. This is not a place for boba but rather for high-quality loose leaf teas from around the world. It’s like a high-end coffee shop with expert baristas, but for tea instead of coffee. You can order tea by the pot and even as a Gongfu tea ceremony. They have plenty of seating for you to work, read, or simply space out while enjoying your tea. I enjoy hopping over to Ballard just to enjoy their fine selection.

Pandy’s recommendation: Try the samples they have to decide what kind of tea you want. Plan to spend at least an hour there instead of taking your tea away in a cup.

Modern Web Testing with Playwright

Playwright is an awesome new web testing framework, and it can help you take a modern approach to web development. In this article, let’s learn how.

Asking tough questions about testing

Let me ask you a series of questions:

Question 1: Do you like it when bugs happen in your code? Most likely not. Bugs are problems. They shouldn’t happen in the first place, and they require effort to fix. They’re a big hassle.

Question 2: Would you rather let those bugs ship to production? Absolutely not! We want to fix bugs before users ever see them. Serious bugs could cause a lot of damage to systems, businesses, and even reputations. Whenever bugs do slip into production, we want to find them and fix them ASAP.

Question 3: Do you like to create tests to catch bugs before that happens? Hmmm… this question is tougher to answer. Most folks understand that good tests can provide valuable feedback on software quality, but not everyone likes to put in the work for testing.

Why the distaste for testing?

Why doesn’t everyone like to do testing? Testing is HARD! Here are common complaints I hear:

Tests are slow – they take too long to run!
Tests are brittle – they break whenever the app changes!
Tests are flaky – they crash all the time!
Tests don’t make sense – they are complicated and unreadable!
Tests don’t make money – we could be building new features instead!
Tests require changing context – they interrupt my development workflow!

These are all valid reasons. To mitigate these pain points, software teams have historically created testing strategies around the Testing Pyramid, which separates tests by layer from top to bottom:

UI tests
API tests
Component tests
Unit tests

Tests at the bottom were considered “better” because they were closer to the code, easier to automate, and faster to execute. They were also considered to be less susceptible to flakiness and, therefore, easier to maintain. Tests at the top were considered just the opposite: big, slow, and expensive. The pyramid shape implied that teams should spent more time on tests at the base of the pyramid and less time on tests at the top.

End-to-end tests can be very valuable. Unfortunately, the Testing Pyramid labeled them as “difficult” and “bad” primarily due to poor practices and tool shortcomings. It also led teams to form testing strategies that emphasized categories of tests over the feedback they delivered.

Rethinking modern web testing goals

Testing doesn’t need to be hard, and it doesn’t need to suffer from the problems of the past. We should take a fresh, new approach in testing modern web apps.

Here are three major goals for modern web testing:

Focus on building fast feedback loops rather than certain types of tests.
Make test development as fast and painless as possible.
Choose test tooling that naturally complements dev workflows.

These goals put emphasis on results and efficiency. Testing should just be a natural part of development without any friction.

Introducing Playwright

Playwright is a modern web testing framework that can help us meet these goals.

It is an open source project from Microsoft.
It manipulates the browser via (superfast) debug protocols
It works with Chromium/Chrome/Edge, Firefox, and WebKit
It provides automatic waiting, test generation, UI mode, and more
It can test UIs and APIs together
It provides bindings for JavaScript/TypeScript, Python, Java, and C#

Playwright takes a unique approach to browser automation. First of all, it uses browser projects rather than full browser apps. For example, this means you would test Chromium instead of Google Chrome. Browser projects are smaller and don’t use as many resources as full browsers. Playwright also manages the browser projects for you, so you don’t need to install extra stuff.

Second, it uses browsers very efficiently:

Instead of launching a full, new browser instance for each test, Playwright launches one browser instance for the entire suite of tests.
It then creates a unique browser context from that instance for each test. A browser context is essentially like an incognito session: it has its own session storage and tabs that are not shared with any other context. Browser contexts are very fast to create and destroy.
Then, each browser context can have one or more pages. All Playwright interactions happen through a page, like clicks and scrapes. Most tests only ever need one page.

Playwright browsers, contexts, and pages

Playwright handles all this setup automatically for you.

Comparing Playwright to other tools

Playwright is not the only browser automation tool out there. The other two most popular tools are Selenium and Cypress. Here is a chart with high-level comparisons:

All three are good tools, and each one has their advantages. Playwright’s main advantages are that offers excellent developer experience with the fastest execution time, multiple language bindings, and several quality-of-life features.

Learning Playwright

If you want to learn how to automate your web tests with Playwright, take my tutorial, Awesome Web Testing with Playwright. All instructions and example code for the tutorial are located in GitHub. This tutorial is designed to be self-guided, so give it a try!

Test Automation University also has a Playwright learning path with introductory and advanced courses:

Playwright is an awesome new framework for modern web testing. Give it a try, and let me know what you automate!