Have you ever attended a developer conference? Or maybe you’ve taken a software product tutorial? Perhaps you’ve listened to a tech podcast, or you follow someone with an avocado emoji 🥑 on Twitter? Chances are that the folks behind lots of that content are involved in Developer Relations.
Many software companies run Developer Relations (or “DevRel”) practices these days. They might go by different names, such as Developer Advocacy, Developer Experience, or Developer Evangelism, but what do folks in DevRel actually do? There’s a lot going on behind the posts and public appearances. In this article, I want to share insights I’ve gained from my time as a Developer Advocate for Applitools.
What’s the main purpose?
The main purpose of Developer Relations is to bridge company and community:
It helps the community understand the value of the company’s technology and how to start using it.
It helps the company understand the community’s needs and facilitates the community’s feedback.
More specifically, DevRel focuses on the community of developers – the practitioners who will actually use the technology. In fact, many DevRel folks like me are developers ourselves. We seek to serve our own communities. Our job is not to sell the C-suite on buying products and services, but rather to help developers get the most value out of what the company offers. DevRel’s goal could be restated as “making life better for developers.”
With that in mind, it is important for DevRel folks to be authentic. If members of the community feel like DevRel is merely sales in disguise, then they will flatly ignore DevRel. Any work DevRel attempts to do will be in vain.
How’s it done?
Saying that DevRel “bridges company and community” sounds very abstract. DevRel encompasses several activities to accomplish that purpose. For example, in my job as a Developer Advocate:
I speak at public conferences and company events.
I write articles about software testing, automation, and quality.
I develop tutorials for my company’s SDKs.
I deliver workshops on test automation and on visual testing.
I talk with customers to learn the challenges they face with testing.
The types of duties done by DevRel naturally cluster around distinct roles. DevRel is a team sport, and it is rare that one person will be highly skilled at all the different roles, let alone have enough time to fulfill them. There are five primary roles that make up a DevRel team.
The evangelist popularizes the technology in an authentic, helpful way. Their job is to make sure people in the community know the big picture. Thus, the evangelist is a very outwardly-focused role. They make many public appearances at events, and they develop loads of content, like videos, articles, and online courses. Travel and strong social media presence are practically required for the job. Since they are on the “front lines,” they also keep their ears open for any feedback they might hear about the technology or the company. Evangelists may be called advocates, thought leaders, and influencers.
Since evangelists are so active in their communities, it’s common for them to have large followings on Twitter, LinkedIn, and YouTube. They may even become somewhat of celebrities in their spaces. More attention on them brings more attention to the companies they represent. Celebrity status must be handled carefully, though. One bad tweet can ruin a reputation. Or, if a company builds their entire DevRel program on one evangelist’s high profile, then the program could collapse if that person ever leaves the company.
The educator teaches members of the community how to use the technology. While the evangelist may give introductions and demos, the educator teaches the finer details. Typically, they write product documentation, quickstart guides, and tutorials. They may even teach courses online or in person. The educator must have strong skills in technical writing and education. In fact, in many DevRel practices, educators transitioned into tech from academia! The educator does not need a high profile like the evangelist. They can be more behind-the-scenes if they choose.
The Community Builder
The community builder fosters scalable relationships with those using the technology. They drive and sustain engagement with the community. A good community builder creates an environment where everyone feels a sense of belonging. They provide a space for organic interactions, vulnerable questions, and helpful camaraderie. For example, folks should be able to ask basic product questions in a Slack room or product forum and get helpful answers from others in the community.
The “scalability” of those relationships is rooted in the programs the community builder runs, such as:
Building a public communication forum (like Slack, Discord, or Discourse)
Hosting events and spaces for folks to gather
Providing newsletters or product updates
Connecting folks together for cool opportunities
Furthermore, the community builder provides front-row seats to what’s happening with the technology. They get folks excited for upcoming features, new releases, and the possibilities for the future. They could provide early access to beta versions of features or hold release parties. In doing so, the community builder also solicits feedback from the community to help make the technology even better.
The Platform Builder
The platform builder creates the systems and infrastructure that maintain DevRel programs. They are essentially developers whose customers are other DevRel folks. For example, the platform builder could:
Create example projects for demos and educational material
Build, deploy, and administer documentation websites
Keep package versions for tutorials and example projects up to date
Implement the infrastructure needed for big events like a hackathon
The platform builder is very much behind-the-scenes and oftentimes the unsung hero.
The visionary casts the vision for the company’s Developer Relations practice. They lead with big ideas, set goals, and guide the team to success. Usually, the visionary carries a title like “director” or “head.” Since a DevRel practice could encompass so many different kinds of activities, they need to be wise in how they align resources to meet their goals. For example, if the company is a startup that’s just starting to roll out early releases, then the visionary should probably focus most on evangelism and community building. Different companies will have different needs.
How do these roles work together?
As I stated previously, DevRel is a team sport! Here’s an example of how these five roles could work together for a program like Test Automation University:
The visionary notices that folks are struggling to use the company’s technology, so they decide to create an online learning platform with free courses about the technology.
The platform builder develops the web app for the learning platform and deploys it to cloud resources.
The educator creates training courses and uploads them to the learning platform.
The evangelist promotes the learning platform when they speak at events and post on social media.
The community builder welcomes new students to the learning platform, sets up a Slack room for them to ask questions, and makes announcements when new courses are published.
These roles have some overlapping responsibilities, but they clearly represent distinct areas of responsibility.
Where does this team belong?
From what I’ve seen, most Developer Relations teams fall under one of two organizations: Product or Marketing. Indeed, DevRel has responsibilities to both. DevRel activities happen at the top of the marketing funnel. The team should work close with marketing folks on messaging. At the same time, DevRel folks must be somewhat independent of sales and marketing in order to be authentic in the community. They should focus on good, valuable content that supports the product and genuinely helps their users. Ultimately, the parent organization will ultimately determine how a DevRel team’s performance is measured.
How should success be measured?
The impact of Developer Relations is tough to measure because it benefits so many aspects of a company. It teaches folks about the company’s technology. It draws them into the marketing funnel. It provides helpful guides to get them started. In many instances, though, it’s hard to measure how one particular conference talk encouraged someone to sign up for a free account, or how one piece of vital documentation prevented a user from rage-quitting.
Overall, the best measure of success is engagement. The more engaged people are in the products and in the community, the more likely they are to pay for more. Here are a few engagement metrics to consider:
How many active users the products have and how they use them
How many people sign up for an account after an event
How many people visit certain blog articles or documentation pages
How many stars, forks, and watchers a GitHub project has
The completion rate for tutorials and courses
There are many other things that could be measured. Just remember that no single metric tells the full story.
Does DevRel really matter?
YES! Developer Relations matters as a practice because developer experience (DX) matters for the consumption of technology. It’s the 2020s. People don’t want to use lousy products. Developers expect tools and services to work well while solving their problems. DevRel is critical for building bridges with developers. It helps them understand why your fancy new tech is worth their consideration, and it helps you as a company understand what they need and what delights them.
It is one of the most recognizable works of art in the world. It is so famous, it has an emoji: 🌊.
The Great Wave Off Kanagawa is a Japanese woodblock print. It is not a painting or a drawing but a print. In Japanese, the term for this type of art is ukiyo-e, which means “pictures of the floating world.” Ukiyo-e prints first appeared around the 1660s and did not decline in popularity until the Meiji Restoration two centuries later. While most artists focused on subjects of people, late masters like Hokusai captured perspectives of landscapes and nature. Here, in The Great Wave, we see a giant wave, full of energy and ferocity, crashing down onto three fast boats attempting to transport live fish to market. Its vibrant blue water and stark white peaks contrast against a yellowish-gray sky. In the distance is Mount Fuji, the highest mountain in Japan, yet it is dwarfed in perspective by the waves. In fact, the water spray from the waves appears to fall over Mount Fuji like snow. If you didn’t look closely, you might presume that Mount Fuji is just the crest of another wave.
The Great Wave is absolutely stunning. It is arguably Hokusai’s finest work. The colors and the lines reflect boldness. The claws of the wave impart vitality. The men on the boat show submission and possibly fear. The spray from the wave reveals delicacy and attention to detail. Personally, I love ukiyo-e prints like this. I travel the world to see them in person. The quality, creativity, and craftsmanship they exhibit inspire me to instill the highest quality possible into my own work.
As software quality professionals, there are several lessons we can learn from ukiyo-e masters like Hokusai. Testing is an art as much as it is engineering. We can take cues from these prolific artists in how we approach quality in our own work. In this article, I will share how we can make our own “Great Waves” using 8 software testing convictions inspired by ukiyo-e prints like The Great Wave. Let’s begin!
Conviction #1: Focus on behavior
Although we hold these Japanese woodblock prints today in high regard, they were seen as anything but fancy centuries ago in Japan. Ukiyo-e was “low” art for the common people, whereas paintings on silk scrolls were considered “high” art for the high classes.
Folks would buy these prints from local merchants for slightly more than the cost of a bowl of noodles – about $5 to $10 US dollars today – and they would use these prints to decorate their homes. By comparison, a print of The Great Wavesold at auction for $1.11 million in September 2020.
These prints weren’t very large, either. The Great Wave measures 10 inches tall by 15 inches wide, and most prints were of similar size. That made them convenient to buy at the market, carry them home, and display on the wall. To understand how the Japanese people treated these prints in their day, think about the decorations in your homes that you bought at stores like Home Goods and Target. You probably have some screen prints or posters on your walls.
Since the target consumer for ukiyo-e prints were ordinary people with working-class budgets, they needed to be affordable, popular, and recognizable. When Hokusai published The Great Wave, it wasn’t a standalone piece. It was the first print in a series named Thirty-six Views of Mount Fuji. Below are three other prints from that series. The central feature in each print is Mount Fuji, which would be instantly recognizable to any Japanese person. The various views would also be relatable.
The features of these prints made them valuable. Anyone could find a favorite print or two out of a series of 36. They made art accessible. They were inexpensive yet impressive. They were artsy yet accessible. Artists like Hokusai knew what people wanted, and they delivered the goods.
This isn’t any different from software development. Features add value for the users. For example, if you’re developing a banking app, folks better be able to log in securely and view their latest transactions. If those features are broken or unintuitive, folks might as well move their accounts to other banks! We, as the developers and testers, are like the ukiyo-e artists: we need to know what our customers need. We need to make products that they not only want, but they also enjoy.
Features add value. However, I would use a better word to describe this aspect of a product: behavior. Behavior is the way one acts or conducts oneself. In software, we define behaviors in terms of inputs and responses. For example, login is a behavior: you enter valid credentials, and you expect to gain access. You gave inputs, the app did something, and you got the result.
My conviction on software testing AND development is that if you focus on good software behaviors, then everything else falls into place. When you plan development work, you prioritize the most important behaviors. When you test the features, you cover the most important behaviors. When users get your new product, they gain value from those features, and hopefully you make that money, just like Hokusai did.
This is why I strongly believe in the value of Behavior-Driven Development, or BDD for short. As a set of pragmatic practices, BDD helps you and your team stay focused on the things that matter. BDD involves activities like Three Amigos collaboration, Example Mapping, and writing Gherkin. When you focus on behavior – not on shiny new tech, or story points, or some other distractions – you win big.
Conviction #2: Prioritize on risk
Ukiyo-e artists depicted more than just views of Mount Fuji. In fact, landscape scenes became popular only during the late period of woodblock printing – the 1830s to the 1860s. Before then, artists focused primarily on people: geisha, courtesans, sumo wrestlers, kabuki actors, and legendary figures. These were all characters from the “floating world,” a world of pleasure and hedonism apart from the dreary everyday life of feudal Japan.
Here is a renowned print of a kabuki actor by Sharaku, printed in 1794:
Sharaku was active only for one year, but he produced some of the most expressive portraits seen during ukiyo-e’s peak period. A yakko was a samurai’s henchman. In this portrait, we see Edobei ready for dirty deeds, with a stark grimace on his face and hands pulsing with anger.
Why would artists like Sharaku print faces like these? Because they would sell. Remember, ukiyo-e was not high-class art. It was a business. Artists would make a series of prints and sell them on the streets of Edo (now Tokyo). They needed to make prints that people wanted to buy. If they picked lousy or boring subjects, their prints wouldn’t sell. No soba noodles for them! So, what subjects did they choose? Celebrities. Actors. “Female beauties.” And some content that was not safe for work, like Hokusai’s The Dream of the Fisherman’s Wife. (Seriously, that link is not safe for work. Click it at your own risk.)
Artists prioritized their work based on business risk. They chose subjects that would be easy to sell. They pursued value. As testers, we should also prioritize test coverage based on risk.
I know there’s a popular slogan saying, “Test all the things!”, but that’s just impossible. It’s like saying, “Print all the pictures!” Modern apps are too complex to attempt any sort of “complete” or “100%” coverage. Instead, we should focus our testing efforts on the most important behaviors, the ones that would cause the most problems if they broke. Testing is ultimately a risk-mitigating activity. We do testing to de-risk problems that enter during development.
So, what does a risk-based testing strategy look like? Well, start by covering the most valuable behaviors. You can call them the MVBs. These are behaviors that are core to your app. If they break, then it’s game over. No soba noodles. For example, if you can’t log in, you’re done-zo. The MVBs should be tested before every release. They are non-negotiable test coverage. If your team doesn’t have enough resources to run these tests, then get more resources.
In addition to the MVBs, cover areas that were changed since the previous release. For example, if your banking app just added mobile deposits, then you should test mobile deposits. Things break where developers make changes. Also, look at testing different layers and aspects of the product. Not every test should be a web UI test. Add unit tests to pinpoint failures in the code. Add API tests to catch problems at the service layer. Consider aspects like security, accessibility, and visuals.
When planning these tests, try to keep them fast and atomic, covering individual behaviors instead of long workflows. Shorter tests are more reliable and give space for more coverage. And if you do have the resources for more coverage beyond the MVBs and areas of change, expand your coverage as resources permit. Keep adding coverage for the next most valuable behaviors until you either run out of time or the coverage isn’t worth the time.
Overall, ask yourself this when weighing risks: How painful would it be if a particular behavior failed? Would it ruin a user’s experience, or would they barely notice?
Conviction #3: Automate
The copy of The Great Wave shown at the top of this article is located at the Metropolitan Museum of Art in New York City. However, that’s not the only version. When ukiyo-e artists produced their prints, they kept printing copies until the woodblocks wore out! Remember, these weren’t precious paintings for the rich, they were posters for the commoners. One set of woodblocks could print thousands of impressions of popular designs for the masses. It’s estimated that there were five to eight thousand original impressions of The Great Wave, but nobody knows for sure. To this day, only a few hundred have survived. And much to my own frustration, museums that have copies do not put them on public display because the pieces are so fragile.
Here are different copies of The Great Wave from different museums:
Print production had to be efficient and smooth. Remember, this was a business. Publishers would make more money if they could print more impressions from the same set of woodblocks. They’d gain more renown if their prints maintained high quality throughout the lifetime of the blocks. And the faster they could get their prints to market, the sooner they could get paid and enjoy all the soba noodles.
What can we learn from this? Automate! That’s our third conviction.
What can we learn from this? Automate! Automation is a force multiplier. If Hokusai spent all his time manually laboring over one copy of The Great Wave, then we probably wouldn’t be talking about it today. But because woodblock printing was a whole process, he produced thousands of copies for everyone to enjoy. I wouldn’t call the woodblock printing process fully “automated” because it had several tedious steps with manual labor, but in Edo period Japan, it was about as automated as you could get.
Compare this to testing. If we run a test manually, we cover the target behavior one time. That’s it: lots of labor for one instance. However, if we automate that test, we can run it thousands of times. It can deliver value again and again. That’s the difference between a painting and a print.
So, how should we go about test automation? First, you should define your goals. What do you hope to achieve with automation? Do you want to speed up your testing cycles? Are you looking to widen your test coverage? Perhaps you want to empower Continuous Delivery through Continuous Testing? Carefully defining your goals from the start will help you make good decisions in your test automation strategy.
When you start automating tests, treat it like full software development. You aren’t just writing a bunch of scripts, you are developing a software system. Follow recommended practices. Use design patterns. Do code reviews. Fix bugs quickly. These principles apply whether you are using coded or codeless tools.
Another trap to avoid is delaying test automation. So many times, I’ve heard teams struggle to automate their tests because they schedule automation work as their lowest priority. They wish they could develop automation, but they just never have the time. Instead, they grind through testing their MVBs manually just to get the job done. My advice is flip that attitude right-side up. Automate first, not last. Instead of planning a few tests to automate if there’s time, plan to automate first and cover anything that couldn’t be automated with manual testing.
Furthermore, integrate automated tests into the team’s Continuous Integration system as soon as possible. Automated tests that aren’t running are dead to me. Get them running automatically in CI so they can deliver value. Running them nightly or even weekly can be a good start, as long as they run on a continuous cadence.
Finally, learn good practices. Test automation technologies are ever-evolving. It seems like new tools and frameworks hit the market all the time. If you’re new to automation or you want to catch up with the latest trends, then take time to learn. One of the best resources I can recommend is Test Automation University. TAU has about 70 courses on everything you can imagine, taught by the best instructors in the world, and it’s 100% FREE!
Now, you might be thinking, “Andy, come on, you know everything can’t be automated!” And that’s true. There are times when human intervention adds value. We see this in ukiyo-e prints, too. Here is Plum Garden at Kameido by Utagawa Hiroshige, Hokusai’s main rival. Notice the gradient colors of green and red in the background:
Printers added these gradients using a technique called bokashi, in which they would apply layers of ink to the woodblocks by hand. Sometimes, they would even paint layers directly on the prints. In these cases, the “automation” of the printing process was insufficient, and humans needed to manually intervene.
It’s always good to have humans test-drive software. Automation is great for functional verification, but it can’t validate user experience. Exploratory testing is an awesome complement to automated testing because it mitigates different risks.
If we use Visual AI to compare these two prints, it will quickly identify the main difference:
The signature block is in a different location! Small differences like small pixel offsets are ignored, while major differences are highlighted. If you apply this style of visual testing to your web and mobile apps, you could catch a ton of visual bugs before they cause problems for your users. Modern test automation can do some really cool tricks!
Conviction #4: Shift left and right
Mokuhanga, or woodblock printing, was a huge process with multiple steps. Artists like Hokusai and Hiroshige did not print their artwork themselves. In fact, printing required multiple roles to be successful: a publisher, an artist, a carver, and a printer.
The publisher essentially ran the process. They commissioned, financed, and distributed prints. They would even collaborate with artists on print design to keep them up with the latest trends.
The artist designed the patterns for the prints. They would sketch the patterns on washi paper and give instructions to the carver and printer on how to properly produce the prints.
The carver would chisel the artist’s pattern into a set of wooden printing blocks. Each layer of ink would have its own block. Carvers typically used a smooth, hard wood like cherry.
The printer used the artist’s patterns and carver’s woodblocks to actually make the prints. They would coat the blocks in appropriately-colored water-based inks and then press paper onto the blocks.
Quality had to be considered at every step in the process, not just at the end. If the artist was not clear about colors, then the printer might make a mistake. If the carver cut a groove too deep, then ink might not adhere to the paper as intended. If the printer misaligned a page during printing, then they’d need to throw it away – wasting time, supplies, and woodblock life – or risk tarnishing everyone’s reputation with a misprint. Hokusai was noted for his stringent quality standards for carvers and printers.
This is just like software development. We can substitute the word “testing” for “inspection” in Deming’s quote. Testers don’t exclusively “own” quality. Every role – business, development, and testing – has a responsibility for high-caliber work. If a product owner doesn’t understand what the customer needs, or a developer skips code reviews, or if a tester neglects an important feature, then software quality will suffer.
How do we engage the whole team in quality work? Shift left and right.
Most testers are probably familiar with the term shift left. It means, start doing testing work earlier in the development process. Don’t wait until developers are “done” and throw their code “over the fence” to be tested. Run tests continuously during development. Automate tests in-sprint. Adopt test-driven and behavior-driven practices. Require unit tests. Add test implementation to the “Definition of Done.”
But what about shift right? This is a newer phase, but not necessarily a newer practice. Shift right means, continue to monitor software quality during and after releases. Build observability into apps. Monitor apps for bugs, failures, and poor performance. Do canary deployments to see how systems respond to updates. Perform chaos testing to see how resilient environments are to outages. Issue different UIs to user groups as part of A/B testing to find out what’s most effective. And feed everything you learn back into development a la “shift left.”
The famous DevOps infinity loop shows how “shift left” and “shift right” are really all part of the same flow. If you start in the middle where the paths cross, you can see arrows pointing leftward for feedback, planning, and building. Then, they push rightward with continuous integration, deployment, monitoring, and operations. We can (and should) take all the quality measures we said before as we spin through this loop perpetually. When we plan, we should build quality in with good design and feedback from the field. When we develop, we should do testing together with coding. As we deploy, automated safety checks should give thumbs-up or thumbs-down. Post-deployment, we continue to watch, learn, and adjust.
Conviction #5: Give fast feedback
The acronym CI/CD is ubiquitous in our industry, but I feel like it’s missing something important: “CT”, or Continuous Testing. CI and CD are great for pushing code fast, but without testing, they could be pushing garbage. Testing does not improve quality directly, but continuous revelation of quality helps teams find and resolve issues fast. It demands response. Continuous Testing keeps the DevOps infinity loop safe.
Fast feedback is critical. The sooner and faster teams discover problems, the less pain those problems will cause. Think about it: if a developer is notified that their code change caused a failure within a minute, they can immediately flip back to their code, which is probably still open in an editor. If they find out within an hour, they’ll still have their code fresh in their mind. Within a day, it’ll still be familiar. A week or more later? Fuggedaboutit! Heaven forbid the problem goes undetected until a customer hits it.
Continuous testing enables fast feedback. Automation enables continuous testing. Test automation that isn’t running continuously is worthless because it provides no feedback.
Japanese woodblock printers also relied on fast feedback. If they noticed anything wrong with the prints as they pressed them, they could scrap the misprint and move on. However, since they were meticulous about quality, misprints were rare. Nevertheless, each print was unique because each impression was done manually. The amount, placement, and hue of ink could vary slightly from print to print. Over time, the woodblocks themselves wore down, too.
On the left, the outline around the title is solid, whereas on the right, the outline has breaks. This is because the keyblock had very fine ridges for printing outlines, which suffered the most from wear and tear during repeated impressions. Furthermore, if you look very closely, you can see that the Japanese characters appear bolder on the right than the left. The printer must have used more ink or pressed the title harder for the impression on the right.
Printers would need to spot these issues quickly so they could either correct their action for future prints or warn the publisher that the woodblocks were wearing down. If the print was popular, the publisher could commission a carver to carve new woodblocks to keep production going.
Conviction #6: Go lean
As I’ve said many times now, woodblock printing was a business. Ukiyo-e was commercial art, and competition was fierce. By the 1840s, production peaked with about 250 different publishers. Artists like Hokusai and Hiroshige were rivals. While today we recognize famous prints like The Great Wave, countless other prints were also made.
Publishers competed in a rat race for the best talent and the best prints. They had to be savvy. They had to build good reputations. They needed to respond to market demands for subject material. For example, Kitagawa Utamaro was famous for prints of “female beauties.”
Ukiyo-e artists also took inspiration from each other. If one artist made a popular design, then other artists would copy their style. Here is a print from Hiroshige’s series, Thirty-Six Views of Mount Fuji. That’s right, Hokusai’s biggest rival made his own series of 36 prints about Mount Fuji, and he also made his own version of The Great Wave. If you can’t beat ‘em, join ‘em!
Publishers also had to innovate. Oftentimes, after a print had been in production for a while, they would instruct the printer to change the color scheme. Here are two versions of Hokusai’s Kajikazawa in Kai Province, from Thirty-six Views of Mount Fuji:
The print on the left is an early impression. The only colors used were shades of blue. This was Hokusai’s original artistic intention. However, later prints, like the one on the right, added different colors to the palette. The fishermen now wear red coats. The land has a bokashi green-yellow gradient. The sky incorporates orange tones to contrast the blue. Publishers changed up the colors to squeeze more money out of existing designs without needing to pay artists for new work or carvers for new woodblocks.
However, sometimes when doing this, artistic quality was lost. Compare the fine detail in the land between these two prints. In the early impression, you can see dark blue shading used to pronounce the shadows on the side of the rocks, giving them height and depth, and making the fisherman appear high above the water. However, in the later impression, the green strip of land has almost no shading, making it appear flat and less prominent.
Ukiyo-e publishers would have completely agreed with today’s lean business model. Seek first and foremost to deliver value to your customers. Learn what they want. Try some designs, and if they fail, pivot to something else. When you find what works, get a full end-to-end process in place, and then continuously improve as you go. Respond quickly to changes.
Going lean is very important for software testing, too. Testing is engineering, and it has serious business value. At the same time, testing activities never seem to have as many resources as they should. Testers must be scrappy to deliver valuable quality feedback using the resources they have.
When I think about software testing going lean, I’m not implying that testers should skip tests or skimp on coverage. Rather, I’m saying that world-class systems and processes cannot be built overnight. The most important thing a team can do is build basic end-to-end feedback loops from the start, especially for test automation.
So many times, I’ve seen teams skew their test automation strategy entirely towards implementation. They spend weeks and weeks developing suites of automated tests before they set up any form of Continuous Testing. Instead of triggering tests as part of Continuous Integration, folks must manually push buttons or run commands to make them start. Other folks on the team see results sporadically, if ever. When testers open bug reports, developers might feel surprised.
I recommend teams set up Continuous Testing with feedback loops from the start. As soon as you automate your first test, move onto running it from CI and sending you notifications for results before automating your second test. Close the feedback loop. Start delivering results immediately. As you find hotspots, add more coverage. Talk with developers about the kinds of results they find most valuable. Then, grow your suite once you demonstrate its value. Increase the throughput. Turn those sidewalks into highways. Continue to iteratively improve upon the system as you go. Don’t waste time on tests that don’t matter or dashboards that nobody reads. Going lean means allocating your resources to the most valuable activities. What you’ll find is that success will snowball!
Conviction #7: Open up
Once you have a good thing going, whether it’s woodblock printing or software testing, how can you take it to the next level? Open up! Innovation stalls when you end up staring at your own belly button for too long. Outside influences inspire new creativity.
Ukiyo-e prints had a profound impact on Western art. After Japan opened up to the rest of the world in the mid-1800s, Europeans became fascinated by Japanese art, and European artists began incorporating Japanese styles and subjects into their work. This phenomenon became known as Japonisme. Here, Claude Monet, famous for his impressionist paintings, painted a picture of his wife wearing a kimono with fans adorning the wall behind her:
Vincent van Gogh in particular loved Japanese woodblock prints. He painted his own versions of different prints. Here, we see Hiroshige’s Plum Garden at Kameido side-by-side with Van Gogh’s Flowering Plum Orchard (after Hiroshige):
Van Gogh was drawn to the bold lines and vibrant colors of ukiyo-e prints. There is even speculation that The Great Wave inspired the design of The Starry Night, arguably Van Gogh’s most famous painting:
Notice how the shapes of the waves mirror the shapes of the swirls in the sky. Notice also how deep shades of blue contrast yellows in each. Ukiyo-e prints served as great inspiration for what became known as Modern art in the West.
Influence was also bidirectional. Not only did Japan influence the West, but the West influenced Japan! One thing common to all of the prints in Thirty-six Views of Mount Fuji is the extensive use of blue ink. Prussian blue pigment had recently come to Japan from Europe, and Hokusai’s publisher wanted to make extensive use of the new color to make the prints stand out. Indeed, they did. To this day, Hokusai is renowned for popularizing the deep shades of Prussian blue in ukiyo-e prints.
It’s important in any line of work to be open to new ideas. If Hokusai had not been willing to experiment with new pigments, then we wouldn’t have pieces like The Great Wave.
That’s why I’m a huge proponent of Open Testing. What if we open our tests like we open our source? There are so many great advantages to open source software: helping folks learn, helping folks develop better software, and helping folks become better maintainers. If we become more open in our testing, we can improve the quality of our testing work, and thus also the quality of the software products we are building. Open testing involves many things: building open source test frameworks, getting developers involved in testing, and even publicly sharing test cases and results.
Conviction #8: Show empathy
In this article, we’ve seen lots of great artwork, and we’ve learned lots of valuable lessons from it. I think ukiyo-e prints remain popular today because their subject matter focuses on the beauty of the world. Artists strived to make pieces of the “floating world” tangible for the common people.
Ukiyo-e prints revealed the supple humanity of the Japanese people, like in this print by Utagawa Kunisada:
Ukiyo-e prints also revealed ordinary people living out their lives, like this print from Hokusai’s Thirty-six Views of Mount Fuji:
Art is compelling. And software, like art, is meant for people. Show empathy. Care about your customers. Remember, as a tester, you are advocating for your users. Try to help solve their problems. Do things that matter for them. Build things that actually bring them value. Be thoughtful, mindful, and humble. Don’t be a jerk.
The Golden Conviction
These eight convictions are things I’ve learned the hard way throughout my career:
Focus on behavior
Prioritize on risk
Shift left and right
Give fast feedback
I live and breathe these convictions every day. Whether you are making woodblock prints or running test cases, these principles can help you do your best work.
If I could sum up these eight convictions in one line, it would be this: Be excellent in all things. If you test software, then you are both an artist and an engineer. You have a craft. Do it with excellence.
Let’s face it: working in software is tough. I’ve worked in software for over a decade, and I’ve learned a lot of things during my journey – oftentimes the hard way.
On April 7, 2022, I tried something new: every hour for 24 hours, I tweeted a lesson I learned during my software career. I covered topics like breaking into the industry, working on projects, and seeking advancement. I used the hashtag #PandyEveryHour to track my tweets. I started at 8am (US Eastern) on April 7, and I concluded at 8am on April 8. It was an awesome experience: lots of folks liked, retweeted, and commented. Some of my insights struck nerves.
In case you missed it while my tweets went live, I compiled all the tweets below in order. Give them a read! If you like any of them in particular, be sure to “like” them on Twitter so I know!
Breaking into the software industry
Working with others
Improving your skills
So, which lessons resonated the most with you? Which did you disagree with? Let me know!
The main photo for this article was taken at DjangoCon 2019 during one of the breaks between sessions.
At my new job as Developer Advocate at Applitools, I now permanently work remotely. Working from home is new to me. At my previous company PrecisionLender (later acquired by Q2), I worked in an office that was a seven-minute drive from my house. My office was a real office: an enclosed room with a door, walls, a large desk, and two huge whiteboards. Working from the office was a joy. We had free snacks and sodas. My colleagues and I all got along great, and we’d go out to lunch almost every day. After COVID-19 hit the USA in March 2020, I was heartbroken to abandon the office. I was so stubborn about eventually returning to the office that I refused to set up a decent workstation at home. For a year and a half, I worked exclusively on my laptop either from a desk in my living room or from the couch – no monitors, no mouse, and no separate keyboard. I mean, come on, the pandemic would be over in a few short months, right?
However, my attitudes about remote work are changing. At PrecisionLender, I was spoiled with the best kind of office situation someone could have. Many others in the software field are not so fortunate. Those who live in the big cities like New York and San Francisco pay thousands of dollars a month to split tiny apartments and spend an hour or more on one-way commutes. Not to mention, many of them move away from family and friends to take those opportunities.
Office layout trends in recent years have also been dismal. The open layout – no walls with clusters of desks sloppily smashed together – is the norm. Perpetual audio and visual distractions are a nightmare for anyone trying to concentrate. Widescreen monitors and noise-cancelling headphones only help a little. I experienced that frustration at MaxPoint and LexisNexis. Cubicles, while passé these days, are only a small step up from open floors. While the “walls” block out visual distractions, you can still hear everything that goes on around you. Plus, the enormous blocks of cubicles on a floor feel brutally confining. I frequently called the IBM 500 complex in RTP “Cubicle City” when I worked there.
Remote work can be a godsend for many hardworking folks who just want to do a good job while doing right by themselves. Folks, especially in software, can literally work anywhere. COVID-19 shutdowns have proven it. My wife and I even took advantage of it: we bought a vacation condo in Seattle, and I worked remotely from there for several weeks. I could automate tests during the day, grab beers with close friends, and catch a Mariners game on the weekend. Even with a 3-hour time zone difference, I was right in step with my team the whole time.
Recently, I saw a Twitter thread by Chris Herd in which he made several predictions on remote work in 2022 and beyond:
This thread really resonated with me. While I still have some concerns about remote work, I do believe it will be a net good for not only my country and people but also for the world. I’d like to highlight some of the points he made that really stuck out to me.
I agree completely with this, and I’m excited to see it happen. The US has so many awesome places, from sea to shining sea. There’s no reason why someone should need to move from their hometown to a big city just to work for a specific company (unless, of course, they want that adventure). Places like Raleigh-Durham, Nashville, Denver, Miami, Pittsburgh, Salt Lake City, Cleveland, and countless other smaller American cities are fantastic places to live. Each has a unique vibe and local specialties. Allowing folks to live where they want with good employment opportunities will rejuvenate local and regional economies. It will bring broader prosperity to all.
This is happening already. The Triangle of NC, where I live, is booming. My wife and I rent local listings on Airbnb, and all through the pandemic, the #1 type of guest we’ve hosted by far is someone interested in relocating to the Triangle.
I hadn’t thought about asynchronous work until Chris mentioned it, but he’s right. Life is 24/7, and people live all around the globe. Remote work can liberate us from a harsh 9-5 and instead embrace more flexible hours. For example, this past week, I had to work around the clock for a few days prepping a big webinar. However, I wasn’t working for 16 hours straight each day. I woke up a bit late, did some light administrative activities before lunch, hit it hard all afternoon, stopped for dinner and time with my puppy, and then burned the midnight oil after dark. That’s what worked for me.
Yes! I’m living it myself. My wife and I just adopted a French bulldog puppy, and I also started restoring a vintage 1970 Volkswagen Beetle.
I sure hope so! Break down those entry barriers. Do away with the gatekeepers. End coastal elitism.
I sure hope this one becomes true, but I’m skeptical.
This one is so ironic. Based on many articles I’ve read, upper management seems to despise work-from-home because they think their workers will work less. However, all the good, hardworking folks I know are actually working more than ever. Not only are folks more efficient, but they tend to keep working just a little extra because there are fewer barriers between work stuff and personal stuff. In my new job, I know I’m certainly working more hours right now than I probably should, and I’m hoping to settle into a more sustainable routine as I acclimate to my new role. Nevertheless, we absolutely need to protect ourselves against unintentional burnout.
One big concern I have with remote work is the loss of in-person contact. Humans need to be around each other, and screens are a poor substitute. I know I’ve felt lonelier working from home than when I worked in an office. That’s why I absolutely love this idea about regular remote retreats. Companies can still bring folks together, and they can hire professionals who know how to maximize that time. I go to software conferences frequently for similar experiences.
I think this goes hand-in-hand with async work. However, unless safeguards are put in place to prevent overworking, then life-work balance could quickly become work-more-work imbalance.
Honestly, this one was never much of a problem for me. The only “paddings” I’ve had in my workdays are long, boring meetings where I participated little. Thankfully, those were confined mostly to my days at IBM and NetApp earlier in my career.
YES! Even though my commutes have always been relatively short, I still appreciate the time I get back in my day. Here are some things I’ve done with my extra, more flexible time:
Sleep in a little later
Play with my puppy when I need a break
Share lunch nearby with friends
Run a quick errand in my Beetle
Grab an impromptu bubble tea with my wife
Take a walk around my neighborhood while thinking about work-related challenges
Documentation is the unspoken superpower of remote teams. I strongly value written communication for its clarity, searchability, and permanence.
Chris had lots of other good tweets in that thread, so be sure to read them. Overall, his tone was bullishly optimistic on remote work – not only in its benefits, but also in its inevitability. Personally, I hope remote work is here to stay, and that it’s not just a phase brought on by the COVID-19 pandemic. It’s a game-changer for individuals and industries alike. However, I’m still somewhat fearful that big companies will reign in wide scale returns to the office after the pandemic. Giants have already sunk millions to billions in commercial real estate for their offices that they can’t get back. I’m also worried about salaries in different geographies. Employers already pay folks who live outside of big cities much less for the same work due to cost-of-living differences. I don’t think that will be fair as people leave big cities in droves for smaller cities or the countryside. Already, the cost of living is skyrocketing in the Triangle, yet our average tech salaries are far below those of Silicon Valley. We’ll see what happens.
Overall, I am optimistic about remote work just like Chris.
So, what do y’all think? Please leave comments below. I’d love to learn different perspectives!
I’ve always been frustrated with poor quality audio recordings. Microphones built into laptops are very convenient, but they usually yield tinny sound lacking the depth of real voices. When I asked my audiophile friends for advice, they recommended studio-level equipment that was beyond my comprehension and my budget. As a software guy, I just wanted an audio setup that captured high-quality audio while still being convenient for everyday usage. I’d need it for remote meetings as well as for recording talks and tutorials. I was willing to pay for good equipment as long as I could use it well. Unfortunately, my biggest frustration was ignorance. I didn’t know anything about recording.
When I did my research, the Blue Yeti microphone was at the top of everyone’s recommendation list. The nicest thing about a Blue Yeti mic is that it connects via USB. You simply plug it right into your laptop and select audio input and output channels. The mic doesn’t need an external power source, either.
Initially, I bought only the Blue Yeti mic and the foam windscreen. Instead of using the boom arm, I used the tabletop stand that came with the mic for all my recordings. This was a big mistake. The tabletop stand picked up a lot of local noise, like typing on a keyboard. The audio quality it picked up also sounded like it had a bit of an echo, which may have been due to sound bouncing off the tabletop or my hardwood floors.
The boom arm and shockmount made a huge improvement in recording quality. Plus, with the boom arm mounted firmly to my desk, I can easily move it towards me for recording or out of the way otherwise. It feels quite sturdy. I chose to use all Blue products so that I could be certain that they’d work together. You can save quite a bit of money if you buy the microphone, boom arm, and shockmount together as a bundle on Amazon (~$200), instead of a la carte like I did (~$250).
I plan to get a docking station for my laptop so that I can plug the microphone’s USB cable into the dock, simplifying desk’s cable management.
Even with this new setup, I still felt like I didn’t understand how to use my Blue Yeti microphone to its full potential. Thankfully, YouTube came to the rescue! This video greatly helped me understand things like tuning the mic’s gain and positioning the mic while speaking:
In summary, here’s what I like about the setup:
It yields very good (maybe professional?) audio recording quality.
It is simple enough for anyone to set up and use.
It is convenient to use for remote meetings, video recordings, etc.
It feels quite sturdy.
It is relatively affordable (compared to other audio equipment).
Please note: I am not an expert in audio equipment, and this article is not sponsored by any company. I simply hope that someone can benefit from the things I learned (and possibly save a few bucks) if they want to improve their own recording game!
Happy Global Testers Day! For 2021, QA Touch is celebrating with webinars, games, competitions, blogs, and videos. I participated by sharing an “upside-down” story from years ago when I accidentally wiped out all of NetApp’s continuous integration testing. Please watch my story below. I hope you find it both insightful and entertaining!
On April 22, 2021, I delivered a talk entitled “Managing the Test Data Nightmare” at SauceCon 2021. SauceCon is Sauce Labs’ annual conference for the testing community. Due to the COVID-19 pandemic, the conference was virtual, but I still felt a bit of that exciting conference buzz.
My talk covers the topic of test data, which can be a nightmare to handle. Data must be prepped in advance, loaded before testing, and cleaned up afterwards. Sometimes, teams don’t have much control over the data in their systems under test—it’s just dropped in, and it can change arbitrarily. Hard-coding values into tests that reference system tests can make the tests brittle, especially when running tests in different environments.
In this talk, I covered strategies for managing each type of test data: test case variations, test control inputs, config metadata, and product state. I also covered how to “discover” test data instead of hard-coding it, how to pass inputs into automation (including secrets like passwords), and how to manage data in the system. After watching this talk, you can wake up from the nightmare and handle test data cleanly and efficiently like a pro!
Here are some other articles I wrote about test data:
In February 2021, Matthew Weeks interviewed me for the Work in Programming podcast. Matthew asked all sorts of questions about my story – how I got into programming, what I learned at different companies, and why I started blogging and speaking. I greatly enjoyed our conversation, so much so that it lasted an hour and a half!
If you’re interested to hear how my career has gone from high school to the present day, please give it a listen. There are some juicy anecdotes along the way. The link is below. Many thanks to Matthew for hosting and editing the interview. Be sure to listed to other Work in Programming interviews, too!
Back in 2011, I was a recent college grad working at IBM as a “performance engineer” for z/OS mainframe software. Now, I didn’t know anything about mainframes, but I was thankful to have a job on the heels of the Great Recession.
At the time, IBM had recently released the Jazz platform with Rational Team Concert (RTC), a collaborative project management tool geared towards Agile software development. Teams company-wide started adopting RTC whether they wanted it or not. My team was no different: we created a team project in RTC and started writing work items in it. In my opinion, RTC was decent. It was very customizable, and its aesthetics and user experience were better than other tools at the time.
One day, I made a typo while trying to assign a work item to myself. When typing a name into the “owner” field, RTC would show a list of names from which to choose. For whatever reason, the list included all IBM employees, not just members from my team. IBM had nearly 400,000 employees worldwide at the time. I accidentally selected someone else with a similar name to mine. Blissfully unaware of my mistake, I proceeded to save the work item and start doing the actual work for it.
About a day later, I received a nastygram from another IBMer named Andrea Knight, demanding to know why I assigned her this work item in RTC. I had never met this person before, and she certainly wasn’t on my team. (To be honest, I don’t remember exactly what her name was, but for the sake of the story, we can call her Andrea.) At first, I felt perplexed. Then, once I read her message, I quickly realized that I must have accidentally listed her as the owner of the work item. I immediately corrected the mistake and humbly replied with an apology for my typo. No big deal, right?
Well, Andrea replied to my brief apology later that day to inform me that she was NOT responsible for that work item because she had NEVER seen it before and that she would NOT do any work for it.
I was quite taken back by her response.
I let it go, but I couldn’t help but wonder why she would answer that way. Perhaps she was having a bad day? Perhaps her manager scrutinized all work items bearing her name? Perhaps the culture in her part of the company was toxic? Was my mistake that bad?
Even though this incident was small, it taught me one important lesson early in my career: a little bit of grace goes a long way. Poor reactions create awkward situations, hurt feelings, and wasted time. If we make a mistake, we should fix it and apologize. If someone else makes a mistake, we should strive to be gracious instead of unpleasant. I try to practice this myself, though, sometimes, I fail.