opportunity

White Guy in a Kurta: My Passage to India

Life never gives us what we want at the moment that we consider appropriate. Adventures do occur, but not punctually.

From “A Passage to India” by E. M. Forster

Back in July of 2013, NetApp sent me on an awesome business trip to Bangalore, India. I had already planned to take a personal trip to China during a company shutdown, and I convinced my senior manager to send me to Bangalore on my way back to the States. The main reason for the trip was to help our offshore contractors build up their test automation skills for an upcoming release. While they benefited from my assistance, I likewise benefited from a much deeper appreciation for international workers. This article documents my experiences and the lessons I learned from my trip.

Transportation

My connecting flight from Kuala Lumpur (on Malaysia Airlines – gasp!) landed in Bangalore at around midnight, local time, on a Monday. NetApp provided a private cab for my whole stay. I felt a little bad because he spoke minimal English with a thick accent, but he was very friendly. I couldn’t see much on the drive to the hotel except a bunch of road blocks and a half-constructed highway. When he dropped me off at the Hyatt, I slipped him a few crumpled rupees my aunt had given me before my trip – she had visited India the previous year and implored me to always leave small tips.

Every morning that week, my cabbie picked me up from the hotel at 8am and returned me at 8pm. (The days were long.) He drove a crossover-like vehicle that felt like a minivan but drove like a small SUV, with flowers and Hindu statues over the dashboard. Even though the office was only a few miles from the hotel, a one-way trip took about 25 minutes. And I saw everything: “two-wheelers” (bikes with or without motors), “three-wheelers” (auto-rickshaws), “four-wheelers” (what we could call “cars” in the USA), pedestrians young and old, buses with people hanging on the side, and even cows just moseying through the streets. It was freaky to see not only everyone driving on the left side of the road, but to realize that the roads did not have any lane markings painted on them. In traffic, you just go with the flow, rather than obey specific conventions. Going to lunch one day, my “substitute” cabbie (not my typical driver) actually hit a two-wheeler, tearing off part of the cab’s bumper. Thankfully, the other guy just spat some pithy words in Hindi and sped away.

Traffic is chaos.  No lanes, few control mechanisms, and they drive on the left.

The view from the backseat of my cab: driver on the right, driving on the left (I think).

india-auto-rickshaw

The quintessential auto-rickshaw, driving rain or shine. And apparently, Bangalore has Taco Bell!

A cow at the bus stop!  Just chillin' there, no big deal...

Moo.

My Team

NetApp had a huge company site in Bangalore, but I spent all but one day at Mindteck, a nearby contracting firm hired by my organization. My team in Raleigh worked extensively with our group of Mindteck contractors. I felt delighted to meet them in person – they became more than broken voices and blurry profile pictures. And the team graciously welcomed me as one of their own. Every afternoon at about three or four o’clock, they tapped me to join them on the roof for masala chai (tea). There was never any shortage of Kinley water bottles, either. Embarrassingly, the team was so large that I struggled to remember who was who, but they never forgot who I was.

india-with-mukesh

Mukesh, one of my team mates from Mindteck, on the right. Me, in my kurta, on the left.

India Kinley.jpg

Kinley, the bottled water choice of champions! We call it “Dasani” in the USA.

During my visit, my team and I worked a lot. Every day, we had meetings, meetings, and more meetings, all about the “big A” – automation. Hearing my Raleigh team mates on the other end of conference calls felt like a strange reversal. I led automation workshops to teach our contractors a new framework we had developed in Raleigh. They also shared a number of problems with me, in the hopes that I could somehow help. Some were simple programming problems that took nothing but a simple proofread to correct. Others were out of my league. One evening, I sat locked in a conference room with an IT guy trying to figure out some obscure network failure, while two managers stared at us silently for almost two hours. I did my best to not feel awkward, assuming this style of management-by-pressure was cultural.

One problem I couldn’t solve for Mindteck was red tape. NetApp put a lot of restrictions on what the contractors could and could not do. They were restricted from accessing certain systems. NetApp’s Bangalore site also wouldn’t give them a direct network connection, which slowed down their network speeds to the point where test scripts would regularly crash from timeouts. Mindteck showed me their lab in disarray and admitted that they didn’t always get the hardware they needed for proper testing. Most frustratingly, I learned that the contractors were not even permitted to ask engineers on the parent team in Sunnyvale, California for help! In Raleigh, getting help from Sunnyvale was always a struggle, but at least we never had our hands slapped for asking. We all vented loudly about that ridiculous rule.

I spent only one day with the employees at the NetApp site. When I arrived, nobody was ready for me, so I poked around the cubicles until I saw plaques with names I recognized. Thankfully, everyone was friendly there, too. I don’t remember much of the NetApp site except meeting a few friends, seeing the Engineering Support center, and getting evening snacks at 6pm.

India NB Team.jpg

Lunch with NetApp employees in my organization. I wish this picture turned out better.

NB gets snacks every afternoon.  What are we doing wrong, Solomon Wube and Warren Campbell?

NetApp gave out snacks every day at 6pm. These were spicy!

The Hotel

NetApp approved only two or three hotels in Bangalore for travel. Arbitrarily, I chose the Hyatt, and they treated me like a king. Every manger, director, and VIP with a business card greeted me throughout the week. Bellhops always assisted me with luggage. Room service was always quick. And it was definitely a luxury hotel. Later, I learned that I was the first NetApper to stay there, and they sought to impress me in the hopes of future business from NetApp.

Exploring on my Own

The evenings were the only chance I had to explore Bangalore. As much as I wanted to hop an auto-rickshaw, I feared getting lost or scammed, so I stayed near my hotel on foot. Around the corner was a fancy shopping mall. The first store I entered was Big Bazaar, India’s version of Walmart. As a dumb American, my first mistake was attempting to enter Big Bazaar through the exit doors – a pair of armed guards quickly corrected my mistake by pointing me to the real entrance. Inside, I meandered through the store amazed by the essentials of Indian life, most of which were not much different from my own in America. When I tried to buy a few snacks from the grocery section, I made my second mistake by presuming the cashier would give me change – nope, I lost a few rupees. Before exiting through the proper exit doors, I detoured through the clothing department and bought myself a souvenir: a black kurta with red embroidery on the collar. I’ve since worn my kurta many times at Indian festivals in Raleigh.

BIG BAZAAR, India's version of a Walmart Super Center.

Big Bazaar – it was pretty big!

Across from Big Bazaar was a movie theater. Since I had nothing else to do, I moseyed over to see what films were playing, hoping to see a Bollywood movie with English subtitles. Instead, they had Despicable Me 2 playing, which I could not resist. The ticket plus samosa and Slice mango drink cost only a few dollars – a fraction of the typical movie cost in the USA. However, when I entered the theater, an armed guard who didn’t speak English forcibly patted me down. In public places, Indians took security very seriously. My coworkers told me the security was necessary to protect against Islamic terrorists.

Bangalore had such stark dichotomies. Near my hotel on MG Road were modern buildings with posh stores and fancy restaurants and security guards, while right around the corner were dilapidated houses with refuse burning on the street. Next to billboards for mobile phone plans were temples with innumerable statues. Two constants, though, were the crowds of people everywhere and, sadly, the pollution they left behind.

fruit stands are common

A mango peddler near Mindteck.

Hinduism

I passed many Hindu temples while riding around in my cab.

The Food

It’s a good thing that I loved Indian food. At work, my Indian counterparts showed much hospitality through their food. Every day, they provided lunch for me. One day, Mindteck catered a full lunch buffet as part of a meeting marathon. On the day I was on-site at NetApp, a friend bought me lunch in the cafe – 100 rupees, or about $1.50. For the remaining two days, both NetApp and Mindteck planned off-site lunches at delicious Indian restaurants. I ate it all: rice, naan, roti, curry of every color, chicken, lamb, paneer, aloo, palak, and gulab jamum for dessert. Nobody was surprised that I was “non-veg”, but they did appreciate my culinary adventurousness. For me, I was surprised to discover that, in India, eating with your bare hands is normal.

india-gulab-jamun

Gulab jamun, hot and fresh at an Indian buffet.

In the evenings, I had two fancy meals on my own. The best was my last night at the hotel. Since NetApp covered all travel expenses, I chose to eat dinner at the Hyatt’s fancy restaurant. It was totally empty, but that meant I received the best service. After serving my chaat and my curry, the head chef himself came out to greet me. The other fancy dinner was at a South Indian restaurant on the top floor of the nearby shopping mall – prawns, yellow-green curry, and round flatbread washed down with a Kingfisher. When the waiter served the food, he put the napkin on my lap for me. A third dinner I ate at the hotel’s standard restaurant while they played live music. On the night I went to the movies, I ate a Chicken Maharaja Mac at McDonald’s – let’s just say the Big Mac is better with beef.  (Come on, I’m a red-blooded American.)

Southern Indian curry.

The South Indian dinner at the fancy shopping mall.

These were the best shrimp I have ever eaten.  Indians always call them "prawns."

Some of the best “prawns” (not shrimp) I’ve ever eaten.

The Maharaja Mac is made using chicken, since it is illegal to kill cows.  The combo cost about $3.25.

The Chicken Maharaja Mac Meal. #MURICA

Goodbyes

On my last day in Bangalore, I wore my kurta. As I left in the morning, the front desk lady at the Hyatt shouted out, “Very nice kurta, Mr. Knight!” My Mindteck team planned a proper sendoff in the afternoon. We took group photos, and they gifted to me a small desk clock that I keep to this day. And they offered one final treat – an ice cream bar. There were six small scoops of various colors, and I had to taste each to guess the flavor. To everyone’s surprise, I got them all right! I didn’t have the heart to tell them that I suffered from lactose intolerance. My belly ached during the whole flight home, but the experience was well worth it.

My flight out of Bangalore departed in the wee hours of the morning. My cab picked me up from the airport shortly after midnight. I flew to Qatar, then NYC, and finally to Raleigh for a nearly day-long transit. My biggest regret was not having enough time for sightseeing in India.

india-mindteck-team

The Mindteck team and me: coworkers and friends.

 

Lactose intolerance in India sucks.  I ate the ice cream anyway.  These flavors are blueberry, mango, baked apple, jackfruit, lychee, and watermelon!

The ice cream bar, from left to right by row: strawberry, banana, lychee (?), jackfruit (?), mango, and blueberry.

Lessons Learned

I gained a genuine appreciation for our offshore teams after my trip. Oftentimes, I hear American workers complain that offshore workers are “stealing jobs,” and “you get what you pay for” in quality of work. But I learned firsthand that these folks are good people just trying to make a living. They aren’t trying to steal our jobs – they’re taking the opportunities in front of them as part of a global economy. Frankly, big companies hire offshore resources so they can throw more bodies at their projects. And quality suffers greatly when any team is not co-located due to challenges with communication, time zone, technology, bureaucracy, etc. Moreover, our guys and girls wanted to be more than just hired help: they sought to learn more, improve their skills, and increase their contribution.

Overall, I was very grateful for the opportunity to visit Bangalore, and I hope to return some day!

 

This post is dedicated to all of my friends from India.

10 Things You Lose Without Automation

Automation has a lot of potential to improve software development. Unfortunately, though, automation is often seen as a luxury. Deadlines in the real word are unforgiving, and since test code isn’t product code, automation tasks are given lower priority and dunked into the black hole of the backlog. Some might argue that this is okay because it is lean or because a new project is just getting started. Once, I even heard it quipped that the first ones cut during a layoff are the automation folks. And it is true that automation requires a nontrivial resource investment.

However, I want to turn the tables. Instead of thinking about automation in terms of the opportunity, think about automation in terms of the opportunity cost. What happens if you don’t automate your tests from the get-go?  There are 10 major things you lose:

#1: Man Hours

Automated tests will automatically run.  Manual tests must be manually run.  That’s ontological.  If you only run a test one time, then automation has no return-on-investment.  But if you run a test more than once, automation saves a tester from repeating themselves. Plus, it’s easy: push the button and wait for results. Automated tests almost always run faster than manual tests, too.  Considering that time is money and engineer salaries aren’t cheap, man hours are a clear opportunity cost.

#2: Coverage

Automated tests can achieve greater coverage than manual tests, particularly for regression testing. As product development progresses, the sheer number of test cases increases. For example, in Agile, new tests will be created every sprint. Older tests must be run periodically to verify that new features don’t break existing features. If regression tests are manual, then testers must burn hours grinding through the same tests repeatedly.  Often, for expediency, this means that they skip some tests – not in the sense of being lazy, but rather as part of a risk-based approach.  Weaker coverage plus risk of missing bugs are accepted for the sake of shorter testing time.  If those regression tests were automated, then there would be no reason to shrink coverage, because they would be easy to run.

#3: Consistency

People make mistakes. It’s human nature – nobody’s perfect. And manual tests are prone to human error because humans run them. I remember how nervous I felt running manual on-call system checks at MaxPoint for the first time, afraid that I would miss a problem that could bring down a million-dollar bidding system.  Automated scripts run the same way every time.

#4: Protection

Continuous integration (CI) protects code against defects by building and testing every code change in real time. A CI system will automatically trigger tests all the time.Tests not running in CI (like manual tests) are effectively dead. At NetApp, failing code changes would immediately be kicked out of the code line, making automated tests act like a vaccine against bugs. On the other hand, I remember a project at MaxPoint that was riddled with bugs and perpetually delayed. When I asked the developers to see their unit tests, they said they never wrote unit tests because “it wasn’t a requirement.”

#5: Delivery Time

Continuous delivery (CD) is the natural extension of continuous integration, in which software products can automatically be delivered (and potentially even deployed) as the final step in a CI pipeline. This is how big companies like Google, Facebook, and Netflix can deliver so rapidly. No automation means no CD.

#6: Results and Metrics

Non-engineers (managers, product owners, scrum masters, oh my!) love to ask questions about tests.  “Are we red or green?” “How many tests do we have for this feature?” “What’s our coverage?” “How often do we run the tests?” Automated tests simply yield more accurate and more comprehensive results. Automation can also generate test reports, so engineers don’t need to waste time drafting emails or updating wiki pages.

#7: Accountability

Numbers don’t lie. Scripts don’t lie. Engineers typically don’t lie, but… results from manual tests can have a fudge factor, or a mistake in reporting, or any other sort of inconsistency. Inaccurate results may lead to poor business decisions. Automated results tell it like it is.

#8: Creativity

Manual testing can devolve into repetitive, menial labor: just follow steps 1-10 again and again and again. It would be much more effective for manual testers to focus on exploratory testing rather than deterministic testing. While automated tests can cover the fixed, repetitive test scenarios, exploratory testing lets testers find creative ways to uncover defects and judge how well a product actually works. Lack of automation ties up human capital.

#9: Peace of Mind

Are you sure that your product is “good”? Can you run enough tests to make sure? I learned the value of peace of mind while I was still in college. In my compiler theory course, I had to develop a simple programming language and build a compiler for it. Every week, we had to add new language features: arithmetic, strings, arrays, functions, etc. And every week, I wrote a slew of mini-programs to test grammar updates to my new language. By the time the project was complete, I had 1000+ automated test cases running through JUnit with 100% coverage, and the entire suite took a mere few minutes to run. And there were many late nights when the tests caught bugs in my language right away before committing code. There was no way I could have passed that class without my automated tests.

#10: Quality

The ultimate purpose of test automation is product quality. Having automation doesn’t necessarily mean product quality is good, but not having automation severely limits how quality can be pursued. Anecdotally, I’ve seen much better code quality come out of projects that have good test automation than ones without it. If I were a product owner, I know what I would want.

My Choice to Automate

Why am I an automation engineer? The answer became starkly clear to me one Tuesday morning that should have been like any other. At that time, I was a “Software Quality Engineer” on the QA Framework team of a fairly small software company. I arrived at the office at my normal time around 9am. The following two hours I spent in my daily groove with my headphones on, blissfully unaware as our director started escorting engineers out the door with boxes in hand. After an interrupted 11am standup, I found myself, heart racing, in a conference room with my manager and the VP of engineering who had flown in from our other site. On the table, I saw a pile of key fobs and bejeweled company phones. By this point, anyone who knows the software industry could predict what would happen next: the dreaded layoff. However, that’s where the story becomes more interesting.

I wasn’t laid off. Surprise!

In a daring reorganization, the VP eliminated all QA positions in the company. The justification given was to make teams purely Agile with no silos of division between development and quality. Everybody would own quality. Manual testers would be removed in order to prioritize automation. I also suspected that company finances may have encouraged the re-org. As a result, people lost their jobs. However, a few QA engineers, myself included, were spared the layoff and rebranded as regular “software engineers.” In that conference room, as the adrenaline rush subsided, the VP outlined a high-level plan in which I would move to a product development team that desperately needed automation help. Apparently, this team didn’t even have unit tests, and their product was weeks behind schedule. Once they got it going, I would become a regular web developer with the others.

So, why me? The survivor’s guilt bothered me, but the answer, apart from God’s grace, was pretty obvious: automation skill. I was the company’s automation champion, and everybody knew it. I consulted with each team to help improve their testing efforts. I gave a lunch-and-learn on behavior-driven development. I wrote several wiki pages and even example projects. Now, that’s not to brag on myself or belittle the skills of others, since many of the engineers who were cut also had automation talent, but my skills had been made highly visible to the people in power. And my skills continued to afford me great opportunity.

However, the last part of the VP’s new plan disconcerted me – did I want to become a standard web UI developer? I pondered the question deeply, but my gut answer never changed: no. Test automation was my specialty. It was the product I made, and I made it well. I knew the ins and outs, the frameworks, the best practices, the fundamental problem of test automation and the lowercase problems that derive. And I loved doing automation because it gave me the opportunity to set things right: to find an eliminate bugs, to protect the code line, and to make it look damn good. I could indeed switch over to web dev, but why? I didn’t have interest in web dev. That would be like making a hardwood floor installer do painting instead. With automation, I had both a specialty and an opportunity.

To conclude the story, I didn’t stick around much longer at that company. I quickly found a better opportunity as a senior-level automation engineer on an exciting new project at a different company. That’s the software industry – so it goes! Ironically, that layoff may have impacted me just as much as if I had actually lost my job. It forced me to make a choice. I definitively chose to be a software quality engineer. Automation was no longer merely a skill set but a career identity.