Sailboats and Submarines, Voyages in Software Testing

The USS Dallas is tracking the soviet submarine, code-named Red October. This classic 90’s Jack Ryan film always has me captivated by the capabilities and tactics of submarine warfare.

Growing up in Southern California allowed me the great pleasure of being a short bike ride away from Redondo Beach. I would spend time with friends looking out at the ocean and watching sailboats cruise the blue waters between shore and Catalina island.

It’s interesting to consider that a voyage to Catalina island via sailboat would be very different than traveling by submarine? Each vessel offers different contexts and capabilities to embark upon the venture.

A journey by sailboat would allow us to skim the ocean’s surface, experiencing the cool ocean breeze, sun shining above. We would observe our destination miles ahead, well before we ever arrived.

A submarine would yield a much different journey. As we entered through watertight doors, our crew would busily prepare to submerge the depths of the sea. Seated on the bridge, we observe glowing red lights while computer screens display sonar images of what lay ahead, behind, and below.

Likewise, the world of software testing also provides many opportunities for execution and exploration. Each Test Engineer comes to similar decision points, choosing to skim the surface or journey to the depths of the application under test. It’s possible to find ourselves hypnotized, glazed over eyes, focusing only on the destination and not on the journey we’re on.

What hypnotizes the Test Engineer into paralysis, where they lose the drive to ask, “Could I test things better?” What leads them to become so focused on the destination that an encounter with the Lockness Monster wouldn’t raise their suspicion that something strange was lurking below?

While the default mindset for many is to approach testing from the Sailboat perspective, I want to challenge you to desire and explore more from the test you write, automate and execute.

Paired Test Planning

What if we began with a way to help the Test Engineer take on a new mindset? One of my favorite things to do on a journey is to be the passenger and navigator of the road trip. When I drive, I focus on my speed (my wife might disagree) while keeping an eye out for obstacles. Sure, this allows me and my passengers to have a safe journey but not much exploration along the way. What if we put into place a Paired-Test-Planning approach? We could put one Test Engineer in the driver seat, following the designated course while the navigator looked ahead, making route corrections past points of interest where the good bugs are always hiding.

Engage the Hyperdrive

Understanding the technology that makes up the software under test would mean we are testing from more of a Whitebox perspective than merely hoping things are the same under the hood as they’ve always been.

A Tool by Any Other Name

Tools, along with technologies, can be a blessing and curse to any technologist. They can easily distract from achieving primary objectives while also providing a greater understanding of how things are working along with way. Take Google Chrome, Developer Tools as an example. This powerful toolset is built into Chrome and offers details for page metrics and asset availability. At the same time, it can be like giving a Five-year-old a garage of power tools and expecting them to build a go-cart. Tools are fantastic when used in a balanced approach.

Consider carefully how your team adopts and gets distracted by tools. It’s easy to justify adding one more tool that clutters up the tester’s toolbox. Some wise Quality Engineers I met at a Google testing conference shared this advice, “Reduce your tools down to a single one that accomplishes the task well.” Put this advice in practice if you have tools that do nearly the same thing. Find one that gets the job done and standardize on it. Create an online reference of the tools your team has standardized on and identify a subject matter expert for that tool if someone needs help.

Permission to change Perspective

Have you ever considered that there is no official school of Software Testing? The majority of Test Engineers learn the Craft of Testing by learning from another Test Engineer. I’ve worked with hundreds of Test Engineers and have encountered a wide variety of styles and values to their testing. Do you recognize any of these on your team?

Casual Carl – This Test Engineer will typically take the path of least resistance, getting to the destination through the easiest way possible. They often live in a world of ad-hoc testing and can find a million and one reasons why they can’t create or maintain their test plan and test cases. Provide measurable opportunities of growth for your Casual Carl’s along with expected timeframes to grow in these disciplines. Casual Carl’s can impact the reputation and brand of your testing organization, so have a plan on how long they can be part of your organization before they need to find a different seat on the bus.

Perfect Polly – This Test Engineer needs to have everything documented before they test anything. I love the heart of this Test Engineer, but they need to find a balance of keeping their test assets manageable and accurate while not becoming obsessed to the point of paralyzation. Encourage Perfect Polly’s to have a more organic approach to creating and maintaining test assets. The METS testing approach (METSTesting.com) is a perfect choice for these Test Engineers to build a living Test Plan.

Tester Tom – This Test Engineer allocates time to keep their test plan and resources healthy. You want most of your team to be in this place if possible. These Test Engineers can execute the Test Plan while knowing just the right amount of exploratory testing needed to reduce risk in the timeframe given.

Given the Tester types above (and of course, there are many others), encourage time to step back and consider how to bring new kinds of testing to their applications. Discuss the concept of what else to validate along the course of the test case.

Conclusion

Find ways to encourage balanced, creative approaches to increased test coverage. Raising confidence that the application is beyond functioning and delivering the experience the business wants the customer to encounter.

Bon Voyage!

Greg Paskal