Menu

How to be an asshole when testing bugs

Published on December 10, 2019 and filed under Software engineering
Written
by Steve Tauber and will take

3 minutes

of your time.
In short
Knowing what to test is sometimes confusing for non-technical product managers that are forced to wear the Quality Assurance hat from time to time. Here are a couple different personas to take on when it's time to get your hands dirty.

Ugh… I can’t believe it! How did he find that?

This is my goal when I test for bugs. I want to find the little edge cases that developers haven’t thought about. I’m a product manager at madewithlove and at times I take on the role of Quality Assurance. This is especially important when building a customer-facing API. I think about testing from a couple of different perspectives to achieve my goals.

As my mom…

What would my mom do? Although she’s from the silent generation, she’s pretty damn tech-savvy. She mostly gets confused when she has to switch between applications or deal with the cleanup. I recently found out her phone was slow because she had about 900 open Safari tabs. This happened because she uses Google Chrome for mobile browsing, but when you open links from Gmail on the iPhone it will launch Safari.

I apply this to testing by not cleaning up after myself. What if there were 100 users in this list? What if instead of a correctly formatted photo, I upload a 20-megapixel behemoth? What happens if I copy and paste the wrong data into a field?

Most importantly, the documentation should match what the experience is. Minor inconsistencies here are very confusing for some users. For instance, is the correct AUTH token name referenced? Are the parameters correctly defined with a clear example?

As a developer…

Development is based on making things work within scope, within a certain set of prerequisites. What if those requirements are no longer true? For these types of logic bugs, I try to get the application into invalid states. What if my last name has s̨̕p͡҉҉e͟͞cia̸͘͞l c̷̵h҉̴̨a̴̸͏r̵á̴͠ćt̴͢e̶ŗ̛́s? What if my age is before Unix epoch? What happens if I delete a parent object from the system then create a child with the same name? Here’s a great example that has to do with accepting query params:

Expected: /users?name_filter=fre

Attempted: /users?name_filter[]=fre

The developer almost certainly expected a string instead of an array. What will happen to the app?

As an asshole…

Beyond testing success states, it’s important to try all sorts of invalid inputs when dealing with an API. I send Unicode, emojis, unescaped strings, single quotes, the words “false” and “null”, integers (positive and negative), and more. In the future, I hope that these types of issues are covered by property-based testing, but it’s something we haven’t tackled yet at madewithlove. What’s most important is that we are consistent with handling inputs. The Big List of Naughty Strings is helpful here.

Sending emojis and malformed requests seem rude but my goal is to help the end-user avoid inconsistent states. It’s important to try these things to have the best possible experience. I feel a bit guilty about returning a ticket for minor documentation changes or issues like this. I promise — I’m not trying to be an asshole.

Written by

Steve Tauber

Steve is an American living in Zagreb. He has experience as a senior programmer, project and product manager, and trainer.

Learn more about Steve Tauber

Related articles

Using MySQL for Testing (on Laravel)

Using MySQL for Testing (on Laravel)

Tony Messias

March 15, 2019

Never talk to strangers

Never talk to strangers

Zvonimir Spajic

January 20, 2020

Releasing with confidence

Releasing with confidence

Jonas Drieghe

December 18, 2019