Quality is delighting customers
Software teams often wonder if a feature is worth testing or if a manual test is worth automating. However, engineers rarely ask themselves what a test teaches others about the application. It’s important that test authors keep in mind the inherent authority their tests possess. An application’s tests are sometimes the first lines of code a new developer will read when acclimating to a new codebase. Always remember that tests have the power to guide, but they also have the power to mislead.
Even a well-written test can be damaging when written for the wrong reasons. I was doing a code review for a coworker recently when I caught something in one of their integration tests that seemed out of place. The review was for a new API endpoint: a simple GET method that returned a list of all saved messages for a user. The test that caught my eye was titled “Should return a 404 if the user has no saved messages.” This seemed odd to me.
Assuming the parent entity existed (in this case, the user), I expected any API that returned a collection on success would return an empty collection if no items were found, not a 404. My coworker agreed with my intuition but said that the acceptance criteria for the story were vague, and the endpoint currently returns a 404. There was a story in the backlog to update the endpoint, but for now he was testing that it worked as implemented.
My coworker argued that the test should validate current functionality and that they could go back and fix it later. The tests were being run automatically, so if the functionality changed, we would find out when the tests started failing. And besides, if anyone had questions about the API testing they could search the software testing forums. .
Please share your views also: