7 non-obvious benefits of automated testing

UpdatedOctober 26, 2020

•5 min read•View as Markdown

7 non-obvious benefits of automated testing

Mario Cervera

Comments (8)

Join the discussion

Reza Lavarian5y ago

Automated tests play the role of code samples.

I'd like to emphasize this sentence, code samples are super helpful.

It even affects the whole design of a feature. Especially, when developing the API, first, and then the implementation, like how we do in TDD.

Good writing style, by the way. Thanks!

Mario Cervera5y ago

Thanks Reza! 🙏

Yes, that's an important benefit. When we write tests, we are effectively using our APIs. If the design is bad, our tests will suffer (e.g. they will be less readable). This is one of the ways testing has a strong positive impact on design.

Darren Finch5y ago

This is a terrific article! I like the writing style and the content.

Honestly I need to establish a better unit testing habit. I don't unit test my personal projects often because TDD feels foreign.

Which also leads me to ask, did TDD feel foreign to you at first?

If so, how did you adapt?

Mario Cervera5y ago

Thanks for your kind words Darren! 🙏

Be careful with TDD. Let me explain. It is a hard practice to master. You need to be good at testing, design, and refactoring, among other things, so it's definitely not the first thing you want to learn when you are inexperienced.

I'd say that you should develop the habit of writing automated tests first, and, eventually, you can try the TDD process, to see if it's a good fit for you.

That's more or less what I did. I didn't try TDD until I was confident with my design and testing skills.

I hope this helps!

Deactivated User5y ago

Thanks Mario, great article!

Tests help you detect errors early, when they are cheapest to fix.

Agreed 100%. The sooner you fix it, the more time you will save in the future.

Mario Cervera5y ago

Thanks Nick! I am happy that you like it 😊.

Yes, short-sighted people usually argue that testing slows the team down, but they fail to consider the savings that come in the future!

Fortune5y ago

I have a few questions?

Is there such a thing as "over testing"?
What is the industry benchmark for percentage coverage?

Mario Cervera5y ago

Yes, there's such a thing. Your time is not infinite, so you cannot possibly test everything or automate everything. You want to automate tests that prove different things, so that they are more useful. For example, to test a "multiply" function, it is probably better to use as test inputs (3,0), (7, 16), (-2,8), and (1242362234, 35674563), which prove different things, than using (1, 1), (2,2), (3,3), and (4,4), which leave some corner cases untested.
No, there is no benchmark. The ideal is 100%, but it is unattainable, and, at the same time, anything less than 100% can be considered bad (because you want a 100% of your app to work, right?). My opinion is that you have to use code coverage as a "motivation metric". That is, always try to improve it, knowing that you won't reach 100%, but the higher the better.

Fortune5y ago

Mario Cervera Thanks for your reply

Matt Moll5y ago

Amazing as always Mario! Looking forward to a nice post about clean code and more examples about testing!

Thanks for sharing!

Mario Cervera5y ago

Thanks Matt! Those posts that you mention will come, eventually 😉

Maxi Contieri5y ago

Amazing as usual, Mario !

Mario Cervera5y ago

Thanks Maxi! I really appreciate it! Getting such positive feedback, no doubt that I will write more articles soon 😉

Apoorv Tyagi5y ago

Nice Article Mario Cervera. Enjoyed reading every point you have mentioned

Very Clear & concise points👍

Mario Cervera5y ago

Thanks Apoorv! Glad to hear that you liked it! This kind of feedback encourages me to write more posts in the future 😊

Jov Mit5y ago

That's an awesome article, I have to admit that I loved it a lot.

Really nice writing style, easy to read and understand. No complex sentences and very good arguments. It would be of a great benefits for the community around you if you keep posting such good content with this writing style.

Little remark about the following:

through the use of test doubles, also known as mocks

I talk about that in my last weekly video on youtube 👇, and basically there is some history of why the term Mocks is used in generic context to refer to all Test-Doubles, but the actual mock is a kind of test-double just like the stub, dummy, spy and fake.

Would be interesting to see your opinions on that especially after watching the video: https://youtu.be/YQ9qlcq6Yyg

Mario Cervera5y ago

Thanks Jovche for the feedback!

I am aware of the different types of test doubles. I typically use the terminology that is presented in the "xUnit Test Patterns" book. But, in this article, I used the term "mock" because it's well-known by many people, despite the fact that, as you rightly suggest, it is just yet another type of test double.

You can expect my feedback on your video soon 😉

Jov Mit5y ago

It's lovely to see we are on the same page :) Indeed the term is widely used and know in the community 💯👌

More from this blog

Is Software Engineering Really Dead in the AI era?

Is Software Engineering really dead in the Artificial Intelligence (AI) era? This question is both provocative and widely debated. It is generating a great deal of discussion on social media and among software developers in general. In this post, I s...

Feb 3, 20268 min read

Is Software Engineering Really Dead in the AI era?

Key Challenges of AI-Assisted Software Engineering

Recently, I published a post on accelerating software engineering with the help of Artificial Intelligence (AI). In that post, I shared my team’s hands-on experience with AI agents, exploring four scenarios where we could easily achieve a significant...

Jan 6, 20267 min read

Key Challenges of AI-Assisted Software Engineering

Leveraging AI to Accelerate Software Engineering

Over the past few months, my team and I have been making extensive use of Artificial Intelligence (AI) – specifically Cursor IDE – in our day-to-day work. It started as a small experiment, but it quickly evolved into a deeper exploration of how Large...

Dec 12, 20257 min read

Leveraging AI to Accelerate Software Engineering

Characterization testing: adding tests to legacy code

Some people feel uneasy when they test-drive code, so they favor the traditional workflow where testing is an after-development activity. Other people, on the contrary, believe that adding automated tests after development is more challenging, so the...

Nov 30, 20257 min read

Characterization testing: adding tests to legacy code

Common test smells

Most of us are familiar with the problems exhibited by the systems that contain design smells (understanding the term “smell” as defined in Martin Fowler’s book Refactoring: Improving the Design of Existing Code). Some of these problems are: The sys...

Sep 21, 20216 min read

Mario Cervera's Blog

12 posts

Software craftsman. I write about Software Engineering (clean code, design, testing, ...) and also about Algorithms & Data Structures.

I started exploring the fascinating world of test automation seven years ago. Right from the start, it was clear to me that testing has important benefits. Anywhere I read back then, I found people describing how testing leads to savings in development time and more robust software, among other things.

However, after all of these years of practice, I have learned some benefits that were less obvious to me when I started.

In this post, I share them.

Maybe, when you read this post, you think that these benefits are basic and well-known. That’s fair. My claim is just that they were not obvious to me when I was writing my first automated tests.

My lack of awareness at that time is the main motivation behind this post. If I can make only one person discover these benefits quicker than I did, then my goal will have been achieved. If I can convince only one unconvinced person of the usefulness of testing, then this post will have gone far beyond my initial expectations.

1. Tests give you code samples.

Have you ever skipped an answer in StackOverflow because it didn't contain a code sample?

We look for code samples because they help us understand how things work.

Automated tests play the role of code samples. Each test represents an example of how the system is used at the code level; therefore, they are of invaluable help when we are trying to understand the system better.

A software system is easier to understand if it has automated tests in place.

2. Tests give you executable specifications.

Written documents become obsolete easily, so they often lie. They specify what the system is supposed to do, not what it really does.

The only truth about the system behavior is in the source code. The code brings this behavior to life and makes it possible; the tests specify, document, and enforce it, in a formal and unambiguous way. Therefore, automated tests are executable specifications of the behavior of the system. If this behavior changes, the tests will fail. If we want to change this behavior, we must adapt the tests.

Tests specify how the system actually behaves and they are always up to date.

3. Tests give you the first users of your code.

When you are writing tests, you become a user of your own code. If the code is bad, you are the first to experience the problem. This makes you put more effort in refactoring, and in writing clean code and clean tests.

When you want your tests to be clean, you will choose better names for your variables, functions and classes. This improves the readability of your tests, which, in turn, improves the readability and design of your code, which, in turn, makes testing easier. It is a self-reinforcing loop.

A good testing strategy leads to cleaner code.

4. Tests give you immediate feedback about code changes.

Have you ever felt the pleasure of misspelling a variable name and getting immediate feedback from your IDE?

Your IDE warns you about this kind of errors because they are syntactic issues that can be detected via static analysis at compile time. But, you can get the same kind of feedback about runtime errors. You only need fast tests that you can run after every change in your code.

For example, if you replace “+” with “-” by mistake, you will change the semantics of the code and therefore its runtime behavior. If the tests are semantically stable, they will detect the error immediately.

Automated tests shorten the feedback loop on coding decisions.

5. Tests prevent the occurrence of bugs.

Having quick feedback on coding decisions has a significant consequence: bug prevention.

Tests specify the behavior of the system. As long as the tests are there, this behavior is preserved. If, sometime in the future, you change this behavior unintentionally, the tests will catch the error. This error, which might have been detected months later, has a lifetime of less than a few seconds.

Tests help you detect errors early, when they are cheapest to fix.

6. Tests give you a safety net.

With a good suite of tests in place, you can modify code, run the tests, and immediately know whether you altered the system behavior. In other words, you can modify code safely.

This is where the safety net metaphor comes from. Automated tests act as a safety net that allows us to refactor confidently, in a similar way to trapeze artists to perform without fear of being hurt. But, nets can contain holes through which we may fall. These holes take the form of untested behaviors, where potential bugs may hide.

Automated tests protect us from unexpected, and potentially harmful, events. But, test suites must be comprehensive, if we want them to meet this goal effectively.

7. Tests have positive architectural implications.

To test a unit of code properly, we must isolate it from its dependencies. This is typically accomplished through the use of test doubles, also known as mocks.

When the code is highly coupled and dependencies are hard-coded, isolation becomes difficult and testing sometimes prohibitive. This implies that, the more loosely coupled a system is, the more of it can be verified in terms of unit tests. In other words, the lower the coupling, the higher the testability.

Automated tests lead to low coupling and higher-quality design.

Conclusion

If we consider all of these benefits together, we can reach an important conclusion:

Automated testing dramatically increases software quality.

And high quality is the only way we can have:

Ability of respond to change quickly.
Huge savings in time and money.
High customer satisfaction.
A motivated development team.

In this new era of uncertainty that we are currently living, all of these advantages may be more necessary than ever before.

Command Palette

Comments (8)

More from this blog

1. Tests give you code samples.

2. Tests give you executable specifications.

3. Tests give you the first users of your code.

4. Tests give you immediate feedback about code changes.

5. Tests prevent the occurrence of bugs.

6. Tests give you a safety net.

7. Tests have positive architectural implications.

Conclusion