Skip to main content

Yet another interpretation of pyramid tests


In recent conversations regarding automating tests, It seems to me that the common understanding of the team is that the only acceptable automated test case is through the workflow tests through the UI. As far as I know, that’s not entirely correct and per “Pyramids of Automation” chapter 15 in More Agile Testing, I think there are lots of different ways we can automate test cases. I would like to take the opportunity to describe the pyramids of automation again with some thoughts/questions.

First, let me try to describe the original pyramid again, and here’s the picture that is worth thousands of words

Per the above pyramid from Mike Cohn with some modification in “More Agile Testing”, the lowest level, the unit tests, gets the fastest feedback on every commit which would provide the most benefits. The higher the tests located in the pyramid texts, the slower the feedback and lower profit we would get from the automated tests.

“The test automation pyramid is a model intended to guide teams in getting the most value from their test automation for the minimum investment and with the least overlap/duplication between each pyramid level.”

Let’s expand the above pyramid with some specific tools/context for our team

expanded pyramid tests per company context

With the expanded pyramid tests, we can see that we need to leverage different tools and techniques to do different automated testings at different levels

Given the above pyramids tests, I think there’s still 1 missing piece in the puzzle.

According to How to implement UI Testing without shooting yourselves in the foot, in order to avoid maintenance costs of UI tests, a very good idea when designing UI level functional tests is to think about describing the UI test and the automation at these 3 different levels:

describe UI tests at 3 different levels

“The stability in acceptance tests comes from the fact that business rules don’t change as much as technical implementations”

Technology moves much faster than business. The closer your acceptance tests are to the business rules, the more stable they are and the more clearer it is to the entire team (QA, Dev, PdM, etc…) on what the systems, products suppose to behave. The acceptance tests could be written in the form of unit tests, integration tests, UI tests, etc… The team usually would decide if they could automate the acceptance criteria and at what test level they want to automate the acceptance criteria.

As far as I can tell, the problems we often encountered with integration, UI tests are slowness in getting the feedback, hard to diagnose the problems when the tests fail (especially performance tests), flaky tests, etc… As a result, I feel that the more acceptance tests could be written in the form of unit behavior tests, the better alignment between the code and the business domain/behavior. This could lead to other benefits such as ubiquitous language for the entire team and design test-ability for the system/product and the better design skills of the team working in the system.

Based on all of the above information, I ask myself, “should we call unit tests as behavior unit tests instead of units tests?”. Yes there is some behavior that is at such a low level that people don’t care about, but there are LOTS of behavior unit tests that in HIGH LEVEL that different stakeholders SHOULD know about

My colleague (David) summarizes this article in the following key points and I couldn’t find a better way to put it together and here it is:

  • If we move our conversation around test to business rules we will all have more clarity and will decouple our conversations from the details. We will gain stability (because our terms don’t change with the technical details) and clarity because we focus on the actual goal
  • Thinking at the business rule level allows us to determine if that business rule can be tested at a level lower than the UI
  • Some business rules can be tested at the unit test level, in which case we should adopt the term behavioral unit test to help people comprehend that those tests are actually testing business rules.

References

Labels:

Comments

Post a Comment

Popular posts from this blog

implementing domain driven design summary

DDD overview anatomy of domain-driven design Putting the model to work Domain-Driven Design is an approach to the development of complex software in which we: Focus on the core domain Explore the models in a creative collaboration of domain practitioners speak a ubiquitous language within an explicitly bounded context. Bounded Context Explicitly define the context within which a model applies. Explicitly set boundaries in team organization, usage within specific application parts, and physical manifestations such as code bases and database schemas. Apply continuous integration to keep the model concepts and terms strictly consistent within these bounds, but don't be distracted or confused by issues outside.  Ubiquitous Language Use the model as the backbone of a language. Then, commit the team to exercise that language relentlessly in all communication within the team and in the code.   Use the same language in diagrams, writing, and speech within a bounded context. Recognize ...
Post a Comment
Read more

Code Retreat 2017 retrospective

  In general, this year is much better compared to previous years There are several important points that I need to write down Typically, the main reason for me to do code retreat to help developers in the community to expose more to TDD/craftsmanship. Recently, I asked myself question: “How come it’s not eas y  to adapt TDD, automated testing which should be the core value of any efficient teams?” I don’t think that people is lazy or people is resistance, I think as the industry, there’s lots of ambiguity in how people should approach TDD (SOLID principles, 4 simple rules of design, etc … are too generic) and agile practices (agile manifesto, etc…) How can I apply SOLID, 4 simple rules of design into my day to day jobs? What’s the most important reasons for developers to write automated tests? is it testability design, is it executable documentations?…. what’s the different between integration/component/API tests? should we care? when I discuss about unit tests with QA, typic...
Post a Comment
Read more

What’s a simple way to share and learn refactor?

  I have been organizing coderetreat and creating screencasts on refactoring and TDD, etc… however, I find it takes lots of time and effort to organize, preparing and facilitate. (it’s not that I complain about it as a matter of fact I love it every single second and I’m amazed at how much I have learned from the community) I have been asking myself, what’s the simpler way for me to learn to refactor and share with others? (why refactor, you may ask? it’s because it’s my favourite topic in the long never-ending list of agile technical practices) and for some reason, I really like to share my boring meme jokes, so I guess why not share a list of animated gifs of refactoring and the motivation of it. I guess since I’m an Asian guy (hopefully some kind of descendants from Bruce Lee) so this saying is pretty cool to me: “I fear not the man who has practiced 10,000 different kicks once, but I fear the man who has practiced one kick 10,000 times.” Another way to look at it, from J.B Rain...
Post a Comment
Read more