Skip to main content

Yet another interpretation of pyramid tests


In recent conversations regarding automating tests, It seems to me that the common understanding of the team is that the only acceptable automated test case is through the workflow tests through the UI. As far as I know, that’s not entirely correct and per “Pyramids of Automation” chapter 15 in More Agile Testing, I think there are lots of different ways we can automate test cases. I would like to take the opportunity to describe the pyramids of automation again with some thoughts/questions.

First, let me try to describe the original pyramid again, and here’s the picture that is worth thousands of words

Per the above pyramid from Mike Cohn with some modification in “More Agile Testing”, the lowest level, the unit tests, gets the fastest feedback on every commit which would provide the most benefits. The higher the tests located in the pyramid texts, the slower the feedback and lower profit we would get from the automated tests.

“The test automation pyramid is a model intended to guide teams in getting the most value from their test automation for the minimum investment and with the least overlap/duplication between each pyramid level.”

Let’s expand the above pyramid with some specific tools/context for our team

expanded pyramid tests per company context

With the expanded pyramid tests, we can see that we need to leverage different tools and techniques to do different automated testings at different levels

Given the above pyramids tests, I think there’s still 1 missing piece in the puzzle.

According to How to implement UI Testing without shooting yourselves in the foot, in order to avoid maintenance costs of UI tests, a very good idea when designing UI level functional tests is to think about describing the UI test and the automation at these 3 different levels:

describe UI tests at 3 different levels

“The stability in acceptance tests comes from the fact that business rules don’t change as much as technical implementations”

Technology moves much faster than business. The closer your acceptance tests are to the business rules, the more stable they are and the more clearer it is to the entire team (QA, Dev, PdM, etc…) on what the systems, products suppose to behave. The acceptance tests could be written in the form of unit tests, integration tests, UI tests, etc… The team usually would decide if they could automate the acceptance criteria and at what test level they want to automate the acceptance criteria.

As far as I can tell, the problems we often encountered with integration, UI tests are slowness in getting the feedback, hard to diagnose the problems when the tests fail (especially performance tests), flaky tests, etc… As a result, I feel that the more acceptance tests could be written in the form of unit behavior tests, the better alignment between the code and the business domain/behavior. This could lead to other benefits such as ubiquitous language for the entire team and design test-ability for the system/product and the better design skills of the team working in the system.

Based on all of the above information, I ask myself, “should we call unit tests as behavior unit tests instead of units tests?”. Yes there is some behavior that is at such a low level that people don’t care about, but there are LOTS of behavior unit tests that in HIGH LEVEL that different stakeholders SHOULD know about

My colleague (David) summarizes this article in the following key points and I couldn’t find a better way to put it together and here it is:

  • If we move our conversation around test to business rules we will all have more clarity and will decouple our conversations from the details. We will gain stability (because our terms don’t change with the technical details) and clarity because we focus on the actual goal
  • Thinking at the business rule level allows us to determine if that business rule can be tested at a level lower than the UI
  • Some business rules can be tested at the unit test level, in which case we should adopt the term behavioral unit test to help people comprehend that those tests are actually testing business rules.

References

Labels:

Comments

Post a Comment

Popular posts from this blog

growing object oriented software, guided by tests summary

Key ideas practice TDD as a design technique and not as a testing technique Software development as a learning process. Expose uncertainly early by testing our assumption as early as possible we use mock objects to identify the essential interactions between modules/objects. This leads to more effective abstractions, which reduces the cost of changing the cost safely over time. Stub queries; expect actions??? Write code that depends as little as possible on its context. programming by intention Test-Driven Development (TDD) and Quality Software development as a learning process. The best approach a team can take is to use empirical feedback to learn about the system and its use and then apply it back to the system. Development is incremental and iterative. Incremental development builds a system feature by feature. Each feature is implemented as an end-to-end slice. As a result, the system is continuously integrated and ready for deployment. A system is deployable when the acceptance t...
Post a Comment
Read more

How to Win Friends and Influence People summary

Principles Fundamental techniques in dealing with people Don't criticize, condemn or complain. This makes them defensive and justifies their actions. Instead, praising them lowers their defense and they will be more receptive to your feedback. give honest and sincere appreciation arouse in the other person an eager want. become genuinely interested in other people i.e.: remember other people's birthdays send linkedin messages of congratulations when someone in your network gets a promotion or professional success coordinate office parties for personal celebrations such as births, accomplishments or birthdays. people crave the feeling of importance. "What is there about him that I can honestly admire?" appeal to the other person's interests. Virtually all people care more about what they want than what you want keep asking yourself - "what is it that this person wants?" everyone has something they can teach you, and you benefit by figuring out what that i...
Post a Comment
Read more

Code Retreat 2017 retrospective

  In general, this year is much better compared to previous years There are several important points that I need to write down Typically, the main reason for me to do code retreat to help developers in the community to expose more to TDD/craftsmanship. Recently, I asked myself question: “How come it’s not eas y  to adapt TDD, automated testing which should be the core value of any efficient teams?” I don’t think that people is lazy or people is resistance, I think as the industry, there’s lots of ambiguity in how people should approach TDD (SOLID principles, 4 simple rules of design, etc … are too generic) and agile practices (agile manifesto, etc…) How can I apply SOLID, 4 simple rules of design into my day to day jobs? What’s the most important reasons for developers to write automated tests? is it testability design, is it executable documentations?…. what’s the different between integration/component/API tests? should we care? when I discuss about unit tests with QA, typic...
Post a Comment
Read more