Painless assertions
A good test provides evidence of how the System Under Test (SUT) transforms inputs into outputs. We set up a test fixture, execute the SUT, and finally, we verify the results (structure known as Setup-Execute-Verify or Arrange-Act-Assert). In this article, I’ll focus on how to verify the outcome of the SUT and mock interactions in a simple and easy way.
Procedural State Verification
If there is a simple and easy way — what are the complex verification techniques? The most common one I encounter in tests is the so-called Procedural State Verification. The test compares each and every property of an object with some expected values. If the SUT returns a collection, each of the collection elements is checked separately:
val invoices = repository.findBy(customer)assertEquals(2, invoices.size())val invoice1 = invoices[0]
assertEquals(12345, invoice1.id)
assertEquals(0.toBigDecimal(), invoice1.totalAmount)val lineItem1 = invoice1.lineItems[0]
assertEquals(“fake SKU 1”, lineItem1.sku)
assertEquals(10.toBigDecima(), lineItem2.amount)// asserts continue…
As the verification part becomes longer, we extract cohesive verification blocks into Custom Assertions methods. We may think you are reducing the complexity, but actually, you are just moving it around from the test method to a helper method. Even worse, you are possibly introducing a test smell known as Mystery Guest. We are losing the relationship between the SUT inputs and the expected result — you see some strange values in assertions and you are asking yourself where the heck they came from?
Moreover, if the SUT starts initializing or updating an existing or newly added property, this new behavior won’t be caught by the test (as a verification failure, since most probably there were no assertions on that property). We expect from our test to sense unintended production code changes but it was not sensitive enough to detect them, leaving us with some untested behaviour.
Even test double specifications are not free from the perils of procedural verification. Concrete implementation of mock specifications using procedural verification depends on the mocking library. In the case of Mockito, it can be done either with argument matchers:
when(invoiceCreator.create(argThat { invoice ->
// property by property asserts on invoice...
}))or even in a more verbose way with captors:
val captor = ArgumentCaptor.forClass(Invoice::class.java)when(invoiceCreator).create(any()).thenReturn(receipt)
verify(invoiceCreator.create(captor.capture))
val invoice = captor.value// property by property asserts on invoice...
Expected object with built-in equality
How can we do better? The answer is simple — create an expected value (or a collection of values) and compare the SUT result to the expected value using one single equality assertion. It seems straightforward, but I still find on many occasions multi-line code performing property-by-property comparisons.
To do so, you need to ensure that the comparison target implements the equals method using all object properties. If there is no such method, just add it, including all class fields. Alternatively, to reduce the boilerplate, you can use a library (like Lombok or Immutable objects if you don’t mind an additional dependency and magic bytecode generation at runtime). Or you can switch to a language that generates equals out-of-the-box in some constructs, like Kotlin data classes or Scala case classes.
What if I cannot perform a full comparison with equals?
Hard-to-modify source code
You are working with a class that has already implemented equals for business equality. Or there is an external library that you cannot modify or it is legacy (if you change it there can be unexpected consequences). Then the only solution is to implement a custom assertion (whatever it means in your assertion library) — an example for assertk:
fun Assert<Invoice>.hasSameProperties(expected: Invoice) = given { actual ->
if (actual.id == expected.id
// and other properties ) return
expected(“Invoice: ${show(expected)} but was ${show(actual)}”)
}Custom equals is already implemented
If you cannot modify equals because it defines business equality, the only solution is to implement a custom assertion (as described previously).
Class constructor is not referentially transparent
Your constructor is referentially transparent if you can replace the constructor invocation expression with the object it creates. Seems straightforward, huh? What about this:
public Invoice() {
createdAt = Instant.now();
}This object is nearly impossible to compare since every time we create a different timestamp.
How to make it referentially transparent? Push the side effect up from the constructor. Either let the caller pass the concrete value:
public Invoice(Instant createdAt) {
this.createdAt = createdAt;
}or the lazy value:
public Invoice(Clock clock) {
createdAt = clock.instant();
}Equality pollution — really a smell?
You may ask yourself — do I need to implement equals only for testing if my production code does not require it? What if I’m following DDD and I need to deeply compare expected and actual Entities? According to the Wikipedia entity definition, an entity “is not defined by its attributes, but rather by a thread of continuity and its identity” and you actually want to compare all the attributes.
XUnit Test Patterns book talks about Equality Pollution as one of the causes of Test Logic in Production. Equality Pollution is defined as equals methods implemented only to support verifications in tests.
We have to distinguish between two situations. One, when you bend the equals definition to specific test requirements (you include only some attributes required to verify the object in a concrete test case). This is indeed a serious smell that must be avoided. It can lead in the best case to fragile tests (tweaking the equals implementation makes one test pass and other tests fail) and production bugs in the worst case.
Another situation is when the equality is not implemented yet. In this case, you have to decide how you want to define built-in equality:
- as business equality — as per entity definition, it includes only the attributes that define the immutable identity over time and entity transformations
- as data equality — by comparing all the constituent parts of the structure
These are two opposing forces pulling your code in two different directions. On one hand, in the production code you may be more interested in business equality (if there is any defined). On the other hand, in the test code, you are more interested in the data equality — you want to check if the SUT result has not only the same business identity but that it was transformed according to the SUT logic. If the business equality is not clearly defined or implemented yet (and maybe it won’t — YAGNI!), the most simple way to increase the testability is to add an equals implementation using all object properties in its definition.
Yes, you may consider it as the Equality Pollution, but the alternatives (custom assertions, test-specific subclasses) spare you little complexity on the production code side and increase the complexity in the test code. When facing this dilemma, I prefer to err on the side of testability.
Testability is sometimes a neglected requirement, raised mostly by developers. We tend to forget that engineers are stakeholders interested in the entrepreneurial success of project/product (we are all in the same boat!). By improving the testing ergonomics of the production code (enabling clear and robust tests) we are increasing the odds of product success. Testability can be improved not only by changing the high-level design (you do this frequently by introducing abstractions or indirections to test some behaviour in isolation) but by making our life easier on the lower level by carefully crafting the information bearers (objects and data structures).
All that I said about testing vs business equality and equality, in particular, loses its importance in languages supporting data classes as first-class citizens (Kotlin, Scala, Java 14 records). When using them, you get the equality implemented for free and you are pushed towards attribute-by-attribute verification in tests.
Note on test-specific subclasses
I mentioned briefly Test-Specific Subclass as a tool to implement equality just for testing. I don’t recommend this approach since it violates the symmetry property of the equals contract:
productionObject.equals(testObject)should returntrueif and only iftestObject.equals(productionObject)returnstrue
Since most probably production code is not implementing equals, the comparison with the test object will result in false. The opposite comparison (test object with the production object) will return true. It is possible albeit tricky to get it right, so better stay off of this.
Summary
- Compare the execution result with the expected value using one single equality assertion
- By default, use all object properties when implementing
equals - Don’t be afraid to add
equalsto increase code testability - Create custom matchers if adding
equalsis not feasible
