Whenever I teach courses that cover testing there’s always something that niggles me and my most recent Dev 450 was no different.
My problem is with the implied behaviour of the
@testSetup annotation in test classes. In particular I always worry about how it interacts with governor limits. So I thought I had better get to the bottom of it in the vain hope that I’ll get a better night’s sleep.
Firstly a quick reminder about what
@testSetup does for us. The idea is that is allows us to provide a method to the unit test engine which is executed before any of the unit tests in a particular class. This allows us to setup the data required for the tests in one place rather than repeating it in each individual unit test.
Check Transaction Boundaries
The next thing to do is to realise that most governor limits are based on a transaction and I therefore need to establish how
@testSetup works with transactions. The easiest way to do this is to make use of the fact that static variables are transactionally scoped. So, we declare a static variable, change it in our @testSetup method and then see what value it has in an actual test method. If the @testSetup method and the test method are in the same transaction then we should see the updated version of the variable in the log. Easy. Here’s the code for that:
Looking at the above if
aTestMethod1() are executed in the same transaction then I am expecting the debug output to be the word setup. If not then it should be init. Here is the debug log generated when I ran that test:
So the evidence says that they are executed in two unique transactions. That is great as it means we now know how the governor limits will work, with both
aTestMethod1() having their own set of limits because they are their own transactions. All that is left to do is verify that.
Verify New Governor Limits
We can verify this pretty quickly, all we need to do is modify the test class that we have to look like this:
What’s going on here? In the
setup() method we execute a DML statement and then a SOQL statement. We then make use of the
Limits class to get a report back from Salesforce as to how many DML and SOQL statements we’ve used. We expect these to each show 1 – after all we’ve done one of each. Then in the
aTestMethod1() method we only call the same methods on the
Limits class again to prove that we’re in a new transaction and therefore have a new set of limits to work with. In this case the output should both be zero. Here are the results:
Hold on. Wait. This is not what we were expecting. The two
TM1 debug statements are showing 1, not 0 as we would have expected. However our static variable still shows that we’re in a different transaction.
What Does This Mean?
The two tests here contradict each other, with static variables behaving like they are in two transactions whilst governor limits behaving as if they were only in one. Something here smells wrong to me, surely it needs to all behave one way or the other.
Putting aside whether the behaviour is correct the most important thing here to be aware of it. Anything done in your
@testSetup method will count towards the governor limits for each of your unit tests. This therefore is not a way to get some extra limits during test data setup. On the flip side if you set something up in your
@testSetup method and assign it to a static variable remember that this will not be visible to the unit test method.
Frustrating for sure but at least now I know and I can sleep just a little easier.