Screaming Penguin - Source Does Matter

Issue 1:

Ok so the other day I noted that JUnit instantiates the test class once for EACH test method therein (not just once).

I noted the TestSetup approach to addressing this (more correctly Martin Fowler and the JUnit docs note it - and I noted that). Yet I still have several major concerns with this whole darn thing. However, TestSetup apparently must be implemented at the *suite* level. This means multiple suites if I simply want different stuff in the run once executed code? Or I could potentially write an abstract base class that all my tests extend that implements this type of TestSetup and leaves the one time run code implementation up to the subclasses. Either way quite a set of hoops just to run something once per test?

Note that the normal "setUp" and "tearDown" are also executed once per test method, not per class.

The only alternatives if you WANT to setup some shared "test fixture state" are basically static code (and possibly a static init block to establish it) or "TestSetup" stuff.

The JUnit docs basically say you are a jackass if your code is so "coupled" that you need to do this:

How can I run setUp() and tearDown() code once for all of my tests?

The desire to do this is usually a symptom of excessive coupling in your design. If two or more tests must share the same test fixture state, then the tests may be trying to tell you that the classes under test have some undesirable dependencies.

Refactoring the design to further decouple the classes under test and eliminate code duplication is usually a better investment than setting up a shared test fixture.

Excessive coupling in my design? I have a web service client that calls a service and creates an object, which other tests then use. I certainly could call that once per test and use it in multiple test methods, but that takes up extra time and resources (in my specific case a LOT of time). Or I have a JDBC connection. Or I have test "seed" data object that is a POJO and is used in multiple tests. This means things are too coupled?

I CAN RECREATE THESE THINGS OVER AND OVER AGAIN, my design allows that and normal usage does that, but I dont want to add 20 seconds and X memory to each "test" method for things I could "centralize" and do once?

The JUnit docs also admit that "A test fixture is useful if you have two or more tests for a common set of objects. Using a test fixture avoids duplicating the test code necessary to initialize and cleanup those common objects for each test." Makes sense to me, I simply want an easy way to create a "text fixture" that is not run each test, but once per test class, and or to understand why thats so crazy and indicates my design is flawed?

Issue 2:

The order of the tests cant be controlled? Again it is stated in the JUnit docs that tests are run in any order, not guaranteed. This is the point of "unit" testing some have argued and if you require a specified order then you are doing "acceptance" tests (which makes no sense if you consider the definition o f acceptance test to be "customer" driven rather than "programmer" driven for unit tests, that may be an oversimplification though).

Then how do I test a CRUD operation? I want to create something, then update it, then delete it. In my brain I want separate tests for these separate operations to keep things tidy. Yet If I want to "delete" a particular thing it has to have been "create"ed? Thus I need to specify the order?

Again specification of the order can be done in JUnit but its quite ugly (IMHO). You can use the old manual method of putting the tests in your suite by hand and put them in in order. Or you can hack it apart as I have and create a monster called "subtest".

Basically I have a single test method I called "testController". There are no other test methods in my class. My other tests are named "subtestWhatever". Thus JUnit sees this as a single test and runs the one test. "testController" then simply calls each "subtest" in the order desired. Yeah, I said it was ugly and it is, but I cant seem to get around it and dont want to use the old style manual test suite (I want the reflection test thing).

So what is the real answer? Is the answer actually I should never have to do either of these things? Or is the answer sure you need to do them, boy arent they hard to do the correct way?