I’m not sure it’s a popular opinion, but I don’t believe in relying strongly on factories in my test suites anymore. Factories, fixtures, blueprints… they’re all very similar. They’re intended to save you from having to create commonly needed objects in your test code.
Like a lot of people, when I was first introduced to the idea of a factory, I thought it was great. It made a lot of sense. Why write the same boilerplate object creation code over and over again in your tests, especially if creating one of those models can be complicated?
Not long ago I was working on maintaining an older ruby project for a client of mine, and I was investigating a bug that led me into reading the source code of our factory library, Machinist. I found that Machinist had a bug in it that was affecting one of our tests, and when I checked to see if a fix had been made in a newer version of the library, I found surprisingly that the author had quit the project and expressed his opinion that factories were a bad idea. (Unfortunately I couldn’t find the old commit where he said that. I would’ve loved to link to it.)
This was a bit shocking to me, and really made me think. It didn’t take me long to begin to question the overuse of factories as well. A few realizations stood out to me and are the main reasons why I’ve strongly cut back using them ever since.
Here are my main points against factories:
- They make your tests brittle.
Just like having some sort of global variable, a change or mistake in one of your factories can break every test that uses them, or worse, render those tests invalid. For the most part, you want your tests, like most of your code, to be self-contained and insulated from changes elsewhere, except for changes in the code that they are testing.
- They almost never “just work”.
In all the years since being introduced to factories, I can say with honesty that I’ve never seen a project of reasonable maturity have a collection of factories that all “just worked”. There have always been a few factories here and there that have had serious issues, where defining and using those factories have been problematic. I’ve often found myself working around issues with these factories, and/or spending too much time trying to get them just right. Note that this is usually a sign of problems with your models, but this leads me to my next point.
- They take you a step away from your actual codebase.
I saved this point for last, but this is really my strongest point against factories. I want to suffer. If it’s hard to instantiate one of the models in my codebase by hand, that’s a strong sign that something is wrong. I shouldn’t hide behind some creation code that’s only available in my tests. If you’re writing your code well, nothing should be that hard to work with. By forcing you to have to go through that pain, it will push you to rethink how your codebase is organized, and hopefully push you to refactor and simplify things. I don’t want to hide from any bad decisions. I want them in my face, front and center, until I fix them. Creating the objects in your codebase should be easy and straightforward enough that you don’t need factories to do so.
Now I don’t mean that you should never use factories. I just believe that we should not do so out of reflex or habit. If you’re going to use a factory, you should be very confident that the object creation code that you are abstracting is worth hiding.
It’s also better if you try to ensure that your factories are more locally defined and used to only the tests that should use them. This will cut down on cases where a change to a factory breaks a disproportionate number of tests.
But whenever possible, I believe you should create objects by hand each time you need to. The result is that you will get more accustomed to how your models are defined, and will more quickly identify bad design in those models.
I’m curious if there are others out there that share my opinion and have moved away from factories. The only other time I’ve seen a similar opinion shared is from the aforementioned author of Machinist. Tell me what you think. Am I crazy? I personally don’t think I am, and since dropping factories I can tell you I’ve never missed them at all. In fact, the reduced complexity of my test suites, and in turn my codebases, have been a relief.