Why Rails bugs me, part 1: test/mocks
UPDATE: As I learn more, I'm collecting relevant links at the bottom of this post. Also, I didn't mean to sound anywhere near as negative as I did. My apologies, in particular to David, and I've reworded a few things.
So here's my dilemma. I heart Ruby and have worked with it whenever possible since I learned it in 2002. Then this Heinemeier Hansson guy comes along and builds a web framework on it, markets the tar out of it, it becomes a killer app, and suddenly everybody from Slashdot to Scoble is making noise about Ruby. It's been astonishing and quite wonderful.
So why am I less than enamored with Rails? Maybe I'm just playing the contrarian, as I'm wont to do, but I think I may have some reasons. Since I keep thinking of specific reasons and forgetting them at key moments, I'll try blogging even smallish issues here with whatever useful discussion I can come up with. Maybe then I can connect the dots more meaningfully.
I'll start with a simple one:test/mocks
I cringed when I saw this introduced. A nice, official-looking directory where I can put code that will screw with my Ruby environment only for tests, and for all tests.
Nooooooo! Why not just start coding with lots of global variables and be done with it?
Well-designed code has minimal dependencies at any one point, and its dependencies are on abstractions rather than implementations. This test/mocks thing blesses the practice of reaching in and hacking on the dependencies of the code under test. This wouldn't be so bad for one or a small group of isolated tests; in fact it's one of the really nice things about Ruby that makes it great for test-driven development. But using test/mocks applies said hacks to all the tests!
Automated tests are worth the most when they can be confidently relied upon: "this test takes A, applies B and C, and I get D." Great. Now say I've changed all the B's in my system, let's call the new version "BB". My production code still uses B, but all of my tests use exclusively BB. How much confidence can a test like that give me when it matters most?
Instead, I should be writing code that can be passed its dependencies so I can use BB only in the tests where it's useful, and have the real McCoy available the rest of the time so my tests catch unexpected integration problems or whatever will happen in the real world where B lurks and my little hacked-up BB ain't.
I wish we could've scored a little chink in the Convention over Configuration armor for a sec back when Jamis wrote up a way to bring ubiquitous dependency injection to Rails. To me, Rails could be so much stronger as a framework with the flexibility and the attention to dependency management Jamis's proposal embodied..
And using the Needle dependency-injection tool this way wouldn't bring about the feared apocalypse of Rails2EE, wherein we do YAML situps to declare ServiceModuleAccessPointTriggers or whatever. It'd just add a spot where we can reach in and change which objects go where for a moment without screwing everything else up.
I really want to know if there's something I can do to help solve this problem. Please fire away with flames, suggestions, questions, or massive groundswell of support in the comments.
Reverie inspired by Stubbing Net::HTTP.get.
Updates
I went back and reread the pages from those few days in November 2004 when all this was shaking out. It looks like it may be possible to make a Needlizing plugin for Rails using Jamis's ideas and hacking the ActionController::Dependencies API.
And now for what might be perhaps the first Rails historical research project (?), wherein I'm trying to figure out who intended what, and what happened to those intentions:
- Jamis's proposal
- Dave Thomas hooks it up with some smart Ruby magic that turns class names into transparent accesses of the registry (looks like this became incorporated into Jamis's paper)
- DHH likes Jamis's idea and especially Dave Thomas' simplification of it, but isn't sure what it will do to Rails internals
- As Sam notes in the comments, Jamis says he's "not as convinced of the necessity of dependency injection in Ruby" as he once was.
- Did anything happen after that? Has there been a definite decision not to incorporate dependency injection into Rails?
11 comments:
HUH. This is like reading one of those blogs you read. I glazed over... sorry.
test/mocks is for the mocks that depend on system services you never want to run directly. This may be a payment gateway or a service API for creation of content somewhere.
If you need mocks for something else, just do them as one-offs in the tests.
So use the automatic wiring in case you have services that should always be mocked, otherwise don't. Nobody is holding a gun to your head to force fit mocks that doesn't fit that description in there.
Wow buddy. Slow down. Hold all the good ideas for when we design our own killer Web framework. :-)
Seems like mocks could be changed so they weren't as global as they are and they would be more useful. But I agree, there are things about Rails which make it hard to test at times. I'm still discovering how to do testing the "Rails way", but feel that it's very hard to test certain kinds of behavior. I'm not testing some things in my current application because I haven't been able to figure out how to test it well, which is bad. I don't think Rails was designed with "test first" in mind, though for some tasks you can do test first development.
John: Of course I agree with you, and I've had this vague restless feeling about it for a good year now. I got tired of being the dilettante about this and decided to dive in. I forgot that DHH has that mind-of-blog Technorati hookup...whoops. ;-)
David: I started this series with the issue in front of me, just in an effort to quantify the said vague restless feeling so I could maybe do something about it or be educated out of it. So, sorry the content of the post doesn't quite match the stated topic -- I'm just closing in on whatever my main issue is. Of course nobody's holding a gun to my head, but right now dependency management isn't up to snuff in Rails, as far as I can tell, and I've felt that pain elsewhere (not in test/mocks) while trying to TDD against Rails. So I'd like to look into it.
Thanks for your patience with a Rails complaint on your Technorati radar while I sort things out. :-)
Did anything happen after that?
Yes, he came to the conclusion that dependency injection in Ruby is unnecessary.
Sam: Thanks for reminding me of that link. As for the conclusion he drew, from my own maddeningly frustrating experiences trying to do true, dependency-free test-first development against Rails, I beg to differ.
Actually, I don't care so much about implementing the Dependency Injection buzzword into Rails as much as I do getting some way to plug dummy objects into hard-to-reach places so I can test-drive just my code without running it through SQL and the whole Rails stack. There was this illustrious hacker who told me DI would be a great way to do it...
My actual real world experience with Rails is limited, so take this with a grain of sald.
However, it seems to me that not being able to create a model object in a unit test with out having some sort of DB (or ActiveRecord hacking) smells very bad to me. It surprises me that this issue doesn't come up more often on blogs or the Rails list, but maybe its just something that isn't as important to Rails TDD as it is to Java TDD.
Rob: Truth be told, it really isn't as nasty a problem as it is in Java, for a variety of reasons, including the lower overhead made possible by Ruby's flexibility. But it is definitely a problem, and I do feel the discussion of this and similar problems has taken a back seat to the desire to press forward and show the enterprise how blazingly fast we can go.
My experience is that I work faster when my tests rock, and that I work slower when I wait on my tests. That's the bottom-line issue for me. I've done the Amazing Rails Demo Before Your Eyes for an audience, but for my real work I find ways around Rails so I can use real dependency-isolating TDD.
Well seeing as it was my code which inspired this...
My code was never supposed to hack Rails internals. It doesn't do this. It also doesn't effect any of my models or really anything of consequence. It just gives a stubbed out xml file back when you ask for it instead of connecting to the web (which isn't always avaliable when I'm running tests). I agree, in general you probably don't want to override most things. This was a specific case.
Kevin: I don't think there's anything wrong with your code. Seeing it and thinking about what I wish was equally easy to do, that's where this post started. And it sounds like we agree that more isolation is better for these things.
I'm sorry I didn't make it clear that I thought your code served its purpose well. I'm learning a lot from this first foray into stirring up the blogosphere. :-)
You might like to have a look at Mocha which allows you to stub methods on concrete objects and classes. By stubbing the new method or ActiveRecord methods like create you can inject mocks and avoid hitting the database in your tests.
Post a Comment