Profilers? We have Tests and the Pause Button!

We ran into a nasty performance problem today. A data load operation that should take a few minutes was taking over an hour. After ruling out the usual suspect (bad database query plans -- we've been burned before), we started wondering just what to do. The data load is supposed to be part of an interactive check-in/check-out process, and spending a couple of hours is not an option.

The first thought was to profile the application. Most of our batch processes can be debugged from the command line, so this wouldn't be too much of a problem, but waiting a couple of hours for answers didn't seem like fun. So we simply stepped through code until we found calls that took "a long time". Of course it was often unclear just what in that code was slow. How to test fixes without firing up a profiler?

Our unit tests. Fortunately most of the code in this module is pretty well covered by unit tests. In a few hours, we were able to isolate some hypotheses, slightly alter some unit tests to use the much larger "real world" dataset, spit out performance results, and make changes. The end result? What once took an hour now takes three minutes. Not bad for an afternoon's work.

We could have done it all with profilers or (smart) trial-and-error easily enough, but I think the ready availability of unit tests made our work faster by a factor of two.

Technorati Tags: programming