The big project I've been working on recently is 100% backend batch XML processing on a rather large scale, and as a consequence of the fact that it's all batch processing, the backend piece of the application (which is the vast majority of it) has no user interface.
The complete lack of UI presents some unique challenges, particularly since tracking every last bit of what goes on in the batch processing, keeping track of when errors occur, etc. is vital to the success of the system as a whole.
First and foremost, if you aren't using cflog regularly in your development process, go read the docs and start using it. In my opinion cflog and the use of custom error types are the two most underutilized things in CFML development, and particularly in a moderately complex system with no UI, this combination is key to knowing exactly what's occuring (and when) in the system.
Moving on to the real point of my post, when a system like this is in production, even given your best efforts you can pretty much depend on the fact that at some point you'll get bad data that will cause processing of said data to fail. When debugging a data problem that's causing a particular iteration of the processing to throw an error, copying a massive database into development every time you need to troubleshoot (sometimes under time pressures) just isn't an option, and I'm rather wary of pointing my local machine to the production database (and in some cases due to permissions, etc. this isn't possible). Also, in some cases what you need to troubleshoot may be in a loop, and only one iteration of the loop is causing issues, so you can't exactly throw in a cfabort to see what's going on.
So what to do?
Back to our friend cflog. Go to the part of the code where the data is causing issues with the processing, which in this case we'll assume is a loop, and use cflog to write out as little or as much information as you need to a log file. Simple, powerful, loop friendly, and no cfaborts needed.
Now for what I think is the really cool part: use this in combination with the tail command on Linux or Mac (or Cygwin on Windows), and you can see the log data immediately as it's being written.
So for example, on OS X I map to the drive on the production Windows server where the CF logs are being written. Then open a terminal window, cd to /Volumes/DRIVE_MAP_NAME and navigate into CF's log directory. Do a tail -F LOG_FILE_NAME and that will "follow" the log file so you'll see updates to the log file as they happen. Again, simple stuff, but just what I needed to debug a system with no UI on a production server without disrupting the system as a whole.
Have you looked into FusionDebug? It is an essential CF Debugging tool for me now.
I've started using cfdump with the output attribute for logging as I get the coolness of cfdump and the ease of specifying the file location in the output attribute.
Great post and suggestion! Just wanted to add that if you don't have Linux/Max or Cygwin, there are several "tail" viewers out there specifically BareMetal's BareTail or BareTailPro (http://www.baremetalsoft.com/baretail/index.php) that let's view files like you suggested. It even has a command line utility to do the same as native tail linux command.
I'd like to second the Mike's recommendation for FusionDebug.
It's a little pricey (I can't remember but I want to say it's $300 or so--about twice that for tek support) but it really gets the job done.
I like using the Console app on OS X to watch log files. It is just a little bit more prettier then using tail. I see it does have some filtering abilities as well.
Just wanted to point out that Mach-II's logging package is independent of the framework itself and can easily be setup with ColdSpring and injected into model components. The nice thing is that logging package has few bundled loggers -- CFLogLogger, EmailLogger and MachIILogger. Because of the unified logging API, you can attach multiple loggers to the logging infrastructure and have them watch and filter the passed logging messages based on logging severity / channel. I've been thinking of writing a logger that logs to a port similar to what you can do with Apache Commons logging as there are tools that you point to a port and watch messages as they come in even if you cannot get access to tail or the log files directly in real time.
I'll 3rd Fusion Debug, but also remember that CF 8 ships with a similar debugging tool now. Don't you get a free one with the Dev Edition of CF?
Also, if you're using CFEclipse, go to Window > Other Views > ColdFusion > CF Log Viewer. Point it at your logs directory and it gives you tail functionality as well.
There are other Eclipse "tail" plugins that work with the standard java log files too (i.e. application.log, exception.log).
Where is your blog entry (or docs) on how to use the logging component? :-)
Matt Williams did a nice Intro to Mach-II Logging on our wiki:
If you are using it independently from Mach-II, you'll be most interested in the Additional Resources section on using the Utility Connector. Although it should how to use Mach-II managed log factories in CS, you can see how to setup up your own log factory in CS.
Thanks for all the comments--well aware of FusionDebug (actually own it!) and the CF 8 debugger, but personally I don't like configuring my production servers for debugging in that way. Handy, yes, so maybe that's just my own feeling about it. Just seems a lot less intrusive (and equally as effective in this case) to throw some logging in there.
Glad Peter brought up the Mach-II logging package--that's a very powerful feature of Mach-II 1.6 but the cool thing is you can use it outside Mach-II as well! We'll have much more information about it very soon.