Friday, July 27, 2007

What's So Funny 'Bout Peace, Love, and Understanding Design Patterns?

There have been some rumblings the last few days about the use of design patterns in the ColdFusion world, all of which seem to have been prompted by Brian Rinaldi's post on the Gateway pattern and its use (some say misuse) in ColdFusion. As usual with things of this nature the discussion got rather heated so I thought I'd add some fuel to the fire. The building's burning down anyway, so what the heck. Actually I love discussions like this because in my opinion anything that gets people this riled up is a good thing. When you question aspects of development at a very fundamental level you either come out of it with a completely different view, or it strengthens your beliefs in the way you're already doing things. Either way much is learned through the process.

In any discussion of design patterns, we must first turn to The Holy Book of Design Patterns, namely Design Patterns: Elements of Reusable Object-Oriented Architecture by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides, a.k.a The Gang of Four, The Four Wisemen, or the Four Horsemen depending on your perspective. I don't think the Biblical comparisons are necessarily misplaced because, like the Bible, this book has many companion books that attempt to explain what the heck the contents of The Book actually mean, the most popular of which is Design Patterns Explained: A New Perspective on Object-Oriented Design by Alan Shalloway and James R. Trott. My copy even has one of those fancy attached ribbon bookmarks in it just like many Bibles. Also, as with religion people get extremely passionate (perhaps unnecessarily so--it's just software after all) about their design pattern beliefs which makes the conversation all the more interesting and in some cases more than a tad bit irrational. Fun stuff.

As a backdrop for this discussion, I'd like to offer up the very first sentence in the introduction of the Gang of Four (GoF for short) book. I do this as a means of ensuring that none of us thinks there is a quick, simple answer to any of this.

"Designing object-oriented software is hard, and designing reusable object-oriented software is even harder."

Now that we know we have an arduous task ahead of us, let's get started with the discussion at hand, the shot heard 'round the world, namely the now-infamous Gateway pattern.

So what is a Gateway?

I think Brian covered this pretty well, and he made some very good points about how "Gateways" in ColdFusion don't necessarily need to be the same thing as Gateways in Java, .NET, or any other language. I do find it funny that some of the same people that are saying ColdFusion shouldn't become Java seem to get bent out of shape that we aren't implementing Java-based design patterns exactly the same way in ColdFusion as they exist in Java. I suspect this may be a semantic argument at some level (how about we call the ColdFusion implementation of this pattern DataSucker and move on?), but I digress.

Many ColdFusion developers, myself included, use objects we call Gateways to encapsulate access to a database table, specifically to retrieve multiple records (meaning methods will typically return either query objects or collections of beans) from specific database tables. Is this a bad thing? Does it violate some commandment that states what a Gateway is? Let's investigate.

To understand what a Gateway is, let us crack open the GoF tome. (Come on, admit it, you bought this book a while back but the binding still cracks when you open it. Mine sure does.) Turn to the index at the back of the book and look for the word "Gateway." Nothing in your copy? Mine either. Must be a misprint. Let's try "Table Gateway," because I think I saw reference to this pattern using that name somewhere. Hmm ... nothing there either. My point, specious as it may be, is that the word "gateway" isn't even important enough to appear in the index of the Bible of design patterns. Either that or they hired a really cut-rate indexer.

We're going to have to look through the entire book ourselves to find the mention of "Gateway" apparently. Go ahead. I'll wait.

Didn't find it? Didn't think so. Me neither. That's because--surprise!--it's not in there. Why not? Well, my own personal theory is because the GoF are higher-level thinkers than something so specific as a Gateway as they're currently implemented in any language. That being said, the Gateway pattern isn't necessarily as concrete as many of us make it, but we'll address that in a moment.

First, in the absence of any mention of Gateway, let's look at an example of what the lofty GoF do consider to be a pattern important enough to be included in their revered book. I'll just choose one more or less at random.

"Pattern: Bridge. Intent: Decouple an abstraction from its implementation so that the two can vary independently."

Based on that definition, if I told you to go build a bridge right now could you do it? If you say "yes," you're either reading far too much into the pattern, you're way smarter than I am, or you thought I meant for you to build a bridge for your Matchbox cars out of popsicle sticks. I suppose my point is that, in the GoF book at any rate, design patterns are so high level that while they're very useful, the say nothing about any specific implementation details. If we could find the word "Gateway" in the GoF book, chances are it wouldn't say anything about hitting database tables, what data types the methods should return, or anything of the sort. Their "intent" description for Gateway, if I may be so bold as to put words in their mouths, would likely be something like "Enapsulate access to an external resource." Keep that high-level definition in mind because we'll come back to it.

The GoF book does provide examples to accompany all their patterns, but remember these are examples that are designed to illustrate one potential implementation of the high-level design pattern. Furthermore, the examples provided are in the context of building a desktop document editor in Smalltalk. In my opinion their examples shouldn't be (and I'll go so far as to say weren't intended to be) taken as gospel. The authors wisely provide examples because, as we see with the Bridge pattern, it's a very lovely pattern but doesn't become very concrete in the mind of developers until we see an example. Examples are indispensable to shedding light on the abstract intent statement of the design. But again, this is an example, not the example.

This brings to mind a bit of an aside, which is one of the key problems I have with the recent posts by Sean Corfield, Peter Bell, and some of the comments on Kola Oeydeji's post (that itself was a response to Brian's original post), namely that they seem to stop quite a bit short of being useful in any practical sense. Saying "people need to review things more" or "I do things completely differently" or "this is a pet peeve of mine" doesn't illuminate anything. I know, we're all busy people, but a specific example such as "I don't use a gateway that way--here's what I do" with some concrete details would add far more substance to the discussion.

I suppose these comments are useful in the limited sense that they urge us to think about what we're doing. That's fine as far as it goes. I think a lot about what I'm doing, however, and I personally find using the Gateway pattern as it has come to be implemented in ColdFusion by numerous developers quite handy. I like it. It's cohesive, it's easy to explain to others, it serves a clear purpose, and has a clear intent. Saying "but it's not really a gateway" doesn't much help the discussion, other than to cause the people who use this pattern and call it a "gateway" to defend their use of it, which as I mentioned at the outset of this post is actually useful for the purposes of facilitating discussion. Will I change my thinking on this in the future? Perhaps. It's certainly happened before. The "it's not really a gateway" comments don't, however, tell me why I shouldn't use it in this way except in an academic sense, and the validity of the academic argument itself is pretty dubious as well see momentarily.

This brings us back to the question concerning what a Gateway is. Since it's not in the GoF book we have to look elsewhere. The obvious next avenue of investigation in my mind is Martin Fowler.

Gateway: A Definition From the Book of Fowler

For whatever reason, unlike the GoF, Fowler does grace us with a definition of Gateway, which comes from his book Patterns of Enterprise Application Architecture. I have to pat myself on the back just a bit, because Fowler's definition is remarkably similar to the one I came up with above:

"An object that encapsulates access to an external system or resource."

As we read on, things get even more compelling:

"Interesting software rarely lives in isolation. Even the purest object-oriented system often has to deal with things that aren't objects, such as relational data-base tables, CICS transactions, and XML data structures."

Relational database tables! Now we're getting somewhere. At a high level this definition, which would be the "intent" section of the GoF book, seems to fit with getting data out of a database, and he goes so far as to give the scenario of retrieving data from a relational database as one possible implementation of this pattern. So far so good. I think at this point we can all agree that an object that retrieves data from a database, at least in Fowler's view, is a perfectly legitimate implementation of the Gateway pattern.

Like the GoF, however, Fowler doesn't drill down into specifics just yet. As we look further into Fowler's pattern definitions we see that there's another instance of the word "gateway" in his pattern definitions, specifically a pattern called the Table Data Gateway, which he defines as follows:

"An object that acts as a Gateway [he references his more generic Gateway pattern here] to a database table. One instance handles all the rows in the table."

So now we see what I consider to be a more specific instance of the generic Gateway pattern. A Gateway merely "encapsulates access to an external system or resource," while the Table Data Gateway pattern explicitly defines the "external system or resource" as a database table. Again, there is no real discussion of the specifics of how the Gateway achieves what it does, but Fowler does provide some sample methods that might be contained in a Person Table Data Gateway such as find(), findForCompany(), insert(), and update(). Pay particular attention to the insert() and update() methods, because they're key to the discussion that follows.

To summarize where we are at this point, we now know that the Gateway pattern (more specifically the Table Data Gateway pattern) is indeed a perfectly acceptable way to encapsulate access to a relational database. This of course isn't the only way to do things, however, so now we'll examine an alternative pattern, compare and contrast, and discuss how the two might not be mutually exclusive.

Data Access Objects (DAOs)

The other pattern many of you may have seen referenced in your OO trials and tribulations is the Data Access Object (DAO) pattern. The most thorough description of the DAO that I've come across, and the one to which most people point when they're discussing this pattern, comes from Sun's "Core J2EE Pattern Catalog." So how is the DAO pattern different from the Gateway pattern? Let's take a look.

After describing the issues with data access in general and the goals of abstraction and encapsulation, Sun's proposed solution is to "use a Data Access Object (DAO) to abstract and encapsulate all access to the data source." Sounds an awful lot like the Gateway pattern, doesn't it? Sun's definition continues, "The DAO manages the connection with the data source to obtain and store data." So the basic idea is that the DAO encapsulates database access.

As a point of comparison, let's prepend the term "DAO" to Fowler's definition of the Table Data Gateway pattern:

The DAO is "an object that acts as a gateway [lowercase mine] to a database table. One instance handles all the rows in the table."

Even by combining the two we see that the general definition remains consistent. So at this point, and considering things only at a high level, the Table Data Gateway pattern and the DAO pattern are essentially the same. They both are defined as objects that encapsulate access to a database table, so it's really just a difference of nomenclature. Furthermore, if you scroll down to some of the examples that Sun offers on their DAO definition page, they also show methods that deal with both single and multiple records, meaning that just as with the Table Data Gateway, you see both an insert() method, which deals with a single record, and a selectCustomers() method, which handles multiple records, in the DAO.

Table Data Gateways vs. Data Access Objects (DAOs)

At this point you might be confused. Don't worry. Whether or not everyone admits it, you're not alone. To review, at this point both the DAO and the Gateway, from a pattern standpoint, serve exactly the same purpose. They both encapsulate access to a database.

So why in many ColdFusion applications do we see both Gateways and DAOs, for example both a PersonGateway and a PersonDAO? To review something I mentioned above, the Gateway pattern is used by many ColdFusion developers to contain methods that handle multiple records. The DAO, on the other hand, is used in many ColdFusion applications to deal solely with single records; many ColdFusion DAOs contain only create(), read(), update(), and delete() methods (affectionately referred to as "CRUD"). To elaborate, in this scenario the PersonGateway would contain methods such as search(), getAll(), and other methods that return multiple records. The PersonDAO would contain the CRUD methods and typically nothing else.

Since in the canonical sources mentioned above (i.e. Fowler and Sun's pattern catalog) the Gateway and DAO are in essence the same thing, where did this distinction in the ColdFusion world originate? I know specifically where it originated for me, and likely for many others as well: the Macromedia Mach-II Development Guide. Since Mach-II was the first fully OO framework for ColdFusion, I suspect that many others (like myself) learned not only Mach-II from this resource, but the relevant OO patterns outlined therein.

Because in my opinion this particular point is so tremendously pertinent to the discussions happening in the various blogs mentioned above, I'm going to quote the relevant section from the Mach-II Development Guide nearly in its entirety. This is not intended to point any fingers at anyone, least of all Sean Corfield who authored the original Mach-II Development Guide, but I do think it answers the question of where the distinction originated for many ColdFusion developers, myself included.

--- Begin quote from the Mach-II Development Guide ---

There are two basic patterns of access to persistent data within most applications:

  • aggregated access - reporting, searching, listing multiple rows,

  • per-object access - creating, editing, working in depth with a single row (object).

ColdFusion has a great built-in idiom for dealing with the first type of access  - the query - which is an efficient way to manipulate (potentially large) sets of data rows retrieved from a database (or other data sources, since you can easily create a query object and populate it with your own data). When you are dealing with aggregated access, it does not make sense to convert every row returned into a fully-encapsulated object (CFC instance) when all you are likely to do with the data is display a few fields with a link to a detail page that will focus on the selected row.

On the other hand, when you are focusing on a single row it usually does make sense to work at the fully-encapsulated object level since you are usually interested in object behavior at that point. This is also the level where you need the standard CRUD (Create, Read, Update, Delete) operations.

Recognizing these two basic patterns, you should design your components accordingly by providing separate components for each pattern. This is best explained through an example:

  • If we have a business model object called Order, then we would provide an OrderGateway component for aggregate access and an OrderDAO component for per-object access (or build the per-object access into the Order object - but see below).

  • The OrderGateway component would provide methods like findAll(), findWhere(), findByID() and they would all return standard query objects (even findByID() which returns a single row).

  • The OrderDAO component would provide CRUD methods like store(), load(), update(), delete() and they would operate on a specific Order object exchanging data via getters/setters on the Order component or via some sort of snapshot of the Order's data, e.g., a bean: the Order component could implement methods like getSnapshot() returning a bean and setSnapshot() taking a bean as an argument - the bean containing the core persistent data for the Order object. Sometimes a more direct data transfer between the business object and the data access object is needed for performance reasons (or because the application can 'trust' the components to exchange less encapsulated data, such as a struct or some opaque data structure e.g., the Memento design pattern). Such optimizations are beyond the scope of this document.

Separating out these operations from the business model object helps it stay persistence-neutral. The gateway components can be optimized for retrieving large record sets, caching etc. The DAO components can be optimized for dirty data updates, pooled object access and so on. Again, the details of these optimizations are beyond the scope of this document but providing for the two distinct patterns of data access will get you started on the right road.

--- End quote from the Mach-II Development Guide ---

To reiterate, the sole purpose for the inclusion of this quote is to point out where I suspect many ColdFusion developers got the idea to separate Gateways from DAOs in this way. Given the recent blog posts and comments, clearly Sean thinks differently about these things now. That's perfectly fine. If we stop thinking about things and changing our minds about things, we stop increasing our knowledge and may as well find another line of work. I personally want to thank Sean for creating this guide not only because I learned so much from it, but because it served as a fantastic foundation on which to base an updated version of the Mach-II Development Guide (forthcoming) for the newer versions of Mach-II.

Also bear in mind that Sean's intent with this guide was not, I assume at any rate, to attempt to provide ColdFusion developers with a design pattern Bible that should be followed to the letter, so it is incumbent upon all of us to study the design patterns on our own and make up our own minds about them. For me, the Mach-II Development Guide was a great starting point for my learning process. I've since learned a great deal more about numerous other design patterns and OO concepts, and I continue to learn more every day. Remember: this stuff is hard to begin with, and doing it well is extremely difficult. If we keep doing things the same way and don't change our minds about things as we increase our knowledge, then we're standing still as developers, so we shouldn't see it as an error or failure on anyone's part if they said something on their blog three years ago that they no longer agree with.

But Aren't You Doing It Wrong?

Given everything I outlined above, specifically how both Fowler and Sun illustrate that methods dealing with single records and methods dealing with multiple records coexist in either the Table Data Gateway or the DAO design pattern, why do I still split out my data access methods into two separate objects? In short, because I can.

Blasphemy! Well, it's actually not only because I can. It's because I like to, and given Joe Rinehart's recent blog post entitled "Defending the Gateway Pattern in ColdFusion," apparently I'm not the only one.

Maybe this all stems from me latching onto Sean's description from the Mach-II Development Guide at a formative stage of my OO ColdFusion development, but as my knowledge has grown I still think there's a tremendous amount of validity to using the Gateway pattern for multiple records and the DAO for single records. No, you won't find any real justification in any fancy books, but in my own real-world development I completely agree with Joe. It's served me well, I've never seen any negative impact by doing things this way, I like the separation of the two concerns, and no one's convinced me that it's wrong to do things this way. Even if there are valid arguments to the contrary, they're still just academic, because the bottom line is these patterns work and continue to work well in many of my applications.

Now I'll get extremely geeky (and getting geeky while writing about OO design patterns is no small feat!) and quote one of my all-time favorite shows, The X-Files:

Deep Throat: "Mr. Mulder, why are those like yourself ... not dissuaded by all the evidence to the contrary?"

Mulder: "Because all the evidence to the contrary is not entirely dissuasive."

Ah, such wisdom from my hero of the 1990s. So why is this seemingly random quote appropriate? Because even given the definitions of such giants of the design pattern world, I'm not convinced that separating data access into two objects is "wrong." Conversely, I'm definitely not convinced that taking these two separate objects and combining them into one is the right thing to do. Many of my thoughts echo what Joe already said, but I'll put them in my own words and expand on them a bit.

First and foremost, I like knowing that my DAOs deal with CRUD and that's it. Once those objects are done they're done, and these types of objects are also fantastic candidates for code generation precisely because they are so singular in purpose and prescriptive in nature.

Gateways, on the other hand, are far more nebulous and end up changing much more during application development. CRUD operations are easy to define. As you build out an application, however, typically you'll find yourself coming up with methods that need to be added to the Gateway that you didn't know about at the outset. Maybe the requirements change as the application evolves, maybe you figure out different or better ways to do things, who knows. Regardless of the reason, in my development I find that I add, remove, and alter methods in my Gateways pretty regularly right up until the application is complete. My DAOs, on the other hand, are built at the outset and they just sit there doing their job.

I also like knowing that when I call methods in my DAOs, they take in an instance of the object with which they are concerned (i.e. the bean) and don't return anything since they're operating directly on the objects passed in. Gateways, by contrast, typically return query objects or, when necessary, collections of beans. It's been lamented in some other recent blog posts that if we don't return a collection of objects, again we aren't using this pattern "properly."

Says who? Fowler? Sun? Now maybe in Java the convention is that you return a collection of objects, but this is ColdFusion, not Java. In ColdFusion we have a wonderful little object called a query that is extremely efficient and darn easy to use. So why not use it? If all I want to do is loop over a query and display the data, and I don't need to interact with each individual object, why on earth would I make things more complex and less efficient just so I'm better meeting another technology's definition of how a Gateway is supposed to behave? Pure nonsense. When I need to deal with each individual object in an aggregate dataset, I can quite easily return an array of CFCs. When I don't need to do that, I won't bother. Some call that bad programming; I call it practical.

Some might argue that having two objects where one will suffice is a bad thing. In one of Sean's comments to Kola's blog post, he makes the point that by combining some of the concerns via "responsibility-driven design" as opposed to "data-driven design" (I take issue with calling the way I do things "data-driven," but I'll leave that for another post), that he winds up with "28-46 CFCs + Transfer compared to 100-150 CFCs which the '5:1' process [meaning controller, service, bean, DAO, and gateway for each object type in the system] creates." In some cases less is more. In OO design this argument doesn't necessarily hold water. First of all, Transfer eliminates the need to create many of these objects because it does it for you. They still exist in some form or another, it's just handled by Transfer.

Regardless of the details, I personally would rather have more objects with fewer methods in each as opposed to fewer objects with more methods in each, and worse, methods that may not be as related to one another as they are in smaller, more cohesive objects. I realize I'm speaking in generalizations a bit here, but I hope you see my point. The "work" of the application needs to be done somewhere. Fewer objects doesn't necessarily mean you're saving any work because if the applications behave the same and contain all the same business processes, then in general terms the same work has to get done regardless of how you organize the objects.

There are also those who believe that based on the strict definition of these design patterns, or maybe just their personal preference, that there should be a single Gateway or DAO as opposed to both existing and serving different purposes. In my mind there's nothing wrong with that approach, it's just not my preference. Again, if the only argument is "fewer objects means easier maintenance," that's not a valid argument in my opinion. Either approach will achieve the goal of abstracting and encapsulating data access, I just prefer having slightly more cohesive objects, and Mulder's statement holds true: all the evidence to the contrary is not entirely dissuasive.

If I'm Doing It Wrong, I Don't Want To Be Right

Maybe that's overstating my case a bit, but as of the time of this writing I still like having both a Gateway and a DAO in the mix, and I have yet to see a convincing enough argument to make me change my mind. Maybe I will at some point, or maybe I'll figure out another way to handle all this that I like even better. Then everyone can point out how gosh-darn eloquent and convincing I was in this post and call me a flip-flopper. I see a career in politics in my future.

I think the biggest problem I had with the recent discussions occurring on the various blogs I mentioned earlier is that many developers who haven't been working with these concepts very long, and particularly those who are just getting into OO development, may be scared off. To anyone who was scared off, don't be. Be brash. Do things completely and utterly wrong and ask for help. Use an utterly inappropriate design pattern and make someone tell you why your implementation isn't right. Crash and burn miserably, then get up and try it again. That's the only way to learn.

The other thing to remember is that the end goal of development is not to use design patterns "correctly," it's to put a functional application into the hands of your users. If your applications work, no one (especially an end user!) is going to know whether or not you're using the "right" design pattern, and they're certainly not going to know if you're using something like a Gateway "incorrectly." If the application works and the objects are cohesive and well-organized, you've likely done your job well, and even another developer can jump in and immediately understand your code.

If you've stuck with me to this point, here's the one pearl of wisdom I was really trying to get across in all of this: overanalysis leads to paralysis. What? I just made you read several thousand words for some cliche fortune-cookie quality wisdom? You betcha. In all seriousness though, it's very easy when reading academic discussions such as those referenced above to get caught up in the right and wrong of it all and not try anything. You're only wrong if you don't try.

I'll wrap this up by saying that while none of the blog posts and comments therein caused me to change my mind (yet ...), I truly appreciate the discussion. It gets all of our minds going, it makes us justify our opinions, offer alternatives, try new things, and grow as developers. And with this post you have yet another person to target if you do things differently or think that my analysis or the way I do things is utterly wrong. Comment away!


Link to a Google doc of this post:

Another great post Matt, you are on a roll. You offer some excellent detail and exposition on many of the points I was trying to make. I think it is great if people learn more patterns and can apply those, but not at the expense of blindly thinking there is something wrong with the bean/dao/gateway structure. It has and continues to work well for me in so many cases. This isn't about making new CF-specific terminology but about adapting patterns to suit the specific advantages and drawbacks inherent in ColdFusion development.

What is all this "Tolerance" stuff? I'm ok, You're ok? You're a real rule-breaker!

Great post, Matt. One of the hardest thing that folks new to (or even very experienced with) OO have to get past is that there usually is not a single "right" way to do things -- instead just a series of options, the value of each dependent on their context. I know that kind of ultra-modernist subjectivism can drive people up the wall, but it's one of the realities of OO. Sometimes there aren't easy answers that apply every where, and you really have to train yourself to always ask "Why am I doing it this way? Is there a better way to do it, or is what I have now actually a good solution for this problem?"

Interesting read, thank you. I was reading Software Architecture With ColdFusion:

Design Patterns and Beyond

Topics Outline Prepared by Simon Horwith for CFUnderground 6 (PDF, google it). Something caught my eye:


Pattern: Row Set DTO

Purpose: Improve

performance by

returning recordset

data to the

presentation tier.

Description: Queries are

returned directly to

the presentation

tier for use rather

than returning



It does sound a lot like "Gateway", what do you think?

Good post, Matt. Thanks for putting the time into this topic and identifying the role that preference plays in software design. I think preference is extremely important topic that is often overlooked (although the post you've linked to may not convey that :) )

I did have a question about your take on the difference between the DAO and a Table Gateway pattern. If the Table Gateway pattern is specific to a database table and the DAO pattern "implements the access mechanism required to work with the data source" (from the link to Sun you provided), can you really use the Gateway pattern on a relational model where a business object doesn't map directly to a single table? If so, could you give me a brief example of how you'd do this? If not, what pattern, in your experience, would you recommend using for this problem? It seems like Fowler's definition for Table Gateway isn't flexible enough for more than a single table, but I could be wrong.

I work with a lot of legacy applications where we don't want the relational model to dictate the object model, so I've always wondered what other developer's perspectives are about this problem.

Absolutely wonderful post Matt! I updated my blog entry to add a plea for everyone to read your blog entry!

It's interesting to see the Mach II Development Guide held up as the possible source of this DAO/Gateway separation so I feel the need to provide some background there.

The original public release was early October 2003 and later that month, in response to "various requesters" I "Expanded Data Access Objects section to better explain DAO, gateway and memento machinery - and added example code" (according to the Revision History - and the memento example code was later removed because too many people followed the example slavishly without understanding it!).

A few months later I "noted that example code takes some shortcuts with the Memento design pattern".

A few months later still I "rewrote confusing DAO/memento paragraph and commented out example implementation of DAO/gateway".

That was early 2004 and that page has not been updated since although a *lot* has changed since then.

Do we disagree on the patterns? No. I still maintain that in ColdFusion it makes sense to keep per-object operations separated from aggregate operations because ColdFusion provides a wonderful abstraction called "query" :)

With the rise of ORM frameworks, it makes even more sense to keep "DAO" and "Gateway" separate because the ORMs replace the DAOs completely but only partially replace Gateways, if at all.

I suspect the only place we differ is in ratio of DAOs and Gateways to the underlying business objects. I offered a 1:1 ratio in the Mach II guide because it was the simplest thing to explain. In reality, I rarely had such a 1:1 ratio but it's much, much harder to explain why, when and how to decide what ratio to use!

Brian Klaas makes a critical point in his comment: "One of the hardest thing that folks new to (or even very experienced with) OO have to get past is that there usually is not a single "right" way to do things". I know I get teased about saying "it depends" but, unfortunately, it's absolutely true. There is no "One True Way" but a lot of people want a black and white, right or wrong answer.

Henry, no, Simon's "Row Set DTO" is merely giving a fancy name to the query object that ColdFusion provides. Nothing to do with gateways specifically, merely describing the return value.

Awesome post Matt. Looking back at my "Mach-ii for newbies" tutorials I see that must be the point where I started separating my database access layer into Gateways and DAOs (only then I called them CRUD components) - and to this very they that is what I do, and feel just like you about this way of doing it.

...."to this very they" should of course be "to this very day"

I find the particular excerpt very intriguing:

"... in my own real-world development I completely agree with Joe. It's served me well, I've never seen any negative impact by doing things this way, I like the separation of the two concerns, and no one's convinced me that it's wrong to do things this way."

Isn't that what its all about? That's all design patterns really are; re-usable solutions to common problems. Until you find an anti pattern that destroys your technique then its a great pattern. The beauty is someone else could do it differently, as long as they do it differently consistently, and have it work just as well for them, still a fine design pattern.

Someone bottle what Matt's been drinking and ration it out to others..

Insightful, useful and I even read the entire X pages of this post :D

You almost made me turn back to CF Fulltime Matt :) (almost) heh.


Scott Barnes.

Developer Evangelist


Incredible post, Matt. I just came back for my second read and I find your writing style pleasing, humorous, and enjoyable -- Not to mention very informative and insightful. When's that first book coming out?

Thoroughly enjoyed -- and mentioned in this blog entry here:

Yorum yazmasak ayip olur

Link to a Google doc of this post:

thnks very good post

Wow, what a insightful read. You hit the head on the nail when you talked about cohesion. I believe separating the two makes more sense and provides a more cohesive approach.

Often times, my gateways do not only reference databases, but also web services or other external resources. In that regard, a gateway makes a lot more sense.

Friday, July 20, 2007

Mach-II Project Supported by Great Biz Tools

Big news for the Mach-II project--the project will now officially be supported by Great Biz Tools, LLC. Read more here.

Monday, July 16, 2007

"Invisible Software," RIAs, and ColdFusion

I saw reference to an article by Robert X. Cringely entitled "An AIR of Invisibility: Adobe has Microsoft in its sights" on Brian Rinaldi's blog, and I'm very glad I did. Robert Cringely is consistently one of the most insightful tech writers and thinkers working today, which is quite contrary to this kind of totally nonsensical, worthless garbage that seems to be the bulk of tech "journalism" online these days, so I'm always interested in what he has to say. In this particular article he discusses the concept of "invisible software," Adobe's products that fall strongly into this category (namely the Adobe Reader and the Flash Player), and what this may mean for the future of Rich Internet Applications as well as their extension onto the desktop.

I strongly recommend reading Cringely's article, but to summarize, "invisible" software is software that is so ubiquitous that you don't even think about it anymore. It's just there, and you know it's going to be there on virtually any given platform. Cringely could only think of two pieces of software that have become invisible in this way, and perhaps not coincidentally they're both Adobe technologies. For me, even though I realized when I first heard about the acquisition of Macromedia by Adobe that it was all about Flash, I think Cringely's discussion really opened my eyes to what this all really means and what a colossally shrewd move the acquisition was for Adobe. Maybe they just got lucky and weren't thinking about where we'd be today when they acquired Macromedia, but I doubt it. I believe they saw where things could go, knew how to get there, and it's all just now starting to be realized and revealed to us.

What we're experiencing right now is one of those points in time in which numerous seemingly disparate elements are actually very much related, and this notion became concrete for me this past week via two conversations. On the face of it these conversations seemed to be addressing completely different issues, but in reality they were coming at the same point from from different angles. Combine this with what's occurring in the RIA and web development space at this moment and things get very, very interesting. Whenever this sort of situation occurs I think it can best be described as zeitgeist. A profound shift is happening, and we as developers may just be too close to the nuts and bolts of it all to see what's happening. This all brought to mind my favorite passage from Hunter S. Thompson's Fear and Loathing in Las Vegas:

"We had all the momentum; we were riding the crest of a high and beautiful wave. So now, less than five years later, you can go up on a steep hill in Las Vegas and look West, and with the right kind of eyes you can almost see the high-water mark--the place where the wave finally broke and rolled back."

We've been hearing about RIAs for years now, but with the addition of the Adobe Integrated Runtime (AIR) and Microsoft Silverlight into the landscape, the web development world is finally about to change. Adobe's acquisition of Macromedia was a substantial underwater seismic event, and we're just now starting to see the waves forming. As in the Hunter S. Thompson passage above, before too long we'll have the perspective of both time and experience to be able to look back and see where the wave broke. What has until now been known as "web development" is no longer solely about the web, and the browser and operating system are both about to become largely irrelevant. As it just so happens, irrelevance is a bit of a theme in this discussion.

Converging Conversations

One of the main things that has me starting to ask rather fundamental questions about what I'm doing these days is the fact that I've started doing quite a lot of Flex development. At first it may seem that Flex development is all about the UI, which in some senses it is, but this different perspective has profoundly changed the way I think about the backend of my applications as well. The mind shift I'm having through my Flex development combined with several conversations I've had in the past couple of weeks has me asking some rather existential questions. What kind of developer am I? A ColdFusion developer? An RIA developer? Is this really so narrowly definable anymore? Does it even matter?

The first recent conversation that plays into these questions occurred on a mailing list. (I'll leave out the details to protect the innocent.) The gist of the conversation concerned Adobe User Group sites not use Adobe technologies, which spilled over into the oft-had discussion concerning Adobe's position on ColdFusion. Adobe Labs doesn't use ColdFusion, for example, and some people get rather perturbed when they see an Adobe web site using PHP as opposed to ColdFusion. does, however, use ColdFusion, which resulted from Adobe's acquisition of Macromedia and the decision by Adobe to use Macromedia's web site, not the pre-existing Adobe web site, as the company's web site moving forward.

The reasons behind technology choice are many and varied. As a ColdFusion developer and proponent, I would of course love to see Adobe use ColdFusion as the technology behind all their web properties. But how realistic is this expectation? In the abstract there seems to be no reason why Adobe wouldn't use their own technology for everything they do. In the real world of business, on the other hand, there are pesky little items such as deadlines and budgets, and in that context the "build vs. buy" decision becomes a very real one of dollars and cents, not to mention time to market. Does Adobe build something like Labs from scratch using ColdFusion, or if something already exists that does exactly what's needed but it's written in PHP, do they use that instead? For everyone except ColdFusion evangelists this choice is a no-brainer.

The second related conversation was one I overheard concerning Flex. In this conversation the topic of Flex development came up, and one person in the conversation said, "I don't see the point of Flex. Until they get it running outside the browser I don't think they have a compelling story there. It would be great if you could mix something like ColdFusion directly in your Flex code. Maybe they'll get there eventually."

Rather than jump into the conversation, which I suppose I could have done, I decided to ponder this for a bit. Is the person who made these statements simply misinformed, or is Flex still largely misunderstood by the masses? AIR is pretty new, so it's not surprising that someone who isn't neck-deep in the Adobe product line may not have heard of it, know what it does, and know where it fits in the development landscape. For that matter, even those of us who know about AIR are still figuring out where it fits and where best to use it.

I was a bit curious, however, as to why someone would think that there's no compelling Flex story simply because it runs in the browser. Maybe I'm missing something, but to me, being able to take a complex application and give users a far simpler, richer interface than could be created with HTML is pretty compelling. Add to that the amazing speed with which you can develop a Flex front end, and I think there's something exciting for both developers and users. Throw AIR into the mix--and yes, I realize I've been rather duplicitous on the value of AIR to this point--and I think this is a key component to the profound change in the air (pun intended) that's occurring right now. As with all technologies they should be used where appropriate, but when I need a desktop-like experience in my "web" applications, I can write something more functional for the user in less time by using Flex, and soon I'll be able to take that exact same application and deploy it on the desktop. That's not just compelling, that's rather mind-blowing.

The comment concerning the ability to mix ColdFusion code in with the MXML code was also interesting to me, and I think this was probably the real spark that started me down this thought process that I'm trying--perhaps in vain--to outline here. Flex is all about the presentation layer. It doesn't care what technology is feeding it data, and it doesn't talk to databases. That's not its job. Giving people the ability to mix CFML in with the already-powerful MXML and ActionScript 3.0 would be an unwarranted and unwanted mixing of concerns with negative benefit. In reality, the fact that Flex doesn't let us throw CFML into our front-ends is a technological enforcement of the way in which we should be building our pure CFML applications. CFML code in the UI should be related solely to the UI. In a CFML application we achieve this through good architecture, whereas Flex has this separation of concerns built into its DNA.

Flex's job, and one it does extremely well, is to pull data in from anywhere and give the user the ability to work with that data in a highly interactive way. As a technologist I think Flex's UI capabilities are just plain cool, but interactive charts and graphs, drag-drop, and slick state transitions aren't just "cool," they allow the user to interact with and visualize data in ways that would otherwise be extremely difficult or impossible. If there is one thing I took away from the recent Flex Maniacs conference, it's something that Macromedia pushed hard and something that Adobe clearly gets and is pushing even harder: Experience Matters.

So back to the crux of this discussion. If Flex and other players in the RIA space don't care where they get their data from, and they can talk to more or less any technology, what does that mean for ColdFusion? Does this better level the playing field for ColdFusion, or does it make ColdFusion invisible? Does invisibility for ColdFusion translate into irrelevance? Hold these thoughts for a moment, because in order to truly pull this all together we need to take a brief detour and discuss converging technologies. This is key to understanding the new Internet application development ecosystem.

Converging Technologies

In addition to these converging conversations, which include "Why isn't Adobe using ColdFusion for everything?" on the one hand, "Why is Flex compelling?" on the other hand, and Cringely's article right in the middle, we have to consider the convergence of technologies in order to get a complete picture of the change that's occurring in the development landscape. We have technologies such as Flex, AIR, and Silverlight on the front end, and technologies like ColdFusion, PHP, Java, Ruby, etc. in the middle-tier and backend space. What's interesting and challenging, however, is the endless combination of options developers now have at their fingertips, as well as the increasing number of "places" (which I'll use as a generic term referring to where applications run) in which our applications will be used.

First let's consider the front end technologies, because where the application runs dictates a lot of decisions that we as developers will be making as we navigate our way through the new development world. In general terms, applications can run in the browser of desktop or mobile devices, natively (meaning as an installed application or via a runtime such as AIR) on desktop or mobile devices, and can either require a mandatory Internet connection or be usable offline through the local caching of data. On a device like a BlackBerry, even a seemingly simple web page can function as an application by deploying it as an icon to the BlackBerry desktop. To the user, clicking on the GMail client versus clicking on an icon that launches a web page from their BlackBerry desktop is the same thing. They don't care what they're launching as long as it gets them the data they want.

There are clearly shades of gray here as well. Even the word "browser" means myriad things because the size and capability of the browser dictate what can and can't be done with the application. In the case of Flash and Silverlight, the browser is merely a container for a virtual machine inside which the application code runs. With AIR we have that exact same application spilling over into the desktop via an OS-specific runtime, and gaining tremendous capabilities (as well as security concerns) in the process. Even the notion of being connected isn't an absolute. Connected full-time or only occasionally? Does the device automatically cache data so it can be displayed when there is no connection? When we say "connected" what do we mean? Traditional high-speed Internet, mobile broadband, or a mix of the two with devices that freely switch back and forth? Does it matter? Should it matter?

We all know the hugely compelling story of Flash's market penetration. Flash is on nearly 99% of all browsers, and new versions of Flash hit a 50% install base within six months of being released. Flash Player 9, which is what's required to run Flex applications, is already on 83% of browsers. As developers we can write applications that run in the Flash player and know to a virtual certainty that they will run. More importantly, they will run in the exact same way regardless of the browser and operating system. The available AJAX frameworks are getting better at helping developers not have to worry about browser compatibility issues, but compatibility issues still exist, and having done both Flex and AJAX recently I can say that for me the development experience in Flex is far better than AJAX. That being said, the right technology for the job needs to be at the forefront of our minds. Flex isn't appropriate in all situations, and neither is AJAX. Both should be tools in our development toolbox.

If the right technology for the job is indeed Flex there's little to argue against it. There's no huge price tag for server software like there used to be, the development environment is fantastic, many parts of Flex are open source, and you can rest assured that what you write in Flex will "just work" because of the invisibility of the Flash player. Not that Flex doesn't have its issues, but even as young as Flex is (Flex 1 and 1.5 were little more than proof-of-concept technologies in my opinion), it's a very strong platform that developers can use to give users better experiences both in the browser and now, with AIR, on the desktop as well.

Where appropriate, developers need to consider that users may want to use our applications in multiple ways on multiple platforms. For example, I'm currently working on an application that will have a full Flex client that will run in the browser, but based on some of the features the users want I can see making this an AIR application at some point in the future. In addition, users want to be able to access the application from a BlackBerry. Does that mean I have to figure out how to fit a 1280 x 1024 Flex application onto the BlackBerry? Even if the BlackBerry did have Flash (which it doesn't), trying to cram the complete application onto a BlackBerry screen doesn't make sense, and honestly that's not even what the users want or need. A very simple subsection of the application is what they want to use on the BlackBerry, so we're focusing on deploying just what they need as a BlackBerry application. To me as a developer this is simply a web page that pulls data from the same backend as the Flex application, but to the BlackBerry user it's just another BlackBerry application they use to access vital data from the application.

As I've heard Hal Helms and others say over and over again, "To the user, the UI is the application." When I first heard this statement I think I interpreted it a bit differently than I do now. It could be taken to mean that the user doesn't care about all the technical wizardry that makes the application work, so whether the application is Flex with a Java backend or a simple series of HTML pages, they just want it to work and be easy to use. This is a perfectly valid interpretation. Viewed in the context of the current discussion, however, the statement could apply equally to developers if we remove "to the user" from it and simply state, "The UI is the application," or perhaps more appropriately, "The UIs are the application."

Clearly this isn't exactly true since we developers still have to build all the backend nuts and bolts to make these various UIs work, but if we start thinking in these different terms it shines a completely new light on how we develop our applications. For example, I just bought a brand-new, fully loaded Toyota Prius, and there is a serious amount of technology going on in this car. The touchscreen navigation system alone is amazing, and when I think about that in combination with all the technology that actually runs the car (it drives too!), it's rather astounding. What's most impressive to me as a technologist, however, is what a phenomenal job they did in making sure that I don't know or care (that much anyway--my geek curiosity still applies to a certain extent) how all this technology works behind the scenes. I just know I can push a button, say "Italian restaurants," and the car will show me where they are. If only all the applications we build worked so well.

I preached in my recent presentation at CFUnited that the business logic of an application needs to be front-end agnostic, meaning any type of front end should be able to talk to the business logic layer of an application. This is accomplished by creating a service layer in your application that serves as a public API to the business layer of the application, and this service layer can be leveraged by multiple types of front-end clients. Not only is this a great way to separate business logic from the presentation layer, it also makes your business logic completely reusable as the front end of your application evolves or exists in multiple forms, as with the Flex client and BlackBerry example mentioned above.

When I gave this presentation the point I was trying to make was that the front end of the application is irrelevant, and that the backend should be detached enough from the front end that it doesn't care what type of front end is talking to it. This perspective probably isn't surprising given that A) I'm far more of a backend developer, and B) the title of the presentation was "OO Architecture Back to Front." Since I gave that presentation I'm regretting the title a bit because I'm starting to think this puts the emphasis where it doesn't belong, even for "backend" developers like myself.

Now that I'm thinking about things in a different way, it could equally be said that the backend of the application is irrelevant. From the perspective of the user, the backend is substantially more irrelevant than the front end. To repeat the wisdom introduced above, to the user, the UI is the application, so we developers need to spend a lot of time on the UI to make it as nice and functional as we can. If the UI is bad, the application is bad, because the user doesn't care how beautiful (or not so beautiful) the code is on the backend.

To the user, the backend is invisible. They don't see it and don't want to see it, because the only time the backend rears its ugly head to the user it means something has gone wrong. They don't care what the backend does, the language in which it was written, or any of the technical details whatsoever. They just want it to work. As developers, once we start thinking of the business logic of our applications as a black box that's exposed via services of various kinds, we stop caring about the backend as well. Once it's written and working it becomes rather invisible even to us. This isn't to say it doesn't take a lot of work to build the business layer of an application, but I hope you're beginning to see my point.

The BlackBerry example above is one very simple yet very real-world instance of converging technologies, but even such a trivial example brings to light how we as developers need to broaden our scope and "think different" (to borrow a phrase from Apple) about what it means to be a developer. Specifically, we need to think about many different types of devices and clients communicating with the same backend, which also means we need to start thinking about our backend as a commodity that's invisible to the user and, once it's written, largely invisible to us as well.

Is ColdFusion "Invisible?"

What does this mean for ColdFusion? Is ColdFusion invisible? The short answer is yes, but unfortunately not in the good way. (The longer answer is maybe; read on for more details.) Adobe Reader and the Flash player are invisible in the sense that they're on everyone's computer so we can rely on them and take advantage of their huge install base. This is positive invisibility. ColdFusion, unfortunately, falls into a different category of invisibility. ColdFusion is invisible because, other than to the developer writing CFML, ColdFusion doesn't matter. For the most part the exact same functionality could be written in any technology and no one but the developer knows or cares. The speed and lack of hair-tearing with which you can build something in ColdFusion may make it a great choice, but again, no one but a developer cares.

Going back to my CFUnited presentation, I showed a concrete example of three completely different UIs all talking to the same ColdFusion backend. First I showed an HTML interface ca. 1996, then I showed an AJAX interface using the AJAX capabilities of ColdFusion 8, and then I showed a Flex interface. You can imagine extending this to a BlackBerry browser or another mobile client quite easily. The intent of this demo was to emphasize the importance of creating a service layer on top of the business logic of an application, because A) it abstracts and encapsulates the business logic, and B) it creates a situation in which swapping out the front end of an application is quite simple. In this sort of architecture you can completely change the front end of an application without the backend knowing or caring that any changes occurred.

Now think about this from the other direction. Suppose I had written three versions of my business logic code, one in PHP, one in Ruby, and one in Java. Would this have changed the substance of the demo? Not in the least. I could have had the 1996-era HTML interface talk to PHP, the AJAX interface talk to Ruby, and the Flex interface talk to Java, and no one would have been the wiser. To the user, and even to the developer of the UI, this doesn't matter. The user really doesn't care, and the UI developer just needs to know what services to talk to and how to talk to them in order to retrieve data and display it to the user.

At this point it may seem that I'm painting a rather grim picture for ColdFusion because in the scenario outlined above, ColdFusion is largely marginalized. Is the picture really this grim for ColdFusion? Yes and no. On the one hand, the technological irrelevance on the backend is good for ColdFusion because it levels the playing field. To put it another way, since the choice of backend technology doesn't for the most part limit what you can do with it, there are fewer reasons not to use ColdFusion than may have previously existed.

ColdFusion also has an extremely strong selling point in this new world order of application development because it can talk to more or less any other technology. ColdFusion 8 vastly improves performance and eliminates many of the shortcomings on which its detractors have been harping on for years, such as image manipulation capabilities, more and better deployment options, server monitoring and profiling, and integration with .NET. ColdFusion has been described in many of the recent ColdFusion 8 presentations as "glue," and when the features are considered as a whole it's some pretty powerful glue.

On the other hand, there are many other brands of glue out there, and while they may not be as nice as ColdFusion in some respects, based on the skillset of the developers on a project, the budget, and so on and so forth, it may make logical sense to use something other than ColdFusion. As I mentioned above, the love of a particular technology doesn't drive the decision-making process from a business perspective, so from a business perspective it has to make sense to use ColdFusion in order for that to be the choice. This could be because your company is full of ColdFusion developers, it could be because you use contractors and that's their technology of choice, or it could be because your company has already standardized on ColdFusion. However, the fact that you can accomplish what you need to accomplish using more or less any technology is bad for ColdFusion. ColdFusion has to sell itself on the how of developers accomplishing their tasks, and to the people with the checkbooks in an organization that's sometimes a hard sell.

Making the Invisible Visible

Given this new development world, one in which in my estimation the backend technology is less important than perhaps it used to be, how does ColdFusion become visible? Visibility for ColdFusion these days means making it the best technology with which to interact with the converging clients and devices outlined above. ColdFusion 7 took a baby step forward on that front with the introduction of SMS and Instant Messenger (IM) event gateways as well as improvements in the handling of web services. The issue with ColdFusion 7 was that these features were more or less all the further it went on this issue. I highly doubt it's as dead simple to create an SMS or IM gateway in any other technology, but the introduction of these features didn't seem to change the game much. ColdFusion was perhaps a bit ahead of its time with these features, so maybe we'll see more done in this area in the future.

With the very small amount of hindsight we now have looking back at ColdFusion 7, the single most important feature for ColdFusion's future viability was the Flex Connectivity capabilities in the 7.0.2 release. This one bit of functionality instantly made ColdFusion the easiest technology to use with Flex. Even with this there are of course solutions that allow Flex to communicate with practically any technology, but ColdFusion is still striving to be the best backend technology for Flex development with the inclusion of the free version of LiveCycle Data Services in ColdFusion 8. This combined with ColdFusion's legendary ease of use and speed of development and you may be asking yourself, "Why isn't everyone using ColdFusion?"

Unfortunately I don't have the answer to that question. If I knew how to get the world to adopt ColdFusion I would have done it by now. As ColdFusion developers we love ColdFusion and know the advantages it offers, so maybe we all need to stop railing against things that we can't change, dump the inferiority complex, continue to use and evangelize ColdFusion, and simply enjoy our technology.

Trying to convince someone whose starting point in the discussion is "ColdFusion sucks!" is, sadly, a waste of breath. The recent comments on following the "Ghost in the Machine" article mentioned at the beginning of this tome showed a level of ignorance of and FUD about ColdFusion that none of us can even begin to fight against. The best we can do is have reasonable conversations when possible and put our money where our mouth is by building great applications with ColdFusion.

To that last point, building great applications with ColdFusion, I think the word "with" needs to be explained. I could have written "... building great applications in ColdFusion," and the fact that I didn't was deliberate. The majority of applications we'll build in the future--and that future is now--probably won't be solely in ColdFusion because a CFML/HTML front end is but one of many options available to us. Flex applications, AIR applications, AJAX applications, and mobile applications can all be built with ColdFusion, and probably be built better and more easily with ColdFusion than with other backend technologies.

This to me is going to be ColdFusion's greatest strength moving forward. The new front-end capabilities of ColdFusion 8 are fantastic, and while this makes ColdFusion a great choice for HTML/AJAX application development, in the end this is a rather limited selling point. It's the glue capabilities of ColdFusion and how well it interacts with Flex that are going to make ColdFusion viable, perhaps even more than viable, as the aforementioned waves begin to crest.

Your Point Being ...

So the upshot of all of this is, well, I'm not sure. Parts of this were admittedly rather rambling, but regardless of their lack of organization and brevity, these are some thoughts that have been going through my head recently so I thought I'd try to put them together in a semi-coherent fashion, share them, and hopefully get your thoughts on them. As I mentioned at the beginning of all of this, I do believe that we're in the midst of a sea change as developers, and if we don't pay attention we might just miss it.

I realize that I may have painted a less than rosy picture for ColdFusion. That wasn't my intent, and actually I believe ColdFusion has a very bright future in this new space. At the same time we all have to be aware that leveling the playing field is both good and bad for ColdFusion, but the good outweighs the bad. ColdFusion isn't going anywhere and will continue to be and perhaps increasingly be a logical choice for the backend on Flex applications.

I think my main point is this: the times they are a-changin'. Going forward I'm not sure where my development path will lead me, what technologies I'll be using, and how they'll all fit together. The one thing I do know for certain is that for the last 10 plus years I've thought of myself as a ColdFusion developer, with brief detours into ASP, Java, .NET, and PHP along the way. I always came back to ColdFusion, however, and I'm always happiest as a developer when I'm using ColdFusion. In that sense I suppose not much is changing. I'll continue to love ColdFusion, and I'll continue to explore other technologies as job requirements or my personal interest dictate. What is changing dramatically, however, is the nature of technologies I use, and the very nature of the development field.

Regardless of how we feel about ColdFusion, we all need to recognize and embrace the change we're in the midst of right now. Most of us won't build pure web applications anymore. Most of us will be building many fewer single-purpose applications than we have in the past. Like it or not, we're going to be spending more time on the front end of applications than we have in the past, and some of this front end technology may be completely foreign to us. We web developers will also find ourselves thinking more like desktop developers with technologies like Flex, which eliminate the traditional request/response model that has become second nature to us, and AIR, which takes our current skills and literally puts us squarely on the desktop.

Frankly I'm excited by these prospects. Taking this different approach has caused me to revisit the fundamentals of my development process and has irrevocably changed how I will think about development moving forward. It also opens up worlds of possibilities that even a year ago were still only on the horizon, only some hazy future possibility yet to be realized.

Now that the future is here we as ColdFusion developers must embrace it and make it part of what we do. ColdFusion is great; always has been, always will be. It's also something that will only be part of what we do moving forward. The great news is that we bet on a pretty solid horse, and it's not one that's getting sent to the glue factory any time soon. Our skills as ColdFusion developers, augmented by various front end technologies, make us important assets to the new development world. We simply need to change our thinking a bit, sharpen our tools and pick up some new ones, and build great applications of all kinds for our users.


Link to a Google doc of this post:

Wow! Quote a tome you wrote... I think there is more to the ColdFusion story in relation to Flex here. The question to me is, is CF really the best/easiest way to build Flex apps? Hmm...well, to be honest, it offers some benefits, but the other connections aren't overly difficult. For our Java developers for instance, CF is *not* the best in their opinion...Java is...and since we already have FDS, they don't need CF.

Now, there is another story here too. Why do we discuss an HTML *or* AJAX *or* Flex app? Why could Flex be a portion of a page that is also HTML and JavaScript. Does making entire pages in Flex make sense for every or even most applications? Of course not. I still say HTML/JavaScript fits many use cases - and you can even see this in AIR.

Anyway, I am rambling. Thanks for taking the time to write this. Good stuff.

See what happens when you put the podcast on vacation? All of those thoughts have to come out somewhere. :)

Seriously, though, you make some interesting points/observations.

I would agree that we're seeing significant changes in the development and delivery of web-based applications (and one day the term "web-based" won't fit anymore). But I would make the argument that a lot of the energy behind this evolution is not coming from the new UI technologies per se, but from the explosion of web services and APIs that has occurred over the past few years. This ability to combine information from multiple sources and in multiple formats has allowed developers to create new applications and extend existing applications with modules and widgets powered by external data.

The UI technologies (Flex, AJAX, etc.) allow for better presentation and control of all of this information and also feed into the development cycle by opening up new ways of viewing and manipulating that information, spawning ideas for new mashups.

As you pointed out, ColdFusion 8 has been designed to act as the "glue" or the central hub in an organization's IT information infrastructure. So in that sense, CF 8 can extend this movement of combining and re-purposing information into new uses and applications by allowing organizations to leverage their own internal data.

With CF 8, you can get at information stored in databases, files on the server, PDF forms on the server, MS Exchange, and internal or external XML files. You can even get at business logic programmed into existing Java or .NET applications.

Doesn't that pretty much cover everywhere an organization stores data electronically? And how many other programming languages can pull that off AND provide all of the web-UI functionality that CF brings with such ease?

(Sorry...realize I'm speaking to the choir here, but I was moved to say it).

So why isn't ColdFusion more popular than it currently is? I think the answer has something to do with the point you made: only developers really care about what technology is being used on the back-end. I would say that your average developer likes to stay in their comfort zone: they're not going to switch from their current programming language to CFML even if working in CFML will ultimately make them more productive. No, the impetus for such change would have to come from a lead programmer or from the business end, folks who feel the pressure (either internally or externally) to do things better or faster.

It'll be interesting to see how it all shakes out.

One last thing: you said that "most of us will be building many fewer single-purpose applications than we have in the past." I'm not so sure about that, at least not in the higher-ed sphere where I work. More often than not, the functional requirements for any given application are so specialized that a custom-built application is the only option: changing one's process or data structure to conform to the functionality of an existing application is something that most groups (again, at least where I work) are loathe to do.

And as I alluded to earlier, the opportunities opened up by the newer UI technologies will only serve to inspire folks to come up with new unique applications.

Will companies be willing to shell out $7,000+ for simple database access middleware?

@Zach--A) CF isn't necessarily $7000+, B) depends on the company, and C) I wouldn't exactly call CF "simple database access middleware."


Saturday, July 14, 2007

Two New CFUG Sites on CHUG App

Two ColdFusion User Group sites--the Bay Area CFUG run by Sean Corfield, and the Boston CFUG run by Brian Rinaldi--recently launched using the user group management application I created for the Capitol Hill User Group. Sean also very graciously contributed some fixes to the code before he went live with it.

Just pointing this out again if you're in need of a user group management app. Also if you want to dive in and help with the code on the project, just let me know.


Just a quick note to let you know that we're using CHUG over here for the Scottish ColdFusion User Group site.

Article - Unlocking the ColdFusion Black Box

I recently wrote an article called "Unlocking the ColdFusion Black Box" for FedTech Magazine, and it's just gone live on their web site. It's a brief overview of the server monitoring and alerting capabilities in ColdFusion 8, which I personally think is one of the most important new features Adobe put into ColdFusion.