I figured out what I made though, and it’s an “Entity-Value-Relationship (EAV)” model, which is aggressively pooh-pooed by most people who mention it on the web. After reading all that’s been said, I still think that it’s justified in my model, as it is only a portion of it (a ‘mixed’ EAV), and for the queries that I’m running, I value extensibility over speed. Coincidentally, what I’m applying EAVs to is exactly what you chose to use as an example. I think everybody who uses EAV modeling successfully is dealing with basically the same thing: many and varying potential fields on many different forms that need to be cross-queried.

For anyone else trying to do something as foolhardy as I, here’s two really useful links:

“Dave’s guide to the EAV” http://weblogs.sqlteam.com/davidm/articles/12117.aspx

And to measure the degree to which you’ll take a performance hit: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=79043

here’s the seminal paper: http://ycmi.med.yale.edu/nadkarni/Introduction%20to%20EAV%20systems.htm

and here’s some unbelievably aggressive pooh-pooing that nevertheless is very educational: http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11QUESTIONID:10678084117056

It’s all personal experience, I’m afraid :). I don’t know of any books that cover this topic, though perhaps “Refactoring Databases”, from the Martin Fowler signature series, does. I haven’t read it (yet), though.

All I want to know is what reference material you were using for this blog entry, if it wasn’t just personal experience. I’ve been hammering google for days trying to get more information on this type of thing, and though I’ve already reinvented the wheel, I don’t want to have to reperfect it. I’m a little freaked out by the joins required for the average query and want an idea of the performance hit I’m going to take before I build it.

thank ye…

For some common scenarios it isn’t a big stretch to just do a little more work to think out the truisms in the domain space. It’s a bit of a chore the first time, but then it’s pretty much the same over and over, and you end up with (1) a pattern that can be easily split across multiple one-to-one tables, if that’s a performance consideration, and (2) a very small number of arbitrarily mapped fields, as in your “Fixed Table” pattern.

The truisms aren’t too hard to figure out, and they are mostly an exercise in careful logic. If you’re talking about a User table, and you know that flexibility in the field set is highly valued, then you just work through the reasonable possibilities. The User table has an unusually high number of truisms to account for. Most other tables have very few. (People are complicated, and we want to dissect data about them in weird ways all the time.)

In the end, with extremely customizable apps, for ASPs, with salespeople that just can’t say no, I’ve never needed more than 8 generic Fixed Table style fields to cover the truly unknowns, and truthfully, using more than 3 or 4 is a “data-smell”.

In an analysis of existing custom clients for an ASP that bends over backwards to touch the floor, only one client exceeded having 5 truly custom fields, and they were all horribly designed life cycle fields. The client was fired by the ASP anyway within the same month of my analysis for apparently different and long-standing “difficult client” issues, but I believe there is a high amount correlation between their many other problems and the fact that their data was hell.

It’s also helpful to consider life cycle event fields as a completely separate issue. Too often life cycle fields are bundled in with properties. It’s serious problem masquerading as a non-issue. Life cycle data is 99 times out of a 100 strictly in the realm of truisms, and if that’s not the case then there’s a very good chance you’re (1) not doing a business application (perhaps scientific?), or (2) need to hire a new head BSA/SA.

Clearly this isn’t for every application and I know people will be disbelieving if they haven’t Just Tried It ™ and will criticize the approach. That’s fine. The Fixed Table approach works pretty well when this one is too big for the application. In my experience with over 15 years in the enterprise space, either Fixed Table or this hybrid are the only two I will select for an RDBMs though.

Clearly the RDBMs vendors don’t do a good job of allowing a multi-paradigm approach to one of the most significant problem in business schema design in existence. Sure they have their proprietary stuff, although there isn’t even much of that. It’s a big hole. LDAP is a useful consideration for some scenarios. And I think the big dream is that Semantic/RDF databases progress fast enough to make up for all this.

I would also encourage critics of this approach, who say that having too many specific columns is a performance problem, that (1) this is a relatively easy performance issue to deal with, and (2) consider the overhead of the alternatives, and (3) why the !$!$! are you putting your CMS data in an RDBMs anyway (taking liberties in my assumptions). Hell! It’s bad enough that you’re putting customer properties in an RDBMs in the first place…but at least it can be reasonably workable. But get documents out of there, in a hurry!