The Canonical Data Model

When I started data modeling it never really occurred to me that I was doing anything more than designing databases. Of course, designing databases is an extremely important and valuable activity, but there is more than can be done with data models. Today, the focus of data management, if not IT, is on the integration of data in order to get value from data. This is a massive paradigm shift from the attitude of IT’s mission being to automate business processes by building transaction applications. And everyone is finding it very difficult.

I believe that data models can play an extremely important role in helping to achieve data integration, and it is a role that is rather different to designing physical databases. Oddly, my attention was drawn to this by reading I had to do for a project involving messaging. Middleware may seem an unlikely area to have any involvement with data models, but it really does.

Middleware delivers messages from senders to recipients in near real time. One of the common strategic architecture options for middleware is to build an Enterprise Service Bus (ESB). This is like a common highway in the enterprise along which all messages flow. This is architecturally superior to having many single point-to-point messaging interfaces between applications. Point-to-points are often built by undisciplined programmers who are essentially hackers. They tightly couple the applications and prevent them from evolving as the business changes.

An ESB by contrast is supposed to loosely couple applications, but to do that it must have common message formats. In a point-to-point environment, an entity like Account will have many (perhaps hundreds) of unique incompatible and undecipherable message formats. In an ESB there should be one. But this requires discipline. If the hackers (sorry, “programmers”) run wild they will simply recreate virtual point-to-points within the ESB for each particular linkage between a producer and consumer. The value of the ESB is then largely lost.

What the messaging gurus recommend is to build a canonical data model (CDM). The term “canon” is used in ecclesiastical regulation, and many people think it means something like “blessed”. In fact, “canonical” is derived from the Greek kanon, i.e. “a rule or practical direction”. It shares its root concept with “normalized” , which means “according to rule”. The canonical data model (for messaging) is a set of XML schemas from which all message schemas can be derived.

So how do you get to a canonical data model? The answer given by the messaging gurus is to talk to the data modelers and derive it from the enterprise data model. Thus the major use case for an enterprise data model today is not for instantiating databases, but to facilitate messaging. It is a use case all enterprise data modelers should embrace.