Metadata in Non-Metamodels

Today most IT professionals have a fairly clear idea of what metadata is and how it differs from regular data.  In data modeling we realize that we are working with metadata as we build the model, but the database that we ultimately create will contain regular data.  The demarcation is easy to see.  But in reality, it is not really practical to keep metadata separated from regular data in a database.   For as long as I can remember - long before the advent of relational databases - there were always fields like "Last Date of Update", even on flat file records.  These were metadata, and could be used for various purposes, such as data quality checks.  

Today, I think we are seeing a greater cross-pollination between metadata and data in data models.   It is now possible to see entire metadata entities in data models, and metadata tables in physical databases.   Of particular importance are the entities Table and Column which can be used for many purposes, from documentation of data to driving business rules.  I have used them quite successfully myself.  It can be argued that "documentation" corresponding to these entities could be put into the system catalog of the DBMS where the database is ultimately materialized.  This can, of course, be done using the DDL produced by data modeling tools like ERwin.  However, it is not possible to add more columns, or even additional tables, to a system catalog in the same way that it is to a declared table or column in a regular database. 

Going back to individual entities, I find myself adding more and more metadata attributes as time goes by.  Perhaps this reflects the kind of databases I am dealing with these days - usually for integration, MDM, or business rules automation.  Take the need for changed data capture in an MDM hub that manages history.  One design approach might be to have one indicator attribute for each regular attribute for which we wish to track history.  Of course, I am aware that there are design alternatives, but I have seen this design implemented.  And with it, the number of metadata attributes is almost the same as the number of regular attributes.

I tend to refer to metadata within a regular table as "embedded metadata" and metadata held in pure metadata tables as "discrete" metadata.   This helps in thinking about how to model a particular metadata need, which often depends on some kind of relationship with a particular cardinality to the regular data.

I think there is a lot of potential in figuring out the relationships between regular data and metadata, and how they should be handled.  However, I also have an additional viewpoint which is that there is no fundamental difference between data and metadata, and that what falls into the two categories depends on your viewpoint.   But that is a topic for a future blog.