What is Unstructured Data through the Eyes of the Data Professional?

Recently I purchased software to help organize my collection of digital photographs. This software had some neat features including the ability to assign tags to each photograph. Tags can help describe the photograph such as who is in it and where was it taken. The software even comes with a reporting system, answering questions such as “How many photographs contain the tag ‘Jamie’ (my youngest)?” (The answer by the way is over 4,000!)

Being a diehard data modeler, I stared at the screen after assigning the tag ‘Jamie’ to her four thousand and first photograph, and wondered this: what does the data model look like behind this software? I started sketching boxes and lines and realized I came to a decision point where I either needed to model the photograph itself or the contents of the photograph. The photograph itself seemed to me as unstructured, yet the photograph’s components appear to have far more structure. I then started asking myself, “What is unstructured data anyway? What is the difference between structured and unstructured?”

A quick search on the Internet turned up many computer-type definitions, such as this one from Wikipedia: Unstructured data refers to (usually) computerized information that either does not have a data structure or has one that is not easily usable by a computer program. Definitions like this have little value to us as a data analyst, data architect, or data modeler. In fact, I could not find a definition that seemed relevant to the field of data management – so…I wrote my own.

Before defining unstructured data, we need to understand the concept of a classword. A classword is the last term in a data element name which defines the high level domain in which the data element belongs. A few examples of classwords are Quantity, Amount, Name, and Code. For example, the data element Gross Sales Amount contains the Amount classword which implies a currency such as US dollars.

Some classwords are simple and some are complex in nature. Simple classwords can only be broken down further through the process of normalization. Simple classwords include Amount, Code, Date, Identifier, Indicator, Name, Number, Percent and Quantity. For example Customer Name can only be broken down further by normalizing Customer Name into Customer First Name and Customer Last Name but this is all within the same classword of ‘Name’.

Complex classwords include Text and Object. Examples of data elements containing the text classword are Order Comments Text and Email Body Text. Examples of data elements containing the object classword (including photographs, music, pdf files, and voice conversations) are Driver License Photo and Book Licensing Agreement. Complex classwords can always be broken down into smaller pieces, such as completely new data elements with simple classwords. The data element Photograph Object for example, can be broken down into Photograph Taken City Name, Person In Photograph Name, and Photograph Caption Text. We can even go further and include simple data elements such as Dots Per Inch Count, Film Speed, and Photograph File Size.

For me as the data modeler, the distinction between structured and unstructured data boils down to a distinction between simple and compound classwords. To be more specific: Structured data is any data which can be referred to with a simple classword. Unstructured data is any data which can be referred to with the complex classwords of Text or Object. Customer Last Name is structured. Order Comments Text is unstructured.

You’re probably wondering, why even distinguish structured from unstructured? If your job function includes analyzing, architecting, modeling, or designing information, there is an impact to your job when the data is unstructured.

Stay tuned for my next blog where I will explain how unstructured data impacts us, the data professional. Until the next blog…