I Never Metadata I Didn't Like
Simba: “What’s a meta?”
Timone: “Nothing, what’s the meta with you?”
- Disney's The Lion King
In its simplest form metadata is data that describes data or facts about the data. For example, if a customer name field contains data like “Joe Smith”, the metadata related to that field would be the name of the field, the data type, and size up to 20 characters. In relational database terms this was called the DDL (data definition language) that represented the schema.
However metadata is used to describe much more than just the specifications of a data store. My first encounter with metadata was when I was an engineer and later product manager for an application development tools vendor Synon back in the 1990’s. The breakthrough product back then utilized metadata to store a logical data model as well as to capture the designs for UI and processing logic that ultimately produced executable code. Upon completion of the design Synon’s customers could choose their target language, environments, (e.g. AS/400, UNIX) even style of UI and code distribution (e.g. Client/server vs. 5250 character only green screens). The metadata provided an abstraction layer for the design so it could be deployed to those environments using a code generator which automatically incorporated the best practices for deploying highly scalable and optimized applications. Customers were also protected and insulated from hardware and software changes, allowing them to take advantage of new capabilities quickly without complete rewrites.
Abstraction also provides benefits such as being able to create recurring models, patterns or blueprints which can be reused, thereby simplifying and accelerating the process for rolling out new applications. Ultimately, write once, run many (one of the original goals of Java) became design once, share, generate & deploy, continuously adapt for many.
Today metadata is a standard means of design artifact exchange, data model designs and other abstracted pieces of logic. The metadata can be freely exported and imported from tools and platforms for creating applications, allowing for wider interoperability and integration. Similarly for the mainstream public, metadata is used to exchange information about photographs – time, data, format, camera settings, and in audio – catalog, ownership or copyrights for digital audio files.
In 1999, during the dotcom boom, while the web was flourishing, another industry Interactive television (iTV) was gaining a second life. iTV took many forms, but the main goal was to allow the viewer to use their remote, in conjunction with their cable or satellite box to be more active with their programming. Using the television to either make purchases, or play along with game shows, and in some cases even surf the web. The same conundrum applied in that iTV applications needed to be created, but app developers could not afford to create customized code for each and every target box or software operating system. Enter my next company, MetaTV. Using metadata we captured all of the logical designs and requirements for iTV applications and once again deployed to all of the set-top boxes in living rooms all across the world.
My third foray into metadata was at Siperian in 2005 (now Informatica MDM) where metadata helped Siperian quickly support multiple data domains and business initiatives in multiple industries. For Siperian, metadata supported the notion of model-driven. A single change at the core definition (e.g. adding a new field) would be reflected in web services, match rules and other aspects of the running MDM Hub. This allowed customers to use the platform to adapt to their changing business needs. At the time this was necessary because of the inflexibility of fixed data models deployed through relational databases. But now there are much more flexible and powerful technology options so metadata can be applied in more interesting and productive ways.
With Reltio Cloud we’ve taken it several steps further. Our modern data management platform uses record level attribution flexibility through an extensible big data columnar store, dimensional analysis such as counts, distribution and frequency on each attribute across any record, and a Commercial Graph to handle complex relationships across all forms of data. But ultimately, we use a configurable metadata schema to bring it all together, rendering data into cohesive business objects.
Reltio customers are shielded from the underlying technology, again allowing them to rapidly deploy data-driven applications for any industry and business context through simple configuration. Meanwhile Reltio seamlessly handles data of any variety, volume and velocity, not just master data. When technology improves, and it always does, business users benefit, with no rewrites and no migration. It’s truly meta-magical!
Now you know why I never metadata I didn’t like.