On 10/4/2016 11:53 AM, Costello, Roger L. wrote:
> Scenario: You are tasked to build a system that will receive data from a
> variety of sources. The data arrives in different formats (e.g., CSV,
> binary, tab-delimited, JSON, XML). The data sources provide different
> kinds of data (e.g., one source provides book data, another source
> provides weather data, another source provides gardening data). The data
> will be converted to a common intermediary form and then from the
> intermediary form, placed into a data store.
One of the problems in tackling the question is that we don't know why
all this data is supposed to end up in a common data store. The data
are all about different kinds of things, so presumably there won't be an
overlap between tables (just as an example, assuming the final data
store even has tables). So why? Is it so that a single query language
can be used over all the data? Is it so that there is only one kind of
data system software to administer or maintain? Is it to satisfy some
manager who insists on this scheme? Is it a research project to see if
RDF is suitable for some task?
But there are some things I notice that relate. Some of the example
formats are essentially the same thing. e.g., CSV and tab-separated
data really the same, and they both provide a row-oriented view of the
data. They can be viewed as representing the contents of a relational
table - each row is a relation. I'm assuming that each row contains the
same columns throughout the file. We can't be sure a priory which
column(s) represent a primary key, or if there even is one, and we can't
be sure about foreign keys, either. There's your data conversion problem.
As relations in a relational table, they can be denormalized into groups
of triples, so basically RDF.
XML, JSON, and binary can potentially contain nested data structures.
So they have some real similarities, even if there could be specialized
things that are common to one format, such as processing instructions in
XML. That specialized data, and the knowledge about it, have to be put
into the specific data converter.
RDF can represent nested data structures. Again, the converter has to
be designed with specialized knowledge to make this work.
Roger quoted this claim: "RDF is the only data model the [sic] enables
such conversions to be done in a way that is computationally tractable."
I think that is silly on its face. However, if not RDF, then you
really want some other existing standard, preferably one that has had a
lot of development, and has practical software available including a
query language. So why not use RDF, which does have all those things?
Well, that might depend a lot on whether you need features that RDF is
weak on, such as reification so that you can make metastatements about
data provenance and history, for example. Since we don't know anything
about the intended usage of this universal data store, can't go further
to answer the question. But in view of the standards issue, and
depending on the usage pattern, it does seem that RDF *could* be in the
running.
TomP
