Re: [xml-dev] What is the general direction you are seeing these days to store and query lots of large complex XML?

From
Simon St.Laurent <>
To
"Liam R. E. Quin" <>
Date
2015-03-08T19:37:35Z
ID
<>
Thread
Re: [xml-dev] What is the general direction you are seeing these days to store and query lots of large complex XML?
On 03/07/2015 09:05 PM, Liam R. E. Quin wrote:
> On Sat, 2015-03-07 at 17:03 -0500, Simon St.Laurent wrote:
>> On 03/05/2015 11:20 AM, Peter Hunsberger wrote:
>>> Without knowing all the details, given relatively unchanging I'd
>>> likely send up parsing them into a database.
>>
>> Yep - this is the answer I keep hearing from people, with the
>> exception of people who have mixed content and tend to think of it
>> all as documents.
>
> Peter didn't say relational database... although he did mention Hadoop
> I think.
>
> There isn't a single right answer.

I agree.  I'm not hearing so much about relational databases per se as I 
am about shredding approaches, with data stored in anything from MySQL 
to Cassandra to Neo4j to custom systems that sound both terrifying and 
unnecessary.

Somehow mixed content leads people to think of chunks as coherent 
pieces, and some of the shredding cases I've discussed with people keep 
large pieces of mixed content, typically (X)HTML, in the same shredded 
bit rather tearing deeper into it.

> So for Roger I'd say that I think the direction I see is more hybrid
> stores (from Virtuoso to MarkLogic..) and more variation being
> acceptable as people come to recognize that different needs are best
> served with different technologies.

Yep.  I suspect the underlying reality is that people are doing so many 
things with so many different kinds of parts that solutions are evolving 
in multiple directions.

Which is pretty cool, actually!  Though it creates vast confusion for 
organizations that just want an answer yesterday.

Thanks,
Simon