Query Use Cases Needed!

This email is a request for additional detail, as well as an attempted summary of the discussion so far (please let me know if I missed or mis-characterized your  contribution). The attempted summary is first, please continue reading through the request for additional detail.

# Summary

Use Cases:
·
Query by ID (e.g., “Give me the object with ID=1234”)
·
Query by property (e.g., “Give me the object(s) where hash=0xABCDEF0123456789”)
o
If “type” (e.g., Threat Actor, Campaign) is considered a property/meta-property, “query by property” can cover John’s “Query by construct + title” and  Jason’s “Query by threat actor + string search options”
·
Query by relationship (e.g., “Given this TTP, please give me all related CoAs”)
·
Query by sub-object (e.g., “Give me incidents with a particular asset, identified by IP”)

Candidate query features:
·
_expression_ of potentially complex/nested AND/OR constructs

Candidate “query languages”:
·
XPath
·
A Key/Value based idea
·
Cypher

Plain-English query examples:
·
Give me all indicators that contain a name of XYZ, that were seen between 2015-07-01T00:00:00 AND 2015-07-01T00:01:00 but not indicators with a TTP  of FOO
·
Give me all TTPs with a name that contains ABC, and have observables in the following net block 4.0.0.0/8 and happened during January or February of  2015

# Request for Detail

In terms of additional detail, I’m not sure I know the Role and Goal for each use case. Who wants to query by ID, and why? I’ll attempt to throw some ideas against  the wall. Recall that answering Roles/Goals for these use cases is not only for those who first mentioned them – Role/Goal question are for the entire community to answer (Note, there can be multiple Roles/Goals per use case).

·
Query by ID
o
Role: Threat Analysis platform, on behalf of threat analyst
o
Goal: Collect known missing information for threat analyst(s) (e.g., platform gets a reference to Observable_1234 but doesn’t have it). Might be automated  or manually initiated.
·
Query by property
o
Role: Threat Analysis platform
o
Goal: Collect additional information about existing objects (e.g., “What more can I learn about file w/ hash 0x123”)
·
Query by property
o
Role:
o
Goal:
·
Query by relationship
o
Role:
o
Goal:
·
Query by Sub-object
o
Role:
o
Goal:

What other Roles/Goals are there for each use case?

Thank you.
-Mark
From:
 [mailto:]  On Behalf Of  Patrick Maroney
Sent:  Thursday, August 13, 2015 10:27 AM
To:  ; Jacobsen, Jasen W. <>
Subject:  Re: [cti-taxii] Query Use Cases Needed!

I was using Cypher as an example of Graph based query languages vs. specifically advocating adoption of same.

However, I would advocate that we consider all viable options with preference/bias for an existing ad hoc/formal standard when we finalize  our Requireme ts fora TAXII Query language.

Just FYI: "Neo4j is an open source product. We support a Community edition under the GPLv3 license. The Enterprise edition is available  under the AGPLv3 license for open source projects otherwise under a commercial license from Neo Technology."


Patrick Maroney  President  Integrated Networking Technologies, Inc.  Desk:  (856)983-0001
Cell:  (609)841-5104

Email:  
On Thu, Aug 13, 2015 at 7:15 AM -0700, "Jacobsen, Jasen W." < 
> wrote:
My concern with Cypher, if offered as a candidate, is that it is proprietary (as far as I know) and would force an implementation to a  particular vendor.

There is also SPARQL.  http://www.w3.org/TR/sparql11-query / It is graph based, but targets  RDF specifically which could be problematic.

- Jasen.
From:
Patrick Maroney < 
>
Date:  Thursday, August 13, 2015 at 9:50 AM
To:  "Wunder, John A." <  >, "  " <  >,  MITRE Employee <  >, "Jordan, Bret" < 
>
Subject:  Re: [cti-taxii] Query Use Cases Needed!

In addition to the other suggestions, presuming we are talking next major release of TAXII, we should look at the Conceptual models for next gen STIX/CybOX which will presumably contain  a much richer Relationships Model when considering approaches for the Query Language.  In other words we might be able to incorporate Graph based queries (i.e., Cypher:  http://neo4j.com/developer/cypher-query-language/ )
Patrick Maroney  President  Integrated Networking Technologies, Inc.  Desk:  (856)983-0001
Cell:  (609)841-5104

Email:  
On Thu, Aug 13, 2015 at 6:22 AM -0700, "Jacobsen, Jasen W." < 
> wrote:
I suggest XQuery.

http://www.w3.org/TR/xquery /
http://www.w3schools.com/xquery /

This will not only allow very rich queries, but would allow the querier to specify what they want returned rather than entire STIX documents.

- Jasen.
From:
< 
> on behalf of "Jordan, Bret" < 
>
Date:  Thursday, August 13, 2015 at 12:12 AM
To:  "Wunder, John A." <  >, "  " < 
>
Subject:  Re: [cti-taxii] Query Use Cases Needed!

All these are really good use cases.  Thanks for taking the time to write them down.  As these are all falling in to a general theme I  think we can now being to look at the ideas that we need to support them....

So what I get from this discussion is:

1) We need a way for a client to present some sort of key/value pair where the key is the object identifier and the type of dataset that  the value exists in and then the value is what it contains.

2) We need the ability to say, if that value in the object equals something, does not equal something, contains, etc.

3) We need the ability to say this object AND this other object or this OR that.



Some examples...

1) Give me all indicators that contain a name of XYZ, that were seen between 2015-07-01T00:00:00 AND 2015-07-01T00:01:00 but not indicators  with a TTP of FOO

2) Give me all TTPs with a name that contains ABC, and have observables in the following net block 4.0.0.0/8 and happened during January  or February of 2015

The trick is going to be making this easy to do... As with all discussions on TAXII, we can easily and rapidly go from high level use-cases  to putting rubber to the road.


Simple Example Structures of a Query (to get the discussion going)

Query:
Key:
stix-indicator-name
Value:      RedHat
Operator: contains

Query
Key: stix-indicator-id
Value:      1111-1234-1234-54321
Operator: equals

This is pretty easy to wrap our brains around....  What gets more tricky is how to do an AND or an OR operator.   Any thoughts?



Thanks,

Bret



Bret Jordan CISSP
Director of Security Architecture and Standards
Office of the CTO
Blue Coat Systems
PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."

On Aug 12, 2015, at 13:47, Wunder, John A. < 
> wrote:

So here’s a few I’ve needed personally (project only partially uses STIX/TAXII now, but the concepts translate):
·
Return all courses of action related to a given set of TTPs (granted this relationship does not exist in STIX now)
·
Return all TTPs for a given set of indicators
·
Return all incidents for a given set of indicators
·
Query by construct type + title (I.e. Give me all campaigns where the name matches Deep Panda)
·
The most advanced one I can think of: return incidents with specific affected asset (identified by IP or ID)

John
From:
< 
> on behalf of Jason Keirstead
Date:  Wednesday, August 12, 2015 at 3:36 PM
To:  Mark Davidson
Cc:  "  "
Subject:  Re: [cti-taxii] Query Use Cases Needed!

Query by threat actor (with expected string search options) I would think to be important.
-  Jason Keirstead  Product Architect, Security Intelligence, IBM Security Systems
www.ibm.com/security
www.securityintelligence.com
Without data, all you are is just another person with an opinion - Unknown
<graycol.gif>  "Davidson II, Mark S" ---2015/08/12 03:22:56 PM---All, Since query was called out yesterday as a potential challenge for the Channel model that's been
From:
"Davidson II, Mark S" < 
>
To:
"  " < 
>
Date:
2015/08/12 03:22 PM
Subject:
[cti-taxii] Query Use Cases Needed!
Sent by:
< 
>
All,
Since query was called out yesterday as a potential challenge for the Channel model that’s been proposed for TAXII, I’d like for us to try and validate the Channel model against query use cases. I’d like for us to start by identifying query use cases that we  think we care about. For now, a one-liner of the use case is probably the right level of abstraction (we can add more detail later).
To get the conversation started, here are some things I’ve heard on the list:
·
Query by ID
·
Query by “observable” (e.g., IP / Hash)
Once we’ve collected the use cases, we can analyze their impact on the channel model. Please offer up your one-liner query use cases!
Thank you.  -Mark
P.S. We have not forgotten the query discussions of old; this is partly to generate discussion and partly to see what’s still on the forefront.
<graycol.gif>

---------------------------------------------------------------------

To unsubscribe from this mail list, you must leave the OASIS TC that
generates this mail.  Follow this link to all your TCs in OASIS at:
https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php