Hi All,
Jeff and I spoke offline and we are in agreement with the hash based approach. Some takeaways:
- cleared up "shallowness" of shallow objects
- conveyed the idea of relationships which contain arrays of ids (he calls them link aggregators)
As we finalize objects across the TC we can go into object-specific required fields. Ex: should every Indicator have an observable?
Keep up the feedback.
-Marlon
From: Mates, Jeffrey CIV DC3/DCCI
Sent: Monday, March 14, 2016 9:37:13 AM
To: Taylor, Marlon;
Cc:
Subject: RE: [cti] RE: Versioning Background Docs
I generally like the idea of using a hash based approach, and helped
implement GUIDs using the UUID v5 spec internally based on this model for
STIX 1.1.1. However I do not think that they will work as expected in STIX
2.
This is because low density STIX objects can cause this model to break. We
worked through this internally in STIX 1.1.1 by tracing any idref and nested
elements and working through their hashes, but in STIX 2.0 links are now
external elements with their own IDs, so this is no longer an option.
For example it is possible for an indicator object in STIX 1.1.1 to have no
properties of its own except for its ID. As such the hash for all shallow
indicator objects will ultimately be the same and this will result in a
massive collision. The same is true for any other link aggregator object.
The effect of this could be mitigated by forcing description fields to be
filled in for STIX 2.0 objects, but if this is filled with random content
then you will end up removing the ability for the GUID to be used for
deduplication purposes. Likewise if this is filled with human generated
content then unintentional collusions will still occur.
Jeffrey Mates, Civ DC3/DCCI
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Computer Scientist
Defense Cyber Crime Institute
410-694-4335
-----Original Message-----
From: [mailto:] On Behalf
Of Taylor, Marlon
Sent: Monday, March 14, 2016 4:07 AM
To: Wunder, John A.;
Cc:
Subject: [Non-DoD Source] RE: [cti] RE: Versioning Background Docs
Hi All,
An email for our international members and US all-nighters.
I was going to type up my strategy for versioning but as I reviewed the top
2 links(see below), I saw Sean summarized it in Option 3 (honestly I was
going to say the same thing, basically). The only difference in my approach
is the use of a HASH rather than a GUID for the ID. So format:
[ObjectType-HASH]
Since Sean did such a good job analyzing and summarizing a point I was going
to make, I'll focus on my reasons for a HASH over a GUID.
HASH based IDs:
- Meaningful IDs: with GUIDs there is no relationship between the
ID and content however hashed based IDs will be directly related to the
content.
- Secret searches: someone could request info on [ObjectType-HASH]
without having to convey what [ObjectType-HASH] is. This could be done with
GUIDs however GUIDs are not based on the content so there's no assurance,
without first requesting that object, that you are getting what you think
you're referencing (you might want to always ask for a copy of what you're
referencing).
- Duplicate detection: since the hashing process will be part of
the standard (part of this idea) consumers/producers/brokers will be able to
determine if a given content is a duplicate or not.
o This is an issue that hasn't been addressed formally but can be resolved
CTI wide right here.
- Data Integrity/Assurance: since the hashing process will be know,
consumers can double-check the provided hash by computing the hash to
determine if there is mismatch.
o This addresses concerns about ensuring all needed data is passed along
with content.
A question about tracking evolving content such as incident/campaign has
been brought up(Sean also mentioned this concern).
Questions/Answers:
1. How will this work for evolving content(or series) such as
incidents?
a. Evolving content can be tracked via relationships
2. Will a new version (hash) keep the relationships of the former
version?
a. Not implicitly. All relationships to the new version must be
explicitly established.
i. This
also has advantages for derivative work concealment (don't worry about how I
got it just know this is what it is)
3. Will a user need to keep all explicit versions of an incident(or
series)?
a. No. Users are free to keep/destroy what they want.
Open Questions:
1. What hash algorithm should be used? (eg.SHA-256)
2. What fields should be included in the hash?
3. What if the provided hash and the computed hash don't match?
a. Reject the content
b. Ignore the provided hash (replace with computed hash) and process
the content
c. Add the provided hash as an alternative id (TBD) re-hash content
These links were sent to me from some one on the mailing list (Thanks!)
Links
1.
http://making-security-measurable.1364806.n2.nabble.com/STIX-Content-Revisio
n-and-Revocation-td7582483.html
2.
http://making-security-measurable.1364806.n2.nabble.com/STIX-Content-Revisio
n-and-Revocation-tp7582483p7582620.html
Image files: (I think these are the images I created, way back, in response
to Sean's diagrams)
3.
http://making-security-measurable.1364806.n2.nabble.com/attachment/7582637/0
/pg4.png
4.
http://making-security-measurable.1364806.n2.nabble.com/attachment/7582637/1
/Sharing.png
Archives
.
http://making-security-measurable.1364806.n2.nabble.com/template/NamlServlet
.jtp?macro=search_page&node=7579090&query=revocation&days=0
What do you think?
Marlon Taylor
Technology Services Section (TSS)
National Cybersecurity & Communications Integration Center (NCCIC) U.S.
Department of Homeland Security
________________________________
From: on behalf of Wunder, John A.
Sent: Friday, March 11, 2016 11:20:02 AM
To: Taylor, Marlon;
Cc:
Subject: Re: [cti] RE: Versioning Background Docs
Thanks Marlon.
This is good background material and I'd encourage everyone participating in
the versioning to read it. It's targeted at STIX 1.1 rather than 2.0 so
you'll have to mentally replace "time" with "serial number" or "revision"
but the questions are still the same.
I'm also planning to either hunt down some existing scenarios or put
together new ones that include real data so we can really dig into what this
would mean in practice. I'll send those out to the list when I'm finished.
John
From: <> on behalf of Marlon Taylor
<>
Date: Friday, March 11, 2016 at 9:20 AM
To: Marlon Taylor <>, ""
<>
Cc: "" <>
Subject: [cti] RE: Versioning Background Docs
I extracted the embedded doc and converted the email body to PDF to ease any
issues opening the previous email archive attachment.
-Marlon
From: [mailto:] On Behalf
Of Taylor, Marlon
Sent: Thursday, March 10, 2016 6:45 PM
To:
Cc:
Subject: [cti] Versioning Background Docs
Hi All,
I posted the attached files on #versioning channel on
Slack(https://cti-tc.slack.com/messages/versioning/). These docs were
generated by the STIX community way back, late 2013 - early 2014, when we
tried to address versioning.
They contain use cases and some explanations of concerns/ideas around the
general topic of versioning like IDs, Versioning, Duplicates, and
Revocation. It would be beneficial to reference these in our strategy for
versioning (not excluding other items). Once everyone has a chance to go
there the previous work we can collaborate and look at strategies for moving
forward.
Looking forward to working with you on this.
Marlon Taylor
Technology Services Section (TSS)
National Cybersecurity & Communications Integration Center (NCCIC)
U.S. Department of Homeland Security
