RE: [cti] Updated Extension Proposal

Hi Allan,

 

I think it would be better/quicker to delay further discussions

until online occasions. I just list two clarifications.

 

- I got your point on what products display for contents they do not understand.

 I am afraid, though, that I have misled you with "raise errors" phrase.

  The case I am talking about is where an implementation dismisses/drops the whole

  STIX file (as an error) when they see custom objects/properties they do not understand.

  I would like to see the valid part (that is what they understand) of the STIX

  gets through to the consumer even if the whole STIX has some unknown (to the consumer)

  custom objects/properties.

 

- The cases, where I thought Custom Objects/Properties or SEP will be used,

  includes communities like ISAC/ISAO other than Intel providers.

  The community members might be using different products.

  It would be difficult for the community to introduce

  community-specific custom contents if some of the products used in the community

  drop the whole STIX with some unknown custom contents.

 

Regards,

 

Ryu

 

From: aa tt <>

Sent: Tuesday, August 25, 2020 4:38 AM

To: Masuoka, Ryusuke/çå
çä <>

Cc: Paul Patrick <>; 

Subject: Re: [cti] Updated Extension Proposal

 

Hi Ryu - I donât believe its practical nor possible to mandate what products do when they encounter custom content from an intel provider.

 

In many cases, there are legitimate reasons why a particular product would *want* to display some warning or error when it fails to understand or import all of an intel providerâs content. Whether those warnings or errors
 are shown in a log file or directly to the user is again a product choice. Not OASIS/standards choice.

 

So I know the approach taken with STIXPreferred was to test that the software being verified for certification would not terminate or fail completely and the verification was done on ensuring non-fatal processing. But
 mandating that the product does not generate an error or warning is not possible. In some cases, a product may prompt the user for guidance on what to do with the intelligence given that in some cases its not possible to automate a decision at all.

 

Allan

On Aug 24, 2020, at 12:05 AM, 
 wrote:

 

Hi Allan,

 

I will wait for the conversation to begin before diving into the deep discussions.

 

Just quickly ...

I am open to the language as long as STIX 2.x implementations do not raise

errors when they encounter SEP contents that they do not understand.

We have given up on using Custom Objects and Properties due to this very concern.

I am afraid that this concern may hamper SEP's idea of the process of gradually

moving custom objects/properties into the standard.

STIXPreferred is for an org that produces or consumes custom objects/properties and

it does not cover other orgs that happen to receive custom objects/properties.

 

Regards,

 

Ryu

 

 

From:  <> On
 Behalf Of aa tt

Sent: Saturday, August 22, 2020 6:00 AM

To: Masuoka, Ryusuke/çå çä <>

Cc: Paul Patrick <>; 

Subject: Re: [cti] Updated Extension Proposal

 

Hi Ryu - Weâre still waiting on the chairs/co-chairs to convene a specific TC working call to discuss the proposal.

 

As soon as that call is scheduled we should include your points in that discussion. 

 

However, I suggest that any language in the spec on things a consumer of STIX does not understand should not state
âMUST processâ. The definition of
âprocess' is ambiguous and therefore not testable without clarification. I would suggest we consider carefully the language used for compliance very carefully so that test specification may be written and easily checked against. This
 will become especially important for STIXPreferred if/when that gets updated for STIX2.1.

 

If you take a look at how STIXPreferred currently expresses the interoperability of solutions handling custom properties/objects that might help guide any future changes to the spec when we are talking about handling
 content defined as an extension.

 

regards

 

Allan

On Aug 21, 2020, at 1:09 AM,  wrote:

 

Hi Allan, Patrick,

 

Thank you for your input and sorry for my late response.

(I am back from my long summer (Obon) break.)

 

I too am supportive of replacing the existing customization mechanism with this proposal

and deprecating the current approach until a future release based on the following reasons.

 

- There should be only one way to do a certain thing.

  I understand this is one of the lessons the TC learned from STIX 1.x experiences.

 

- The current approach ("11 Customizing STIX" of STIX 2.1) says

  

  A consumer that receives STIX content containing Custom Properties, Objects or Extensions

  it does not understand MAY refuse to process the
 content or MAY ignore those properties 

  or objects and continue processing the content.

 

 and some STIX 2.x implementation actually raises errors when they see

 Custom Properties/Objects that it does not understand.

 When this happens, it is really difficult to use Custom Properties/Objects and

 it limits their utility/practicality. They can ignore what they do not understand, but

  I would like the implementations to process the STIX as long as it is conformant.

 

 I hope that SEP is going to be like "A consumer MAY ignore

  the STIX it does not understand, but MUST process the STIX as long as

  it is conformant (to SEP)." Am I right to expect that?

 

Regards,

 

Ryu

 

From: Paul
 Patrick <> 

Sent: Friday, August 7, 2020 12:48 AM

To: aa tt <>; Masuoka, Ryusuke/çå çä <>

Cc: 

Subject: Re: [cti] Updated Extension Proposal

 

Ryu,

 

I generally agree with Allan in that I see the STIX Extension proposal as a candidate to replace the way customization of STIX 2.1 is done today.  Like Allan,
 I too would be supportive of replacing the existing customization mechanism with this proposal but would recommend that the TC deprecate, not remove, the current approach until a future release.

 

To that end, I believe the section in the spec on Customizing STIX be updated to not only provide guidance on how to use the STIX Extension approach but provide
 guidance how to use it IN PLACE OF the existing schema.  I can also imagine what an update to STIX Elevator could be to aid in stepping-up from STIX 2.1 to STIX 2.1.1 to make that transition easier for people to move to the new scheme.

 

Paul Patrick

DarkLight, Inc.

 

 

From: <> on behalf of aa tt <>

Date: Thursday, August 6, 2020 at 11:33 AM

To: "" <>

Cc: "" <>

Subject: Re: [cti] Updated Extension Proposal

 

Hi Ryu -

 

My personal feeling is that customization of STIX will remain in the standard as-is but industry best-practices will steer orgs towards using the extension proposal as the primary mechanism to change STIX.

 

The rationale: JSON content can always be modified regardless of what the CTI TC decides and the customization language in the current specification can just remain as most people already have done customization in products.

 

However, if the TC felt that it was better to remove the customization language and replace with this extension proposal then I would be supportive of that also.

 

Other proponents of the extension proposal, as well as other TC members should speak up on this topic.

 

regards

 

allan

 

On Aug 5, 2020, at 11:35 PM,  wrote:

 

Hi Allan,

 

I might have missed some discussions, but

what are the relationships between

"11 Customizing STIX" of STIX 2.1 (custom properties,

custom objects, and custom extensions) and

this SEP proposal?

 

Regards,

 

Ryu

 

From:  <> On
 Behalf Of aa tt

Sent: Thursday, August 6, 2020 12:28 AM

To: 

Subject: [cti] Updated Extension Proposal

 

CTI TC - 

 

Please find attached an updated STIX extension proposal that the mini-group worked with IBM (Jason/Emily) to incorporate an acceptable compromise to their concerns.

 

In summary:

 

1) Previously presented extension proposal is maintained and has been named âsub-component extensionâ to
 clarify the distinction between when it would/should be used vs the additional mechanism added for top-level extension.

2) Added the concept of Top-Level object extension to allow extension properties directly with existing STIX standard objects.

3) Clarified language/definitions around use for both aspects.

 

The google doc containing the specification changes are here: 

https://docs.google.com/document/d/1TkXv1btieo33BAEcVsprgbVob5qeHOHVAQU0bg9AK-c/edit#heading=h.y2hw6f789m5b

 

This proposal represents the agreement by all mini-group participants. We would like to encourage the TC to consider this proposal and incorporate the changes to the STIX standard.

 

Allan Thomson