Yes, and I would love to see a SC setup to address this. We need people that can focus on this and drive it to completion, if it stays at the TC level, it will get lost just like before.
I am not a fan of massive unit tests, for all the reasons Eric points out, and I do not think we need a formal testing body like the WiFi Alliance (maybe in 10 years). But I would like to know things like:
Say for TAXII
1) Support Inbox Services?
2) Support Query?
3) Support Delete Requests?
4) Support Push Services?
5) Support Data Feeds?
6) Support Data Collections?
7) Support Subscriptions?
8) Support Authentication and if so what kinds?
etc etc
For STIX
1) It would be things like how much of each Idiom do they support? 2) Do they support data handling / data marking?
3) Do they support source integrity?
4) Support profiles?
etc etc
In my mind, the SC would come up with a list of things that people would do to self certify their implementations / products. They would also come up with a way to track / publish who is doing what. Kind of like what Interworks has done on their meta-server about taxii servers.
Thanks,
Bret
Bret Jordan CISSP
Director of Security Architecture and Standards | Office of the CTO
Blue Coat Systems
PGP Fingerprint: 63B4 FC53 680A 6B7D 1447 F2C0 74F8 ACAE 7415 0050
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."
On Jul 6, 2015, at 14:09, Struse, Richard <> wrote:
I think we are actually in agreement. I'm not advocating heavyweight "suites" of exhaustive tests but I do think we will benefit from a shared understanding of the basic outlines of what it means to be interoperable. Definitely a question that the SC would need to tackle.-----Original Message-----From: [mailto:] On Behalf Of Eric BurgerSent: Monday, July 06, 2015 3:47 PMTo: : Re: [cti] CTI TC Adoption and Interoperability SCsBelieve it or not, I would shy away from any measurements short of what we say someone needs to do to meet a profile, which is the profile definition itself.Likewise, I would shy away from test or verification suites. The industry’s experience with standards with elaborate verification suites has been the protocols fail because it takes ages to build the verification suites, they are obsolete when published, which ossifies the underlying specification.In a decade after everyone is using STIX and TAXII, we can think about formal acceptance suites.If the specifications are so obfuscated we need specifications to specify what the specifications specify, we have failed.On Jul 6, 2015, at 3:39 PM, Struse, Richard <> wrote:Makes sense.If we (the CTI TC) focus on defining what interoperability means forSTIX/TAXII/CybOX and how to measure/verify it, that leaves the door open forthird-party organizations such as testing labs to conduct interoperabilitytests using the CTI TC-defined benchmarks.-----Original Message-----From: [mailto:] On BehalfOf Eric BurgerSent: Monday, July 06, 2015 3:18 PMTo: : Re: [cti] CTI TC Adoption and Interoperability SCsWhen I was Chair of the SPEECHSC IETF Work Group (speech server control), wesetup a self-reported interoperability matrix:http://standardstrack.com/ietf/speechsc/MRCPv2-Plans.htmlSame for LEMONADE (mobile messaging):http://standardstrack.com/ietf/lemonade/Lemonade-Plans.htmlThe idea is OASIS will *not* become the protocol police. That is tooexpensive and carries unlimited liability. What a self-reporting resourcedoes do is (1) advertise who is running with the standards and (2) who hastested against whom. That is not a guarantee of interoperability, but it islightyears ahead of publishing and praying.On Jul 6, 2015, at 2:31 PM, Tony Rutkowski <> wrote:It would be useful if CTI maintained the equivalent ofthe MILE Implementation Report ID just announced below.Additionally, in a world of dramatic scaling of cloud datacenter virtualization, especially NFV, instantiating a CTIdedicated focus on the subject is important.--tony---------------https://tools.ietf.org/html/draft-ietf-mile-implementreport-05MILE C. InacioInternet-Draft CMUIntended status: Informational D. MiyamotoExpires: January 6, 2016 UTokyoJuly 5, 2015 MILE Implementation Report draft-ietf-mile-implementreport-05Abstract This document is a collection of implementation reports from vendors, consortiums, and researchers who have implemented one or more of the standards published from the IETF INCident Handling (INCH) and Management Incident Lightweight Exchange (MILE) working groups.---------------------------------------------------------------------To unsubscribe from this mail list, you must leave the OASIS TC thatgenerates this mail. Follow this link to all your TCs in OASIS at:https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php
Attachment:
signature.asc
Description: Message signed with OpenPGP using GPGMail
