RE: [mqtt-sn] Re: Packet Type Field Change

I agree

Andy

Andy Stanford-Clark

Innovation Leader - IBM SPEED team

  Distinguished Engineer, Master Inventor, IBM Quantum Ambassador,

  Member IBM Academy of Technology, Fellow BCS
  Visiting Professor at Newcastle, Southampton and UEA.
Tel: +44 (0)7801 787096

twitter: @andysc
From:  Ian Craggs <>
  Sent:  11 January 2023 12:43
To:  Andy Stanford-Clark <>
Cc:  Simon Johnson <>; Davide Lenzarini <>;  <>;  <>
Subject:  [EXTERNAL] Re: [mqtt-sn] Re: Packet Type Field Change


 If we have a new packet type for publish QoS -1, then that packet type can contain a byte for the protocol version to allow for any further protocol version changes. That would be my favoured option currently of the ones suggested. There may

 ZjQcmQRYFpfptBannerStart

 This Message Is From an External Sender

 This message came from outside your organization.



 ZjQcmQRYFpfptBannerEnd

  If we have a new packet type for publish QoS -1, then that packet type can contain a byte for the protocol version to allow for any further protocol version changes. That would be my favoured option currently of the ones suggested. There may
  be some "spare" bits in the protocol byte, but in my opinion we should make the protocol field match in publish QoS -1 and connect packets. So if we wanted to change the approach to protocol versioning, we should do it in both cases.

  On Tue, 10 Jan 2023 at 11:01, Andy Stanford-Clark < 
> wrote:

Thanks Si
  ð

Andy

Andy Stanford-Clark

Innovation Leader - IBM SPEED team

  Distinguished Engineer, Master Inventor, IBM Quantum Ambassador,

  Member IBM Academy of Technology, Fellow BCS
  Visiting Professor at Newcastle, Southampton and UEA.
Tel: +44 (0)7801 787096

twitter: @andysc
From:
    < 
> on behalf of Simon Johnson < 
>
  Sent:  10 January 2023 10:50
To:  Andy Stanford-Clark < 
>
Cc:  Davide Lenzarini <  >; Ian Craggs <  >;
    <  >;
    < 
>
Subject:  [EXTERNAL] [mqtt-sn] Re: Packet Type Field Change


 Agreed, we need to ensure the PUBLISH -1 is available out of session as this has huge benefit for less typical use cases. I also hear Davide s point about avoiding extra bytes where possible. I think we should probably discuss this one on a

 ZjQcmQRYFpfptBannerStart

 This Message Is From an External Sender

 This message came from outside your organization.



 ZjQcmQRYFpfptBannerEnd
 Agreed, we need to ensure the PUBLISH -1 is available out of session as this has huge benefit for less typical use cases. I also hear Davide s point about avoiding extra bytes where possible. I think we should probably discuss this one on a call as I would
  really like input from Ian, Tara et Al as well. I will consolidate this feedback and put together a couple of worked up proposals for discussion on next STC meeting (hopefully next week).

  Thanks everyone

  S

  Sent from my iPhone

  On 10 Jan 2023, at 09:04, Andy Stanford-Clark < 
> wrote:

  ï

On 5... if I've understood what you're saying, then I would say that the original intent of QoS -1 was for devices that don't
  have a receive capability - only a transmit - hence the "lower than qos 0" - it's literally fire and forget, and even worse, have no idea if it got there or not.

So we can't limit it to Active and Asleep states.

In my CAN bus project, I'm thinking of it as "drive-by publishing"
  ð

Andy

Andy Stanford-Clark

Innovation Leader - IBM SPEED team

  Distinguished Engineer, Master Inventor, IBM Quantum Ambassador,

  Member IBM Academy of Technology, Fellow BCS
  Visiting Professor at Newcastle, Southampton and UEA.
Tel: +44 (0)7801 787096

twitter: @andysc
From:  Davide Lenzarini < 
>
  Sent:  10 January 2023 08:48
To:  Simon Johnson < 
>
Cc:  Ian Craggs <  >;
    <  >;
    <  >; Andy Stanford-Clark < 
>
Subject:  [EXTERNAL] RE: Packet Type Field Change


 Hi Simon, Ok to add a new packet type for v2 PUBLISH-1 and leave 0x0C for v1. 2 PUBLISH. If I have correctly understood, this option may limit the variants of PUBLISH-1 to only 2. This option increases the overhead of PUBLISH-1 by 1 byte and

 ZjQcmQRYFpfptBannerStart

 This Message Is From an External Sender

 This message came from outside your organization.



 ZjQcmQRYFpfptBannerEnd

  Hi Simon,

  Ok to add a new packet type for v2 PUBLISH-1 and leave 0x0C for v1.2 PUBLISH.  If I have correctly understood, this option may limit the variants of PUBLISH-1 to only 2.  This option increases the overhead of PUBLISH-1 by 1 byte and I think we should avoid this.  .  We may limit the usage of PUBLISH-1 only to the Active and Asleep states, so only after a session has been established (eventually lasting for a very long time). In any case for security reasons we need to identify the sender and
  provide it with a token to authorize the data published as QoS-1.
  Bye
  Davide
From:  Simon Johnson < 
>

  Sent:  Monday, January 9, 2023 4:03 PM
To:  Andy Stanford-Clark < 
>
Cc:  Davide Lenzarini <  >; Ian Craggs <  >;
   ;
  
Subject:  Re: Packet Type Field Change



**** This is an EXTERNAL email. It was sent from outside of u-blox. ****

  I fear we may have our hand forced actually; the ENCAPSULATED message type sits at 0xFE right at the end of the reserved range in 1.2 which precludes the use of bits 0-1 for any nefarious purposes such as this so the available solutions to this as I see
  it are as follows:



  Create a NEW packet type for version 2 which is a new PUBLISH_MINUS_1 specific packet format such that version 1.2 PUBLISH -1's received by a gateway will decode successfully in the old format and can be differentiated  Modify the meaning of flags (byte 3) field bit 3 (presently reserved in the current PUBLISH spec) where 0 means "old style PUBLISH" and 1 means "new style -1 PUBLISH" - this mean old style packets will still decode ok since they are forced to send 0 here,
  and there is no extra byte required -  this is important since many old client impls still exist and we will want to be able to accept valid 1.2 traffic on new gateways
Use one of those reserved bits above to mean "protocol" version byte follows, which when set to 1 will decode the a protocol byte inserted at Byte 4  Ignore and accept the broken change  Other suggestions....

  Now we have the packet versions broken out for QoS -1 & 0 in the document anyway, this seems less impactful - but as Andy said, we need to ensure we're good going forward.



  S





  On Mon, 9 Jan 2023 at 09:26, Andy Stanford-Clark < 
> wrote:

Well spotted, Si
  ð



Isn't this just kicking the can down the road, Davide?

 Perhaps we should bite the bullet now and take the extra byte for the protocol version?

Our future selves will one day thank us

ð



I really like the "drive-by publish" of QoS -1, but given that it is completely "self contained" it should not surprise us that it's a bit bigger than a normal "in session" publish.



I admit to being torn on this one, as "spare bits" always make me unhappy!



Andy



Andy Stanford-Clark

Innovation Leader - IBM SPEED team

Distinguished Engineer, Master Inventor, IBM Quantum Ambassador,

Member IBM Academy of Technology, Fellow BCS

Visiting Professor at Newcastle, Southampton and UEA.
Tel: +44 (0)7801 787096       twitter: @andysc
From:
   < 
> on behalf of Davide Lenzarini < 
>
  Sent:  09 January 2023 08:52
To:  Simon Johnson < 
>
Cc:  Ian Craggs <  >;
    <  >;
    < 
>
Subject:  [EXTERNAL] [mqtt-sn] RE: Packet Type Field Change



Great catch Simon! I agree with your approach as consistent with MQTT5. I am anyway scared about the fact that the  Protocol Version  in CONNECT is represented over 1 byte and in PUBLISH-1 is represented over 2 bits.
  I propose the following

ZjQcmQRYFpfptBannerStart

This Message Is From an External Sender

This message came from outside your organization.



ZjQcmQRYFpfptBannerEnd

  Great catch Simon!
  I agree with your approach as consistent with MQTT5.
  I am anyway scared about the fact that the  Protocol Version  in CONNECT is represented over 1 byte and in PUBLISH-1 is represented over 2 bits.

  I propose the following 2 options to have a future-proof solution:

  Option 1 (Packet type 2 bits flag extension):

  00 means MQTT-SN v1.2  01 means MQTT-SN v2  10 is reserved  11 means there is a following dedicated byte providing the  Protocol Version

  Option 2 (Packet type refactoring)

PUBLISH 0x0C (MQTT-SN v2)

PUBLISHv1_2 0x1E (MQTT-SN v1.2)

  Bye
  Davide
From:  Simon Johnson < 
>

  Sent:  Sunday, January 8, 2023 11:31 AM
To:

Cc:  Ian Craggs <  >;
   ; Davide Lenzarini < 
>
Subject:  Re: Packet Type Field Change



**** This is an EXTERNAL email. It was sent from outside of u-blox. ****

  ERRATUM: That should be bits 7 &  6 for the flags and 5 - 0 for packet type



  On Sun, 8 Jan 2023 at 10:13, Simon Johnson < 
> wrote:

  Good morning folks,



  I have been working with PUBLISH -1 in v2. I have found a flaw with the new packet type layouts. Presently you can PUBLISH -1 without a session, and it is the SESSION that determines the protocol version you have decided to use. Without a session, the GATEWAY
  has no mechanism to determine the version of the PUBLISH packet to use to decode it. So to maintain the ability for a GW to support both protocol versions, we must have a mechanism in the PUBLISH packet to note protocol version WITHOUT adding any further bytes.



  I was looking, the packet type field seems to be a great chance to squeeze some more space out of - in a similar fashion to tt5, so we coolocate a generic flags field in bits 1 & 0, using 6 bits of space (7-2) for packet type (64 available types - we currently
  use ~30) so plenty of growing room - then the 2 new flags can be used for flags on the message without needing a new flags field. In the case of PUBLISH this can be protocol version, but also it could be used for a few other types specifying and using only
  1 or 2 flags saving space elsewhere. I feel this is a fundamental enough issue to warrant this change - and give us further flexibility for saving space elsewhere.



  I still havn't gained access to the ticketing system, but will press OASIS next week and raise a ticket when I can. I'm happy to create a WD version (25) with just this change applied to all the message types this week for review.



  Further, Davide, I have made the changes to the doc RE: network connection and will submit WD 24 this coming week.



  Best,

  Si





  Unless otherwise stated above:

 IBM United Kingdom Limited
 Registered in England and Wales with number 741598
 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU

  Unless otherwise stated above:

 IBM United Kingdom Limited
 Registered in England and Wales with number 741598
 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU

  Unless otherwise stated above:

 IBM United Kingdom Limited
 Registered in England and Wales with number 741598
 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU

--
Ian Craggs CEng MBCS CITP
  Eclipse Paho project lead; Eclipse IoT PMC; MQTT-SN OASIS TC chair

Unless otherwise stated above:
IBM United Kingdom Limited  Registered in England and Wales with number 741598  Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU