amqp — archive

AW: [amqp] Size-based link flow control

From: Rob Godfrey <[email protected]>
To: [email protected]
Date: Mon, 11 Mar 2019 09:00:00 -0000

On Sat, 9 Mar 2019 at 00:06, Clemens Vasters < [email protected] > wrote: The session flow control has frame-size resolution and we also use that to cap memory use in the gateway. The links end up interacting with the respective link target s backend processor/storage. We do have the scenario where a single inbound session carries a bundle of links where each link talks to a different backend. For that, the team wants finer grained flow control metric than a message count. Von: Alan Conway < [email protected] > Gesendet: Freitag, 8. MÃrz 2019 12:46 An: Clemens Vasters < [email protected] > Cc: [email protected] Betreff: Re: [amqp] Size-based link flow control Qpid Dispatch uses the existing byte-based session flow control to bound its memory use. It does effectively block clients that send a lot of bytes - regardless of whether it is many small messages or some very large ones. Why do you need a per-link limit on top of that? A per-link limit allows senders that create a ton of links to blow up the receiver. Of course a malicious sender could create a ton of sessions, but that's much less likely to happen by accident, and the max number of sessions is easy to limit giving you a guaranteed upper-bound on memory per connection. On Fri, Mar 8, 2019 at 2:34 PM Clemens Vasters < [email protected] > wrote: Our engineering team has the strong desire to add a flow control mode where link-credit is counted in bytes rather than in number of messages. We re occasionally getting into the situation where we re being hammered with a ton of large messages all at once and team is convinced that a byte-oriented credit counter could allow for tighter control of what the client are allowed to push. For session-based flow control, the permitted windows we need are also too big. The Flow control section 2.6.7. in the core spec is surprisingly vague and doesn t have a hard reference to link-credit being a message counter. The only hard reference seems to be is in 2.7.4. where the link-credit section says that The current maximum number of messages that can be handled at the receiver endpoint of the link When designing AMQP 1.0 originally we deliberately didn't add a literal "bytes" level flow control based on prior experience in earlier protocols - bytes accounting is hard, both in agreeing what is and isn't included in the count (headers, framing overhead, etc) and also in determining behaviour when only partial credit is available. As Alan described, the session-level flow control based on frames was added as a way of performing some form of "byte-like" flow control, frame sizes can be made arbitrarily small if the desire is to closely account for the number of bytes to be sent/received. If 2.6.7 doesn't explicitly state that link-credit is a message count that would be because it was thought to be obvious that this was the meaning :-), e.g. the definition of link credit in 2.6.7 states: link-credit The link-credit variable defines the current maximum legal amount that the delivery-count can be increased by. This identifies a delivery-limit that can be computed by adding the link-credit to the delivery-count . Only the receiver can independently choose a value for this field. The sender's value MUST always be maintained in such a way as to match the delivery-limit identified by the receiver. This means that the sender's link-credit variable MUST be set according to this formula when flow information is given by the receiver: link-credit snd := delivery-count rcv + link-credit rcv - delivery-count snd . Which I think makes it pretty clear that that link-credit is to be calculated in terms of the delivery count. Now if you tried really hard to pretend this was ambiguous then you could, I suppose, claim that delivery count only talks of being incremented when ever a message is sent rather than explicitly stating "incremented by 1"... but that's a big reach. I'd be strongly against trying to change the meaning of link credit - this would, I think lead to horrible incompatibilities between implementations from different vendors. If you are absolutely dead set on adding some sort of bytes based flow control, I would suggest making use of the properties field on the flow performative as a way of conveying this additional flow control mechanism (along with some sort of initial connection capability exchange at open so that both sides can declare their ability to use this feature). -- Rob So we re thinking about an extra spec with a layered-on capability that allows for reinterpretation of link-credit and delivery-count as counts of bytes. Thoughts? -- ______________________________ ______________________________ _________________ Red Hat GmbH, www.de.redhat.com , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: ,Charles Cachera, Michael Cunningham, Michael O'Neill, Eric Shander