virtio — archive

[Luiz Capitulino <mst@redhat.com>]

On Thu, 30 Jan 2014 20:42:00 +0200 "Michael S. Tsirkin" <mst@redhat.com> wrote: > On Thu, Jan 30, 2014 at 11:47:19AM -0500, Luiz Capitulino wrote: > > On Thu, 30 Jan 2014 12:16:29 +0200 > > "Michael S. Tsirkin" <mst@redhat.com> wrote: > > > > > Also copy virtio-dev since this in clearly implementation ... > > > > > > On Thu, Jan 30, 2014 at 07:34:30PM +1030, Rusty Russell wrote: > > > > Hi, > > > > > > > > I tried to write a new balloon driver; it's completely untested > > > > (as I need to write the device). The protocol is basically two vqs, one > > > > for the guest to send commands, one for the host to send commands. > > > > > > > > Some interesting things come out: > > > > 1) We do need to explicitly tell the host where the page is we want. > > > > This is required for compaction, for example. > > > > > > > > 2) We need to be able to exceed the balloon target, especially for page > > > > migration. Thus there's no mechanism for the device to refuse to > > > > give us the pages. > > > > > > > > 3) The device can offer multiple page sizes, but the driver can only > > > > accept one. I'm not sure if this is useful, as guests are either > > > > huge page backed or not, and returning sub-pages isn't useful. > > > > > > > > Linux demo code follows. > > > > > > > > Cheers, > > > > Rusty. > > > > > > More comments: > > > - for projects like auto-ballooning that Luiz works on, > > > it's not nice that to swap page 1 for page 2 > > > you have to inflate then deflate > > > besides overhead this confuses the host: > > > imagine you tell QEMU to increase target, > > > meanwhile guest inflates temporarily, > > > QEMU thinks okay done, now you suddenly deflate. > > > > Yes. Just to give more context: one of my auto-ballooning versions broke > > when virtballoon_migratepage() ran. The reason was that my host-side code > > in the balloon device did not expect guest initiated operations. And the > > current spec does imply that all operations are initiated by the host. > > > > So, first suggestion: if the current spec is still valid, we have to add > > a note there that balloon operations can be initiated by the guest. > > > > My current code is different, but something it does that could also brake > > due to guest initiated inflate/deflate is that it keeps track of the > > current balloon size. This is done by a counter which is incremented > > on inflate and decremented on deflate. I did that because the device just > > doesn't have this information ('actual' is unreliable, besides it's > > only updated every 256 pages inflated/deflated). > > > > Second suggestion: I think we need a reliable way to know the current > > balloon size on the host. My counter does work, btw. > > > > As far as the guest is concerned, my current code just informs the host > > that the guest is facing pressure. This is done through a "message" virtqueue, > > but I think this could just use the guest command virtqueue. > > > > > A couple of other suggestions: > > > > > > - how to accomodate memory pressure in guest? > > > Let's add a field telling host how hard do we > > > want our memory back > > > > I agree we have to accommodate pressure in the guest some way, but what > > you proposed is more or less related to auto-ballooning. > > > > My suggestion would be for the host to tell the guest what to do in > > case of pressure. Like, it could tell the guest to just keep trying like > > it does today or it could ask the guest to stop inflation on pressure > > (which would require an ack from the host, which complicates the > > protocol a bit). > > If we need ack anyway, it seems enough to notify host > and then host can ask guest to stop inflating? Yep, that's exactly what auto-ballooning does. When the guest gets into pressure while inflating, it holds the inflation and sends a message to QEMU saying "hey, I'm under pressure". The guest doesn't continue until QEMU acks that message. This gives QEMU enough time to reset num_pages to the current balloon size (which cancels inflate) then QEMU acks the message and the guest stops inflating, as num_pages == balloon size. > > Also, there are two ways to know the guest is under pressure: 1. when > > alloc_page() fails or 2. use in-kernel vmpressure notification like > > auto-balloon does. > > 2 can detect pressure earlier so seems nicer. Yeah, exactly. > > > - assume you want to over-commit host and start > > > inflating balloon. > > > If low on memory it might be better for guest to > > > wait a bit before inflating. > > > Also, if host asks for a lot of memory a ton of > > > allocations will slow guest significantly. > > > But for guest to do the right thing we need host to tell guest what > > > are its memory and time contraints. > > > Let's add a field telling guest how hard do we > > > want it to give us memory (e.g. time limit) > > > > I think this is also related to auto-ballooning. > > Maybe we should start > > with a simple device/driver and add all these features on top. > > > Just shows these are good ideas :). > > I tried to propose building blocks in a generic way that will > be useful for multiple features. > autoballoon can build on top but these features > are useful even with manual inflate. Agreed.