netkvm: does it need to wait when StopFlows

[wangyan0507]

When shutdown the windows vm with netkvm driver, shoud it wait the m_Counter to zero for m_RxStateMachine?
The shutdown stack:

ndis!ndisDispatchIoWorkItem
->netkvm!OnSetPowerWorkItem
  ->netkvm!ParaNdis_PowerOff
    ->netkvm!CMiniportStateMachine::NotifySuspended
      ->netkvm!CMiniportStateMachine::StopFlows
        ->netkvm!CNdisList<CDataFlowStateMachine,CRawAccess,CNonCountingObject>::ForEach
          ->netkvm!CFlowStateMachine::Stop
            ->m_NoOutstandingItems.Wait();  // which will wait the m_Counter to become StoppedMask.

For shutdown vm scene, when there is a lot of incomping packets, the m_Counter will be released as follows for m_RxStateMachine:

ParaNdis6_ReturnNetBufferLists()
  ->pContext->m_RxStateMachine.UnregisterOutstandingItems(nofNetkvm);  // which will release for m_Counter.
    ->m_Counter.Release(NumItems)
    -> CheckCompletion(value)
      ->CompleteStopping()
        ->m_NoOutstandingItems.Notify(); // which will awake the upper 'm_NoOutstandingItems.Wait()'.

But from the microsoft doc for OID_PNP_SET_POWER:
The miniport driver that supports NDIS 6.30 and later versions of NDIS must also do the following:
Not wait for the completion of pending receive indications through calls to its MiniportReturnNetBufferLists function.

Thanks in advance!
Yan Wang

[YanVugenfirer]

@wangyan0507 Unfortunately, paravirtualizaed "HW" behaves differently than actual HW in this case, so we must wait.
If we release the memory (the NET_BUFFER_LISTs), the host might still continue to write to memory that can potentially be reused by the guest.

[wangyan0507]

@YanVugenfirer Sorry, I am not understand very well.
The shutdown process:

ndis!ndisDispatchIoWorkItem
->netkvm!OnSetPowerWorkItem
  ->netkvm!ParaNdis_PowerOff
    ->netkvm!CMiniportStateMachine::NotifySuspended
      ->netkvm!CMiniportStateMachine::StopFlows
        ->netkvm!CNdisList<CDataFlowStateMachine,CRawAccess,CNonCountingObject>::ForEach
          ->netkvm!CFlowStateMachine::Stop
            ->m_Counter.SetMask(StoppedMask); // Or with StoppedMask to m_Counter.
            ->m_NoOutstandingItems.Wait();    // Wait the m_Counter to become StoppedMask.

The normal rx process:

ndis!ndisMiniportDpc
  ->netkvm!MiniportMSIInterruptDpc
    ->netkvm!ParaNdis_RXTXDPCWorkBody
      ->netkvm!RxDPCWorkBody
        ->pContext->m_RxStateMachine.RegisterOutstandingItems // Add reference for m_Counter.
        ->ndis!NdisMIndicateReceiveNetBufferLists
          ->...
            ->netkvm!ParaNdis6_ReturnNetBufferLists
              ->pContext->m_RxStateMachine.UnregisterOutstandingItems
                ->m_Counter.Release(NumItems) // Release NumItems for m_Counter.
                -> CheckCompletion(value)
                  ->CompleteStopping()
                    ->m_NoOutstandingItems.Notify(); // Wake the upper 'm_NoOutstandingItems.Wait()'.

In "Not wait for the completion of pending receive indications through calls to its MiniportReturnNetBufferLists function.", does it mean no need to wait the call for netkvm!ParaNdis6_ReturnNetBufferLists? If it is, the call m_NoOutstandingItems.Wait() may be get stuck, because it depends on the m_Counter to become StoppedMask.
You mentioned "If we release the memory (the NET_BUFFER_LISTs)", I don't know where do we release the memory?

Thanks!
Yan Wang

[YanVugenfirer]

@wangyan0507 Do you see an actual case when we are stuck waiting for completed buffers, or is it a theoretical question?

[wangyan0507]

@YanVugenfirer Yes, I got a stuck waiting for completed buffers in the older netkvm drivers(commit 315c637). The bsod RIVER_POWER_STATE_FAILURE (9f) happend.
The stuck stack is as follows, the m_Counter is 1 with the StoppedMask.

[YanVugenfirer]

Can you please send more details, like what OS, what drivers version, and the exact scenario?

Thank you.