Can I turn off buffer sharing among ports on a switch?

[daihuichen]

Hi Yibo,
From your replies in the issue Some questions about how modules work in simlator, I learnt that all the ports on a switch share the same queue buffer by "A node can have multiple qbb-net-device (especially on a switch), which share the same m_broadcom and m_queue."
I would like to know, can I turn off the buffer sharing and let each port have its own fixed-size buffer? In this way I can control the buffer resource allocated to each switch port.

Thanks~

[bobzhuyb]

Yes, you can configure per port threshold here

ns3-rdma/src/network/model/broadcom-node.cc

Line 83 in c414c6b

m_port_max_shared_cell = 4800 * 1030; //max buffer for an ingress port

Right now it's set to a large threshold so it won't really trigger.

You may also need to look at a few other parameters around this line.

[daihuichen]

Got it.
Where do you mark the packets (the ECN bits), the input queue or the output queue? (I think it's in the input queue, just want to confirm.)

m_port_max_shared_cell = 4800 * 1030; //max buffer for an ingress port
I feel a little bit confused. By the parameter name, it is the buffer size that shared by all the ports, but according to your reply (and also the comment) it is the queue size threshold for each port. I want to confirm that this is where I should adjust.

Besides the (private) queue size threshold for each port specified by this parameter, is there any switch I need to turn off to prevent the packets from using the shared buffer when the queue is full?
I found this line commented out:
//m_buffer_cell_limit_sp_shared=4000*1030; //ingress sp buffer shared threshold, nonshare -> share
Maybe I should uncomment this line?

Thanks~

[bobzhuyb]

"shared" means it's shared by all ingress pg (priority groups) of that port. sp means service pool.

These are Broadcom terminology. It's roughly Total buffer = guarantee + sp + headroom; all ports' shared buffer is in sp.

For more concrete details you can read the code
https://github.com/bobzhuyb/ns3-rdma/blob/master/src/network/model/broadcom-node.cc#L118
and
https://github.com/bobzhuyb/ns3-rdma/blob/master/src/network/model/broadcom-node.cc#L143

To turn off ingress sharing, make sure m_dynamicth is false and this is not called:
https://github.com/bobzhuyb/ns3-rdma/blob/master/src/network/model/broadcom-node.cc#L412
(config.txt has a knob for this).

I don't think I implemented egress sharing.