set queue rate/depth commands with priority queues

[carlos-hfm]

Hi, I am working on simple_switch target with priority queues.

I am generating traffic to the queues, and trying to change the capacity and rate of each queue, to see the impact of it in the QoS and in the occupation of each one. But, in my tests, it seems that the set queue rate/depth commands are changing the values of the inverse queue of the input!

For example, when I have 3 priority queues, and I generate traffic (UDP) to queue/priority 2, and I want to reduce the rate of that queue, I use the command set_queue_rate 100 0 2 (on simple_switch_CLI), I can't see any difference in the QoS or in the queue occupation. But if I use the set_queue_rate 100 0 0, then the QoS and occupation of queue 2 change! The same thing happens when I change the depth/capacity of the queues.

Then I repeated the test with 4 priority queues, and in this case I had to "change queue 1" to see the difference in queue 2 occupation.

In the past I did a little change in simple_switch.cpp code, to return the occupation of the specific priority queue of each packet pass in the enq_qdepth metadata, but I think it is not the reason of the problem.

Can anybody repeat these tests to confirm?

[jafingerhut]

I have not tested to verify what you are seeing, but I did look for some code that might be the root cause of it, and found some:

• https://github.com/p4lang/behavioral-model/blob/main/targets/simple_switch/simple_switch.cpp#L421-L423
• https://github.com/p4lang/behavioral-model/blob/main/targets/simple_switch/simple_switch.cpp#L673

In particular, the expressions that look like ((# of queues) - 1 - priority) for selecting which queue to put a packet into.

It has been like this since 2017, when multiple queues were introduced. I do not know the reason why, but probably because of some desire that queue number 0 in a P4 program corresponds to the highest priority, or the lowest priority (whichever one it is -- I have not dug further to find out). Perhaps this was done in hopes of making the behavior more closely match some other switch's behavior.

I see two potential changes that would make things a bit more clear:

(a) make the set_queue_rate and any other CLI commands that reference a queue number, have the same ((# of queues) - 1 - priority) formula in them.
(b) change the ((# of queues) - 1 - priority) formula in the packet processing code to priority.

If either of those changes are made, I would strongly suggest that the relative scheduling priority of the queue numbers as viewed from P4 code priority value and set_queue_rate queue numbers be documented clearly in these files:

• https://github.com/p4lang/behavioral-model/blob/main/docs/simple_switch.md for the interpretation of the priority metadata field.
• https://github.com/p4lang/behavioral-model/blob/main/docs/runtime_CLI.md for the set_queue_rate CLI command

e.g. it should explicitly say that the behavior is whatever of the following two statements is correct (I do not know which one, or whether it is actually some other statement not listed below that is the truth):

(1) If Q > 1 queues per port are configured, then queue number 0 is scheduled with strict highest priority, followed by queue 1, then queue 2, etc. up to queue number (Q-1).
(2) If Q > 1 queues per port are configured, then queue number Q-1 is scheduled with strict highest priority, followed by queue Q-2, then queue Q-3, etc. down to queue 0.

[jafingerhut]

According to some of the comments and code in this file, it claims that the queue numbered 0 has strict priority over queue 1, queue 1 has strict priority over 2, etc. If you set a maximum packet rate on a queue, then if that queue reaches its sending limit, it is then possible for a lower priority queue to send even though the higher priority queue has packets waiting. If a queue has no rate limit, though, then it always wins over lower priority queues if it has packets waiting to send.

Because of the ((# of queues) - 1 - priority) formula in the packet enqueueing logic, that suggests to me that from the point of view of the P4 code selection the priority values, queue (Q-1) has strict priority over queue (Q-2), which has priority over (Q-3), etc. down to queue 0 which is the lowest priority.

It would be good to confirm that with running live tests before documenting it, though.

[carlos-hfm]

Thank you @jafingerhut, your answer helped me a lot.

I have another suggestion: it would be very nice and useful if we could have a command to print the current values of rate and depth (capacity) for each queue.

[jafingerhut]

If you are interested in implementing any of this, that would be excellent. Let us know. If not, this issue may have to wait until someone else interested comes along that would like to do so.