CPU C-states disabling ruins network performance

Question

With intel_idle.max_cstate=1:

root@debian:~# curl -o /dev/null http://xx.xx.38.35:999/speedtest/1000mb
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1000M  100 1000M    0     0  41.1M      0  0:00:24  0:00:24 --:--:-- 41.0M

With intel_idle.max_cstate=3 (download speed is much better):

root@debian:~# curl -o /dev/null http://xx.xx.38.35:999/speedtest/1000mb
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1000M  100 1000M    0     0  75.9M      0  0:00:13  0:00:13 --:--:-- 76.0M

Without C-state tuning (download speed virtually unchanged):

root@debian:~# curl -o /dev/null http://xx.xx.38.35:999/speedtest/1000mb
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1000M  100 1000M    0     0  76.1M      0  0:00:13  0:00:13 --:--:-- 73.4M

Any idea why intel_idle.max_cstate=1 network performance is so bad?

OS: Linux (Debian Buster)
Kernel version: tried several, current 5.5.17-1~bpo10+1
CPU: Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz (one installed)
Motherboard: Quanta S2S-MB
NIC: Mellanox ConnectX-3 EN (MCX341A-XCEN)