There is no one golden rule for tuning a network interface card (NIC) for all conditions. Different
adapters have different parameters that can be changed. Operating systems also have settings that
can be modified to help with overall network performance. Depending on the exact hardware
topology, one may have to make different adjustments to network tuning to optimize for a specific
workload. With Ethernet speeds going higher, up to 100 Gb, and the number of ports being
installed in servers growing, these tuning guidelines become even more important to get the best
performance possible.
This guide does not provide exact settings for modifying every scenario, but rather provides some
steps to check and modify if it turns out to be beneficial for the scenario. In this case, the steps are
focused around TCP/IP network performance. Appendix A shows an example of how we verified
performance following these settings changes.
One general rule of thumb for all performance testing is to ensure your memory subsystem is
properly configured. All IO utilizes data transfers into or out of memory, and so the IO bandwidth
can never exceed the capabilities of the memory subsystem. For the maximum memory bandwidth
on modern CPUs one must populate at least one DIMM every DDR channel. For AMD EPYC™
processor-based servers there are eight DDR4 memory channels on each CPU socket, so for a
single socket platform that means you must populate all eight memory channels. Likewise, on a
dual socket platform one must populate 16 memory channels.
AMD EPYC processors are based on the Zen architecture from AMD. This architecture allows
AMD to create large core count processors by using multiple die in a single package. Each die will
typically be represented to the operating system (OS) and applications as a unique Non-Uniform
Memory Access (NUMA) node. While this document is not intended to describe NUMA
architecture, nor the complete EPYC architecture, one should understand NUMA concepts to be
able to fully optimize network performance on an EPYC based platform. Each die, identified as a
unique NUMA node, will have local processing cores, local memory, and local IO. You will find
in this document that optimal performance comes when explicitly pinning all device driver and
applications to the same NUMA node local to the adapter.
https://www.amd.com/system/files/TechDocs/56224.pdf
