| here is a quote from Nvidia Ampere GA102 GPU Architecture: | |
| Third-Generation NVLink® | |
| GA102 GPUs utilize NVIDIA’s third-generation NVLink interface, which includes four x4 links, | |
| with each link providing 14.0625 GB/sec bandwidth in each direction between two GPUs. Four | |
| links provide 56.25 GB/sec bandwidth in each direction, and 112.5 GB/sec total bandwidth | |
| between two GPUs. |