tun: AMD64 optimized checksum
This adds AMD64 assembly implementations of IP checksum computation, one for baseline AMD64 and the other for v3 AMD64 (AVX2 and BMI2). All performance numbers reported are from a Ryzen 7 4750U but similar improvements are expected for a wide range of processors. The generic IP checksum implementation has also been further improved to be significantly faster using bits.AddUint64 (for a 64KiB buffer the throughput improves from 15,000MiB/s to 27,600MiB/s; similar gains are also reported on ARM64 but I do not have specific numbers). The baseline AMD64 implementation for a 64KiB buffer reports 32,700MiB/s and the AVX2 implementation is slightly over 107,000MiB/s. Unfortunately, for very small sizes (e.g. the expected size for an IPv4 header) setting up SIMD computation involves some overhead that makes computing a checksum for small buffers slower than a non-SIMD implementation. Even more unfortunately, testing for this at runtimen in Go and calling a func optimized for small buffers mitigates most of the improvement due to call overhead. The break even point is around 256 byte buffers; IPv4 headers are no more than 60 bytes including extensions. IPv6 headers do not have a checksum but are a fixed size of 40 bytes. As a result, the generated assembly code uses an alternate approach for buffers of less than 256 bytes. Additionally, buffers of less than 32 bytes need to be handled specially because the strategy for reading buffers that are not a multiple of 8 bytes fails when the buffer is too small. As suggested by additional benchmarking, pseudo header computation has been rewritten to be faster (benchmark time reduced by 1/2 to 1/4). Updates tailscale/corp#9755 Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
This commit is contained in:
parent
ec6f23b33e
commit
88b11b4a0d
10 changed files with 2266 additions and 115 deletions
45
tun/checksum_amd64_test.go
Normal file
45
tun/checksum_amd64_test.go
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
//go:build amd64
|
||||
|
||||
package tun
|
||||
|
||||
import (
|
||||
"golang.org/x/sys/cpu"
|
||||
)
|
||||
|
||||
var archChecksumFuncs = []archChecksumDetails{
|
||||
{
|
||||
name: "generic32",
|
||||
available: true,
|
||||
f: checksumGeneric32,
|
||||
},
|
||||
{
|
||||
name: "generic64",
|
||||
available: true,
|
||||
f: checksumGeneric64,
|
||||
},
|
||||
{
|
||||
name: "generic32Alternate",
|
||||
available: true,
|
||||
f: checksumGeneric32Alternate,
|
||||
},
|
||||
{
|
||||
name: "generic64Alternate",
|
||||
available: true,
|
||||
f: checksumGeneric64Alternate,
|
||||
},
|
||||
{
|
||||
name: "AMD64",
|
||||
available: true,
|
||||
f: checksumAMD64,
|
||||
},
|
||||
{
|
||||
name: "SSE2",
|
||||
available: cpu.X86.HasSSE2,
|
||||
f: checksumSSE2,
|
||||
},
|
||||
{
|
||||
name: "AVX2",
|
||||
available: cpu.X86.HasAVX && cpu.X86.HasAVX2 && cpu.X86.HasBMI2,
|
||||
f: checksumAVX2,
|
||||
},
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue