In its latest ISA instruction reference, Intel has stated that the Alder Lake client CPUs won’t support AVX-512 or AMX instructions, and both the ISAs will be limited to the server-class Sapphire Rapids CPUs. While the reason for the exclusion of the former is the hybrid (big-little) architecture of the Alder Lake CPUs, meaning both the high performance and low power cores need to support it. This isn’t the case, however. The little cores “Tremont” only support vector instructions up to 256-bit or AVX2 similar to AMD’s Zen 2 cores.
The lack of AMX support on Alder Lake is a real shame though. AMX or Advanced Matrix Extensions brings tensor support to Intel CPUs for the first time. It allows operations on matrices (rank-2 tensors). This is done via a 2D register file with tiles (registers) using accelerators that operate on them.
<Insert grain of salt>
As per rumors, AMX is a major step up from traditional AVX based vector instructions. It supports BF16 and INT8 multiplication as well as FP32/INT32 FMA. AMX based BF16 is supposed to be 5x faster than Cooper Lake’s BF16 while AMX-INT8 is 8x faster than AVX-VNNI (512-bit). As such, it’s possible that Sapphire Lake which will be the first CPU to support these instructions will be the first with 100TFLOP of theoretical FP throughput, and 300TOPs of simultaneous INT8 compute performance.
</Insert grain of salt>
