THE ESSENTIALS IN 30 SECONDS

  • ZYFA provides causal forward (prefill) attention kernels in FP8 and BF16 for AMD Instinct MI355X, at head dimension 128 and DeepSeek-V2's latent-attention shape.
  • On attention inputs captured from three models, Zyphra reports FP8 speedups of 1.16x to 1.76x over AMD's default AITER kernels; BF16 gains are marginal.
  • Zyphra says its FP8 error is 62–78% lower than AITER's at head dimension 128; all figures come from Zyphra's own tests on a single eight-GPU node.

Zyphra, the San Francisco AI company that has built much of its stack on AMD hardware, published ZYFA on 5 November: a set of attention kernels for AMD Instinct MI355X GPUs, released alongside a technical report on arXiv and code on GitHub. Coverage of the release reports that the code is under the Apache 2.0 licence. The name stands for Zyphra FlashAttention on AMD.

What the kernels cover

ZYFA implements causal forward attention, the prefill stage of inference, in FP8 and BF16 precision. It supports a head dimension of 128 and the multi-head latent attention shape used by DeepSeek-V2. According to Zyphra, the kernels form part of the attention engine behind its own inference cloud and on-premises deployments. Decoding, paged KV caches and variable-length batches are not supported yet; the company lists them as future work.

The problem Zyphra targets is that FP8 matrix units offer roughly twice the peak throughput of BF16, while the softmax step between the two attention matrix multiplications does not get faster. In the report, the authors note that AMD's AITER kernels reach 39% of FP8 peak at 32,000 tokens, against 49% of BF16 peak. ZYFA's main techniques are a cheaper way to compute FP8 probabilities without an exponential, a schedule in which one group of threads runs softmax while another runs matrix operations, and per-block power-of-two scaling before rounding.

The claimed results

On attention inputs captured from Mistral-7B, Qwen3-8B and DeepSeek-V2-Lite at 16,000 tokens, Zyphra reports FP8 speedups of 1.16x, 1.17x and 1.76x respectively over AITER's default kernels. BF16 gains are only 1–2%. On synthetic inputs, the figures rise to 1.26x–1.37x at head dimension 128 and 1.99x at the DeepSeek shape. On accuracy, Zyphra says its FP8 error is within a few percent of AMD's Composable Kernel and Triton implementations and 62–78% lower than AITER's.

Caveats

The headline 1.76x is the best of three real-model results, as FourWeekMBA points out, and full forward passes showed median gains of about 1.25x. All numbers come from Zyphra's tests on one eight-GPU node with ROCm 7.2.3 and AITER 0.1.19, which the authors describe as point-in-time. They also report that the MI355X's sustained clock depends on the data it processes, so random-input benchmarks can misstate real-world speed. FourWeekMBA found no independent test of the kernels and no response from AMD.