From d224ae8f735d0a8fc997bc9a717cb611fab559c0 Mon Sep 17 00:00:00 2001 From: Michael Melesse Date: Fri, 24 Jan 2025 05:12:17 -0800 Subject: [PATCH] Update readme --- README.md | 6 +++--- flash_attn/flash_attn_triton_amd/README.md | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index f25465af1..776824f75 100644 --- a/README.md +++ b/README.md @@ -137,16 +137,16 @@ These features are supported in Fwd and Bwd 2) Variable sequence lengths 3) Arbitrary Q and KV sequence lengths 4) Arbitrary head sizes +5) Multi and grouped query attention +6) Dropout +7) Rotary embeddings These features are supported in Fwd for now. We will add them to backward soon. -1) Multi and grouped query attention 2) ALiBi and matrix bias These features are in development 1) Paged Attention 2) Sliding Window -3) Rotary embeddings -4) Dropout 5) Performance Improvements #### Getting Started diff --git a/flash_attn/flash_attn_triton_amd/README.md b/flash_attn/flash_attn_triton_amd/README.md index fc411425f..353b493f6 100644 --- a/flash_attn/flash_attn_triton_amd/README.md +++ b/flash_attn/flash_attn_triton_amd/README.md @@ -11,16 +11,16 @@ These features are supported in Fwd and Bwd 2) Variable sequence lengths 3) Arbitrary Q and KV sequence lengths 4) Arbitrary head sizes +5) Multi and grouped query attention +6) Dropout +7) Rotary embeddings These features are supported in Fwd for now. We will add them to backward soon. -1) Multi and grouped query attention 2) ALiBi and matrix bias These features are in development 1) Paged Attention 2) Sliding Window -3) Rotary embeddings -4) Dropout 5) Performance Improvements #### Getting Started