[Frontend] Lower attention op to FlashAttention vectorized kernel.

[GuoningHuang]

1. Implement FlashAttention kernel in MLIR using affine loops + vector ops.
2. Replace the existing Attention → TOSA lowering path with Attention → FlashAttention vectorized kernel.
3. Tagrget at single op, performance were compared against the original TOSA lowering，achieves ~3× speedup on CPU.
4. achieve ~1.17x speed up in total decode phrase, corresponds to the performance tested in issue [TASK] Locate Performance Bottlenecks in DeepSeek R1 Decode Subgraph #604.
   
   baseline:

apply for decode phase:

apply for both prefill and decode phase：

[zhanghb97]

• Please format your code and install pre-commit tool.
• No obvious performance improvement observed
• Please use vector.load/store, vector.transfer_read/write as sometimes they cannot guarantee the generation of optimal hardware code
• Can the splat inside the loop be moved outside? You can verify whether it affects performance

Before:

After:

[GuoningHuang]

• Please format your code and install pre-commit tool.
• No obvious performance improvement observed
• Please use vector.load/store, vector.transfer_read/write as sometimes they cannot guarantee the generation of optimal hardware code
• Can the splat inside the loop be moved outside? You can verify whether it affects performance

Before:

After:

Hi, thanks for the feedback.
I've addressed the issues:

• The code has been reformatted and the pre-commit tool is now installed and working properly.
• I have replaced the vector.transfer_read/write operations with vector.load/store as suggested, you can see in next-flash-attention-vec.mlir.
• As for the splat inside the loop, I have moved some splat outside the loop and it brings measurable performance improvements on single operator as below :
  \
• After these optimizations, I still gain the same result as before, a performance improvement observed in the decode phase compare with no opt in my machine.
  Before:

After:

• I guess the performance improvement may vary across different machines, since memory bandwidth and per-core compute capability differ between systems, which can lead to different performance bottlenecks. below is my machine:

[GuoningHuang]

@zhanghb97
I implemented a new FlashAttention version featuring tiled KV processing and vectorization.
The implementation is available in next-flash-attention-vec-tiled.mlir.
Based on theoretical analysis and my previous experiments, I think this version maybe close to the performance ceiling on CPU.
It achieves better cache utilization then previous one. It also delivers small but measurable performance improvements, as shown below:
Before:

After:

[GuoningHuang]

This is a longer prompt test result：
Before：
After，KV tiled size = 32：
After，KV tiled size = 64：

[zhanghb97]

@GuoningHuang It looks pretty good. Please resolve the conflicts, then I'll pull it down to test the performance.