Jafner/Jafner.net
1
0
Fork 0
Code Issues 16 Pull Requests Actions Packages Projects Releases Wiki Activity
efcec78ac4
Jafner.net/docker-llm-amd/FLASH-ATTENTION.md
Joey Hafner 0858ce927a Init
2024-06-04 12:42:14 -07:00

1.9 KiB
Raw Blame History

Flash Attention in Docker on AMD is Not Yet Working

Below are my notes on the efforts I've made to get it working.

FROM rocm/pytorch-nightly:latest
COPY . .
RUN git clone --recursive https://github.com/ROCm/flash-attention.git /tmp/flash-attention
WORKDIR /tmp/flash-attention
ENV MAX_JOBS=8
RUN pip install -v .

Resources

  1. What is Flash-attention? (How do i use it with Oobabooga?) :...
  2. Adding flash attention to one click installer · Issue #4015 ...
  3. Accelerating Large Language Models with Flash Attention on A...
  4. GitHub - Dao-AILab/flash-attention: Fast and memory-efficien...
  5. GitHub - ROCm/llvm-project: This is the AMD-maintained fork ...
  6. GitHub - ROCm/AITemplate: AITemplate is a Python framework w...
  7. Stable diffusion with RX7900XTX on ROCm5.7 · ROCm/composable...
  8. Current state of training on AMD Radeon 7900 XTX (with bench... Current state of training on AMD Radeon 7900 XTX (with benchmarks) rLocalLLaMA
  9. llm-tracker - howto/AMD GPUs
  10. RDNA3 support · Issue #27 · ROCm/flash-attention · GitHub
  11. GitHub - ROCm/xformers: Hackable and optimized Transformers ...
  12. [ROCm] support Radeon™ 7900 series (gfx1100) without using...