# Using AMD'S RocM with accelerate library

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-amds-rocm-with-accelerate-library%2F69551%3E%5C
**Category:** 🤗Accelerate\
**Created:** [January 17, 2024, 11:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-amds-rocm-with-accelerate-library%2F69551 "2024-01-17T23:20:49Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![imamcsiro](https://avatars.discourse-cdn.com/v4/letter/i/e99b99/32.png) [@imamcsiro](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fimamcsiro)%5C
**Post date:** [January 17, 2024, 11:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-amds-rocm-with-accelerate-library%2F69551%2F1 "2024-01-17T23:20:49Z")

</div>

I have two AMD GPUs with ROCm. I want to use the SFTTrainer class with the accelerate library to fine-tune an LLM on the two GPUs with distributed data parallelism (DDP). However, I keep running into out of memory (OOM) errors, despite fine tuning running fine on one GPU. Does the accelerate library support ROCm? Is this an issue with something else?

---

<div class="post-metadata">

**Author:** ![marcsun13](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marcsun13/32/19059_2.png) [@marcsun13](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmarcsun13)%5C
**Post date:** [January 24, 2024, 6:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-amds-rocm-with-accelerate-library%2F69551%2F2 "2024-01-24T18:32:27Z")

</div>

Hi @imamcsiro , could you try with a smaller model and try to see how much is the increase of memory ? It should work since Pytorch supports AMD ROCm GPU
