# CUDA out of memory for Longformer

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcuda-out-of-memory-for-longformer%2F1472%3E%5C
**Category:** Beginners\
**Created:** [October 11, 2020, 4:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcuda-out-of-memory-for-longformer%2F1472 "2020-10-11T16:03:27Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshainaraza)%5C
**Post date:** [October 11, 2020, 4:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcuda-out-of-memory-for-longformer%2F1472%2F1 "2020-10-11T16:03:27Z")

</div>

I have issue training the longformer on custom dataset, even on a small batch number, it says CUDA out of memory,

RuntimeError Traceback (most recent call last)  
 in ()  
----\> 1 trainer.train()

18 frames  
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in \_pad(input, pad, mode, value)  
3550 assert len(pad) // 2 \<= input.dim(), ‘Padding length too large’  
3551 if mode == ‘constant’:  
-\> 3552 return \_VF.constant\_pad\_nd(input, pad, value)  
3553 else:  
3554 assert value == 0, ‘Padding mode “{}”" doesn’t take in value argument’.format(mode)

RuntimeError: CUDA out of memory. Tried to allocate 1.13 GiB (GPU 0; 15.90 GiB total capacity; 11.40 GiB already allocated; 659.81 MiB free; 14.39 GiB reserved in total by PyTorch)

---

<div class="post-metadata">

**Author:** ![Emanuel](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/emanuel/32/20406_2.png) [@Emanuel](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FEmanuel)%5C
**Post date:** [October 20, 2021, 1:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcuda-out-of-memory-for-longformer%2F1472%2F2 "2021-10-20T13:32:47Z")

</div>

Did you try smaller batch sizes? What is the size of single batch size in your RAM?

---

<div class="post-metadata">

**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshainaraza)%5C
**Post date:** [October 20, 2021, 1:44pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcuda-out-of-memory-for-longformer%2F1472%2F3 "2021-10-20T13:44:37Z")

</div>

I tried 8 batch size, do not remember the single batch size. I use colab so it has its own limitations even the Pro version. any thoughts?

---

<div class="post-metadata">

**Author:** ![Emanuel](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/emanuel/32/20406_2.png) [@Emanuel](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FEmanuel)%5C
**Post date:** [October 20, 2021, 2:50pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcuda-out-of-memory-for-longformer%2F1472%2F4 "2021-10-20T14:50:19Z")

</div>

Your input sentences are being limited to a maximum size?

---

<div class="post-metadata">

**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshainaraza)%5C
**Post date:** [October 22, 2021, 11:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcuda-out-of-memory-for-longformer%2F1472%2F5 "2021-10-22T11:26:09Z")

</div>

I truncated to lesser as well.

---

<div class="post-metadata">

**Author:** ![Emanuel](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/emanuel/32/20406_2.png) [@Emanuel](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FEmanuel)%5C
**Post date:** [October 22, 2021, 12:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcuda-out-of-memory-for-longformer%2F1472%2F6 "2021-10-22T12:00:07Z")

</div>

Could you provide a simple snippet to reproduce the OOM?

---

<div class="post-metadata">

**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshainaraza)%5C
**Post date:** [October 22, 2021, 1:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcuda-out-of-memory-for-longformer%2F1472%2F7 "2021-10-22T13:06:21Z")

</div>

if you see my post above, its months ago, its not actively something I am working on, I didn’t get answer but I solved that time by having smaller batches and less input.  
I have few more issues like batch processing, having predictions at the end with a score, do you mind discussing that?
