# Memory Requirements for Running LLM
**URL:**
**Author:**  [@pjp94](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpjp94)%5C
**Post date:** [October 3, 2023, 7:05pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmemory-requirements-for-running-llm%2F57282%2F1 "2023-10-03T19:05:39Z")
Are there any rule of thumb calculations for determining memory requirement (as a function of number of model parameters) for an LLM model. I’m referring to a base model (no quantization) with full fine-tuning vs no fine-tuning that I would like to run inference on. Lets use Llama2 7B as an example. Below is what I’ve seen for pre-training, but not sure how this exactly translates over to loading in the pre-trained base model and working with it. The disconnect here for me is I’m unsure what components of the model exactly are getting stored in memory. Any help appreciated!
---
**Author:**  [@MattiLinnanvuori](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMattiLinnanvuori)%5C
**Post date:** [October 4, 2023, 4:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmemory-requirements-for-running-llm%2F57282%2F2 "2023-10-04T04:31:12Z")
[https://huggingface.co/spaces/hf-accelerate/model-memory-usage](https://huggingface.co/spaces/hf-accelerate/model-memory-usage) is a tool.
---
**Author:**  [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [May 8, 2024, 7:30pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmemory-requirements-for-running-llm%2F57282%2F3 "2024-05-08T19:30:35Z")
There’s also this nice blog post: [Calculating GPU memory for serving LLMs | Substratus.AI](https://www.substratus.ai/blog/calculating-gpu-memory-for-llm)