# Wav2vec2 Acces Feature Layers Performance

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwav2vec2-acces-feature-layers-performance%2F31105%3E%5C
**Category:** 🤗Transformers\
**Created:** [February 4, 2023, 11:41am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwav2vec2-acces-feature-layers-performance%2F31105 "2023-02-04T11:41:31Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![codexseraph](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/codexseraph/32/13981_2.png) [@codexseraph](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcodexseraph)%5C
**Post date:** [February 4, 2023, 11:41am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwav2vec2-acces-feature-layers-performance%2F31105%2F1 "2023-02-04T11:41:31Z")

</div>

Hey All,

I am rather new with Wav2Vec 2.0 working with it since december for a speech emotion recognition project. I started with torchaudio.pipelines bundle of Wav2vec 2 base model and put a Classification Head on top of it and freeze the transformers CNN-Layers, but fine-tune the rest on my emotion recognition task. I could work with the last layer (12) output, but I’d like to rather acces output of layer 10.  
I did this with model.extract\_features(). How to use the model without storing each layers Output? I have a batch of 32 of length 400000… Therefore ( the reason I suspect) I can’t make it run on a GPU (Feature Layers leading to 12 x 32 x 1400 x 768 tensor storage). There is the argument num-layers, which outputs only the layers output up to a number (why only int instead of list??). I manipulated components.py to only return the layer I want ( nice! I am able to run batches of 32 instead of previous maximum of 25 on my local CPU). I also tried the transformer method and used model(x).last\_hidden\_state inside my Classification head model… With everything I tried I still get Memory overload of more than 80GB on gpu. I guess I am not using the right method for what I want to achieve… please help me! 🙂

---

<div class="post-metadata">

**Author:** ![marccasalssalvador](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marccasalssalvador/32/33970_2.png) [@marccasalssalvador](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmarccasalssalvador)%5C
**Post date:** [May 7, 2025, 7:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwav2vec2-acces-feature-layers-performance%2F31105%2F2 "2025-05-07T07:31:51Z")

</div>

Hi @codexseraph! Can you provide us the code?