# Adafactor step() give me "tensors must be 2-D" error

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadafactor-step-give-me-tensors-must-be-2-d-error%2F8376%3E%5C
**Category:** Beginners\
**Created:** [July 15, 2021, 9:08am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadafactor-step-give-me-tensors-must-be-2-d-error%2F8376 "2021-07-15T09:08:18Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![kouohhashi](https://avatars.discourse-cdn.com/v4/letter/k/45deac/32.png) [@kouohhashi](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkouohhashi)%5C
**Post date:** [July 15, 2021, 9:08am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadafactor-step-give-me-tensors-must-be-2-d-error%2F8376%2F1 "2021-07-15T09:08:18Z")

</div>

Hi, I want to try Adafactor optimizer but when I use optimizer.step() , I got an error like below.

I instantiate Adafactor optimizer like below.

```auto
optimizer = Adafactor(model.parameters(), scale_parameter=True, relative_step=True, warmup_init=True, lr=None)

```

And when I execute `optimizer.step()` , I got error like below.

```auto
Traceback (most recent call last):
  File "pretrain_v2.py", line 371, in <module>
    optimizer.step()
  File "~/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/optim/optimizer.py", line 88, in wrapper
    return func(*args, **kwargs)
  File "~/anaconda3/envs/pytorch/lib/python3.6/site-packages/transformers/optimization.py", line 584, in step
    update = self._approx_sq_grad(exp_avg_sq_row, exp_avg_sq_col)
  File "~/anaconda3/envs/pytorch/lib/python3.6/site-packages/transformers/optimization.py", line 515, in _approx_sq_grad
    return torch.mm(r_factor.unsqueeze(-1), c_factor.unsqueeze(0))
RuntimeError: tensors must be 2-D

```

The shape of r\_factor and c\_factor is like below.

```auto
r_factor: torch.Size([512, 1])
c_factor: torch.Size([512, 10])
r_factor.unsqueeze(-1): torch.Size([512, 1, 1])
c_factor.unsqueeze(0): torch.Size([1, 512, 10])

```

It looks actually my tensors are not 2-D tensors.  
But I don’t know where things went wrong.

Is this error something to do with this warning?

```auto
~/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/_tensor.py:575: UserWarning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (Triggered internally at /pytorch/aten/src/ATen/native/BinaryOps.cpp:467.)
  return torch.floor_divide(self, other)

```

Thanks in advance.