# Remove a row/specific index from the dataset
**URL:**
**Author:**  [@zilong](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fzilong)%5C
**Post date:** [December 16, 2021, 12:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fremove-a-row-specific-index-from-the-dataset%2F12875%2F1 "2021-12-16T00:57:34Z")
Given the code
```auto
from datasets import load_dataset
dataset = load_dataset("glue", "mrpc", split='train')
idx = 0
```
How can I remove row 0 (dataset[0]) from this dataset?
The only way I can think of for now is using dataset.select(), and then selecting every index except 0, but that doesn’t seem efficient.
---
**Author:**  [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [December 17, 2021, 11:25am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fremove-a-row-specific-index-from-the-dataset%2F12875%2F2 "2021-12-17T11:25:29Z")
Hi!
You can do `dataset = load_dataset("glue", "mrpc", split='train[1:]')` to skip the first example while loading the dataset.
> The only way I can think of for now is using dataset.select(), and then selecting every index except 0, but that doesn’t seem efficient.
Why do you think `select` is not efficient? It depends on the ops you use afterward, but `select` alone is very efficient as it only creates an indices mapping, which is (almost) equal to `list(indices)`, and not a new PyArrow table.
`
---
**Author:**  [@zilong](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fzilong)%5C
**Post date:** [December 17, 2021, 2:44pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fremove-a-row-specific-index-from-the-dataset%2F12875%2F3 "2021-12-17T14:44:02Z")
Hi Thank you for your reply.
The issue is that I need to remove random rows from the dataset. So not just idx = 0. But more like `idxs =[76, 3, 384,10]`. Currently I do this by selecting every index that is not in `idxs`. Which works, but I feel like there should be a better way to do it.
---
**Author:**  [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [December 21, 2021, 1:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fremove-a-row-specific-index-from-the-dataset%2F12875%2F4 "2021-12-21T13:08:55Z")
> Which works, but I feel like there should be a better way to do it.
“Better way” in terms of the API design? If yes, do you have an API in mind? Or better in terms of speed?
Removing rows is not easy to implement (efficiently) because PyArorw tables, which `datasets` use to store data, are immutable. You could use `pandas` for that (`ds.to_pandas()`) if your dataset is not too big and can fit in memory.
---
**Author:**  [@mkdeeperinsights](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmkdeeperinsights)%5C
**Post date:** [September 21, 2022, 4:47pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fremove-a-row-specific-index-from-the-dataset%2F12875%2F5 "2022-09-21T16:47:45Z")
In summary, it seems the current solution is to [`select`](https://huggingface.co/docs/datasets/v2.5.1/en/package_reference/main_classes#datasets.Dataset.select) all of the ids except the ones you don’t want.
So in this example, something like:
```nohighlight
from datasets import load_dataset
# load dataset
dataset = load_dataset("glue", "mrpc", split='train')
# what we don't want
exclude_idx = [76, 3, 384, 10]
# create new dataset exluding those idx
dataset = dataset.select(
(
i for i in range(len(dataset))
if i not in set(exclude_idx)
)
)
```
---
**Author:**  [@ykeselman](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fykeselman)%5C
**Post date:** [December 4, 2024, 3:04am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fremove-a-row-specific-index-from-the-dataset%2F12875%2F6 "2024-12-04T03:04:50Z")
For my reasonably sized dataset of 15K rows, converting it to a `pandas.DataFrame` first, doing filtering there, then converting back, worked the fastest. Use `dataset.to_pandas` and `Dataset.from_pandas`.
---
**Author:**  [@ChIrish06](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FChIrish06)%5C
**Post date:** [February 8, 2025, 6:05am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fremove-a-row-specific-index-from-the-dataset%2F12875%2F7 "2025-02-08T06:05:06Z")
This is the best solution I found on this page. Converting to pandas seemed to take a long time, I used a mask instead of an exclusion set and it ran through 30K rows in less than a second.