# Python pandas Filtering out nan from a data selection of a column of strings

Without using `groupby`

how would I filter out data without `NaN`

?

Let say I have a matrix where customers will fill in 'N/A','n/a' or any of its variations and others leave it blank:

```
import pandas as pd
import numpy as np
df = pd.DataFrame({'movie': ['thg', 'thg', 'mol', 'mol', 'lob', 'lob'],
'rating': [3., 4., 5., np.nan, np.nan, np.nan],
'name': ['John', np.nan, 'N/A', 'Graham', np.nan, np.nan]})
nbs = df['name'].str.extract('^(N/A|NA|na|n/a)')
nms=df[(df['name'] != nbs) ]
```

output:

```
>>> nms
movie name rating
0 thg John 3
1 thg NaN 4
3 mol Graham NaN
4 lob NaN NaN
5 lob NaN NaN
```

How would I filter out NaN values so I can get results to work with like this:

```
movie name rating
0 thg John 3
3 mol Graham NaN
```

I am guessing I need something like `~np.isnan`

but the tilda does not work with strings.

Just drop them:

```
nms.dropna(thresh=2)
```

this will drop all rows where there are at least two non-`NaN`

then you could then drop where name is `NaN`

:

```
In [87]:
nms
Out[87]:
movie name rating
0 thg John 3
1 thg NaN 4
3 mol Graham NaN
4 lob NaN NaN
5 lob NaN NaN
[5 rows x 3 columns]
In [89]:
nms = nms.dropna(thresh=2)
In [90]:
nms[nms.name.notnull()]
Out[90]:
movie name rating
0 thg John 3
3 mol Graham NaN
[2 rows x 3 columns]
```

**EDIT**

Actually looking at what you originally want you can do just this without the `dropna`

call:

```
nms[nms.name.notnull()]
```

**UPDATE**

Looking at this question 3 years later, there is a mistake, firstly `thresh`

arg looks for at leas `n`

non-`NaN`

values so in fact the output should be:

```
In [4]:
nms.dropna(thresh=2)
Out[4]:
movie name rating
0 thg John 3.0
1 thg NaN 4.0
3 mol Graham NaN
```

It's possible that I was either mistaken 3 years ago or that the version of pandas I was running had a bug, both scenarios entirely possible

From: stackoverflow.com/q/22551403

**★ Back to homepage or read more recommendations:**