Skip to content

Instantly share code, notes, and snippets.

@cjauvin
Created November 17, 2015 15:23
Show Gist options
  • Select an option

  • Save cjauvin/ca51e4d9999a5224cd4a to your computer and use it in GitHub Desktop.

Select an option

Save cjauvin/ca51e4d9999a5224cd4a to your computer and use it in GitHub Desktop.
from sklearn.preprocessing import LabelEncoder
import pandas as pd
import bisect
le = LabelEncoder()
train_c = le.fit_transform(['a', 'b', 'c', 'a'])
test_c = pd.Series(['a', 'b', 'c', 'd']).map(
lambda s: '<unk>' if s not in le.classes_ else s
)
print(test_c.tolist()) # ['a', 'b', 'c', '<unk>']
# We need to insert <unk> in the sorted classes order, because
# le.transform uses np.searchsorted:
# https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/preprocessing/label.py#L149
# However, this feels really clunky!
le_classes = le.classes_.tolist()
bisect.insort_left(le_classes, '<unk>')
le.classes_ = le_classes
print(le.transform(test_c)) # [1, 2, 3, 0]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment