Fastest way to uniqify a list in Python

Maxime Biais February 14, 2007 Reply

I always use this one (order preserving, generative, concise, but don't work on no hashable).

from itertools import ifilter
def _f14(iterable):
....m = set()
....return ifilter(lambda x: not (m.__contains__(x) or m.add(x)), iterable)

def f14(lst):
....return list(_f14(lst))

My results:
* f5 1.94
* f5b 1.89
* f8 1.18
* f10 1.24
* f11 1.23
* f13 1.31
* f14 1.87
f1 0.67
f3 0.64
f6 0.68
f7 0.38
f9 0.41

Not so bad ;)