I have a list that contains list of tuples as follows.
mylist = [['xxx', 879], ['yyy', 315], ['xxx', 879], ['zzz', 171], ['yyy', 315]]
I want to remove the duplicate tuples from mylist and get an output as follows.
mylist = [['xxx', 879], ['yyy', 315], ['zzz', 171]]
It seems like set in python does not work for it.
mylist = list(set(mylist))
Is there any fast and easy way of doing this in python (perhaps using libraries)?
It seems like you want to preserve order. In that case you can keep a set that keeps track of what lists have been added.
Here is an example:
mylist = [['xxx', 879], ['yyy', 315], ['xxx', 879], ['zzz', 171], ['yyy', 315]]
# set that keeps track of what elements have been added
seen = set()
no_dups = []
for lst in mylist:
# convert to hashable type
current = tuple(lst)
# If element not in seen, add it to both
if current not in seen:
no_dups.append(lst)
seen.add(current)
print(no_dups)
Which Outputs:
[['xxx', 879], ['yyy', 315], ['zzz', 171]]
Note: Since lists are not hashable, you can add tuples instead to the seen set.
The reason that you're not able to do this is because you have a list of lists and not a list of tuples.
What you could do is:
mytuplelist = [tuple(item) for item in mylist]
mylist = list(set(mytuplelist))
or
mylist = list(set(map(tuple, mylist)))
You need to write code that keeps the first of the sub-lists, dropping the rest. The simplest way to do this is to reverse mylist, load it into an dict object, and retrieve its key-value pairs as lists again.
>>> list(map(list, dict(mylist).items()))
Or, using a list comprehension -
>>> [list(v) for v in dict(mylist).items()]
[['zzz', 171], ['yyy', 315], ['xxx', 879]]
Note, that this answer does not maintain order! Also, if your sub-lists can have more than 2 elements, an approach involving hashing the tuplized versions of your data, as @JohnJosephFernandez' answer shows, would be the best thing to do.
mylist = [['xxx', 879], ['xxx', 200]][list(v) for v in dict([['xxx', 879], ['xxx', 200]]).items()] is not list(set(tuple(item) for item in [['xxx', 879], ['xxx', 200]])) and I think the latter (like John Joseph wrote) is what is wanted. But the OP accepted your answer so I may be wrong! Perhaps this situation never arises in their data anywayAnother option:
>>> mylist = [['xxx', 879], ['yyy', 315], ['xxx', 879], ['zzz', 171], ['yyy', 315]]
>>> y = []
>>> for x in mylist:
... if not x in y:
... y+=[x]
...
>>> y
[['xxx', 879], ['yyy', 315], ['zzz', 171]]
