How to remove duplicates from a Python list?

/ #Python


Sometimes we have a list of objects we want to clean for duplicates. Here's how you do that.

Example

>>> # Create a list
>>> animals = ["Giraffe", "Lion", "Horse", "Lion", "Giraffe", "Hippo", "Cow"]
>>>
>>> # Remove duplicates
>>> animals = list(dict.fromkeys(animals))
>>>
>>> # Print the list
>>> print(animals)
['Giraffe', 'Lion', 'Horse', 'Lion', 'Giraffe', 'Hippo', 'Cow']

What happened?

First, we create a list with animals. Some of those are duplicates.

Next, we create a new list, based on a dictionary we make from the list.

If we just run the "dict.fromkeys(animals)", we get this:

>>> animals = dict.fromkeys(animals)
>>> print(animals)
{'Giraffe': None, 'Lion': None, 'Horse': None, 'Hippo': None, 'Cow': None}

So here we have a dictionary with keys and values. Well, the values is None as you can see.

When we convert this to a list, the value will be removed and the key will be preserved.

The reason why the duplicates are removed is that a Python dictionary can't have duplicate keys, so they are just ignored.

Comments

Must Have | Dec 19, 21 09:01

>>> animals = ["Giraffe", "Lion", "Horse", "Lion", "Giraffe", "Hippo", "Cow"]
>>> a_set = set(animals)
>>> print(list(a_set))
['Lion', 'Giraffe', 'Hippo', 'Horse', 'Cow']

Add comment

Newsletter

Subscribe to my weekly newsletter. One time per week I will send you a short summary of the tutorials I have posted in the past week.