Python Program to Remove Duplicate Elements From an Array

An array is a collection of elements of the same data type, where each element is identified by an index value. It is the simplest data structure where each data element can be accessed directly using its index number.

Arrays in Python

Python does not have a specific data structure to represent arrays. Here, we can use a List as an array ?

[6, 4, 1, 5, 9]
 0  1  2  3  4

The indexing in Python starts from 0. In the above example, the integers 6, 4, 1, 5, 9 are the array elements and 0, 1, 2, 3, 4 are the respective index values.

An array can have duplicate elements. In this article, we will discuss different ways to remove duplicate elements from an array.

Input Output Scenario

Let's assume we have an input array with duplicate values. The resultant array will contain unique elements only ?

Input array:
A = [1, 5, 3, 6, 3, 5, 6, 1]

Output array:
[1, 5, 3, 6]

The elements 1, 5, 3, 6 are the unique elements from the given array.

Using For Loop

We use a for loop to iterate through all array elements. In each iteration, we check for duplicates using the not in operator ?

arr = [1, 5, 3, 6, 3, 5, 6, 1] 
print("The original array is:", arr) 

# Remove repeated elements from array 
result = []

for element in arr: 
   if element not in result: 
      result.append(element) 

print("The array after removing repeated elements:", result)

The original array is: [1, 5, 3, 6, 3, 5, 6, 1]
The array after removing repeated elements: [1, 5, 3, 6]

The not in operator checks if the current element exists in the result list. If it doesn't exist, the element is appended to the result list; otherwise, it's ignored.

Using Sets

A Set is a data structure in Python that stores only unique data. It cannot store duplicate elements ?

arr = [1, 5, 3, 6, 3, 5, 6, 1] 
print("The original array is:", arr) 

# Remove repeated elements from array 
result = list(set(arr)) 

print("The array after removing repeated elements:", result) 

The original array is: [1, 5, 3, 6, 3, 5, 6, 1]
The array after removing repeated elements: [1, 3, 5, 6]

Note: Sets don't maintain the original order of elements, so the output may have a different order than the input.

Using enumerate() Function

The enumerate() function returns a tuple containing the index and value for each element in an iterable ?

Syntax

enumerate(iterable, start=0)

arr = [1, 5, 3, 6, 3, 5, 6, 1] 
print("The original array is:", arr) 

# Remove repeated elements from array 
result = [element for index, element in enumerate(arr) if element not in arr[:index]]

print("The array after removing repeated elements:", result) 

The original array is: [1, 5, 3, 6, 3, 5, 6, 1]
The array after removing repeated elements: [1, 5, 3, 6]

This approach checks if each element already exists in the array up to the current index. If it exists, the element is ignored; otherwise, it's included in the result.

Using dict.fromkeys()

The dict.fromkeys() method creates a dictionary from a given sequence of keys. Since dictionary keys are unique, this method automatically removes duplicates ?

Syntax

dict.fromkeys(keys, value)

Parameters

• keys Required parameter. An iterable specifying the keys of the new dictionary.

• value Optional parameter. The value for all keys. Default is None.

arr = [1, 5, 3, 6, 3, 5, 6, 1] 
print("The original array is:", arr) 

# Remove repeated elements from array
result = list(dict.fromkeys(arr))

print("The array after removing repeated elements:", result) 

The original array is: [1, 5, 3, 6, 3, 5, 6, 1]
The array after removing repeated elements: [1, 5, 3, 6]

Since dictionary keys cannot be duplicated, the fromkeys() method automatically removes repeated values while preserving the original order.

Comparison

Conclusion

Use dict.fromkeys() for the most efficient duplicate removal while preserving order. Use set() when order doesn't matter and you need the fastest performance.