Python Pandas – Find the common rows between two Data Frames

To find the common rows between two DataFrames in Pandas, use the merge() method with how='inner'. This returns only the rows that have identical values across all columns in both DataFrames.

Creating Sample DataFrames

Let us first create two DataFrames with car data ?

import pandas as pd

# Create DataFrame1
dataFrame1 = pd.DataFrame({
    "Car": ['BMW', 'Lexus', 'Audi', 'Tesla', 'Bentley', 'Jaguar'],
    "Units": [100, 150, 110, 80, 110, 90]
})

print("DataFrame1:")
print(dataFrame1)

DataFrame1:
      Car  Units
0     BMW    100
1   Lexus    150
2    Audi    110
3   Tesla     80
4 Bentley    110
5  Jaguar     90

import pandas as pd

# Create DataFrame2
dataFrame2 = pd.DataFrame({
    "Car": ['BMW', 'Lexus', 'Audi', 'Mustang', 'Bentley', 'Jaguar'],
    "Units": [100, 250, 150, 80, 130, 90]
})

print("DataFrame2:")
print(dataFrame2)

DataFrame2:
      Car  Units
0     BMW    100
1   Lexus    250
2    Audi    150
3 Mustang     80
4 Bentley    130
5  Jaguar     90

Finding Common Rows

Use merge() with how='inner' to find rows that exist in both DataFrames with identical values ?

import pandas as pd

# Create DataFrames
dataFrame1 = pd.DataFrame({
    "Car": ['BMW', 'Lexus', 'Audi', 'Tesla', 'Bentley', 'Jaguar'],
    "Units": [100, 150, 110, 80, 110, 90]
})

dataFrame2 = pd.DataFrame({
    "Car": ['BMW', 'Lexus', 'Audi', 'Mustang', 'Bentley', 'Jaguar'],
    "Units": [100, 250, 150, 80, 130, 90]
})

# Check if DataFrames are equal
print("Are both DataFrames equal?", dataFrame1.equals(dataFrame2))

# Find common rows
common_rows = dataFrame1.merge(dataFrame2, how='inner', indicator=False)
print("\nCommon rows between two DataFrames:")
print(common_rows)

Are both DataFrames equal? False

Common rows between two DataFrames:
     Car  Units
0    BMW    100
1 Jaguar     90

How It Works

The merge() method compares all columns in both DataFrames. Only rows where all column values match exactly are returned:

• BMW, 100 ? exists in both DataFrames
• Jaguar, 90 ? exists in both DataFrames
• Lexus ? different Units values (150 vs 250)
• Audi ? different Units values (110 vs 150)

Alternative Method Using Set Operations

For finding common rows using set intersection ?

import pandas as pd

dataFrame1 = pd.DataFrame({
    "Car": ['BMW', 'Lexus', 'Audi', 'Tesla', 'Bentley', 'Jaguar'],
    "Units": [100, 150, 110, 80, 110, 90]
})

dataFrame2 = pd.DataFrame({
    "Car": ['BMW', 'Lexus', 'Audi', 'Mustang', 'Bentley', 'Jaguar'],
    "Units": [100, 250, 150, 80, 130, 90]
})

# Convert to tuples and find intersection
set1 = set([tuple(row) for row in dataFrame1.values])
set2 = set([tuple(row) for row in dataFrame2.values])
common_tuples = set1.intersection(set2)

# Convert back to DataFrame
common_df = pd.DataFrame(list(common_tuples), columns=dataFrame1.columns)
print("Common rows using set intersection:")
print(common_df)

Common rows using set intersection:
     Car  Units
0    BMW    100
1 Jaguar     90

Conclusion

Use merge(dataFrame2, how='inner') to find common rows between DataFrames. This method compares all columns and returns only rows with identical values across both DataFrames.