Pandas dataframe values equality test

Question

Another Pandas question!

I am writing some unit tests that test two data frames for equality, however, the test does not appear to look at the values of the data frame, only the structure:

dates = pd.date_range('20130101', periods=6)

df1 = pd.DataFrame(np.random.randn(6, 4), index=dates, columns=list('ABCD'))
df2 = pd.DataFrame(np.random.randn(6, 4), index=dates, columns=list('ABCD'))

print df1
print df2
self.assertItemsEqual(df1, df2)

-->True

Do I need to convert the data frames to another data structure before asserting equality?

STerliakov · Accepted Answer · 2023-11-09 02:46:05Z

44

Ah, of course there is a solution for this already:

from pandas.util.testing import assert_frame_equal

EDIT: pandas.util.testing was deprecated in 2020. From version 1.0 forward, use:

from pandas.testing import assert_frame_equal

The API documentation for assert_frame_equal can be found here

edited Nov 9, 2023 at 2:46

STerliakov

9,1343 gold badges27 silver badges59 bronze badges

answered Nov 12, 2013 at 11:45

MarkNS

4,1414 gold badges49 silver badges68 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Rustam Aliyev · Accepted Answer · 2015-04-09 10:15:14Z

6

While assert_frame_equal is useful in unit tests, I found the following useful on analysis as one might want to further check which values are not equal: df1.equals(df2)

answered Apr 9, 2015 at 10:15

Rustam Aliyev

6491 gold badge12 silver badges20 bronze badges

Comments

ankostis · Accepted Answer · 2014-09-14 17:55:15Z

4

Also numpy's utilities work:

import numpy.testing as npt

npt.assert_array_equal(df1, df2)

answered Sep 14, 2014 at 17:55

ankostis

9,6803 gold badges54 silver badges68 bronze badges

Comments

Surya Chhetri · Accepted Answer · 2017-02-15 06:43:35Z

In [62]: import numpy as np

In [63]: import pandas as pd

In [64]: np.random.seed(30)

In [65]: df_old = pd.DataFrame(np.random.randn(4,5))

In [66]: df_old
Out[66]: 
          0         1         2         3         4
0 -1.264053  1.527905 -0.970711  0.470560 -0.100697
1  0.303793 -1.725962  1.585095  0.134297 -1.106855
2  1.578226  0.107498 -0.764048 -0.775189  1.383847
3  0.760385 -0.285646  0.538367 -2.083897  0.937782

In [67]: np.random.seed(30)

In [68]: df_new = pd.DataFrame(np.random.randn(4,5))

In [69]: df_new
Out[69]: 
          0         1         2         3         4
0 -1.264053  1.527905 -0.970711  0.470560 -0.100697
1  0.303793 -1.725962  1.585095  0.134297 -1.106855
2  1.578226  0.107498 -0.764048 -0.775189  1.383847
3  0.760385 -0.285646  0.538367 -2.083897  0.937782

In [70]: df_old.equals(df_new) #Equality check here, returns boolean expression: True/False
Out[70]: True

Collectives™ on Stack Overflow

Pandas dataframe values equality test

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related