recfunctions.append_fields fails on arrays containing objects (Trac #1751) #2346
Description
Original ticket http://projects.scipy.org/numpy/ticket/1751 on 2011-02-28 by trac user aickley, assigned to unknown.
The problem is that append_fields calls array.view() which does not accept arrays containing objects (See [http://projects.scipy.org/numpy/ticket/674])
Test case (to include in numpy/lib/tests/recfunctions.py):
class TestAppendFieldsObj(TestCase):
"""
Test append_fields with arrays containing objects
"""
def setUp(self):
from datetime import date
self.data = dict(obj=date(2000, 1, 1))
def test_append_to_objects(self):
"Test append_fields when the base array contains objects"
obj = self.data['obj']
x = np.array([(obj, 1.), (obj, 2.)], dtype=[('A', object), ('B', float)])
y = np.array([10, 20], dtype=int)
test = append_fields(x, 'C', data=y, usemask=False)
control = np.array([(obj, 1.0, 10), (obj, 2.0, 20)],
dtype=[('A', object), ('B', float), ('C', int)])
assert_equal(test, control)
#
def test_append_with_objects(self):
"Test append_fields when the appended data contains objects"
obj = self.data['obj']
x = np.array([(10, 1.), (20, 2.)], dtype=[('A', int), ('B', float)])
y = np.array([obj, obj], dtype=object)
test = append_fields(y, 'C', data=y, dtypes=object, usemask=False)
control = np.array([(10, 1.0, obj), (20, 2.0, obj)],
dtype=[('A', int), ('B', float), ('C', object)])
assert_equal(test, control)
======================================================================
ERROR: Test append_fields when the base array contains objects
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/kolpakov/lib/python2.6/site-packages/numpy/lib/tests/test_recfunctions.py", line 406, in test_append_to_objects
test = append_fields(x, 'C', data=y, usemask=False)
File "/home/kolpakov/lib/python2.6/site-packages/numpy/lib/recfunctions.py", line 629, in append_fields
base = merge_arrays(base, usemask=usemask, fill_value=fill_value)
File "/home/kolpakov/lib/python2.6/site-packages/numpy/lib/recfunctions.py", line 399, in merge_arrays
return seqarrays.view(dtype=seqdtype, type=seqtype)
TypeError: Cannot change data-type for object array.
======================================================================
ERROR: Test append_fields when the appended data contains objects
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/kolpakov/lib/python2.6/site-packages/numpy/lib/tests/test_recfunctions.py", line 416, in test_append_with_objects
test = append_fields(y, 'C', data=y, dtypes=object, usemask=False)
File "/home/kolpakov/lib/python2.6/site-packages/numpy/lib/recfunctions.py", line 627, in append_fields
for (a, n, d) in zip(data, names, dtypes)]
TypeError: Cannot change data-type for object array.
----------------------------------------------------------------------
Ran 33 tests in 0.062s
Regarding the first failure, append_fields calls merge_arrays on the base array:
base = merge_arrays(base, usemask=usemask, fill_value=fill_value)
Actually this line can be completely removed without breaking existing unit tests. I don't like this call because if we are calling merge_array on a single array we are are not actually merging anything, but rather making sure that the array conforms to certain criteria. Maybe this code should be refactored out from merge_arrays...