Skip to content

Fatal Stack Overflow Stemming From Misuse of astype('category') #23049

@filpia

Description

@filpia

Code Sample

import pandas as pd
foo = pd.DataFrame([{'a':'hello'}])
bar = pd.concat([foo,foo], axis=1)
bar.astype('category') # this will make it go boom

Problem description

As evidenced above, I hit a stack overflow from an error on my end when attempting to cast a column that appears twice in a DataFrame from a careless pd.concat(, axis=1). This error led to a fatal stack overflow error that kills the Python session with the following error...

Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00007fff9d7f83c0 (most recent call first):
  File "<frozen importlib._bootstrap>", line 165 in _get_module_lock
  File "<frozen importlib._bootstrap>", line 148 in __enter__
...
  File "/Users/filippiasevoli/anaconda/lib/python3.6/site-packages/pandas/util/_decorators.py", line 178 in wrapper
  File "/Users/filippiasevoli/anaconda/lib/python3.6/site-packages/pandas/core/generic.py", line 4996 in <genexpr>
  File "/Users/filippiasevoli/anaconda/lib/python3.6/site-packages/pandas/core/reshape/concat.py", line 256 in __init__
  ...
Abort trap: 6

Expected Output

Informative error, not a fatal.

Details

INSTALLED VERSIONS

commit: None
python: 3.6.1.final.0
python-bits: 64
OS: Darwin
OS-release: 16.7.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.4
pytest: 3.0.7
pip: 9.0.1
setuptools: 40.2.0
Cython: 0.25.2
numpy: 1.15.2
scipy: 1.1.0
pyarrow: None
xarray: 0.10.0
IPython: 5.3.0
sphinx: 1.5.6
patsy: 0.5.0
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: 1.2.1
tables: 3.3.0
numexpr: 2.6.5
feather: None
matplotlib: 2.0.2
openpyxl: 2.4.7
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.3
bs4: 4.6.0
html5lib: 0.999
sqlalchemy: 1.2.9
pymysql: None
psycopg2: 2.7.3.1 (dt dec pq3 ext lo64)
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    CategoricalCategorical Data TypeNeeds TestsUnit test(s) needed to prevent regressions

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions