ListColumnData::ScanCount - internal list scan offset is out of range #10212
Description
What happens?
I've started testing DuckDb and have imported some 90 million rows of entities from https://dumps.wikimedia.org/wikidatawiki/entities/
This is what my table looks like:
select * from items where id > 'P' limit 10;
┌─────────┬─────────┬───────────────────┬──────────────────────┬──────────────────────┬───────────────────────────────────┬─────────────────────┬───────────┬────────────────────────────────────────────────────────────┬───────────────────┐
│ id │ type │ label │ description │ aliases │ instance_of │ subclass_of │ facet_of │ part_of │ wiki_title │
│ varchar │ varchar │ varchar │ varchar │ varchar[] │ varchar[] │ varchar[] │ varchar[] │ varchar[] │ varchar │
├─────────┼─────────┼───────────────────┼──────────────────────┼──────────────────────┼───────────────────────────────────┼─────────────────────┼───────────┼────────────────────────────────────────────────────────────┼───────────────────┤
│ Q31 │ item │ Belgium │ country in western… │ [Kingdom of Belgiu… │ [Q3624078, Q43702, Q6256, Q2018… │ [] │ [] │ [Q215669, Q458, Q46, Q1065, Q13116, Q476033, Q7184, Q8932] │ Belgium │
│ Q8 │ item │ happiness │ mental or emotiona… │ [joy, happy] │ [Q331769, Q60539479] │ [Q16748867] │ [] │ [Q77468620, Q13100823] │ Happiness │
│ Q23 │ item │ George Washington │ president of the U… │ [Father of the Uni… │ [Q5] │ [] │ [] │ [Q186539] │ George Washington │
│ Q24 │ item │ Jack Bauer │ character from the… │ │ [Q15632617, Q15773317, Q20085850] │ [] │ [] │ [] │ Jack Bauer │
│ Q42 │ item │ Douglas Adams │ English author and… │ [Douglas Noël Adam… │ [Q5] │ [] │ [] │ [] │ Douglas Adams │
│ Q1868 │ item │ Paul Otlet │ Belgian author, li… │ [Paul Marie Ghisla… │ [Q5] │ [] │ [] │ [] │ Paul Otlet │
│ Q2013 │ item │ Wikidata │ free knowledge gra… │ [wikidata.org, m.w… │ [Q33120876, Q638153, Q36509592,… │ [] │ [] │ [Q54837, Q43984865] │ Wikidata │
│ Q45 │ item │ Portugal │ country in Southwe… │ [Portuguese Republ… │ [Q3624078, Q6256, Q20181813, Q1… │ [] │ [] │ [Q50807777, Q458, Q8932] │ Portugal │
│ Q51 │ item │ Antarctica │ polar continent in… │ [Antarctic Selecti… │ [Q5107, Q82794, Q312461] │ [] │ [] │ [Q205895, Q1555938, Q2] │ Antarctica │
│ Q58 │ item │ penis │ primary sexual org… │ │ [Q712378] │ [Q4620674, Q168552] │ [] │ [Q842083, Q181100] │ Penis │
├─────────┴─────────┴───────────────────┴──────────────────────┴──────────────────────┴───────────────────────────────────┴─────────────────────┴───────────┴────────────────────────────────────────────────────────────┴───────────────────┤
│ 10 rows 10 columns │
└────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
Ids are strings that can either contain a 'P'+<number> or 'Q'+<number>, so I was interested in what's in there:
D select count(*) from items where id > 'Q';
┌──────────────┐
│ count_star() │
│ int64 │
├──────────────┤
│ 90170415 │
└──────────────┘
However this fails
D select * from items where id < 'Q' LIMIT 10;
Error: INTERNAL Error: ListColumnData::ScanCount - internal list scan offset is out of rangeTo Reproduce
I can share the file, but it's rather large: 8 GB
OS:
Ubuntu 23.10
DuckDB Version:
v0.9.2 3c695d7 (official binary from DuckDb.org)
DuckDB Client:
DuckDb command line client (SQL)
Full Name:
Peter Kese
Affiliation:
/
Have you tried this on the latest main branch?
I have tested with a main build
Have you tried the steps to reproduce? Do they include all relevant data and configuration? Does the issue you report still appear there?
- Yes, I have