acceptance: many nightlies failing with cockroach SIGILL #19909
Description
Example failures:
- https://teamcity.cockroachdb.com/viewLog.html?buildId=401964&tab=buildResultsDiv&buildTypeId=Cockroach_Nightlies_Steady_6Medium
- https://teamcity.cockroachdb.com/viewType.html?buildTypeId=Cockroach_Nightlies_SchemaChangeTest3to510g
- https://teamcity.cockroachdb.com/viewLog.html?buildId=401954&tab=buildResultsDiv&buildTypeId=C_BW_4Nodes_6Hours
Using gdb on the core dump, I got this stack trace:
Program terminated with signal SIGILL, Illegal instruction.
#0 0x000000000193b75d in rocksdb::LRUCache::LRUCache(unsigned long, int, bool, double) ()
(gdb) where
#0 0x000000000193b75d in rocksdb::LRUCache::LRUCache(unsigned long, int, bool, double) ()
#1 0x000000000193b9c7 in rocksdb::NewLRUCache(unsigned long, int, bool, double) ()
#2 0x0000000001a355b0 in rocksdb::BlockBasedTableFactory::BlockBasedTableFactory(rocksdb::BlockBasedTableOptions const&) ()
#3 0x0000000001a1b71e in rocksdb::ColumnFamilyOptions::ColumnFamilyOptions() ()
#4 0x000000000057422c in ?? ()
#5 0x0000000001c25aed in __libc_csu_init ()
#6 0x00007f7a40ddd7bf in __gconv (cd=0x7f7a40845010, inbuf=0x800000, inbufend=0x63c4d5 <_start> "1\355I\211\321^H\211\342H\203\344\360PTI\307\300\020[\302\001H\307\301\240Z\302\001H\307ǀ\245i", outbuf=0x0, outbufend=0x0,
irreversible=0x23) at gconv.c:47
#7 0x0000000001c25aa0 in ?? ()
#8 0x00007ffe57493288 in ?? ()
#9 0x0000000000000034 in ?? ()
#10 0x000000000069a580 in ?? ()
#11 0x0000000001c25b10 in ?? ()
#12 0x00007f7a41aed8e0 in _dl_sort_fini (maps=0x0, nmaps=0, used=0x0, ns=<optimized out>) at dl-fini.c:117
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
I can't get more info at the moment, because the nightly tests download a stripped binary.