Skip to content

LocalSchema.migratedFileIds is not persisted — compaction crashes silently drop in-flight transactions #4333

@ruispereira

Description

@ruispereira

Affected version: 26.4.2 (confirmed still present in 26.5.1)
Component: com.arcadedb.schema.LocalSchema / com.arcadedb.engine.TransactionManager

Summary

LocalSchema.migratedFileIds (the oldFileId → newFileId map written by LSM/vector index compaction) is a ConcurrentHashMap in JVM heap. It is not included in schema.json (verified by reading toJSON() at line 1749 of LocalSchema.java). After a restart the map is empty. WAL records addressed to the old fileId hit !existsFile(...) in applyChanges and are silently skipped with only a FINE-level log.

Code

engine/com/arcadedb/schema/LocalSchema.java:119, 1861–1868
engine/com/arcadedb/engine/TransactionManager.java:307–315

// LocalSchema:
private final Map<Integer, Integer> migratedFileIds = new ConcurrentHashMap<>();

// TransactionManager.applyChanges:
if (!database.getFileManager().existsFile(txPage.fileId)) {
  LogManager.instance().log(this, Level.FINE, "Skipping page for deleted file %d during transaction apply", null, txPage.fileId);
  continue;
}

Impact

A crash during or shortly after an index compaction loses any committed mutation whose WAL record still references the old fileId. No SEVERE-level log; "recovery completed" message lies.

Suggested fix

Persist the migration map (as part of schema.json, with fsync of the schema file before announcing the migration to clients). Consult it in
applyChanges to redirect old-fileId pages to the new file. At minimum log SEVERE (not FINE) when a recovery WAL page references a missing file.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions