Skip to content

ScaleoutStore doesn't update MinMappingId properly, resulting in lost messages #4121

@analogrelay

Description

@analogrelay

When a fragment is overwritten in ScaleoutStore, we update _minMappingId with this code:

// Move the minimum id when we overwrite
if (overwrite)
{
_minMessageId = (long)(existingFragment.MaxId + 1);
_minMappingId = existingFragment.MaxId;
}
else if (idxIntoFragmentsArray == 0)
{
_minMappingId = mapping.Id;
}

However, existingFragment.MaxId is a Message ID not a Mapping ID!

This _minMappingId value is used to detect a really old cursor (older than the oldest mapping in the store) here:

// If we're expired or we're at the first mapping or we're lower than the
// min then get everything
if (mappingId < _minMappingId || mappingId == UInt64.MaxValue)
{
return GetAllMessages(minMessageId);
}

Once Mapping IDs go over 65,535, this completely breaks and any client with a cursor below the current minimum mapping is completely unable to get any new messages.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions