Skip to content

Fix assertion error for messages_on_hold._activate_ordering_keys #913

@acocuzzo

Description

@acocuzzo

It is possible to call activate_ordering_keys with keys which are not present in messages_on_hold._pending_ordered_messages. This can happen if:

  1. A message (or messages) lease exceeds max_lease_duration, it is dropped from maintain_leases

  2. We call drop in the dispatcher

  3. If there are no messages remaining in messages_on_hold._pending_ordered_messages, therefore the pending_ordered_messages queue for the ordering key is removed:
    https://github.com/googleapis/python-pubsub/blob/main/google/cloud/pubsub_v1/subscriber/_protocol/messages_on_hold.py#L154

  4. The message is acked from the user callback, the ack completes and drop is called in the dispatcher:

  5. We call _activate_ordering_keys with a key that does not exist in messages_on_hold._pending_ordered_messages:
    https://github.com/googleapis/python-pubsub/blob/main/google/cloud/pubsub_v1/subscriber/_protocol/messages_on_hold.py#L122

Reproduction:

  1. Publish a single message with an ordering key:
  2. Subscribe with flow control max_lease_duration < time to ack the message, to ensure that the library will drop leasing prior to the message being acked.

Results:

Listening for messages...
Received b'message1'.
Dropping 1 items because they were leased too long.
Acking message with ordering key: key1 
AssertionError: A message queue should exist for every ordered message in flight.

Metadata

Metadata

Assignees

Labels

api: pubsubIssues related to the googleapis/python-pubsub API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions