Skip to content

[improve] [broker] make consumer and producer can work even if schema ledger lost #18010

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
poorbarcode wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[improve] [broker] make consumer and producer can work even if schema ledger lost #18010

poorbarcode wants to merge 1 commit into from

Conversation

@poorbarcode
Copy link
Contributor

@poorbarcode poorbarcode commented Oct 11, 2022 •
edited
Loading

Fixes: #17221

Master Issue: #17221

Motivation

In the current implementation, if the schema ledger is lost, the pulsar client create consumer and pulsar client create producer will fail case by could not read the origin schema.

Modifications

If enabled autoSkipNonRecoverableData, when the schema ledger is lost, the consumer and producer can add new schemas without compatibility check(because the original schema definition cannot be found).

Verifying this change

When the schema ledger is lost, if the newly added schema is not compatible with the schema in the historical message, consumers will still fail (because the compatibility check has been skipped).

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@poorbarcode poorbarcode mentioned this pull request Oct 11, 2022
Closed
@github-actions
Copy link

github-actions bot commented Oct 11, 2022

@poorbarcode Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

@poorbarcode poorbarcode changed the title [ci-run][improve][broker]make consumer and producer can work even if schema ledger lost [improve] [broker] make consumer and producer can work even if schema ledger lost Oct 11, 2022
@github-actions github-actions bot added doc-not-needed Your PR changes do not impact docs and removed doc-label-missing labels Oct 11, 2022
@github-actions
Copy link

github-actions bot commented Nov 12, 2022

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions bot added the Stale label Nov 12, 2022
@poorbarcode poorbarcode force-pushed the broken_ledger/schema branch from 5f71523 to 44f9bc6 Compare March 17, 2023 17:07
@poorbarcode poorbarcode force-pushed the broken_ledger/schema branch from 44f9bc6 to 45b787e Compare March 17, 2023 17:17
@poorbarcode poorbarcode mentioned this pull request Apr 23, 2023
Open
15 tasks
@Denovo1998 Denovo1998 mentioned this pull request May 27, 2023
Open
2 tasks
@Technoboy- Technoboy- added this to the 3.2.0 milestone Jul 31, 2023
@Technoboy- Technoboy- modified the milestones: 3.2.0, 3.3.0 Dec 22, 2023
This was referenced Apr 16, 2024
Open
Merged
@coderzc coderzc modified the milestones: 3.3.0, 3.4.0 May 8, 2024
@lhotari lhotari modified the milestones: 4.0.0, 4.1.0 Oct 14, 2024
@coderzc coderzc modified the milestones: 4.1.0, 4.2.0 Sep 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@Denovo1998 Denovo1998 Denovo1998 left review comments

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

doc-not-needed Your PR changes do not impact docs Stale

Projects

None yet

Milestone

4.2.0

Development

Successfully merging this pull request may close these issues.

Make the autoSkipNonRecoverableData can work for schema Ledger

5 participants

@poorbarcode @Denovo1998 @lhotari @Technoboy- @coderzc