Skip to content

[FW] 17.0 pypdf trixie compat moc#185282

Closed
fw-bot wants to merge 8 commits intoodoo:saas-17.4from
odoo-dev:saas-17.4-17.0-pypdf-trixie-compat-moc-nhHz-fw
Closed

[FW] 17.0 pypdf trixie compat moc#185282
fw-bot wants to merge 8 commits intoodoo:saas-17.4from
odoo-dev:saas-17.4-17.0-pypdf-trixie-compat-moc-nhHz-fw

Conversation

@fw-bot
Copy link
Contributor

@fw-bot fw-bot commented Oct 25, 2024

As Debian wants to remove pypdf2 and keep only pypdf (4.3) in
trixie, so we need to be compatible otherwise Odoo could not be released in the next Debian.

Forward-Port-Of: #184787
Forward-Port-Of: #183165

@robodoo
Copy link
Contributor

robodoo commented Oct 25, 2024

Pull request status dashboard

@fw-bot
Copy link
Contributor Author

fw-bot commented Oct 25, 2024

@d-fence cherrypicking of pull request #183165 failed.

stdout:

Auto-merging addons/account/models/ir_actions_report.py
Auto-merging addons/account/models/ir_attachment.py
Auto-merging addons/sale_pdf_quote_builder/models/ir_actions_report.py
CONFLICT (content): Merge conflict in addons/sale_pdf_quote_builder/models/ir_actions_report.py
Auto-merging addons/website_slides/models/slide_slide.py
Auto-merging odoo/addons/base/models/ir_actions_report.py
CONFLICT (content): Merge conflict in odoo/addons/base/models/ir_actions_report.py
Auto-merging odoo/tools/pdf/__init__.py
CONFLICT (content): Merge conflict in odoo/tools/pdf/__init__.py

Either perform the forward-port manually (and push to this branch, proceeding as usual) or close this PR (maybe?).

In the former case, you may want to edit this PR message as well.

⚠️ after resolving this conflict, you will need to merge it via @robodoo.

More info at https://github.com/odoo/odoo/wiki/Mergebot#forward-port

@robodoo robodoo added forwardport This PR was created by @fw-bot conflict There was an error while creating this forward-port PR labels Oct 25, 2024
@C3POdoo C3POdoo added the RD research & development, internal work label Oct 25, 2024
@fw-bot
Copy link
Contributor Author

fw-bot commented Oct 26, 2024

@d-fence this forward port of #183165 is awaiting action (not merged or closed).

3 similar comments
@fw-bot
Copy link
Contributor Author

fw-bot commented Oct 27, 2024

@d-fence this forward port of #183165 is awaiting action (not merged or closed).

@fw-bot
Copy link
Contributor Author

fw-bot commented Oct 28, 2024

@d-fence this forward port of #183165 is awaiting action (not merged or closed).

@fw-bot
Copy link
Contributor Author

fw-bot commented Oct 29, 2024

@d-fence this forward port of #183165 is awaiting action (not merged or closed).

xmo-odoo and others added 8 commits October 29, 2024 09:58
Motivation: Debian wants to kill pypdf2 and keep only pypdf (4.3) in
trixie, so we need to be compatible or risk being dropped.

- move all imports from PyPDF2 to odoo.tools.pdf
- add two-ways compatibility shims and aliases

X-original-commit: fddf53c
In 3.x the PDF header is the header *only*, as it's decoded to a
string. However the mess is mostly unnecessary as since 1.27 pypdf2
always adds a binary comments line when writing out documents. So only
add (or copy) our own garbage when using 1.x.

Also this means our heuristic to detect pdf/a documents makes no
sense, as any pdf document can have a binary sig, but hey...

Also fix _ID handling: in pypdf 3.x it's a property which reads from
the trailer *and* it's automatically copied over by
`clone_reader_document_root`.

X-original-commit: 3dd4a34
Try to override both old and new API, although for the new version it
might actually be useless as pypdf seems to have added multiple
attachments support circa 3.5? (py-pdf/pypdf#1611)

X-original-commit: c78870c
The test relied on reader and writer sharing data because
`clone_reader_document_root` would just copy the reference to the root
object, so the writer would immediately impact the reader.

pypdf apparently now copies the document deeply (possibly since 3.2.0
/ py-pdf/pypdf#1520) so this is broken, we need to serialize and
reload the document every time.

X-original-commit: d5807f4
Next up in the series "`clone_document_from_reader` does stuff now",
`clone_document_from_reader` now properly clones over document
metadata. This means any metadata set *before* the cloning are nuked.

Update the branded writer to write its metadata at the last possible
moment, to ensure it always applies (this might actually be
undesirable as it means callers can't trivially override it anymore).

X-original-commit: e391863
`test_download_with_encrypted_pdf` tried to corrupt the PDF by
redirecting the /Encrypt indirect ref to an invalid entry, but it did
so by replacing a hard-coded index. Except the encryption object is
not stored at a fixed offset, where it's stored depends on what pypdf
wants to do with it or what other objects it stores in the file.

Update the replacement to not be completely hard-coded (and also not
assume /Encrypt is at the end of the trailer), and point it back to a
very early object (specifically the first one) which is *extremely*
unlikely to be a valid encryption object (it's very likely to be
either the catalog or the page tree root, though it could also be the
metadata map). This leads pypdf to not find a `/Filter` entry, which
triggers a parse error.

Note: the indirect ref can't be pointed at a nonsensical entry (e.g. 0
or 5479) because in that case pypdf just ignores the ref entirely.

X-original-commit: f37d753
Partial cherry-pick of 59709ca to be able to build a debian package in
Trixie.

X-original-commit: 72b419e
@d-fence d-fence force-pushed the saas-17.4-17.0-pypdf-trixie-compat-moc-nhHz-fw branch from 0236122 to a1756c7 Compare October 29, 2024 09:10
@C3POdoo C3POdoo requested review from a team, HydrionBurst and rco-odoo and removed request for a team October 29, 2024 09:12
@d-fence
Copy link
Contributor

d-fence commented Oct 29, 2024

robodoo override=ci/security

was already overriden by xmo in the initial PR

@d-fence
Copy link
Contributor

d-fence commented Oct 29, 2024

robodoo override=ci/style
This is the PR for the tools that this ci is asking for 🐍

@d-fence
Copy link
Contributor

d-fence commented Oct 29, 2024

robodoo r+

robodoo pushed a commit that referenced this pull request Oct 29, 2024
Motivation: Debian wants to kill pypdf2 and keep only pypdf (4.3) in
trixie, so we need to be compatible or risk being dropped.

- move all imports from PyPDF2 to odoo.tools.pdf
- add two-ways compatibility shims and aliases

X-original-commit: fddf53c
Part-of: #185282
Related: odoo/enterprise#72781
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Oct 29, 2024
In 3.x the PDF header is the header *only*, as it's decoded to a
string. However the mess is mostly unnecessary as since 1.27 pypdf2
always adds a binary comments line when writing out documents. So only
add (or copy) our own garbage when using 1.x.

Also this means our heuristic to detect pdf/a documents makes no
sense, as any pdf document can have a binary sig, but hey...

Also fix _ID handling: in pypdf 3.x it's a property which reads from
the trailer *and* it's automatically copied over by
`clone_reader_document_root`.

X-original-commit: 3dd4a34
Part-of: #185282
Related: odoo/enterprise#72781
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Oct 29, 2024
Try to override both old and new API, although for the new version it
might actually be useless as pypdf seems to have added multiple
attachments support circa 3.5? (py-pdf/pypdf#1611)

X-original-commit: c78870c
Part-of: #185282
Related: odoo/enterprise#72781
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Oct 29, 2024
The test relied on reader and writer sharing data because
`clone_reader_document_root` would just copy the reference to the root
object, so the writer would immediately impact the reader.

pypdf apparently now copies the document deeply (possibly since 3.2.0
/ py-pdf/pypdf#1520) so this is broken, we need to serialize and
reload the document every time.

X-original-commit: d5807f4
Part-of: #185282
Related: odoo/enterprise#72781
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Oct 29, 2024
Next up in the series "`clone_document_from_reader` does stuff now",
`clone_document_from_reader` now properly clones over document
metadata. This means any metadata set *before* the cloning are nuked.

Update the branded writer to write its metadata at the last possible
moment, to ensure it always applies (this might actually be
undesirable as it means callers can't trivially override it anymore).

X-original-commit: e391863
Part-of: #185282
Related: odoo/enterprise#72781
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Oct 29, 2024
X-original-commit: 0180654
Part-of: #185282
Related: odoo/enterprise#72781
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Oct 29, 2024
`test_download_with_encrypted_pdf` tried to corrupt the PDF by
redirecting the /Encrypt indirect ref to an invalid entry, but it did
so by replacing a hard-coded index. Except the encryption object is
not stored at a fixed offset, where it's stored depends on what pypdf
wants to do with it or what other objects it stores in the file.

Update the replacement to not be completely hard-coded (and also not
assume /Encrypt is at the end of the trailer), and point it back to a
very early object (specifically the first one) which is *extremely*
unlikely to be a valid encryption object (it's very likely to be
either the catalog or the page tree root, though it could also be the
metadata map). This leads pypdf to not find a `/Filter` entry, which
triggers a parse error.

Note: the indirect ref can't be pointed at a nonsensical entry (e.g. 0
or 5479) because in that case pypdf just ignores the ref entirely.

X-original-commit: f37d753
Part-of: #185282
Related: odoo/enterprise#72781
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Oct 29, 2024
Partial cherry-pick of 59709ca to be able to build a debian package in
Trixie.

closes #185282

X-original-commit: 72b419e
Related: odoo/enterprise#72781
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
@robodoo robodoo closed this Oct 29, 2024
@d-fence d-fence deleted the saas-17.4-17.0-pypdf-trixie-compat-moc-nhHz-fw branch November 7, 2024 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conflict There was an error while creating this forward-port PR forwardport This PR was created by @fw-bot RD research & development, internal work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants