Skip to content

[FW] 17.0 pypdf trixie compat moc#186301

Closed
fw-bot wants to merge 7 commits intoodoo:masterfrom
odoo-dev:master-17.0-pypdf-trixie-compat-moc-cFSx-fw
Closed

[FW] 17.0 pypdf trixie compat moc#186301
fw-bot wants to merge 7 commits intoodoo:masterfrom
odoo-dev:master-17.0-pypdf-trixie-compat-moc-cFSx-fw

Conversation

@fw-bot
Copy link
Contributor

@fw-bot fw-bot commented Nov 5, 2024

As Debian wants to remove pypdf2 and keep only pypdf (4.3) in
trixie, so we need to be compatible otherwise Odoo could not be released in the next Debian.

Forward-Port-Of: #185736
Forward-Port-Of: #183165

@robodoo
Copy link
Contributor

robodoo commented Nov 5, 2024

Pull request status dashboard

@fw-bot
Copy link
Contributor Author

fw-bot commented Nov 5, 2024

@d-fence cherrypicking of pull request #183165 failed.

stdout:

Auto-merging addons/account/models/ir_actions_report.py
Auto-merging addons/account/models/ir_attachment.py
Auto-merging addons/sale_pdf_quote_builder/models/ir_actions_report.py
CONFLICT (content): Merge conflict in addons/sale_pdf_quote_builder/models/ir_actions_report.py
Auto-merging addons/snailmail/models/snailmail_letter.py
Auto-merging addons/website_slides/models/slide_slide.py
Auto-merging odoo/addons/base/models/ir_actions_report.py
CONFLICT (content): Merge conflict in odoo/addons/base/models/ir_actions_report.py
Auto-merging odoo/tools/pdf/__init__.py
CONFLICT (content): Merge conflict in odoo/tools/pdf/__init__.py

Either perform the forward-port manually (and push to this branch, proceeding as usual) or close this PR (maybe?).

In the former case, you may want to edit this PR message as well.

⚠️ after resolving this conflict, you will need to merge it via @robodoo.

More info at https://github.com/odoo/odoo/wiki/Mergebot#forward-port

@robodoo robodoo added forwardport This PR was created by @fw-bot conflict There was an error while creating this forward-port PR labels Nov 5, 2024
@C3POdoo C3POdoo added the RD research & development, internal work label Nov 5, 2024
@d-fence d-fence force-pushed the master-17.0-pypdf-trixie-compat-moc-cFSx-fw branch from b1aed18 to 02a0f5d Compare November 6, 2024 08:51
@C3POdoo C3POdoo requested review from a team, rco-odoo and xmo-odoo and removed request for a team November 6, 2024 08:54
@d-fence d-fence force-pushed the master-17.0-pypdf-trixie-compat-moc-cFSx-fw branch from 02a0f5d to 420145d Compare November 6, 2024 10:07
Motivation: Debian wants to kill pypdf2 and keep only pypdf (4.3) in
trixie, so we need to be compatible or risk being dropped.

- move all imports from PyPDF2 to odoo.tools.pdf
- add two-ways compatibility shims and aliases

X-original-commit: fddf53c
In 3.x the PDF header is the header *only*, as it's decoded to a
string. However the mess is mostly unnecessary as since 1.27 pypdf2
always adds a binary comments line when writing out documents. So only
add (or copy) our own garbage when using 1.x.

Also this means our heuristic to detect pdf/a documents makes no
sense, as any pdf document can have a binary sig, but hey...

Also fix _ID handling: in pypdf 3.x it's a property which reads from
the trailer *and* it's automatically copied over by
`clone_reader_document_root`.

X-original-commit: 3dd4a34
Try to override both old and new API, although for the new version it
might actually be useless as pypdf seems to have added multiple
attachments support circa 3.5? (py-pdf/pypdf#1611)

X-original-commit: c78870c
The test relied on reader and writer sharing data because
`clone_reader_document_root` would just copy the reference to the root
object, so the writer would immediately impact the reader.

pypdf apparently now copies the document deeply (possibly since 3.2.0
/ py-pdf/pypdf#1520) so this is broken, we need to serialize and
reload the document every time.

X-original-commit: d5807f4
Next up in the series "`clone_document_from_reader` does stuff now",
`clone_document_from_reader` now properly clones over document
metadata. This means any metadata set *before* the cloning are nuked.

Update the branded writer to write its metadata at the last possible
moment, to ensure it always applies (this might actually be
undesirable as it means callers can't trivially override it anymore).

X-original-commit: e391863
`test_download_with_encrypted_pdf` tried to corrupt the PDF by
redirecting the /Encrypt indirect ref to an invalid entry, but it did
so by replacing a hard-coded index. Except the encryption object is
not stored at a fixed offset, where it's stored depends on what pypdf
wants to do with it or what other objects it stores in the file.

Update the replacement to not be completely hard-coded (and also not
assume /Encrypt is at the end of the trailer), and point it back to a
very early object (specifically the first one) which is *extremely*
unlikely to be a valid encryption object (it's very likely to be
either the catalog or the page tree root, though it could also be the
metadata map). This leads pypdf to not find a `/Filter` entry, which
triggers a parse error.

Note: the indirect ref can't be pointed at a nonsensical entry (e.g. 0
or 5479) because in that case pypdf just ignores the ref entirely.

X-original-commit: f37d753
@d-fence d-fence force-pushed the master-17.0-pypdf-trixie-compat-moc-cFSx-fw branch from 420145d to 2cd4210 Compare November 6, 2024 12:14
@d-fence
Copy link
Contributor

d-fence commented Nov 6, 2024

robodoo override=ci/security

was already overriden by xmo in the initial PR

@d-fence
Copy link
Contributor

d-fence commented Nov 6, 2024

robodoo override=ci/style
This is the PR for the tools that this ci is asking for 🐍

@d-fence
Copy link
Contributor

d-fence commented Nov 6, 2024

robodoo r+

robodoo pushed a commit that referenced this pull request Nov 6, 2024
Motivation: Debian wants to kill pypdf2 and keep only pypdf (4.3) in
trixie, so we need to be compatible or risk being dropped.

- move all imports from PyPDF2 to odoo.tools.pdf
- add two-ways compatibility shims and aliases

X-original-commit: fddf53c
Part-of: #186301
Related: odoo/enterprise#73330
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Nov 6, 2024
In 3.x the PDF header is the header *only*, as it's decoded to a
string. However the mess is mostly unnecessary as since 1.27 pypdf2
always adds a binary comments line when writing out documents. So only
add (or copy) our own garbage when using 1.x.

Also this means our heuristic to detect pdf/a documents makes no
sense, as any pdf document can have a binary sig, but hey...

Also fix _ID handling: in pypdf 3.x it's a property which reads from
the trailer *and* it's automatically copied over by
`clone_reader_document_root`.

X-original-commit: 3dd4a34
Part-of: #186301
Related: odoo/enterprise#73330
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Nov 6, 2024
Try to override both old and new API, although for the new version it
might actually be useless as pypdf seems to have added multiple
attachments support circa 3.5? (py-pdf/pypdf#1611)

X-original-commit: c78870c
Part-of: #186301
Related: odoo/enterprise#73330
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Nov 6, 2024
The test relied on reader and writer sharing data because
`clone_reader_document_root` would just copy the reference to the root
object, so the writer would immediately impact the reader.

pypdf apparently now copies the document deeply (possibly since 3.2.0
/ py-pdf/pypdf#1520) so this is broken, we need to serialize and
reload the document every time.

X-original-commit: d5807f4
Part-of: #186301
Related: odoo/enterprise#73330
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Nov 6, 2024
Next up in the series "`clone_document_from_reader` does stuff now",
`clone_document_from_reader` now properly clones over document
metadata. This means any metadata set *before* the cloning are nuked.

Update the branded writer to write its metadata at the last possible
moment, to ensure it always applies (this might actually be
undesirable as it means callers can't trivially override it anymore).

X-original-commit: e391863
Part-of: #186301
Related: odoo/enterprise#73330
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Nov 6, 2024
X-original-commit: 0180654
Part-of: #186301
Related: odoo/enterprise#73330
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
robodoo pushed a commit that referenced this pull request Nov 6, 2024
`test_download_with_encrypted_pdf` tried to corrupt the PDF by
redirecting the /Encrypt indirect ref to an invalid entry, but it did
so by replacing a hard-coded index. Except the encryption object is
not stored at a fixed offset, where it's stored depends on what pypdf
wants to do with it or what other objects it stores in the file.

Update the replacement to not be completely hard-coded (and also not
assume /Encrypt is at the end of the trailer), and point it back to a
very early object (specifically the first one) which is *extremely*
unlikely to be a valid encryption object (it's very likely to be
either the catalog or the page tree root, though it could also be the
metadata map). This leads pypdf to not find a `/Filter` entry, which
triggers a parse error.

Note: the indirect ref can't be pointed at a nonsensical entry (e.g. 0
or 5479) because in that case pypdf just ignores the ref entirely.

closes #186301

X-original-commit: f37d753
Related: odoo/enterprise#73330
Signed-off-by: Christophe Monniez (moc) <moc@odoo.com>
@robodoo robodoo closed this Nov 6, 2024
@robodoo robodoo added the 18.1 label Nov 6, 2024
@d-fence d-fence deleted the master-17.0-pypdf-trixie-compat-moc-cFSx-fw branch November 7, 2024 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

18.1 conflict There was an error while creating this forward-port PR forwardport This PR was created by @fw-bot RD research & development, internal work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants