Skip to content

Buffer LongDouble + ndim#6460

Merged
youknowone merged 2 commits intoRustPython:mainfrom
youknowone:buffer-ndim
Dec 20, 2025
Merged

Buffer LongDouble + ndim#6460
youknowone merged 2 commits intoRustPython:mainfrom
youknowone:buffer-ndim

Conversation

@youknowone
Copy link
Copy Markdown
Member

@youknowone youknowone commented Dec 20, 2025

Summary by CodeRabbit

  • Improvements
    • Extended buffer format type support for enhanced data serialization compatibility.
    • Relaxed buffer validation constraints to properly handle edge cases with zero-dimensioned data structures.
    • Updated test execution logic to correctly identify and run package-style test modules.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Dec 20, 2025

Walkthrough

Three distinct code modifications: extended FormatType enum with LongDouble and PyObject variants in buffer format handling, relaxed itemsize validation in BufferDescriptor for zero-length cases, and updated test naming logic to support package-style test discovery via parent directory patterns.

Changes

Cohort / File(s) Summary
Buffer format support
crates/vm/src/buffer.rs
Added FormatType enum variants LongDouble (b'g') and PyObject (b'O') with corresponding mappings for both native and non-native endianness. Extended format string parsing to skip additional marker characters: '{', '}', '&', '<', '>', '@', '=', '!'
Buffer validation
crates/vm/src/protocol/buffer.rs
Relaxed itemsize zero-check in BufferDescriptor::validate (debug build) to allow itemsize=0 when len=0 or for 1D scalar-like cases. Added has_zero_dim check to permit zero strides when any dimension is zero. Preserved final shape_product * itemsize == len invariant.
Test utilities
scripts/fix_test.py
Updated test_name computation to detect package-style tests by checking if parent directory name starts with "test_"; if matched, test_name becomes "parent_stem.stem", otherwise remains file stem.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • crates/vm/src/protocol/buffer.rs: Review the validation logic changes carefully to ensure the relaxed itemsize checks don't introduce unintended edge cases; verify the has_zero_dim condition correctly handles all dimension scenarios.
  • scripts/fix_test.py: Confirm the package-style test detection logic (parent directory prefix "test_") aligns with actual test structure and doesn't create false positives.
  • crates/vm/src/buffer.rs: Verify the new FormatType variants' mappings are correct for both endianness cases and that the expanded character-skipping set doesn't inadvertently skip valid format markers.

Poem

🐰 Format types hop in with grace so fine,
LongDouble, PyObject—in perfect line!
Validation bends where zero leads the way,
Test names find packages come what may,
With stricter bounds now softly play,
The buffer dances through the fray! 🎉

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Buffer LongDouble + ndim' directly references the two main changes: adding LongDouble format type support and relaxing ndim/itemsize constraints.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

📜 Recent review details

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 46c61a6 and 1c0321f.

📒 Files selected for processing (3)
  • crates/vm/src/buffer.rs (4 hunks)
  • crates/vm/src/protocol/buffer.rs (1 hunks)
  • scripts/fix_test.py (1 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.py

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.py: In most cases, Python code should not be edited; bug fixes should be made through Rust code modifications only
Follow PEP 8 style for custom Python code
Use ruff for linting Python code

Files:

  • scripts/fix_test.py
**/*.rs

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

**/*.rs: Follow the default rustfmt code style by running cargo fmt to format Rust code
Always run clippy to lint Rust code (cargo clippy) before completing tasks and fix any warnings or lints introduced by changes
Follow Rust best practices for error handling and memory management
Use the macro system (pyclass, pymodule, pyfunction, etc.) when implementing Python functionality in Rust

Files:

  • crates/vm/src/buffer.rs
  • crates/vm/src/protocol/buffer.rs
🧬 Code graph analysis (1)
crates/vm/src/protocol/buffer.rs (3)
crates/stdlib/src/array.rs (1)
  • itemsize (738-740)
crates/vm/src/builtins/memory.rs (2)
  • itemsize (584-586)
  • shape (594-603)
crates/vm/src/builtins/dict.rs (2)
  • len (716-718)
  • len (747-749)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
  • GitHub Check: Run snippets and cpython tests (ubuntu-latest)
  • GitHub Check: Run snippets and cpython tests (windows-latest)
🔇 Additional comments (7)
scripts/fix_test.py (1)

162-166: LGTM!

The package test detection logic correctly handles nested test structures. The conditional check for test_ prefix in the parent directory name appropriately distinguishes between package-style and single-file tests.

crates/vm/src/buffer.rs (4)

153-155: LGTM with note on long double representation.

The non-native format mappings are reasonable. Note that LongDouble => f64 is a simplification—actual long double varies by platform (80-bit on x86, 128-bit on some architectures)—but this is consistent with the native mapping approach used elsewhere in the code.


190-192: LGTM!

The native format mappings correctly use platform-specific types: raw::c_double for LongDouble and *mut raw::c_void for PyObject pointer representation.


315-326: Verify PEP 3118 compliance for non-standard format characters.

PEP 3118 specifies that endian markers ('!', '@','=','>','<', '^') are allowed inside format strings and can change as needed. The code correctly skips these standard markers (<, >, @, =, !) to allow them mid-string. However, the code also skips & as a "pointer prefix," which is not documented in PEP 3118. The standard format code for pointers is P, not &. Clarify whether & is a custom extension for this implementation beyond PEP 3118, or document what specification it implements.


91-93: Format codes 'g' (LongDouble) and 'O' (PyObject) align with proposed PEP 3118 but use pragmatic approximations.

While PEP 3118 proposed additions to struct string-syntax, only ? (_Bool) was officially implemented in CPython's struct module. However, NumPy's PEP 3118 parser includes both 'g' and 'O' in its supported format codes. The implementation correctly maps LongDouble to f64 (treating it as double) and PyObject to void* pointers. Verify that these pragmatic approximations—particularly the comment "long double same as double for now"—are appropriate for downstream buffer protocol consumers and document any deviations from full PEP 3118 compliance if this implementation is exposed as a standards-conformant buffer protocol provider.

crates/vm/src/protocol/buffer.rs (2)

206-210: LGTM!

The relaxed validation correctly handles empty scalar structures (len=0, itemsize=0) while maintaining the invariant itemsize == len for zero-dimensional cases. The conditional check if self.len > 0 appropriately enforces non-zero itemsize only when necessary.


213-220: LGTM!

The has_zero_dim flag correctly permits zero strides for empty arrays (any dimension is zero) while preserving the stride validation for non-empty arrays. Computing the flag before iteration is efficient, and the final invariant shape_product * itemsize == len (line 222) remains enforced.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@youknowone youknowone marked this pull request as ready for review December 20, 2025 09:03
@youknowone youknowone merged commit 7bfa5d9 into RustPython:main Dec 20, 2025
13 checks passed
@youknowone youknowone deleted the buffer-ndim branch December 20, 2025 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant