Conversation
…nt transforms - Changed protocol version from 2 to 3 in BinarySerializer.cs. - Implemented new encoding for client transforms, including physical, head, right, left, and virtual transforms. - Added quantization and compression for quaternion data to reduce payload size. - Introduced encoding flags to manage the validity of different transform types. - Enhanced test coverage for transform serialization, ensuring fidelity and error bounds. - Updated TransformSyncManager to optimize sending logic based on pose signature changes. - Adjusted NetSyncAvatar to correctly set pose flags based on available transforms.
…on in documentation
There was a problem hiding this comment.
Pull request overview
This pull request implements protocol version 3 for transform synchronization, introducing significant bandwidth reduction through quantization and compression. The changes affect both Unity (C#) and Python implementations, removing backward compatibility with protocol v2 and JSON fallback.
Changes:
- Updated protocol version from 2 to 3 with no backward compatibility
- Implemented int16 quantization for positions (0.01m absolute, 0.005m relative) and yaw (0.1°)
- Added 32-bit smallest-three quaternion compression for rotations
- Switched to head-relative encoding for right/left hand and virtual transforms
- Enhanced flag consistency: Right/Left/Virtual transforms now require HeadValid
- Removed JSON fallback in server for protocol v3-only handling
- Added comprehensive test coverage with 10,000+ randomized roundtrip tests
- Implemented only-on-change sending with 1Hz heartbeat for idle scenarios
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| STYLY-NetSync-Server/src/styly_netsync/binary_serializer.py | Core v3 serialization with quantization, quaternion compression, and head-relative transforms |
| STYLY-NetSync-Server/tests/test_binary_serializer.py | Comprehensive test suite with 10K+ random poses, boundary tests, and error bound verification |
| STYLY-NetSync-Server/src/styly_netsync/server.py | Removed JSON fallback, fixed type annotation for pose_time field |
| STYLY-NetSync-Unity/.../NetSyncAvatar.cs | Enforces HeadValid requirement for Right/Left/Virtual transforms at flag building |
| STYLY-NetSync-Unity/.../BinarySerializer.cs | Unity C# implementation matching Python v3 serialization (not in diff, verified via search) |
| STYLY-NetSync-Server/benchmark/.../styly_client.py | Added one-shot payload size comparison logging |
| Multiple CLAUDE.md and README.md files | Updated documentation with v3 protocol details and compatibility notes |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…and validation flags
|
@claude Review this PR. |
This comment was marked as outdated.
This comment was marked as outdated.
… handling in serialization
…ons and enhance quantization handling
|
@claude Review the latest code of this PR. |
|
Claude finished @from2001's task in 2m 4s —— View job Code Review: Protocol v3 Transform SerializationI've completed a comprehensive review of this PR. This is an excellent implementation of protocol v3 with significant improvements to bandwidth efficiency and code quality. Here are my findings: ✅ Strengths1. Excellent Compression Implementation
2. Clean Architecture
3. Proper Flag Handling
4. Documentation
🔍 Issues & SuggestionsCritical IssuesNone found - The implementation is solid. Minor Issues & Suggestions1. Type Annotation Inconsistency (Python)Location: The # Current: implicitly expects float
buffer.extend(struct.pack("<d", client.get("poseTime", 0.0)))
# Suggestion: Add explicit type comment or ensure consistency
pose_time: float = client.get("poseTime", 0.0)
buffer.extend(struct.pack("<d", pose_time))This is a very minor style issue and doesn't affect functionality. 2. Quaternion Decompression Warning Threshold (Unity)Location: The decompression warning triggers when // Current:
if (sumSquares > 1f)
{
Debug.LogWarning($"...");
}
// Suggestion:
const float EPSILON = 1e-6f;
if (sumSquares > 1f + EPSILON)
{
Debug.LogWarning($"...");
}This prevents spurious warnings from normal float precision limits. 3. Virtual Transform Count Validation (Both)Location: Both implementations handle # Python: binary_serializer.py:912-916
if not virtual_valid and virtual_count > 0:
logger.warning(
"Virtual count %d but VirtualsValid flag unset - malformed payload",
virtual_count,
)Suggestion: Rate-limit these warnings or use 4. Physical Yaw Extraction DocumentationLocation: The yaw extraction assumes Y-up right-handed coordinates (Unity standard). Consider adding a comment about coordinate system assumptions for future maintainers: def _quaternion_to_yaw_degrees(qx: float, qy: float, qz: float, qw: float) -> float:
"""Extract yaw in degrees from quaternion.
Assumes Y-up right-handed coordinate system (Unity standard).
Yaw is rotation around the Y axis.
"""5. Test Coverage for Edge CasesLocation: The tests are excellent, but consider adding:
Example: def test_denormalized_quaternion_handling(self) -> None:
"""Verify that non-unit quaternions are normalized before compression."""
# Non-unit quaternion (magnitude != 1)
q_unnormalized = (0.5, 0.5, 0.5, 0.5) # magnitude = 1.0, actually unit!
q_unnormalized = (1.0, 1.0, 1.0, 1.0) # magnitude = 2.0
packed = binary_serializer._compress_quaternion_smallest_three(*q_unnormalized)
result = binary_serializer._decompress_quaternion_smallest_three(packed)
# Should be normalized (magnitude = 1)
mag = math.sqrt(sum(x*x for x in result))
assert abs(mag - 1.0) < 1e-66. Server Type Annotation FixLocation: Server diff shows type annotation update for The server.py change from: tuple[str, list[tuple[int, dict[str, Any] | None, bytes]]]
# to
tuple[str, list[tuple[int, float, dict[str, Any] | None, bytes]]]This is correct and fixes a type annotation bug. Good catch! 🎯 Performance ConsiderationsPositive
Potential Optimization (Future)
📋 Checklist
🚀 RecommendationAPPROVE - This is high-quality work that significantly improves the networking protocol. The minor suggestions above are optional refinements and don't block merging. The implementation demonstrates:
Great job! 🎉 Suggested Pre-Merge Actions:
Post-Merge:
|
…normalization and yaw extraction
…g and related computations
|
@claude Review the latest code. |
|
Claude finished @from2001's task in 1m 48s —— View job Code Review: Protocol v3 Transform Serialization (Latest Code)I've completed a comprehensive review of the latest code on this PR. The implementation is excellent with strong architecture, thorough testing, and proper optimization. Here are my findings: Todo List
OverviewThis PR implements protocol version 3 with significant improvements to transform serialization:
✅ Strengths1. Excellent Compression & Encoding
2. Robust TestingThe test suite is comprehensive (
3. Clean Architecture
4. Proper Flag Handling
5. Documentation
🔍 Issues & RecommendationsCritical IssuesNone found - The implementation is production-ready. Minor Issues & Suggestions1. Quaternion Decompression Warning ThresholdLocation: The warning triggers when // Current:
if (sumSquares > 1f + 1e-6f)
{
Debug.LogWarning($"...");
}
// Suggestion: Use slightly larger epsilon to match Python (1e-6f vs 1e-6)
const float EPSILON = 1e-6f;
if (sumSquares > 1f + EPSILON)
{
Debug.LogWarning($"...");
}Note: Python already uses 2. Virtual Transform Count Validation LoggingLocation: Both implementations log warnings when virtual count is set but the flag is unset. In a malformed-client scenario, this could spam logs: # Python: binary_serializer.py:968-972
if not virtual_valid and virtual_count > 0:
logger.warning(
"Virtual count %d but VirtualsValid flag unset - malformed payload",
virtual_count,
)Suggestion: Consider rate-limiting these warnings or adding client context (e.g., device ID) to help identify misbehaving clients: logger.warning(
"[%s] Virtual count %d but VirtualsValid flag unset - malformed payload",
client.get("deviceId", "unknown"),
virtual_count,
)3. Physical Reconstruction Warning ContextLocation: The warning when physical delta is received without head pose could benefit from client context: # Current:
logger.warning(
"Physical delta received without head pose; physical reconstruction skipped"
)
# Suggestion:
logger.warning(
"[%s] Physical delta received without head pose; physical reconstruction skipped",
client.get("deviceId", "unknown"),
)This helps identify which client is sending malformed data. 4. Test Coverage Enhancement (Optional)Location: The test suite is already excellent, but consider adding these edge cases for completeness:
Example: def test_denormalized_quaternion_normalized_before_compression(self) -> None:
"""Verify that non-unit quaternions are normalized before compression."""
# Non-unit quaternion (magnitude = 2.0)
q_unnormalized = (1.0, 1.0, 1.0, 1.0)
packed = binary_serializer._compress_quaternion_smallest_three(*q_unnormalized)
result = binary_serializer._decompress_quaternion_smallest_three(packed)
# Should be normalized (magnitude ≈ 1)
mag = math.sqrt(sum(x*x for x in result))
assert abs(mag - 1.0) < 1e-65. Type Annotation Precision (Python)Location: The # Current (implicit float):
buffer.extend(struct.pack("<d", client.get("poseTime", 0.0)))
# Suggestion (explicit type for clarity):
pose_time: float = client.get("poseTime", 0.0)
buffer.extend(struct.pack("<d", pose_time))This is a very minor style issue and doesn't affect functionality. 🎯 Performance AnalysisPositive
Measured Characteristics
📋 Architecture ReviewCorrectness
Code Quality
Protocol Design
🚀 RecommendationAPPROVE - This is high-quality production-ready code. The minor suggestions above are optional refinements that don't block merging. The implementation demonstrates:
Great work! The protocol v3 implementation significantly improves bandwidth efficiency while maintaining excellent fidelity for XR tracking data. Suggested ActionsPre-Merge:
Post-Merge:
|
Close #329