NetSync Wire Protocol V3 (Breaking Change Within V3)

“Physical block replaced with XROrigin Δ (3DoF: XZ translation + yaw)”

Goal

Reduce bandwidth while keeping correct reconstruction of the Physical pose on receivers by sending only:

• Head as absolute pose (virtual transform)
• Right/Left/Virtual as head-relative pose
• Physical as XROrigin delta (SE(2): XZ + yaw) instead of absolute physical transform

Receivers reconstruct Physical from HeadAbs + Δ.

---

1) Compatibility / Versioning Policy

• messageType IDs remain unchanged:

  • MSG_CLIENT_POSE_V2 = 11
  • MSG_ROOM_POSE_V2 = 12

• protocolVersion remains 3

• No backward compatibility

  • This is a breaking wire change even among V3.
  • To prevent silent corruption, we introduce a mandatory encoding flag bit:
    If PhysicalValid is set but the bit is missing → fail fast.

---

2) Flags and Semantics

2.1 PoseFlags

• PoseFlags.PhysicalValid is redefined:

  • Now means: “XROrigin Δ is present”
  • (In current Unity code, BuildPoseFlags() may always set PhysicalValid, meaning Δ will typically be sent every frame subject to send policy.)

• PoseFlags.IsStealth behavior remains:

  • All transform-valid bits must be 0
  • virtualCount = 0
  • Minimal payload only (no unnecessary transform data)

2.2 encodingFlags (V3)

Keep existing bits and add a new required bit:

Existing (from prior V3 draft):

• bit0: physicalYawOnly = 1 (kept; Physical is reconstructed as yaw-only)
• bit1: rightRelHead = 1
• bit2: leftRelHead = 1
• bit3: virtualRelHead = 1

New (mandatory when PhysicalValid=1):

• bit4: ENCODING_PHYSICAL_IS_XRORIGIN_DELTA = 1 << 4

Rules:

• ENCODING_FLAGS_DEFAULT MUST include bit4.

• Deserialization:

  • If PhysicalValid == 1 AND bit4 is not set → throw (Unity: InvalidDataException; Server: exception strongly recommended).

---

3) Quantization / Compression (V3)

3.1 Positions

• Head absolute position: i16 x3, 0.01 m per unit
• Right/Left/Virtual relative position (to head): i16 x3, 0.005 m per unit

3.2 Rotations

• Quaternion compression: “Smallest-three” packed into u32 (32-bit)
• Head rotation: absolute
• Right/Left/Virtual rotation: relative to head
  rotRel = compress( inv(headRotAbs) * partRotAbs )

3.3 Physical (NEW): XROrigin Δ (SE(2))

Replaces the old “physical absolute transform” block with:

• dx : i16 (meters, 0.01 m scale)
• dz : i16 (meters, 0.01 m scale)
• dyaw: i16 (degrees, 0.1 deg scale)

Quantization:

• Δ_POS_SCALE = 0.01 m
  dx_q = round(dx / 0.01) clamp to int16
  dz_q = round(dz / 0.01) clamp to int16
• Δ_YAW_SCALE = 0.1 deg (reuse existing yaw scale)
  dyaw_q = round(dyaw / 0.1) clamp to int16
  dyaw must be normalized via DeltaAngle to (-180..180] equivalent.

Note: Δ is XZ + yaw only. No Y translation is sent.

---

4) Wire Layout

4.1 Common Header

• messageType: unchanged (11/12)
• protocolVersion: 3

4.2 Client Pose Body (V3, raw-cached by server)

Field order is:

1. poseSeq — u16 — always
2. flags — u8 — always (PoseFlags)
3. encodingFlags — u8 — always (must include bit4 if PhysicalValid)

Physical block (when PhysicalValid)

4. dx — i16
5. dz — i16
6. dyaw — i16

Head (when HeadValid)

7. headPosAbs — i16 x3 — 0.01 m
8. headRotAbs — u32 — Smallest-three

Right (when RightValid)

9. rightPosRel — i16 x3 — (right - head), 0.005 m
10. rightRotRel — u32 — compress(inv(head)*right)

Left (when LeftValid)

11. leftPosRel — i16 x3 — (left - head), 0.005 m
12. leftRotRel — u32 — compress(inv(head)*left)

Virtual transforms

13. virtualCount — u8 — always (0..MAX_VIRTUAL_TRANSFORMS)
14. virtuals[i].posRel — i16 x3 — 0.005 m
15. virtuals[i].rotRel — u32 — compress(inv(head)*virtual)

4.3 Room Pose (server → clients)

Same envelope as today; only the client body is the V3 body above.

Per client concatenate:

• u16 clientNo + f64 poseTime + clientBodyV3

Server behavior:

• Continue raw body caching and rebroadcast to minimize re-serialization.

---

5) Definition of XROrigin Δ (SE(2)) — MUST MATCH ON ALL SIDES

This is not simply (p1 - p0). It must handle the rotation center correctly.

Let:

• Initial XROrigin (captured at Start / reference time):

  • p0 = (offsetPos.x, 0, offsetPos.z)
  • yaw0 = offsetYawY

• Current XROrigin (at send time):

  • p1 = (xrOrigin.pos.x, 0, xrOrigin.pos.z)
  • yaw1 = xrOrigin.eulerAngles.y

Compute:

• dyaw = DeltaAngle(yaw0, yaw1) (degrees)

• R = RotY(dyaw) (Quaternion yaw-only)

• translation component

  • t = p1 - (R * p0)
  • dx = t.x, dz = t.z

This ensures that for yaw-only rotations, translation becomes t = p0 - R*p0, i.e. rotation is applied around the correct center (snap-turn consistency).

---

6) Physical Reconstruction (Receiver Side)

Given:

• received dx,dz,dyaw
• received headPosAbs, headRotAbs (HeadValid must be true to reconstruct)

Let:

• t = (dx, 0, dz)
• R = RotY(dyaw)
• invR = RotY(-dyaw) (or Quaternion.Inverse(R))

6.1 Physical position

• physicalPos = invR * (headPosAbs - t)

6.2 Physical yaw (yaw-only)

• headYaw = Yaw(headRotAbs) (extract yaw from head quaternion)
• physicalYaw = NormalizeAngle(headYaw - dyaw)
• physicalRot = RotY(physicalYaw) (yaw-only quaternion)

If HeadValid is false in a frame, Physical should not be reconstructed and hands/virtual must be suppressed (see flag consistency).

---

7) Sending Policy (Traffic Reduction)

On Unity sender:

• Keep SendRate as maximum Hz cap

• Add:

  • Only-on-change (send only when pose signature changes)
  • Heartbeat at 1 Hz (send at least once per second even if unchanged)

This is independent of the wire format and is required to reduce idle traffic while preventing timeouts/disconnects.

---

8) Size Impact (Guidance)

• Physical block reduction (relative to prior “V3 physical-absolute” implementation that used 11 bytes):
  11 bytes → 6 bytes (−5 bytes)
• Example (as provided in JP spec): virtualCount=0 full body
  49 bytes → 44 bytes (raw body size)

(Exact totals depend on enabled flags and existing implementation details, but the physical delta block is strictly 6 bytes when present.)

---

9) Unity Implementation Plan (STYLY-NetSync-Unity)

9.1 Data Structure Changes

File

• Packages/com.styly.styly-netsync/Runtime/Internal Scripts/DataStructure.cs

Change
Add delta fields to ClientTransformData:

• public Vector3 xrOriginDeltaPosition; // only x,z used; y fixed 0
• public float xrOriginDeltaYaw; // degrees

Keep existing physical: TransformData as the reconstructed physical container on receiver side.

---

9.2 Compute Δ on Sender (SE(2) definition)

Files

• Runtime/NetSyncManager.cs
• Runtime/Internal Scripts/NetSyncAvatar.cs

Implementation

• In NetSyncManager.Start() you already store:

  • _physicalOffsetPosition (→ p0)
  • _physicalOffsetRotation (→ yaw0)

• At send time, read _XrOriginTransform for p1/yaw1, compute dx/dz/dyaw per Section 5.

Recommended:

• Add an internal helper on NetSyncManager:

  • internal void ComputeXrOriginDelta(out Vector3 tXZ, out float dyawDeg)

• In NetSyncAvatar.GetTransformData():

  • _tx.xrOriginDeltaPosition = tXZ;
  • _tx.xrOriginDeltaYaw = dyawDeg;

---

9.3 BinarySerializer.cs (Core Wire Change)

File

• Runtime/Internal Scripts/BinarySerializer.cs

A) Add new encoding flag bit

• private const byte ENCODING_PHYSICAL_IS_XRORIGIN_DELTA = 1 << 4;
• Ensure ENCODING_FLAGS_DEFAULT always includes it.

B) Add / confirm scales

• private const float LOCO_POS_SCALE = 0.01f; // 1 cm
• Reuse yaw scale 0.1 deg as currently used for physical yaw quantization.

C) Serialize Physical block as Δ

In SerializeClientTransformInto(...), when physicalValid:

• Write dx_q (short), dz_q (short), dyaw_q (short)

• Source:

  • data.xrOriginDeltaPosition.x
  • data.xrOriginDeltaPosition.z
  • data.xrOriginDeltaYaw

D) Pose signature hashing must match wire

In ComputePoseSignature(...):

• Replace old “physical absolute pos/yaw” contribution with:

  • quantized dx_q, dz_q, dyaw_q

• Quantization/clamp MUST be byte-for-byte identical to serializer logic.

E) Deserialize and reconstruct Physical

In DeserializeRoomTransform(...) (or equivalent):

• If physicalValid:

  • verify encoding bit4; if missing → InvalidDataException
  • read dx_q,dz_q,dyaw_q and dequantize to dx,dz,dyaw
  • you may need to temporarily store until HeadAbs is read

• After HeadAbs is available:

  • if headValid && physicalValid, reconstruct client.physical using Section 6.

---

9.4 TransformSyncManager size estimation

File

• Runtime/Internal Scripts/TransformSyncManager.cs

Update EstimateClientTransformSize() physical portion:

• Old: 11 bytes
• New: 6 bytes

---

9.5 HumanPresence correction must apply SE(2) consistently (Important)

File

• Runtime/NetSyncManager.cs (UpdateHumanPresenceTransform(...))

Problem:

• A simple worldPos += xrOriginPos - offsetPos does not rotate around the correct center.

Fix:

• Compute local t_local, dyaw_local using the same SE(2) definition as Section 5.

• Apply:

  • worldPos = R_local * worldPos + t_local
  • worldRotYaw = R_local * worldRotYaw (or equivalent yaw-only composition)

This keeps HumanPresence stable during snap turns and locomotion.

---

10) Server Implementation Plan (STYLY-NetSync-Server)

10.1 binary_serializer.py (Wire update + decode reconstruction)

File

• src/styly_netsync/binary_serializer.py

A) Add encoding bit and default

• ENCODING_PHYSICAL_IS_XRORIGIN_DELTA = 1 << 4
• Ensure included in ENCODING_FLAGS_DEFAULT

B) Serialize Physical block as Δ

In _serialize_client_body(...), when physical_valid:

• read delta values from the high-level input (e.g. keys like):

  • xrOriginDeltaX, xrOriginDeltaZ, xrOriginDeltaYaw (names may vary; standardize)

• quantize:

  • 1 cm for dx/dz, 0.1 deg for dyaw

• pack: struct.pack("<hhh", dx_q, dz_q, dyaw_q)
  (order MUST match Unity)

C) Deserialize Physical block as Δ and reconstruct physical

In _deserialize_client_body(...), when physical_valid:

• verify encoding bit4 present; else fail
• unpack dx_q, dz_q, dyaw_q
• store raw deltas in result for debugging/REST
• if head_valid, reconstruct physical transform dict using Section 6

---

11) Public Interface and API Stability

• Wire interface: still protocolVersion = 3 but breaking payload

• Message IDs remain 11/12

• High-level APIs should remain absolute:

  • Unity: NetSyncAvatar public interface unchanged
  • Python: send_transform() remains compatible at the API level
    Internally, the serializer uses Δ + head-relative coding.

---

12) Tests & Acceptance Criteria

12.1 Serializer roundtrip (V3-breaking)

• 10,000 randomized poses

• Encode → decode → reconstruct

• Pass criteria:

  • HeadAbs position error ≤ 0.01 m
  • Rel (hands/virtual) position error ≤ 0.005 m
  • Rotation error ≤ 1.0° (or agreed threshold)
  • No NaN / zero quaternions from smallest-three codec

12.2 Δ correctness (SE(2) invariants)

• Pure yaw rotation around offset center:

  • Verify translation t = p0 - R*p0 behavior (no drift)

• Pure translation:

  • dyaw=0 and t = p1 - p0

12.3 Mixed-client protection

• If PhysicalValid=1 but encoding bit4 missing:

  • Unity: throws InvalidDataException
  • Server: rejects / errors (fail fast)

12.4 Relay integrity

• client → server raw-cache → broadcast → client
• poseSeq, flags, and reconstructed pose match expected within quantization bounds

12.5 Traffic behavior

• Idle 30s: ~1 Hz heartbeat
• Motion: up to SendRate cap, only-on-change works (no visible stutter)

12.6 Bandwidth benchmark

• 10 clients, 10 Hz, virtualCount=0
• Room bytes/sec reduced by ≥55% vs baseline (or update target if baseline is the previous V3)