Looks like XRNode3D and XRCamera3D are returning the wrong non-updated transforms when you try to access it from physics process with this approach, will add a fix soon
Fixed

Firstly I've noticed you've used get_global_transform_interpolated() for XROrigin, XRCamera and XRNode. I have very little familiarity with XR and I may have misunderstood your description of XROrigin, I had somehow got the impression earlier that there was only one that was always on, but the docs say there may be multiple.

For Cameras and likely XRNodes (I'm guessing like hand markers?) you may need to have better control over the prev/curr xforms and may end up needing to handle this data manually as in Camera3D. I suspect now this may also be the case in XROrigin.

To be clear, using get_global_transform_interpolated() is easier to use and easy to get going with, but there are subtle issues to do with the need for priming of the data stream before use in order to prevent glitches when turning nodes on and off (e.g. Cameras), so you may end up having to go with the longhand approach.

If we end up needing several nodes with full client interpolation, it's possible we can "genericize" some of the boiler plate to make it easier and less bug prone to add to nodes.

Combining interpolated and non-interpolated on same node

My first impressions are the approach of trying to have XRCamera and XRNode both interpolated AND non-interpolated seems a bit hacky - there's no other node that does this. It is likely to introduce physics interpolation warnings and send incorrect / superfluous data to VisualServer.

Nodes are currently intended to be either physics interpolated (operating on the physics tick) or non-interpolated (operating on the frame) but not both. You are essentially in uncharted waters here, other parts of Godot may not work correctly.

I suspect this needs a proper discussion of what problems are trying to be solved to be sure that this is the correct avenue to solve them, probably via a proposal.

I must apologize as you are on a difficult boundary here - effectively I'm the goto guy for interpolation, but I don't have a clue how the current XR stuff works, and I don't have XR equipment, so you have to explain it like to a newbie and explain what problems you are trying to solve.

Essentially the current paradigm for 3D interpolation in the engine is this in a nutshell:

• Root node and branches leading from it are interpolated (operating on physics ticks)
• Some specific edge case nodes towards the end of branches (such as Cameras) can be non-interpolated, and do a client side (scene tree) manual interpolation of their parent node or target in order to achieve consistent look with the interpolated scene (there will be one tick of lag), but also allowing some zero lag control (e.g. mouse look)
• Branches that are non-interpolated cannot then be later set back to interpolated further on in the branch, as the tick based xforms are no longer there (or rather they are, but 1 tick behind)
• Any interpolated node that you use to follow another independent non-interpolated target, is going to introduce a one tick lag

Now, there may be some specific leeway to move outside this if a specific use case requires it, but there isn't existing code to work outside this framework.

How XR is meant to work (at a guess)

At a guess for how things are meant to operate with XR:

• You want these XR nodes to take their xform from the hardware and reflect that in the nodes in as near realtime as possible, so you don't get lag
• You want to get some of the benefits of physics interpolation where possible

If this is correct:

• You want a main branch of the scene tree that is interpolated
• A branch with the XR origin and an XRCamera, that is used to view the interpolated scene
• You may want some XR objects totally defined by their XR xform and non interpolated (say a gun)
• You may want an arm to line up to this XR node via e.g. IK (we have a similar issue open for IK)
• You may want to perform some physics tests on an object that approximates the xform of the XR node

This is just trying to explain some of how things currently work interpolation-wise, but I think I would need some further description / research into how XR currently works in order to come up with some coherent design here.

Firstly I've noticed you've used get_global_transform_interpolated() for XROrigin, XRCamera and XRNode. I have very little familiarity with XR and I may have misunderstood your description of XROrigin, I had somehow got the impression earlier that there was only one that was always on, but the docs say there may be multiple.

There can be several of them but only one will be active at a time: https://docs.godotengine.org/en/stable/classes/class_xrorigin3d.html#class-xrorigin3d-property-current

For Cameras and likely XRNodes (I'm guessing like hand markers?) you may need to have better control over the prev/curr xforms and may end up needing to handle this data manually as in Camera3D. I suspect now this may also be the case in XROrigin.

To be clear, using get_global_transform_interpolated() is easier to use and easy to get going with, but there are subtle issues to do with the need for priming of the data stream before use in order to prevent glitches when turning nodes on and off (e.g. Cameras), so you may end up having to go with the longhand approach.

I went with this approach due to this comment: #103232 (comment)

My first impressions are the approach of trying to have XRCamera and XRNode both interpolated AND non-interpolated seems a bit hacky - there's no other node that does this. It is likely to introduce physics interpolation warnings and send incorrect / superfluous data to VisualServer.

Nodes are currently intended to be either physics interpolated (operating on the physics tick) or non-interpolated (operating on the frame) but not both. You are essentially in uncharted waters here, other parts of Godot may not work correctly.

I suspect this needs a proper discussion of what problems are trying to be solved to be sure that this is the correct avenue to solve them, probably via a proposal.

This approach comes from this comment: #103232 (comment)
The local position of XRCamera3D and XRNode3D isn't interpolated and it's polled every render frame, however we need them to "follow" the interpolated parent, the reason was explained here: #103232 (comment)
If we just disable interpolation entirely for XRNode3D the hands will just jitter as soon as you move the parent, if you do the intended approach and only update the hands on physics frame and interpolate, it's practically unusable due to the latency.

• You want these XR nodes to take their xform from the hardware and reflect that in the nodes in as near realtime as possible, so you don't get lag
• You want to get some of the benefits of physics interpolation where possible

This is correct, the benefit from physics interpolation being: interpolate the global transform so it follows the parent correctly while not interpolating the local transform so the tracked movement is in realtime.

This current approach works fine because it just acts as any other non interpolated node, but setting its global transform to "follow" the interpolated transform of the parent. This shouldn't have any side effect as it's the same as changing the transform of an non-interpolated node inside _process. All childs of these nodes will also act as any non-interpolated node following its parent on every render frame.

The only issue is the transform being updated on physics frame by one of the interpolated ancestors changing its own transform, but this is a separate issue that happens outside of XR as you already know: #103351

I think there are some language issues between us (I admit it's quite difficult to put across in words so apologies for any confusion 😁 ), I wasn't clear on the definitions I was using, and now I can see it was ambiguous, so to be more clear:

When I mention "interpolated / non-interpolated" I was trying to refer to the physics interpolation mode of the node.

Interpolated

By saying a node that is interpolated I was meaning interpolation_mode ON / inherited ON.
Nodes with interpolation_mode ON should be moved and have transform set on the TICK.

Non-interpolated

A node that is non-interpolated is a node that is interpolation_mode OFF / inherited OFF.
Nodes with interpolation_mode OFF should be moved and have transform set on the FRAME.

An interpolation_mode OFF node can therefore use e.g. get_global_transform_interpolated() on the frame to follow / derive an interpolated transform from another node, such as a parent, or a target node (if following a target).

From now on I'll try and refer to the physics_interpolation_mode to make this clearer.

get_global_transform_interpolated()

In terms of the pros and cons of this I highly recommend reading the c++ functions in Node3d:
_get_global_transform_interpolated()
update_client_physics_interpolation_data() which includes timeouts (and the reason for the need for priming)

I think there are some language issues between us (I admit it's quite difficult to put across in words so apologies for any confusion 😁 ), I wasn't clear on the definitions I was using, and now I can see it was ambiguous, so to be more clear:

When I mention "interpolated / non-interpolated" I was trying to refer to the physics interpolation mode of the node.

Interpolated

By saying a node that is interpolated I was meaning interpolation_mode ON / inherited ON. Nodes with interpolation_mode ON should be moved and have transform set on the TICK.

Non-interpolated

A node that is non-interpolated is a node that is interpolation_mode OFF / inherited OFF. Nodes with interpolation_mode OFF should be moved and have transform set on the FRAME.

An interpolation_mode OFF node can therefore use e.g. get_global_transform_interpolated() on the frame to follow / derive an interpolated transform from another node, such as a parent, or a target node (if following a target).

From now on I'll try and refer to the physics_interpolation_mode to make this clearer.

get_global_transform_interpolated()

In terms of the pros and cons of this I highly recommend reading the c++ functions in Node3d: _get_global_transform_interpolated() update_client_physics_interpolation_data() which includes timeouts (and the reason for the need for priming)

I misunderstood you, yeah, sorry 😅
I didn't know you were talking about me changing the transform inside both physics and render frame (or tick and frame as you referred here), I replied to that here: #103233 (comment)

Been discussing and learning a bit more how XR works with @BastiaanOlij on rocket chat (where the data comes in before the ticks etc). I'm kind of coming around to your idea of setting the transform on the ticks as a bodge for XR as being a possibility (best of a difficult situation) but would be good to discuss on a follow up PR.

In particular it might even be possible to do a light version of set_transform() that doesn't invoke the heavy machinery (propagation etc) but gets the benefit required. Not sure yet.

Been discussing and learning a bit more how XR works with @BastiaanOlij on rocket chat (where the data comes in before the ticks etc). I'm kind of coming around to your idea of setting the transform on the ticks as a bodge for XR as being a possibility (best of a difficult situation) but would be good to discuss on a follow up PR.

In particular it might even be possible to do a light version of set_transform() that doesn't invoke the heavy machinery (propagation etc) but gets the benefit required. Not sure yet.

I've just read the conversation, I want to point out that the reason why I'm doing it also on the camera is because childs have the same problem as in the hands when following it, for example a raycast.

About possible alternatives to setting the transform at physics process I'm all ears 🫡
I will investigate too and I can test possible solutions since you don't have XR equipment, feel free to let me know when needed.

About a followup PR, you want me to separate this into 2 PRs one for the world_origin and one for the nodes, or you mean a new PR with possible alternatives like making a light set_transform

About a followup PR, you want me to separate this into 2 PRs one for the world_origin and one for the nodes, or you mean a new PR with possible alternatives like making a light set_transform

Essentially only setting the transform on the frame for this PR. I understand that this means some physics aspects may not work correctly with physics interpolation.

That's fine, a PR doesn't have to fix all outstanding bugs. The engine is (as I understand it) broken already with physics interpolation, so we are just making it slightly less broken.

Essentially only setting the transform on the frame for this PR. I understand that this means some physics aspects may not work correctly with physics interpolation.

That's fine, a PR doesn't have to fix all outstanding bugs. The engine is (as I understand it) broken already with physics interpolation, so we are just making it slightly less broken.

Alright that makes sense, with this PR even if we remove the transform change on the physics frame it should be easy to workaround if needed from GDScript, although it should work out of the box
I will remove it and update the PR

I've updated the PR to remove what was requested.
As a workaround, this works fine:

extends Node3D

@onready var controller: XRController3D = $".."

var origin: XROrigin3D

func _ready() -> void:
	set_physics_interpolation_mode(Node.PHYSICS_INTERPOLATION_MODE_ON)
	
	if controller.get_parent() is XROrigin3D:
		origin = controller.get_parent() as XROrigin3D

	# set as top level to ignore parent transform!!!
	top_level = true
	pass

func _physics_process(delta: float) -> void:
	if !origin || !controller || !controller.get_pose():
		return
	
	global_transform = origin.global_transform * controller.get_pose().transform
	pass

This can be added to a node child of the XRNode3D (or camera) we want to follow, in this case a controller (XRController3D).
This node will be interpolated and only update its transform on physics process as intended, all childs of it will follow just fine.
Here is a demo of it working, left is without the workaround, you can see how it easily gets behind. Right is with the workaround, following the hand perfectly.
It's running at 10 ticks/s that's why the physics are a bit choppy but it's easier to see the behavior this way.

This looks fine to me although will need a second pass by an XR guy, and maybe a check for style.

Just to throw a spanner in the works (sorry! 😁 ), partly as a result of this XR issue and IK and ragdoll, I've revisited the overall 3D physics interpolation design.

After doing some testbed work today and speaking with other maintainers, we've decided to evaluate changing the 3D physics interpolation paradigm to scene side rather than server side.

If this works out ok, it will make this XR code much easier to do and simpler for XR maintainers to understand (and will need to be remodified), but I don't think there's a major problem with this version being available in the interim. (Alternatively you guys may be happy with the workaround, but this PR is fine too.)

The current provisional plan is to try out new version in 3.x then if successful, look at porting to 4.x.

Interpolation on the scene side makes more sense to me, and will indeed be easier to mantain without having to make workarounds on some parts like we're doing here.

Thank you for the review 🤝

I will wait for @BastiaanOlij to review so I can make a final commit and don't trigger CI/CD for a comment change

Sorry it took me awhile to get to this.

I think this is as good as we'll get it, it solves the core problem and the only question mark I have is related to a scenario that wouldn't work well with interpolation anyway (using physics following hand tracking).

It is also important to note that the approach used in XR Tools, and described in the OpenXR origin centric movement demo will not play nice with interpolation regardless of what we do.

Users must use the OpenXR character centric movement approach, which is superior anyway.

We may wish to add a remark around this to the room scale page in the manual that details the two approaches.

Thank you for the review 🤝
As I mentioned in the review comment, it is actually possible to have physics based hands with interpolation enabled using this workaround: #103233 (comment)
I'm using this on a project running at 50 physics fps and it works as good as without interpolation 😀

Looks like it needs a rebase.

Also reminder to anyone that can to review #103685, as the sooner we get that merged in 3.x the sooner we can get it in 4.x and it should make the XR stuff a lot easier and perhaps open up opportunities to do things with less lag.

Rebased and fixed the comments, should be ready to merge

You'll need to squash the commits, a single commit is preferred. See PR workflow docs for how-to.

You'll need to squash the commits, a single commit is preferred. See PR workflow docs for how-to.

Fixed.
I did 2 commits on purpose because I thought it would be more clear the 2 changes being done here, will do just 1 commit next time 🫡

Thanks! And congrats for your first merged Godot contribution 🎉

Cherry-picked for 4.4.1.