Update integration to support VTK 9.1 by jcfr · Pull Request #90 · KitwareMedical/SlicerVirtualReality

jcfr · 2022-02-26T10:40:30Z

In addition of the API changes documented below, this commit also updates
vtkVirtualRealityViewInteractor to allow retrieving the last tracked device
position stored in the array vtkOpenVRRenderWindowInteractor::Trackers.

The Trackers array contains TrackerActions each being a struct with
two entries:

Source of type vr::VRInputValueHandle_t
LastPose of type vr::TrackedDevicePose_t

Obsolete branches

branch	Notes
cpinter@slicervr-vtkrenderingvr-build	Remove, change integrated in this PR (#90)
cpinter@fix-build-against-vtk-9.1.20211022	Remove, change integrated in this PR (#90)
cpinter@gui-widgets-module	Remove, change integrated in this new branch: jcfr@fix-build-against-vtk-9.1.20211022-with-gui-widgets-module

Remaining issues

(1) Retrieval of last position associated with generic tracker

Retrieval of last position associated with generic tracker in
vtkVirtualRealityViewInteractor::GetTrackedDevicePose given
the index is not possible.

⚠️ TODO: Handle retrieval of tracker position associated with generic tracker associated with index.

Background:

Add support for generic tracker originally contributed in these commits:

Kitware/VTK@6083532cf8 (ENH: Exposing the generic tracker openvr device type to downstream projects)
59b7d1bf3 (Adding interface and logic to maintain MRML transforms for each generic VR tracker)

but removed following the refactoring done in these commits:

Kitware/VTK@1aac3fe5f (Create VRInteractorStyle and VRRenderWindowInteractor)
Kitware/VTK@af0fab486 (Clean up VR and OpenVR classes)
Kitware/VTK@9bd64d666 (Cleanup and rework of the VRInteractorStyle and subclasses)
Kitware/VTK@09a478a22 (Cleanup VR camera code and subclasses)

(2) RecognizeComplexGesture is not virtual anymore in base VTK class

⚠️ TODO: Assess with this function should still be overriden with VTK 9.1

Background: The function became non virtual in Kitware/VTK@af0fab486 (Clean up VR and OpenVR classes)

API changes

vtkEventData API changes:

vtkEventDataButton3D -> vtkEventDataDevice3D

RenderWindow API changes

GetNumberOfTrackedDevicesForDevice -> GetNumberOfDeviceHandlesForDevice
GetTrackedDeviceIndexForDevice -> GetDeviceHandleForDevice
GetTrackedDeviceModel -> GetModelForDevice
GetTrackedDeviceIndexForDevice -> GetDeviceHandleForDevice

InteractorStyle API changes

PositionProp sinature updated to include "lwpos" and "lwori"

…_INFO Co-authored-by: Csaba Pinter <pinter.csaba@gmail.com>

The check converts the usage of null pointer constants (eg. NULL, 0) to use the new C++11 nullptr keyword. (1) Compile project using Ninja specifying -DCMAKE_EXPORT_COMPILE_COMMANDS:BOOL (2) Run run-clang-tidy specifying path to inner-build BUILD_DIR=/home/jcfr/Projects/SlicerVirtualReality-Release/inner-build run-clang-tidy-10 -p ${BUILD_DIR} -extra-arg=-D__clang__ -checks=-*,modernize-use-nullptr -header-filter=.* -fix Adapted from Slicer/Slicer@526643353 and https://www.slicer.org/wiki/Documentation/Nightly/Developers/Tutorials/MigrationGuide#C.2B.2B11:_Update_source_code_to_use_nullptr Co-authored-by: Csaba Pinter <pinter.csaba@gmail.com>

List of VTKExternalModule changes: $ git shortlog 3bae71e5e..50f1c5be9 --no-merges Jean-Christophe Fillion-Robin (3): Add support for specifying path to vtk.kit file vtkmodule-config: Skip include of properties if python wrappers are disabled Add support for externally building a VTK modules and its dependencies Co-authored-by: Csaba Pinter <pinter.csaba@gmail.com> Co-authored-by: Rafael Moreta <rmoretamartinez@gmail.com>

…rStyle Based of Slicer/Slicer@364ff7b6d (ENH: Use common interactor style for slice and 3D views) Co-authored-by: Jean-Christophe Fillion-Robin <jchris.fillionr@kitware.com>

This change is done anticipating the transition to the updated API associated with VTK >= 9.1

In addition of the API changes documented below, this commit also updates "vtkVirtualRealityViewInteractor" to allow retrieving the last tracked device position stored in the array "vtkOpenVRRenderWindowInteractor::Trackers". The "Trackers" array contains "TrackerActions" each being a struct with two entries: - Source of type "vr::VRInputValueHandle_t" - LastPose of type "vr::TrackedDevicePose_t" Remaining issues ---------------- ### (1) Retrieval of last position associated with generic tracker Retrieval of last position associated with generic tracker in "vtkVirtualRealityViewInteractor::GetTrackedDevicePose" given the index is not possible. TODO: Handle retrieval of tracker position associated with generic tracker associated with index. Background: Add support for generic tracker originally contributed in these commits: * Kitware/VTK@6083532cf8 (ENH: Exposing the generic tracker openvr device type to downstream projects) * KitwareMedical/SlicerVirtualReality@59b7d1bf3 (Adding interface and logic to maintain MRML transforms for each generic VR tracker) but removed following the refactoring done in these commits: * Kitware/VTK@1aac3fe5f (Create VRInteractorStyle and VRRenderWindowInteractor) * Kitware/VTK@af0fab486 (Clean up VR and OpenVR classes) * Kitware/VTK@9bd64d666 (Cleanup and rework of the VRInteractorStyle and subclasses) * Kitware/VTK@09a478a22 (Cleanup VR camera code and subclasses) ### (2) RecognizeComplexGesture is not virtual anymore in base VTK class TODO: Assess with this function should still be overriden with VTK 9.1 Background: The function became non virtual in Kitware/VTK@af0fab486 (Clean up VR and OpenVR classes) API changes ----------- vtkEventData API changes: * vtkEventDataButton3D -> vtkEventDataDevice3D RenderWindow API changes * GetNumberOfTrackedDevicesForDevice -> GetNumberOfDeviceHandlesForDevice * GetTrackedDeviceIndexForDevice -> GetDeviceHandleForDevice * GetTrackedDeviceModel -> GetModelForDevice * GetTrackedDeviceIndexForDevice -> GetDeviceHandleForDevice InteractorStyle API changes * PositionProp sinature updated to include "lwpos" and "lwori"

jcfr · 2022-02-26T11:57:06Z

@cpinter @adamrankin

Would be great if you could review the issues I outlined above.
- Retrieval of last position associated with generic tracker
- RecognizeComplexGesture is not virtual anymore in base VTK class
Once, this PR is integrated, we can then move forward with the rebased branch adding the gui widget support.
- See jcfr@fix-build-against-vtk-9.1.20211022-with-gui-widgets-module
- Note that while it has been rebased (along with few fixes like removing extra space, ordering includes, ...), and despite of few warnings it is building.

cpinter · 2022-03-01T15:36:53Z

I can confirm that the branch jcfr/SlicerVirtualReality@fix-build-against-vtk-9.1.20211022...fix-build-against-vtk-9.1.20211022-with-gui-widgets-module builds. I'll test it tomorrow in the office where I have access to VR hardware.

cpinter · 2022-03-04T14:20:06Z

I finally could test it with a Slicer build from March 1st. There are a few issues, but it's a bit tricky to know what is because of what.

Scenario 1: HP Reverb 2 on Windows 11 (Windows updated from 10 without asking)

Virtual reality rendering works, as well as moving objects and two-handed navigation (i.e. pinch 3D)
Volume rendering does not work, meaning that it shows up in 2D but not in 3D. However (!) it does not work either in my previous build that we used (Jan 31) it successfully before the machine updated to Win11, so it seems to be due to this update.
Flying does not work

Scenario 2: HTC Vive Pro on Windows 10

Virtual reality rendering DOES NOT WORK. It shows up for around half a second, then an error message pops up, see below
On this configuration with the Jan 31 build, virtual reality works
- Also, flying works, as well as volume rendering

cpinter · 2022-03-11T09:43:42Z

@jcfr I can work a bit on this today. I'll check out the flying issue, and see if there is anything else to fix before integration (volume rendering works, it was simply a driver issue - see here).
Update: volume rendering does NOT show up in VR, we'll need to fix that.

Please let me know if I should check out anything specific.

jcfr · 2022-03-11T12:37:49Z

re: action manifest error

I suggest you try copying the relevant file in the current working directory.

Looking at https://github.com/Slicer/VTK/blob/slicer-v9.1.20220125-efbe2afc2/Rendering/OpenVR/vtkOpenVRRenderWindowInteractor.cxx#L34, we can see that the filename is looked using ./vtk_openvr_actions.json

The files are the following:

vtk_openvr_actions.json
vtk_openvr_binding_hpmotioncontroller.json
vtk_openvr_binding_knuckles.json
vtk_openvr_binding_oculus_touch.json
vtk_openvr_binding_vive_controller.json

and should be copied in the current working directory.

Source: https://github.com/Slicer/VTK/tree/slicer-v9.1.20220125-efbe2afc2/Rendering/OpenVR

lassoan · 2022-03-22T17:47:44Z

@cpinter We are working on fixing all extension builds for Slicer5. There are now only two extensions - SlicerVirtualReality and SlicerRadiomics - that must be fixed before the new release: https://docs.google.com/spreadsheets/d/1GC4DWDpOXhuDYdfYOjJ6PmjHTeMM3-K7SoiC5ZR1GYg/edit?usp=sharing

It would be great if you could finalize this pull request so that we can start the Slicer5 release process.

cpinter · 2022-03-22T19:42:38Z

This is on my list every single day, but I have a very hard time finding time to work on this these weeks. I still haven't been able to reinstall the computer that I use for VR. Hopefully tomorrow I can do that, but to spend even two hours fixing real issues is quite unlikely over the next two weeks or so.

I'd say that we integrate this to fix the build, and then fix the remaining issues afterwards. I cannot say with certainty that there are actual issues with the extension that haven't been there before, because my VR computer acted up as well as @dgmato 's, and nobody else seems to have tested this. Having the extension build successfully would at least facilitate this.

lassoan · 2022-03-22T19:46:56Z

OK, thanks, I'll merge the changes then.

cpinter · 2022-03-22T20:09:51Z

Great, let's see the dashboard tomorrow, it should be green and I'll at least try to test it with the preview after reinstall.

Sorry I haven't been more help with this, but sometimes projects pile up.

jcfr · 2022-03-22T20:58:11Z

I copied the summary of remaining issues related to the integration with VTK 9.1 in #91

These would still need to be addressed.

jcfr and others added 6 commits February 26, 2022 03:27

COMP: Fix -Wunused-parameter in qMRMLVirtualRealityView::onButton3DEvent

de25d0f

STYLE: qMRMLVirtualRealityView: Consistently log message using Q_FUNC…

05302e0

…_INFO Co-authored-by: Csaba Pinter <pinter.csaba@gmail.com>

DOC: Improve vtkMRMLVirtualRealityViewNode Magnification docstring

3d63fab

COMP: Remove support for building against Slicer < 4.11

bc1d663

jcfr mentioned this pull request Feb 26, 2022

COMP: Attempt to fix build including vtkRenderingVR WIP #89

Closed

cpinter and others added 3 commits February 26, 2022 06:04

ENH: Convert the interactor style to use the new vtkMRMLViewInteracto…

51d1d1f

…rStyle Based of Slicer/Slicer@364ff7b6d (ENH: Use common interactor style for slice and 3D views) Co-authored-by: Jean-Christophe Fillion-Robin <jchris.fillionr@kitware.com>

STYLE: Update updateTransformNodeWithPose to accept post as a pointer

2da7ad9

This change is done anticipating the transition to the updated API associated with VTK >= 9.1

jcfr force-pushed the fix-build-against-vtk-9.1.20211022 branch from 7fab306 to 83beb31 Compare February 26, 2022 11:26

jcfr mentioned this pull request Feb 28, 2022

Get OpenVR working with latest Slicer Slicer/Slicer#6214

Closed

jcfr mentioned this pull request Mar 2, 2022

Update extensions for Slicer-4.13 Slicer/Slicer#5269

Closed

20 tasks

lassoan merged commit 1c36e40 into KitwareMedical:master Mar 22, 2022

jcfr mentioned this pull request Mar 22, 2022

Address issues related to integration with VTK 9.1 #91

Closed

jcfr deleted the fix-build-against-vtk-9.1.20211022 branch March 22, 2022 20:57

jcfr mentioned this pull request Oct 13, 2022

Add support for OpenXR & OpenXRRemoting #80

Closed

3 tasks

jcfr mentioned this pull request Dec 21, 2023

BUG: Fix integration with updated Slicer event delegation and VTK OpenVR API #131

Merged

jcfr mentioned this pull request Dec 30, 2023

Enhancements and Refactoring for XR Runtimes Integration #149

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update integration to support VTK 9.1#90

Update integration to support VTK 9.1#90
lassoan merged 9 commits intoKitwareMedical:masterfrom
jcfr:fix-build-against-vtk-9.1.20211022

jcfr commented Feb 26, 2022 •

edited

Loading

Uh oh!

jcfr commented Feb 26, 2022 •

edited

Loading

Uh oh!

cpinter commented Mar 1, 2022

Uh oh!

cpinter commented Mar 4, 2022

Uh oh!

cpinter commented Mar 11, 2022 •

edited

Loading

Uh oh!

jcfr commented Mar 11, 2022 •

edited

Loading

Uh oh!

lassoan commented Mar 22, 2022

Uh oh!

cpinter commented Mar 22, 2022

Uh oh!

lassoan commented Mar 22, 2022

Uh oh!

cpinter commented Mar 22, 2022

Uh oh!

jcfr commented Mar 22, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jcfr commented Feb 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Obsolete branches

Remaining issues

(1) Retrieval of last position associated with generic tracker

(2) RecognizeComplexGesture is not virtual anymore in base VTK class

API changes

Uh oh!

jcfr commented Feb 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cpinter commented Mar 1, 2022

Uh oh!

cpinter commented Mar 4, 2022

Uh oh!

cpinter commented Mar 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jcfr commented Mar 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lassoan commented Mar 22, 2022

Uh oh!

cpinter commented Mar 22, 2022

Uh oh!

lassoan commented Mar 22, 2022

Uh oh!

cpinter commented Mar 22, 2022

Uh oh!

jcfr commented Mar 22, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jcfr commented Feb 26, 2022 •

edited

Loading

jcfr commented Feb 26, 2022 •

edited

Loading

cpinter commented Mar 11, 2022 •

edited

Loading

jcfr commented Mar 11, 2022 •

edited

Loading