Why?

Sharing the roadmap with everyone is always good for transparency and best way to drive the development in order to bring the most value in each step.

What?

All feedback is more than welcome !
And we'd love to know more about you and get some insights:

• Main platform are you using? (MacOS, Win, Linux)
• Could you explain bit more about your use case? What are you looking to build with the Unity plugin?
• What part of the Unity plugin you find most interesting? (OAK for Creators, OAK for Developers, OAK for CV/AI)
• Do you have experience with OAK API/SDK? What platform are you using the most? (C/C++ , Python)

Features and Roadmap:

v1.0.0 Initial Release to AssetStore (free):

• No code approach. Just drag and drop some prefabs. Bullet-proof.
• Access to OAK device streams:
  • RGB and depth (point cloud viz currently ongoing dev).
    • About point cloud: Add support for external libraries: PCL, ... and VFX integration.
  • Access to IMU if available and retrieve system information.
  • Record and Replay capability.
• OAK Device Manager and multiple OAK devices support.
• OAK For Unity Creators: High-Level API - Unlock “Myriad” applications with predefined and ready-to-use pretrained models (with depth support if you run OAK-D family):
  • Face Detector, 3D Head Pose Estimation, Face emotion, Body Pose (MoveNet), Hand Tracking and Gesture recognition (Blaze Hands), Object detector (Tiny Yolo v3/4/5 and MobileNet SSD). DeepLabv3 segmentation. Background removal, Depth (only OAK-D family) - point cloud visualization
  • Multiple camera demo.
  • Example how to integrate your own pipeline
  • Integration with some paid assets (demos)
• OAK For CV/AI/Robotics: Unity Virtual OAK Camera

Version 1.0 is just the top of the iceberg. Foundations to build applications on top. Beta testing along the path to get v1.0 ready for assetstore.

Some topics for next versions (big blocks without much detail)

• More complex demos on top of pretrained models (see list below)
• Send unity virtual camera images to OAK pipeline.

Do you miss some models / use cases / integration with some specific paid asset? Please let us know.

• Android support. Integration with AR.
• End-to-end workflows for synthetic dataset generation, training and deploy.
• Integration with Unity Robotics Hub (ROS) / SystemGraph / SensorSDK. Publish to ROS.
• Integration with Unity Simulation and Unity Simulation Pro.
• Intergration with Unity ML-Agents.
• Define custom pipelines inside Unity (visual tool).
• Implement partial C# wrapper

How?

We're opening beta testing on the repo. The plan is start delivering all the features gradually following initial order and always covering full feature (unity plugin lib + unity project/demos) and for all platforms (Win, Mac and Linux)

Following features (v1.0) are ready:

• Device Manager
• Demo menu scene
• Demos:
• Device streams (rgb, depth), Point cloud visualization using VFX
• Object detector demo
• Face detector demo
• Body pose demo
• Head pose demo
• Face emotion demo
• Hand tracking demo

Please tell us about your project !

More resources:
unity forums: https://forum.unity.com/threads/oak-for-unity-spatial-ai-meets-the-power-of-unity.1205764/
youtube unity plugin playlist: https://youtu.be/CSFOZLBV2RA?list=PLFzqMMJPSNSbsHp7QeJpOHrZu_1BAdDms