Classification application for Windows on Snapdragon® with MobileNet-V2 using ONNX runtime.

The app demonstrates how to use the QNN execution provider to accelerate the model using the Snapdragon® Neural Processing Unit (NPU).

• Snapdragon® Platform (e.g. X Elite)
• Windows 11+

• Visual Studio 22
  • Download any variant of Visual Studio here
  • Make sure Desktop development with C++ tools are selected during installation or installed separately later

Download the float/onnx variant of MobileNet-V2 from AI Hub. Rename it and place it into:

<project directory>/assets/models/classification.onnx

1. Open Classification.sln

2. Setting up dependencies

   • NuGet packages

     • NuGet packages should automatically restore in Visual Studio during build
     • If packages are not restored automatically, try the following:
       • If prompted by Visual Studio to restore NuGet packages
         • Click on restore to restore all NuGet packages
       • Otherwise,
         • Go to Project -> Manage NuGet packages in Visual studio
         • Install ONNX-Runtime-QNN 1.19.0

   • vcpkg packages

     • Project is configured to work with vcpkg in manifest mode

     • If opencv headers are missing, vcpkg is not setup correctly.

     • Integrate vcpkg with Visual Studio:

       • Go to View -> Terminal in Visual studio
       • Run the following command in terminal

       vcpkg integrate install

3. Build and run project in Visual Studio

Visual Studio project is configured with the following command arguments:

--model .\assets\models\classification.onnx --image .\assets\images\keyboard.jpg

You can simply run the app from Visual Studio to run classification on sample image.

.\ARM64\Debug\Classification.exe --model .\assets\models\classification.onnx --image .\assets\images\keyboard.jpg

You can additionally run --help to get more information about all available options:

.\ARM64\Debug\Classification.exe --help

Please refer to QNN EP options that can be provided as --qnn_options to the app.

1. Model input resolution: 224x224
   • If input image is of different shape, it's resized to 224x224
   • You can override model input dimensions if model uses different spatial image dimensions
2. App is built to work with post-processed outputs
   • App processes output logits and produces consumable output as Class Label.
   • If you want to try out any other model than Yolo (with post-processing included in model), please update output handling accordingly.

1. If you get a DLL error message upon launch (for instance that opencv_core4d.dll was not found). Try Build -> Clean Solution and re-build. If this still happens, please go over the NuGet and vcpkg instructions again carefully.
2. How do I use a model with different input shape than 224x224?
   • Use --model_input_ht / --model_input_wt to model input dimensions.
3. I have a model that does have different post-processing. Can I still use the app?
   • You will have to modify the app and add the necessary post-processing to accommodate that models.

Following section describes how to configure your own project with NuGet and vcpkg from scratch:

1. Start empty Visual Studio project
2. Set up NuGet to install ONNXRuntime QNN Execution provider
   • Go to Project -> Manage NuGet Packages
   • Look up and install the following packages
     • Microsoft.ML.OnnxRuntime.QNN
   • This NuGet package does not properly copy all necessary files to the build folder, so we use a batch script copy_missing_qnn_ep_files.bat to do this for us. Copy this script into your project and make sure it runs by adding this to your .vcxproj file:

     <Target Name="PostBuildCopyQnn" AfterTargets="Build">
       <Exec Command="&quot;$(ProjectDir)copy_missing_qnn_ep_files.bat&quot; &quot;$(OutDir)&quot;" />
     </Target>
     

3. Set up Visual Studio for vcpkg

   • Enable vcpkg manifest mode

     • Go to Project Setting
     • General -> vcpkg
     • Enable Manifest mode

   • Add OpenCV dependency in vcpkg

     • Run the following commands in Visual Studio powershell:

     vcpkg —new application
     vcpkg add port opencv

   • Important: To get around a vcpkg bug affecting ARM64, please add the following to your .vcxproj file:

     <PropertyGroup>
       <VcpkgPlatformTarget Condition="'$(Platform)'=='ARM64'">arm64</VcpkgPlatformTarget>
     </PropertyGroup>
     

     If this is not added, you will see an error message like this:

     error: Invalid triplet name. Triplet names are all lowercase alphanumeric+hyphens.
     

4. Now project is setup to work with the vcpkg and NuGet package managers