AI Image descriptions progress #19807
Replies: 3 comments 7 replies
-
|
What do you mean by a high-quality graphics card? For example, if I bought an Ultrabook with an NPU but only an integrated graphics card, would I still be unable to benefit from high-quality AI? |
Beta Was this translation helpful? Give feedback.
-
|
Where is that said? I only mentioned NPU and GPUs together |
Beta Was this translation helpful? Give feedback.
-
|
I apologize, I'm using a translation tool, and I think there's a translation error. What I meant was: For quality explanations, it requires an NPU/GPU. Is an integrated GPU sufficient, or is a dedicated GPU also required? |
Beta Was this translation helpful? Give feedback.
-
|
I would imagine a dedicated GPU or NPU is required for responsive high quality image descriptions. To be clear though, this is just a rough idea. We have no specific models proposed, we just want a cheap CPU model and a more expensive model for better hardware. Minimum and recommended hardware for the models is entirely dependent on what models we actually end up using |
Beta Was this translation helpful? Give feedback.
-
|
Hi Sean, I apologize if my questions are bothering you, but I'm trying to understand the situation. To summarize: I was one of the users arguing that the AI planned for version 2026.1 of NVDA would be insufficient and that we needed a higher-quality AI. While arguing this, I understood that a computer with an NPU would be required. However, the mention of NPU/GPU confused me. I definitely understood that an NPU was necessary. But I wasn't sure if the GPU referred to an integrated GPU or a dedicated GPU capable of handling demanding games. However, from your recent statements, I understand that the NPU is more important. After all, GPUs are present in almost every computer, good or bad. I'm going to upgrade my computer just to be able to use high-quality AI in NVDA. I don't want to buy a computer with an NPU and be deprived of it simply because its GPU is standard. That's why I've been focusing on this issue. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Harun, to put it simply, we don't know, as we haven't decided on a high quality model yet. |
Beta Was this translation helpful? Give feedback.
-
|
I have already implemented multi-model support in the add-on using a model manager, and have experimentally added support for the Qwen 3.5 series ONNX models. The 0.8B model can produce results in about one minute with reasonably good quality, and it supports multiple languages using preset prompt Localized string (currently Chinese and English). It is available for manual download here Regarding hardware acceleration, I have also made some attempts. However, I was not able to successfully integrate DirectML acceleration. It seems that DirectML has compatibility issues with certain model operators, and resolving them turned out to be more complex than expected. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @tianzeshi-study |
Beta Was this translation helpful? Give feedback.
-
|
I don't think building ollama from source is something in scope of NVDA / something we want to maintain at NV Access |
Beta Was this translation helpful? Give feedback.
-
|
Also Mozilla Lllamafile project could be interesting for this. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
The latest good NVDA installer of AI image descriptions can be found here.
After early alpha testing feedback, on-device AI image descriptions were removed from 2026.1.
This is due to these main reasons:
To reintroduce the feature in alpha, we want to fix the following things first:
So far we have done this, by recently improving the model slightly.
The next biggest priorities are:
After:
A big technical challenge here is the lag importing numpy introduces, which the python onnxruntime requires.
We investigated creating a C++ layer, but the implementation is still experimental and not working for ARM64EC: microsoft/onnxruntime#15403
We could consider offloading onnxruntime, numpy and the describer to a separate process, similar to the 32bit shim.
Related issues and PRs:
Beta Was this translation helpful? Give feedback.
All reactions