Conversation
|
So there is a concern that some of these helper functions should remain in the implementation so that they can be inlined efficiently in the internal ggml code - some of them can be called extensively in hot loops. I think we can do the following:
This way, internal ggml code will continue to use the inlined With this new pattern established, we can continue to migrate the rest of the auxilary calls, such as |
|
Yes, this is the best solution, inline for performance and an auxiliary for API. The only shortcoming is that API names become longer with |
Yes, it won't be a breaking change this way. Though it would require a massive change in the internal ggml implementation to switch to the aux calls. But at the same time, it can be done incrementally. So your suggestion might be the better option indeed - let me give this a bit of more thought. |
* upstream/master: (88 commits) ci : bump komac version (ggml-org#19682) build : link ws2_32 as PUBLIC on Windows (ggml-org#19666) build : cleanup library linking logic (ggml-org#19665) convert : add JoyAI-LLM-Flash (ggml-org#19651) perplexity: add proper batching (ggml-org#19661) common : inline functions (ggml-org#18639) ggml : make `ggml_is_view` as API (ggml-org#19539) model: Add support for Tiny Aya Models (ggml-org#19611) build : rework llama_option_depr to handle LLAMA_CURL (ggml-org#19658) Adjust workaround for ROCWMMA_FATTN/GFX9 to only newer ROCm veresions (ggml-org#19591) models : deduplicate delta-net graphs for Qwen family (ggml-org#19597) graph : fix KQ mask, lora, cvec reuse checks (ggml-org#19644) ggml: aarch64: Implement SVE in Gemm q4_k 8x8 q8_k Kernel (ggml-org#19132) sync : ggml ggml : bump version to 0.9.7 (ggml/1425) ggml : bump version to 0.9.6 (ggml/1423) cuda: optimize iq2xxs/iq2xs/iq3xxs dequantization (ggml-org#19624) docs: update s390x build docs (ggml-org#19643) build : remove LLAMA_HTTPLIB option (ggml-org#19623) cmake : check if KleidiAI API has been fetched (ggml-org#19640) ...
* make `ggml_is_view` as API * introduce `ggml_aux_is_view` as inline version for internal use. * change `ggml_aux_is_view` to `ggml_impl_is_view`
* make `ggml_is_view` as API * introduce `ggml_aux_is_view` as inline version for internal use. * change `ggml_aux_is_view` to `ggml_impl_is_view`
* make `ggml_is_view` as API * introduce `ggml_aux_is_view` as inline version for internal use. * change `ggml_aux_is_view` to `ggml_impl_is_view`
Make sure to read the contributing guidelines before submitting a PR
This PR makes
ggml_is_viewinto an API.Motivation
Some backends complain (example) when an input is a view with offsets. It helps to have an
is_viewAPI to make these backends happy.