Skip to content

Allow use of hip SDK (if installed) dlls on windows#27

Merged
YellowRoseCx merged 2 commits intoYellowRoseCx:mainfrom
one-lithe-rune:allow-sdk-dll-loading
Oct 9, 2023
Merged

Allow use of hip SDK (if installed) dlls on windows#27
YellowRoseCx merged 2 commits intoYellowRoseCx:mainfrom
one-lithe-rune:allow-sdk-dll-loading

Conversation

@one-lithe-rune
Copy link

Motivation

I had trouble getting things to work via the koboldcpp.py script after building the source on windows. This was down to the the hipBlas/rocBlas .dlls (or their dependencies, I forget which) not loading up even though I had the hip 5.5.1 sdk installed to do build, with the relevant folders on my path.

I turns out windows/python ignores the path unless winmode=0 is set in python CDLL call. But since that setting had been explicitly commented out in the script I didn't feel comfortable changing it. Here's an alternative, FWIW.

Changes

  • If the rocm/hip sdk is installed on windows, as indicated by the HIP_PATH environment variable then explicitly include the sdk bin folder as a potential location to load the hipBlas/rocBlas .dlls from. The other existing .dll paths are included prior to to this, so if they relevant .dlls are already there the sdk ones should be ignored.

* If the rocm/hip sdk is installed on windows, then include the sdk
as a potential location to load the hipBlas/rocBlas .dlls from. This
allows running koboldcpp.py directly with python after building
work on windows without having to build the .exe and run that or
copy .dlls around.
@YellowRoseCx
Copy link
Owner

I originally had winmode=0 in there before But concedo didn't want it in there since it could have unintended effects, and he added this instead, but when I tested it, I might have had the .dll and /rocblas/ residually in my folder still accidentally and didn't catch it not working:

    if os.name=='nt':
        os.add_dll_directory(dir_path)
        os.add_dll_directory(abs_path)
        os.add_dll_directory(os.getcwd())

I think adding your code would be a good addition; I'll just have to switch over to Windows later today and test it out first. Thanks!

@YellowRoseCx YellowRoseCx merged this pull request into YellowRoseCx:main Oct 9, 2023
YellowRoseCx pushed a commit that referenced this pull request Oct 9, 2023
…low use of hip SDK (if installed) dlls on windows

* If the rocm/hip sdk is installed on windows, then include the sdk
as a potential location to load the hipBlas/rocBlas .dlls from. This
allows running koboldcpp.py directly with python after building
work on windows without having to build the .exe and run that or
copy .dlls around.
YellowRoseCx pushed a commit that referenced this pull request Oct 9, 2023
…low use of hip SDK (if installed) dlls on windows

* If the rocm/hip sdk is installed on windows, then include the sdk
as a potential location to load the hipBlas/rocBlas .dlls from. This
allows running koboldcpp.py directly with python after building
work on windows without having to build the .exe and run that or
copy .dlls around.
@zeeroh
Copy link

zeeroh commented Oct 9, 2023

Hello; thanks for merging this in. I had tested it earlier this week and found it built nicely for me and created a new kobold hipblas dll file which was actually a different filesize than the one in the precompiled executable that I was previously using.

I had hoped this new dll was the solution I was looking for since I assumed my problems were due to not having a properly built dll for gfx1032… but unfortunately it still gives me the exact same error as before. When selecting my model in the UI and then running, it errors when trying to load/use the dll and seems to be unhappy with the ctype stuff. I’m afraid I don’t know enough to say why that might be the case or even what the ctype stuff is. Any suggestions?

I’m using win 10, have rocblas installed, and trying to run with my radeon rx 6600 (gfx1032). I got kobold working in Ubuntu (although oddly enough still no luck with llama.cpp which is confusing since I believe kobold is built with that), but no luck in Windows yet unfortunately.

Also: apologies for sneaking my comment into the merged PR here… I didn’t see a more appropriate place to leave it. Thanks for your work on expanding support to those of us using Windows with AMD GPUs. It is very much appreciated!

YellowRoseCx added a commit that referenced this pull request Oct 21, 2023
…low use of hip SDK (if installed) dlls on windows (LostRuins#470)

* If the rocm/hip sdk is installed on windows, then include the sdk
as a potential location to load the hipBlas/rocBlas .dlls from. This
allows running koboldcpp.py directly with python after building
work on windows without having to build the .exe and run that or
copy .dlls around.

Co-authored-by: one-lithe-rune <skapusniak@lithe-runes.com>
@one-lithe-rune one-lithe-rune deleted the allow-sdk-dll-loading branch March 10, 2024 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants