Makefile.msc only makes release builds and provides no way to make a debug build, which makes it harder to use generated libraries in debug configurations because VC++ CRT (C/C++ runtime) for debug and release builds are different. Specifically, -MD is used unconditionally on this line in Makefile.msc:

CFLAGS  = -nologo -MD -W3 -O2 -Oy- -Zi -Fd"zlib" $(LOC)

The fix is fairly simple and is to remove -MD from CFLAGS and add it conditionally, based on whether DEBUG=1 is defined.

CFLAGS  = -nologo -W3 -O2 -Oy- -Zi -Fd"zlib" $(LOC)
...
!IFDEF DEBUG
CFLAGS = $(CFLAGS) /MDd
!ELSE
CFLAGS = $(CFLAGS) /MD /DNDEBUG
!ENDIF

This also adds NDEBUG, which is needed for assert calls to work properly for release builds (i.e. should be no-op in release builds).

This would build debug and release libraries.

nmake -f win32\Makefile.msc DEBUG=1
nmake -f win32\Makefile.msc

You can find a full patch here:

https://github.com/StoneStepsInc/zlib-nuget/blob/master/patches/Makefile.msc.patch

I do realize that there's a CMake build, but I find Makefile.msc cleaner for my purposes and since it's a part of the core project, it probably should work for commonly used configurations.