Without pkg-config it is not possible to build libgit2 with ssh support.
In libgit2/CMakeLists.txt the detection of libssh2library is done with PKG_CHECK_MODULES:

IF (USE_SSH)
        PKG_CHECK_MODULES(LIBSSH2 libssh2)
ENDIF()
IF (LIBSSH2_FOUND)
        ADD_DEFINITIONS(-DGIT_SSH)
        INCLUDE_DIRECTORIES(${LIBSSH2_INCLUDE_DIRS})
        LINK_DIRECTORIES(${LIBSSH2_LIBRARY_DIRS})
        SET(LIBGIT2_PC_REQUIRES "${LIBGIT2_PC_REQUIRES} libssh2")
        SET(SSH_LIBRARIES ${LIBSSH2_LIBRARIES})
ELSE()
        MESSAGE(STATUS "LIBSSH2 not found. Set CMAKE_PREFIX_PATH if it is installed outside of the default search path.")
ENDIF()

If pkg-config package does not exist, it'll continue without adding libssh2 support.