I first reported this issue in rbenv/ruby-build#601. Basically, my boxen/homebrew setup on OS X 10.9.4 installs a modified java wrapper that sets DYLD_FALLBACK_LIBRARY_PATH. From /opt/boxen/bin/java:

#!/bin/sh
#
# Invoke java with the $DYLD_LIBRARY_PATH set with the homebrew lib dir.
# This allows java to load native libraries installed via homebrew.

export DYLD_FALLBACK_LIBRARY_PATH="$BOXEN_HOME/homebrew/lib:$DYLD_FALLBACK_LIBRARY_PATH"

if [ -x /usr/libexec/java_home ]; then
  export JAVA_HOME=`/usr/libexec/java_home`
fi

exec /usr/bin/java "$@"

When DYLD_FALLBACK_LIBRARY_PATH is setup, jgem has the following error:

~/Downloads/jruby-1.7.13/bin $ ./jgem install bundler
ERROR:  Loading command: install (LoadError)
    Could not open library 'libc.dylib' : dlopen(libc.dylib, 5): image not found
ERROR:  While executing gem ... (NoMethodError)
    undefined method `invoke_with_build_args' for nil:NilClass

This error occurs using the binary 1.7.13 and 1.7.12 packages from the website. 1.7.11 and below do not seem affected. A work around is to put export DYLD_FALLBACK_LIBRARY_PATH=/usr/lib in your profile. However, it seems like this is a bug and should not need the work around, as "FALLBACK" implies that it should only be consulted if the normal search paths come up empty.