Note that these tests are written without using
performance_test_fixture. Because the parameter server is running in the
same process, any allocations happening in the spin thread for the
server get picked up by the allocation statistics even though those
functions aren't invoked in the tests.

If we can find a way to turn off the memory tracking on a per-thread
basis, we can enable memory tracking. Until then, leaving the memory
statistics enabled could be misleading.

Cherry-picked from f5e35bd

Signed-off-by: Scott K Logan <logans@cottsay.net>