-
Notifications
You must be signed in to change notification settings - Fork 18.6k
Description
Hi, currently I encounter a problem. I run the train_net.bin on my own data, the testing seems to crash. The logged information is:
I0111 00:39:27.884718 8026 solver.cpp:84] Testing net
F0111 00:39:28.925695 8026 syncedmem.cpp:45] Check failed: (cudaMalloc(&gpu_ptr_, size_)) == cudaSuccess (2 vs. 0)
*** Check failure stack trace: ***
@ 0x7f1b7e10bb5d google::LogMessage::Fail()
@ 0x7f1b7e10fb77 google::LogMessage::SendToLog()
@ 0x7f1b7e10d9f9 google::LogMessage::Flush()
@ 0x7f1b7e10dcfd google::LogMessageFatal::~LogMessageFatal()
@ 0x436d57 caffe::SyncedMemory::mutable_gpu_data()
@ 0x4208fe caffe::Blob<>::mutable_gpu_data()
@ 0x445dd4 caffe::ConvolutionLayer<>::Forward_gpu()
@ 0x42842a caffe::Net<>::ForwardPrefilled()
@ 0x41d319 caffe::Solver<>::Test()
@ 0x41e705 caffe::Solver<>::Solve()
@ 0x40b8dd main
@ 0x30b9c1ecdd (unknown)
@ 0x40b739 (unknown)
Aborted (core dumped)
I exactly exploit the network architecture defined in "imagenet.prototxt" and "imagenet_val.prototxt". My training and testing datasets, respectively, are out 20G and contain 200,000 images cropped to 256*256.
Anyway, I feel a little confused now...Thus I would like to ask for help here. Much thanks!