/home/user/cuda-ubuntu-16.04-ec2/pytorch/aten/src/THCUNN/BCECriterion.cu:30: Acctype bce_functor<Dtype, Acctype>::operator()(Tuple) [with Tuple = thrust::detail::tuple_of_iterator_references<thrust::device_reference<float>, thrust::device_reference<float>, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type, thrust::null_type>, Dtype = float, Acctype = float]: block: [0,0,0], thread: [223,0,0] Assertion `input >= 0. && input <= 1.` failed.
Traceback (most recent call last):
  File "train_model.py", line 138, in <module>
    train_model(config)
  File "train_model.py", line 105, in train_model
    worker.train(train_loader, plot_lr=plot_lr, on_iter=on_iter)
  File "/home/user/src/worker.py", line 204, in train
    time_loss = F.binary_cross_entropy(time_pred, time_hist.float())
  File "/home/user/miniconda3/envs/cuda/lib/python3.6/site-packages/torch/nn/functional.py", line 1507, in binary_cross_entropy
    return torch._C._nn.binary_cross_entropy(input, target, weight, size_average, reduce)
RuntimeError: reduce failed to synchronize: device-side assert triggered