nn modules have a self.state_dict() and self.load_state_dict() function to save and load the state respectively. However, optimizer only has a self.state_dict() and no way to load the state. This would be nice to have to be able to pause the training by saving both the model/optimizer state and resume where left off.