Could you please add a test as well?

@peterjc123
I tried to modify the existing test to check read / write of files over 2GB. Any concerns with this approach?

@yn4bit That's great! Thank you.

It's a little bad since Windows tests are failing on master but I'm going to optimistically go ahead and land this.

@ezyang merged this pull request in 1afe3fc.

There's something wrong with the tests:

16:36:07 ======================================================================
16:36:07 ERROR: test_serialization_offset (__main__.TestTorch)
16:36:07 ----------------------------------------------------------------------
16:36:07 Traceback (most recent call last):
16:36:07   File "test_torch.py", line 5034, in test_serialization_offset
16:36:07     b_loaded = torch.load(f)
16:36:07   File "C:\Jenkins\workspace\pytorch-builds\pytorch-win-ws2016-cuda9-cudnn7-py3-test2\build\win_tmp\build\torch\serialization.py", line 426, in load
16:36:07     return _load(f, map_location, pickle_module, **pickle_load_args)
16:36:07   File "C:\Jenkins\workspace\pytorch-builds\pytorch-win-ws2016-cuda9-cudnn7-py3-test2\build\win_tmp\build\torch\serialization.py", line 620, in _load
16:36:07     deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
16:36:07   File "C:\Jenkins\Miniconda3\lib\tempfile.py", line 485, in func_wrapper
16:36:07     return func(*args, **kwargs)
16:36:07 OSError: [Errno 22] Invalid argument
16:36:07 

I'm going to revert this for now until we figure out what exactly the problem is

@yn4bit This one gets reverted, could you please figure out a new one?

Update: Let me do this for you.

@peterjc123 Thank you for your comment!

I think lseek is certainly suspicious. Python portable_lseek () is as follows, so I think that it needs to be changed.

#ifdef MS_WINDOWS
     res = _lseeki64(fd, pos, whence);
#else
     res = lseek(fd, pos, whence);
#endif

However, since OSError has finally occurred, I think that the cause of the direct error is the value passed to the Python seek or a restriction inside Python. I would like to correct this after analyzing the generation mechanism, but there is no Windows development environment at hand ...

@yn4bit I'm testing the fix at #27125. You could comment there if you have more thoughts.

@peterjc123 Thank you for your testing. Wow, this is a surprise to solve the problem. I don't understand the behavior of Windows ...
I checked the patch and it's looks good to me.