Skip to content

Storage: mtime of downloaded file is incorrect by UTC offset #4

@mcsimps2

Description

@mcsimps2

Google Cloud Storage v1.25.0
Python 3.7.3
OS: OSX & Win7

Issue: If I upload a file to Google Cloud Storage and then immediately download it, the mtime is incorrect - for me, I'm in EST, so I'm 5 hours behind UTC. That's the exact timedelta that occurs between the file's original mtime and the recorded mtime after the file is downloaded.

Here's an example screenshot:
Screen Shot 2020-01-27 at 9 35 52 PM
The original file mtime in Google Cloud Storage is 1/23/20 9:04 PM (which is correct from the file I uploaded), but when I download the file, the mtime becomes 1/24/20 2:04 AM, which is 5 hours ahead of what is supposed to be (the UTC offset from my timezone).

The issue is here in blob.download_to_filename:

updated = self.updated
if updated is not None:
            mtime = time.mktime(updated.timetuple())
            os.utime(file_obj.name, (mtime, mtime))

In my example, updated is the timezone-aware datetime corresponding to 2020-01-24 02:04:11.184000+00:00 (it has tzinfo==UTC). The updated.timetuple() is

time.struct_time(tm_year=2020, tm_mon=1, tm_mday=24, tm_hour=2, tm_min=4, tm_sec=9, tm_wday=4, tm_yday=24, tm_isdst=0)

The problem, I believe, is that the timetuple doesn't know this is a UTC date, nor did it convert to my timezone. The docs of mktime note, "Its argument is the struct_time or full 9-tuple (since the dst flag is needed; use -1 as the dst flag if it is unknown) which expresses the time in local time, not UTC." Perhaps, we should do this instead:

if updated is not None:
   mtime = updated.timestamp() # For Python3, not sure of the Python2 version
   os.utime(file_obj.name, (mtime, mtime))

The timestamp() function accounts for the timezone information in the datetime object.
I've just been doing this manually in my code after downloading a file because my application is sensitive to mtimes, and it seems to fix the issue.

Metadata

Metadata

Labels

api: storageIssues related to the googleapis/python-storage API.priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions