Skip to content

Invalid TRPOS<n> keyword not generated for FITS time column#8784

Merged
taldcroft merged 7 commits intoastropy:masterfrom
aaryapatil:time_location_warning
Jun 23, 2019
Merged

Invalid TRPOS<n> keyword not generated for FITS time column#8784
taldcroft merged 7 commits intoastropy:masterfrom
aaryapatil:time_location_warning

Conversation

@aaryapatil
Copy link
Member

@aaryapatil aaryapatil commented Jun 2, 2019

This fixes issue #8773. I had to make a few minor changes to the FITS TIME machinery to handle default values. Have added a small test too!

@taldcroft Do check!

Fixes #8773

@bsipocz bsipocz added the io.fits label Jun 2, 2019
@bsipocz bsipocz added this to the v3.2.1 milestone Jun 2, 2019
@bsipocz bsipocz requested a review from taldcroft June 2, 2019 22:08
@pllim
Copy link
Member

pllim commented Jun 3, 2019

Please fix PEP 8:

astropy/io/fits/tests/test_fitstime.py:63:1: W293 blank line contains whitespace
1

I think doc build error is unrelated?

@bsipocz
Copy link
Member

bsipocz commented Jun 3, 2019

docbuild error is indeed unrelated and should be fixed after a rebase.

@aaryapatil aaryapatil force-pushed the time_location_warning branch from 7dbf9ff to b7ea093 Compare June 3, 2019 18:27
@codecov
Copy link

codecov bot commented Jun 3, 2019

Codecov Report

Merging #8784 into master will increase coverage by 0.27%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #8784      +/-   ##
==========================================
+ Coverage    86.7%   86.97%   +0.27%     
==========================================
  Files         403      400       -3     
  Lines       59965    59461     -504     
  Branches     1100     1100              
==========================================
- Hits        51992    51719     -273     
+ Misses       7332     7101     -231     
  Partials      641      641
Impacted Files Coverage Δ
astropy/io/fits/fitstime.py 84.42% <100%> (+0.15%) ⬆️
astropy/io/fits/convenience.py 85.9% <100%> (+0.14%) ⬆️
astropy/visualization/units.py 68.08% <0%> (-5.5%) ⬇️
astropy/constants/__init__.py 94.87% <0%> (-5.13%) ⬇️
...coordinates/builtin_frames/icrs_cirs_transforms.py 78.1% <0%> (-4.76%) ⬇️
astropy/utils/iers/iers.py 92.16% <0%> (-0.75%) ⬇️
astropy/io/fits/hdu/base.py 90.36% <0%> (-0.16%) ⬇️
astropy/io/fits/column.py 90.37% <0%> (-0.1%) ⬇️
astropy/wcs/wcsapi/low_level_api.py 97.05% <0%> (-0.09%) ⬇️
astropy/coordinates/baseframe.py 92.93% <0%> (-0.07%) ⬇️
... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a22c724...4ad4644. Read the comment docs.

Copy link
Member

@taldcroft taldcroft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @aaryapatil - sorry for the delay. Here is my first look, please excuse that I have entirely forgotten how your code works and the FITS standard, if I ever understood the first time. 😄

CHANGES.rst Outdated
astropy.io.fits
^^^^^^^^^^^^^^^

- Invalid TRPOS<n> keyword is not generated for FITS time column. [#8784]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

Fixes bug where an invalid TRPOS<n> keyword was being generated for FITS
time column when no location was available. [#8784]

@@ -259,13 +259,20 @@ def _verify_column_info(column_info, global_info):
# considered.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is no longer correct, as the code change below also now ignores global reference position. That does not seem like the right thing, but I don't understand the FITS time spec in detail.

column_info['location'] = global_info['location']
if column_info['location'] is None:
if global_info['location'] is None:
warnings.warn(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't see why this needs to generate a warning. At least I think this simple case should work without warning:

In [7]: tm = Time(['2000:001'])
   ...: t = Table([tm])
   ...: t.write('time.fits', overwrite=True)
   ...: t2 = t.read('time.fits', astropy_native=True)
   ...: 
   ...: 
WARNING: Time column reference position "TRPOSn" is not specified. The default value for it is "TOPOCENTER", but due to unspecified observatory position, reference position will be ignored. [astropy.io.fits.fitstime]

I don't see any reason to assume that Time data written to FITS has anything to do with an observatory.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We call is "space-time" because technically time alone does not have a meaning, e.g. if you see two signals in two bands (let's say you discover a supernova in I and R band), you might want to see which band detected it first. Due to the light-travel time, the position of the two observatories is required to do this for small time differences.
However, in practice, most observations are done somewhere on the earth and on a time precision much less than the light travel time, so location does not matter. Thus, I agree that a warning is needed in principle ,but I think it will be annoying for the vast majority of users where it does not matter, so I recommend "no warning".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "no warning" makes sense. I was inclined to atleast warn the user following @hamogu's argument, but it definitely would be annoying.

'has been specified. However, for supporting column-specific location, '
'reference position will be ignored for this column.',
AstropyUserWarning)
column_info['location'] = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, not clear why this behavior is being changed to always ignore global location.

Copy link
Member Author

@aaryapatil aaryapatil Jun 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is when there are multiple time columns in a table, one with a location associated with it and one without one.

t = Table()
t['a'] = Time(['2000:001'], format='isot', scale='utc', location=EarthLocation(-2446354,
                      4237210, 4077985, unit='m'))
t['b'] = Time([1], format='cxcsec', scale='tt')

The FITS time standard requires us to write the EarthLocation(-2446354, 4237210, 4077985, unit='m') in global time keywords OBSGEO-X, OBSGEO-Y, OBSGEO-Z. TRPOSn is a column-specific keyword and we would set it for the column 'a' as TRPOS1 = 'TOPOCENTER'. But what FITS would do is that if there is no value for TRPOS2 (for column 'b'), by default it would treat it as TRPOS2 = 'TOPOCENTER' and use the global keywords OBSGEO-X, OBSGEO-Y, OBSGEO-Z as its location. But, we want reproducibility with astropy, so instead when TRPOSn is not specified, I treat the default as None instead of 'TOPOCENTER'. I think this makes sense, as the user should ideally specify TRPOSn to make sure there is no ambiguity because of default values.

@bsipocz bsipocz modified the milestones: v3.2.1, v3.2.2 Jun 15, 2019
@bsipocz
Copy link
Member

bsipocz commented Jun 20, 2019

@aaryapatil - If you get there, please rebase this and also remove the capture of the warning from here: https://github.com/astropy/astropy/pull/8754/files#diff-ff07dd01aa949e3fe71d5431d0f7d2bcR167

@aaryapatil aaryapatil force-pushed the time_location_warning branch from b7ea093 to 4ad4644 Compare June 20, 2019 16:16
@bsipocz bsipocz requested a review from taldcroft June 23, 2019 02:19
Copy link
Member

@taldcroft taldcroft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Dropping those warnings for typical use cases is good, test changes look good, and I explicitly confirmed that my original issue is fixed. Thanks @aaryapatil !!

@taldcroft taldcroft merged commit 0efa346 into astropy:master Jun 23, 2019
bsipocz pushed a commit that referenced this pull request Jun 25, 2019
Invalid TRPOS<n> keyword not generated for FITS time column
@aaryapatil aaryapatil deleted the time_location_warning branch July 12, 2019 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Invalid TRPOS<n> keyword generated for FITS time column

5 participants