Skip to content

Conversation

@jl-wynen
Copy link
Member

@jl-wynen jl-wynen commented Sep 27, 2023

Part of #3166.

I did not yet touch the data creation scripts prepare_data_rhessi.py and prepare-filtering-data.py. The former should wait until #3258 is merged and I haven't looked in detail at the latter yet.

Also, I did not touch ADRs after talking to Neil about it. We agreed that ADRs should not change after they were accepted.

EDIT
Migrated the event filtering example.

EDIT
Migrated the solar flares example.
Unfortunately, NASA updated the flare list and it is now filtered such that the tutorial doesn't make much sense. So I stuck with the old data. But since I don't have a copy of the old flare list, I modified our old hdf5 file.

@jl-wynen jl-wynen requested a review from nvaytet September 27, 2023 14:40
@jl-wynen jl-wynen force-pushed the remove-attrs-from-docs branch from 2e4b890 to 720542d Compare October 17, 2023 12:00
@jl-wynen jl-wynen marked this pull request as ready for review October 17, 2023 12:02
@jl-wynen jl-wynen force-pushed the remove-attrs-from-docs branch 2 times, most recently from 0cb8171 to f55658b Compare October 17, 2023 12:32
@jl-wynen jl-wynen requested review from nvaytet and removed request for nvaytet October 19, 2023 08:41
Copy link
Member

@nvaytet nvaytet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. Some small comments. I have to say I did not go through the scipp source to see if any other mentions of attributes were missed, I only checked that what is in this PR looks ok.

"ds.coords['x'] # the coords *dict* is copied, but the 'x' coordinate references same buffer"
"# the coords *dict* is copied, \n",
"# but the 'x' coordinate references same buffer\n",
"ds.coords['x'] "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"ds.coords['x'] "
"ds.coords['x']"

)
new['flares'].coords['max_energy'] = new['flares'].attrs.pop('max_energy')
new['flares'].coords['min_energy'] = new['flares'].attrs.pop('min_energy')
new.save_hdf5("docs/tutorials/data/rhessi_flares.h5")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
new.save_hdf5("docs/tutorials/data/rhessi_flares.h5")
new.save_hdf5("rhessi_flares.h5")

?
Or have a common path prepended to both input and output files?

"- `peak_time`: Date and time of the highest x-ray flux.\n",
"- `x`, `y`: Position in the image seen by RHESSI.\n",
"- `min_energy`, `max_energy`: Energy band that a flare was observed in. Bands do not overlap."
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing non-solar? I can see we are still using it below.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, is it because it's not longer a coordinate? Do we need to describe its meaning anyway if we are using it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, is it because it's not longer a coordinate? Do we need to describe its meaning anyway if we are using it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, is it because it's not longer a coordinate?

Yes. I meant to explain it separately but forgot. Did it now.

"# Move min_energy from attributes to coordinates\n",
"flares.coords['min_energy'] = flares.attrs.pop('min_energy')\n",
"flare_data = sc.io.load_hdf5(filename)\n",
"flare_data"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's slightly confusing for new-comers between flare_data and flares.data. Can we make the names more different?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I always struggle to come up with good names in this situation. It's similar to loading nexus files:

<needs-a-name> = load()
detector = <needs-a-name>['instrument']['detector']

Do you have a good idea what to use for <needs-a-name>?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a good idea what to use for <needs-a-name>?

No.

I see in this PR you changed to flare_datagroup, which is fine. I guess you could also have used something like flare_database, if you wanted to stay away from the Scipp types, but either is fine.

"| mask | combine with `or` |\n",
"| attr | typically ignored or dropped |\n",
"\n",
"In the special case of in-place operations such as `+=` or `*=` Scipp preserves existing attributes and ignores attributes of the right-hand-side.\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this sentence as well? It's talking about attributes.

"When slicing a data array the following additional rule applies:\n",
"\n",
"- Meta data (coords, masks, attrs) that *do not depend on the slice dimension* are marked as *readonly*\n",
"- Meta data (coords, masks) that *do not depend on the slice dimension* are marked as *readonly*\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What should be the consistent spelling? "Meta data" or "Metadata"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to my grammar checker, Wikipedia, and Merriam-Webster it's 'metadata'.
It's inconsistent in our docs.

Copy link
Member

@nvaytet nvaytet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but there are merge conflicts....

@jl-wynen jl-wynen force-pushed the remove-attrs-from-docs branch from 1141f2b to 39d7645 Compare October 25, 2023 14:34
@jl-wynen jl-wynen merged commit 92389ff into main Nov 1, 2023
@jl-wynen jl-wynen deleted the remove-attrs-from-docs branch November 1, 2023 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants