Skip to content

Gather outdated snapshots#7739

Merged
baermat merged 10 commits intomasterfrom
gather-outdated-snapshots
Feb 24, 2023
Merged

Gather outdated snapshots#7739
baermat merged 10 commits intomasterfrom
gather-outdated-snapshots

Conversation

@baermat
Copy link
Member

@baermat baermat commented Feb 23, 2023

This adds a scripts that enables to gather which snapshots are older than a given date. The data is gathered within a JSON object

The data includes at the moment:

  • The provided date to check the snapshot date against
  • The number of outdated snapshots
  • The name/node-ids of the outdated snapshots as list
  • If invoked via cli, the names in a whitespace-separated string instead of a list, to make automated execution via pytest possible if wanted

@baermat baermat marked this pull request as ready for review February 23, 2023 15:20
@baermat baermat requested a review from steffyP February 23, 2023 15:20
@baermat
Copy link
Member Author

baermat commented Feb 23, 2023

Failing tests are unrelated since this doesn't actually affect the code 🙂

Copy link
Member

@steffyP steffyP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work 🚀
Finally, we can stop guessing how old the most snapshot tests really are 😄 This will be very helpful for automating the snapshot updates in the future 🤩

Just one thing I discovered: we have some snapshot-tests "hidden" in subfolders, it would be great if we could also iterate sub directories and search for additional snapshot-files 🙂

Fetches all snapshots in PATH that were recorded before the given DATE_LIMIT.
Format of the DATE_LIMIT-string must be "DD-MM-YYYY".

Example usage: python get_snapshots.py ../tests/integration 24-12-2022
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: I think you renamed the file to gather_outdated_snapshots.py, would be nice to change the example usage here :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

outdated_snapshots = []
if not path.endswith("/"):
path = f"{path}/"
for file in os.listdir(path):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we might need to check sub-folders as well, e.g. for s3 there are some snapshot files in a sub-directory

@coveralls
Copy link

Coverage Status

Changes Unknown when pulling 49c29b4 on gather-outdated-snapshots into ** on master**.

@baermat baermat requested a review from steffyP February 24, 2023 07:18
Copy link
Member

@steffyP steffyP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome 🚀 Thanks for the changes @baermat, it will make the snapshot updates much easier 👍

@baermat baermat merged commit 39dea38 into master Feb 24, 2023
@alexrashed alexrashed deleted the gather-outdated-snapshots branch March 27, 2023 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants