Skip to content

Make tests pass some more years#3045

Merged
EnricoMi merged 6 commits intoPyGithub:mainfrom
bmwiedemann:main
Oct 27, 2024
Merged

Make tests pass some more years#3045
EnricoMi merged 6 commits intoPyGithub:mainfrom
bmwiedemann:main

Conversation

@bmwiedemann
Copy link
Copy Markdown

Without this patch, tests fail after 2024-11-25.

Background:
As part of my work on reproducible builds for openSUSE, I check that software still gives identical build results in the future.
The usual offset is +16 years, because that is how long I expect some software will be used in some places.
This showed up failing tests in our python-PyGithub package build.
See https://reproducible-builds.org/ for why this matters.

Without this patch, tests fail after 2024-11-25.

Background:
As part of my work on reproducible builds for openSUSE, I check that software still gives identical build results in the future.
The usual offset is +16 years, because that is how long I expect some software will be used in some places.
This showed up failing tests in our python-PyGithub package build.
See https://reproducible-builds.org/ for why this matters.
@EnricoMi
Copy link
Copy Markdown
Collaborator

These expiration timestamps are evaluated server-side. I think there is no issue with the tests past this time.

Have you tested this breaks with local time changed past that date?

@bmwiedemann
Copy link
Copy Markdown
Author

Yes, that is how I found this.

 =================================== FAILURES ===================================
 _____________ Authentication.testAppInstallationAuthAuthentication _____________
 
 self = <tests.Authentication.Authentication testMethod=testAppInstallationAuthAuthentication>
 
     def testAppInstallationAuthAuthentication(self):
         # test data copied from testAppAuthentication to test parity
         installation_auth = github.Auth.AppInstallationAuth(self.app_auth, 29782936)
         g = github.Github(auth=installation_auth)
     
         # test token expiry
         # token expires 2024-11-25 01:00:02
         token = installation_auth.token
 >       self.assertFalse(installation_auth._is_expired)
 E       AssertionError: True is not false
 
 tests/Authentication.py:182: AssertionError
 =========================== short test summary info ============================
 FAILED tests/Authentication.py::Authentication::testAppInstallationAuthAuthentication
 ======================== 1 failed, 936 passed in 31.00s ========================

@EnricoMi
Copy link
Copy Markdown
Collaborator

Then we should mock the clock used by _is_expired:

diff --git a/tests/Authentication.py b/tests/Authentication.py
index ffd419c6..071a2147 100644
--- a/tests/Authentication.py
+++ b/tests/Authentication.py
@@ -176,16 +176,15 @@ class Authentication(Framework.BasicTestCase):
         installation_auth = github.Auth.AppInstallationAuth(self.app_auth, 29782936)
         g = github.Github(auth=installation_auth)
 
-        # test token expiry
         # token expires 2024-11-25 01:00:02
         token = installation_auth.token
-        self.assertFalse(installation_auth._is_expired)
         self.assertEqual(
             installation_auth._AppInstallationAuth__installation_authorization.expires_at,
             datetime(2024, 11, 25, 1, 0, 2, tzinfo=timezone.utc),
         )
 
-        # forward the clock so token expires
+        # test token expiry
+        # control the current time used by _is_expired
         with mock.patch("github.Auth.datetime") as dt:
             # just before expiry
             dt.now = mock.Mock(return_value=datetime(2024, 11, 25, 0, 59, 3, tzinfo=timezone.utc))

@bmwiedemann
Copy link
Copy Markdown
Author

OK. I can test your patch.

@bmwiedemann
Copy link
Copy Markdown
Author

I tried that patch and got a different error/trace:
https://rb.zq1.de/other/python-PyGithub/pr3045-log1.txt

@bmwiedemann
Copy link
Copy Markdown
Author

There is around one month left to solve this. What are the plans?

@EnricoMi
Copy link
Copy Markdown
Collaborator

I tried that patch and got a different error/trace: https://rb.zq1.de/other/python-PyGithub/pr3045-log1.txt

Probably because you did not revert your changes of the replay data.

Reverting your commit and applying the proposed patch fixes the issue.

@bmwiedemann
Copy link
Copy Markdown
Author

bmwiedemann commented Oct 21, 2024

You can have a look at
https://rb.zq1.de/other/python-PyGithub/build-log-0c41ff94d.txt
in line 85 you see that it runs 13 months from now with -rtc base=2025-11-26
in line 656, you can see that only 0c41ff9.patch gets applied
and in line 3552 you can see the failed assertion.

It does not throw the assertion with the current date.

@bmwiedemann
Copy link
Copy Markdown
Author

Looking more closely at the test log, the problem seems to be that it tries to connect a live server, but our builds run without access to the internet. So maybe something wrong with the mocking?

@EnricoMi
Copy link
Copy Markdown
Collaborator

Of course, using the token has to be guarded against real time as well: 6898f52

How is this now?

@bmwiedemann
Copy link
Copy Markdown
Author

Yes, that makes it pass.

Copy link
Copy Markdown
Collaborator

@EnricoMi EnricoMi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@EnricoMi EnricoMi added this pull request to the merge queue Oct 27, 2024
Merged via the queue into PyGithub:main with commit 352c55a Oct 27, 2024
@bmwiedemann
Copy link
Copy Markdown
Author

Can we please get a new release well ahead of 2024-11-25 ?

@EnricoMi
Copy link
Copy Markdown
Collaborator

EnricoMi commented Nov 6, 2024

Why is that?

@bmwiedemann
Copy link
Copy Markdown
Author

So that we (openSUSE and all other distributions) don't need to cherry-pick this patch onto our 2.4.0 package.

@EnricoMi
Copy link
Copy Markdown
Collaborator

EnricoMi commented Nov 6, 2024

Do you execute unit tests in your pipeline?

@bmwiedemann
Copy link
Copy Markdown
Author

@EnricoMi
Copy link
Copy Markdown
Collaborator

EnricoMi commented Nov 6, 2024

Released as v2.5.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants