Integration test & unit tests jobs separation by auvipy · Pull Request #6649 · celery/celery

auvipy · 2021-02-28T05:15:01Z

No description provided.

auvipy · 2021-02-28T05:25:07Z

@xirdneh you can push improvement on this branch as use this as a base. I am checking how service containers going to work here for integration tests

auvipy · 2021-02-28T05:52:06Z

pypy tests were failing before this PR.

codecov · 2021-02-28T05:56:26Z

Codecov Report

Merging #6649 (a1ae742) into master (3cf5072) will decrease coverage by 0.15%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #6649      +/-   ##
==========================================
- Coverage   89.23%   89.08%   -0.16%     
==========================================
  Files         138      138              
  Lines       16623    16623              
  Branches     2099     2099              
==========================================
- Hits        14834    14808      -26     
- Misses       1570     1597      +27     
+ Partials      219      218       -1

Flag	Coverage Δ
unittests	`89.07% <ø> (-0.16%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
celery/backends/couchbase.py	`40.62% <0.00%> (-42.19%)`	⬇️
celery/backends/asynchronous.py	`60.64% <0.00%> (+0.46%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d3e5df3...a1ae742. Read the comment docs.

auvipy · 2021-02-28T05:59:00Z

redis connection is throwing errors. the service container might need some adjsutment

auvipy · 2021-03-05T04:30:59Z

@matusvalo please suggest modification too.

matusvalo · 2021-03-22T20:49:20Z

Let me check CI in celery. It should be pretty straightforward to port kombu/py-amqp CI to celery also...

maybe-sybr · 2021-05-23T23:34:37Z

Just freshened this on master. I think I'll spend a little time on it today to see if I can get the redis suite quite working as a first step and then the rabbitmq/rpc prior to merge

maybe-sybr · 2021-05-24T00:20:07Z

The suite starts running now, but we have a busted test which fails the run after a few retries.

lgtm-com · 2021-05-24T01:05:54Z

This pull request introduces 1 alert when merging 1dac185 into 025bad6 - view on LGTM.com

new alerts:

1 for Unused import

maybe-sybr · 2021-05-24T01:47:01Z

t/integration/test_inspect.py

                    'routing_key': 'celery',
                    'priority': 0,
-                    'redelivered': False
+                    'redelivered': ANY,


I'm not sure why these tests stopped passing. AFAICT the redelivered seems to be set to None implying that we "don't know" (or don't bother to default to False) if a task has been redelivered or not. These tests /probably/ (??) don't care about that so changing this to an ANY seems like it is okay.

I'd like for someone a bit more familiar with redelivery logic to think about this if possible. I just don't know what we should be expecting here and why it's not False like it must have been in the past.

I'll see if I can find some time later this week

lgtm-com · 2021-05-24T02:04:22Z

This pull request introduces 1 alert and fixes 2 when merging bfab125 into 025bad6 - view on LGTM.com

new alerts:

1 for Unused import

fixed alerts:

1 for Non-exception in 'except' clause
1 for Module is imported with 'import' and 'import from'

maybe-sybr · 2021-05-24T03:46:47Z

Not sure why that inspect test is failing. It passes on my box reliably. Leaving this diff for the moment, someone else can feel free to take a look at it in the meantime.

lgtm-com · 2021-05-24T03:49:36Z

This pull request introduces 1 alert when merging 8f8d402 into 025bad6 - view on LGTM.com

new alerts:

1 for Unused import

maybe-sybr · 2021-05-24T03:50:19Z

.github/workflows/python-package.yml

+        python-version: ['3.7', ] #'3.8', '3.9', 'pypy3']
+        # XXX: Does each matrix variation get their own copy of the service
+        # containers? Test cross-talk may be an issue...
+        toxenv: ['redis', ] #'rabbitmq']


This should probably actually be called backend since it's only the tail of the toxenv name.

tarpas · 2021-06-11T08:23:35Z

Hi. I'm curious what's the motivation for this change. What problem is it solving? Automated tests, their speed and UX for developer is an area of my (informal) research so I'm genuinely interested.

maybe-sybr · 2021-06-13T00:13:08Z

Hi. I'm curious what's the motivation for this change. What problem is it solving? Automated tests, their speed and UX for developer is an area of my (informal) research so I'm genuinely interested.

Our integration suits aren't being run as CI at the moment due to the slow migration we've had away from Travis which is where our CI jobs used to run. This change is mostly just intended to get them running again. There is an outstanding discussion around fast feedback and preferring to run the unit tests prior to the integration tests (as a prerequisite) which are intended to avoid running the more CI minute intensive suite(s) when the unit tests can show that something is clearly broken.

I think you'd need to ask a more specific question for any more detail.

tarpas · 2021-06-14T06:00:08Z

That's good information! Do GitHub actions have some build minutes limits for OS projects? I wasn't able to google any mention of that.

Another way to save CI minutes would be to use pytest -x (stop after first failure). Pro: saves probably even more minutes, Con: only indicates the first error/failure so the PR author might need to do more back and forth if not careful.

maybe-sybr · 2021-06-16T04:19:26Z

Freshened this on current master, I assume that inspect test is probably still busted

maybe-sybr · 2021-06-16T04:22:20Z

That's good information! Do GitHub actions have some build minutes limits for OS projects? I wasn't able to google any mention of that.

I'm not sure tbh, I do know that minutes on GitLab shared runners are limited per project/namespace so maybe GitHub is similar.

Another way to save CI minutes would be to use pytest -x (stop after first failure). Pro: saves probably even more minutes, Con: only indicates the first error/failure so the PR author might need to do more back and forth if not careful.

We do use -x :) You might be able to see it in this diff, or have a look at the pytest commands we run via the tox.ini file. I find it more annoying than anything since I also use tox to test locally and find myself editing those lines to remove the -x pretty much any time I'm devving.

lgtm-com · 2021-06-16T04:47:41Z

This pull request fixes 1 alert when merging 6b18fdd into 5d72aee - view on LGTM.com

fixed alerts:

1 for Non-exception in 'except' clause

maybe-sybr · 2021-06-18T05:00:57Z

Looks like the 3.7-redis suite passed last time I pushed. I've restored the unit tests (for sanity) and the integration matrix of python versions. I'm yet to re-enable the rabbitmq backend matrix element since I'm not sure if/how to avoid spinning up service containers which are for a backend not being tested in a particular job...

lgtm-com · 2021-06-18T05:28:19Z

This pull request introduces 2 alerts and fixes 1 when merging ada7d5a into ee9a251 - view on LGTM.com

new alerts:

1 for Non-exception in 'except' clause
1 for Unused import

fixed alerts:

1 for Module is imported with 'import' and 'import from'

The non-eager tests needed the manager fixture so that there was a worker to actually run the async tasks.

These come back as `None` rather than the previously expected `False`.

lgtm-com · 2021-08-08T14:50:15Z

This pull request fixes 2 alerts when merging 762668e into d3e5df3 - view on LGTM.com

fixed alerts:

2 for Wrong name for an argument in a call

lgtm-com · 2021-08-09T10:49:54Z

This pull request fixes 2 alerts when merging a1ae742 into d3e5df3 - view on LGTM.com

fixed alerts:

2 for Wrong name for an argument in a call

maybe-sybr · 2021-08-11T00:30:50Z

     File "/home/runner/work/celery/celery/celery/app/trace.py", line 358, in build_tracer
      push_request = request_stack.push
  AttributeError: 'NoneType' object has no attribute 'push'

This seems pretty bad 😬

auvipy · 2021-11-11T11:38:03Z

     File "/home/runner/work/celery/celery/celery/app/trace.py", line 358, in build_tracer
      push_request = request_stack.push
  AttributeError: 'NoneType' object has no attribute 'push'

This seems pretty bad grimacing

was the rabbitmq tests passing? I cant remember, I can only remember redis was failing. I will be doing some experiment in another PR to see if rabbit services & tests are running. I goose step at a time, lets see

auvipy added this to the 5.1.0 milestone Feb 28, 2021

auvipy requested a review from xirdneh February 28, 2021 05:23

maybe-sybr mentioned this pull request May 18, 2021

test: Improve use of test fixtures in integration tests #6773

Closed

maybe-sybr force-pushed the ci branch from 6f99337 to 2f576ce Compare May 23, 2021 23:15

maybe-sybr force-pushed the ci branch 4 times, most recently from 33d5350 to a080602 Compare May 24, 2021 00:06

maybe-sybr force-pushed the ci branch from a080602 to 1dac185 Compare May 24, 2021 00:35

maybe-sybr force-pushed the ci branch from 1dac185 to bfab125 Compare May 24, 2021 01:44

maybe-sybr reviewed May 24, 2021

View reviewed changes

maybe-sybr force-pushed the ci branch from 8f8d402 to 6b18fdd Compare June 16, 2021 04:18

maybe-sybr and others added 11 commits August 8, 2021 17:05

test: Fix test_link_error integration suite

aee2f1d

The non-eager tests needed the manager fixture so that there was a worker to actually run the async tasks.

test: Allow any redelivered in inspect tests

5b45684

These come back as `None` rather than the previously expected `False`.

Integration test & unit tests jobs separation

df5d3f5

rabbitmq service container

84f5531

disbale couchbase temporarily

9ed6e05

redis service container

3fdac3c

redis port

8210dff

redis host to localhost

ed47bfd

DO NOT MERGE: hack: Use tox for redis integration suite

3320eba

ci: Add -vv to pytest integration test cmd

93659df

squash! DO NOT MERGE: hack: Use tox for redis integration suite

762668e

Omer Katz added 2 commits August 9, 2021 12:31

Fix rebase blunder.

12a67c2

Increase integration jobs timeout.

a1ae742

auvipy modified the milestones: 5.2, 5.2.x Oct 31, 2021

auvipy self-assigned this Oct 31, 2021

auvipy mentioned this pull request Jan 11, 2022

[WIP] - intial tox-docker config for dockerised integration tests #7216

Draft

dobosevych mentioned this pull request Apr 11, 2022

Binary can't be passed as argument #7448

Open

17 tasks

auvipy mentioned this pull request Apr 14, 2022

Integration test fix #7460

Merged

auvipy modified the milestones: 5.2.x, 5.3 Apr 23, 2022

auvipy closed this Oct 12, 2022

auvipy deleted the ci branch June 2, 2023 00:17

Uh oh!

Conversation

auvipy commented Feb 28, 2021

Uh oh!

auvipy commented Feb 28, 2021

Uh oh!

auvipy commented Feb 28, 2021

Uh oh!

codecov bot commented Feb 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

auvipy commented Feb 28, 2021

Uh oh!

auvipy commented Mar 5, 2021

Uh oh!

matusvalo commented Mar 22, 2021

Uh oh!

maybe-sybr commented May 23, 2021

Uh oh!

maybe-sybr commented May 24, 2021

Uh oh!

lgtm-com bot commented May 24, 2021

Uh oh!

maybe-sybr May 24, 2021

Choose a reason for hiding this comment

Uh oh!

maybe-sybr May 24, 2021

Choose a reason for hiding this comment

Uh oh!

lgtm-com bot commented May 24, 2021

Uh oh!

maybe-sybr commented May 24, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lgtm-com bot commented May 24, 2021

Uh oh!

maybe-sybr May 24, 2021

Choose a reason for hiding this comment

Uh oh!

tarpas commented Jun 11, 2021

Uh oh!

maybe-sybr commented Jun 13, 2021

Uh oh!

tarpas commented Jun 14, 2021

Uh oh!

maybe-sybr commented Jun 16, 2021

Uh oh!

maybe-sybr commented Jun 16, 2021

Uh oh!

lgtm-com bot commented Jun 16, 2021

Uh oh!

maybe-sybr commented Jun 18, 2021

Uh oh!

lgtm-com bot commented Jun 18, 2021

Uh oh!

lgtm-com bot commented Aug 8, 2021

Uh oh!

lgtm-com bot commented Aug 9, 2021

Uh oh!

maybe-sybr commented Aug 11, 2021

Uh oh!

auvipy commented Nov 11, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Feb 28, 2021 •

edited

Loading

maybe-sybr commented May 24, 2021 •

edited

Loading