Skip to content
This repository was archived by the owner on Sep 17, 2025. It is now read-only.

Use Django 2.0 DB instrumentation too#358

Closed
zoidyzoidzoid wants to merge 18 commits intocensus-instrumentation:masterfrom
zoidyzoidzoid:django-db-instrumentation
Closed

Use Django 2.0 DB instrumentation too#358
zoidyzoidzoid wants to merge 18 commits intocensus-instrumentation:masterfrom
zoidyzoidzoid:django-db-instrumentation

Conversation

@zoidyzoidzoid
Copy link
Copy Markdown

@zoidyzoidzoid zoidyzoidzoid commented Oct 22, 2018

Closes #356

I'm not sure whether we want to use the {mysql,postgresql,sqlite3}.queryor a standard sql.query?

I'm also probably missing some other useful attributes we could add, like affected rows and last inserted ID.

Though it doesn't seem like we have recommended/standard attributes for databases/sql attributes, yet.

I also still need to add tests, and make it fail gracefully on Django <2.0.

@c24t
Copy link
Copy Markdown
Member

c24t commented Nov 16, 2018

@zoidbergwill if you add some tests to show _trace_db_call working I can review.

@zoidyzoidzoid
Copy link
Copy Markdown
Author

Sorry for the delay. Will update this week.

span)

if django.VERSION >= (2,):
connection.execute_wrappers.append(self._trace_db_call)
Copy link
Copy Markdown

@hazzadous hazzadous Jan 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this re-append even if the connection already has self._trace_db_call?

I tested out locally and it seems like this is the case, I added a simple check to ensure it doesn't do so in a naive way (just checking if it is already in connection.execute_wrappers) although I'm sure there is a better way to do so.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like a great idea. Thanks @hazzadous

@reyang
Copy link
Copy Markdown
Contributor

reyang commented Mar 7, 2019

@zoidbergwill, would you update the PR and fix the tests? I can review and get this merged.

@zoidyzoidzoid
Copy link
Copy Markdown
Author

Hi @reyang

Thanks, I'll take a look at updating this PR and addressing the comments and fixing the tests this week

@zoidyzoidzoid zoidyzoidzoid force-pushed the django-db-instrumentation branch from 8026400 to e26626f Compare March 11, 2019 15:28
@zoidyzoidzoid zoidyzoidzoid requested review from a team, c24t, reyang and songy23 as code owners March 11, 2019 15:28
@zoidyzoidzoid
Copy link
Copy Markdown
Author

I'm not sure where the source of googleapis/nox is, but we need to add Python 3.7 support to it. If someone can link it to me, I'm happy to make a PR to do it.

@zoidyzoidzoid
Copy link
Copy Markdown
Author

Also, a caveat of the current Travis CI and Circle CI config, with my current changes which allow people to use Django 1.11 or 2, we're not testing both, which we should.

@zoidyzoidzoid
Copy link
Copy Markdown
Author

Whoops, I need to fix Django 1 support.

@reyang
Copy link
Copy Markdown
Contributor

reyang commented Mar 26, 2019

There are two CI error:

  1. code coverage.
  2. system test under python 3.6.
Traceback (most recent call last):
  File "/root/repo/.nox/sys-3-6/lib/python3.6/site-packages/django/utils/module_loading.py", line 20, in import_string
    return getattr(module, class_name)
AttributeError: module 'django.contrib.auth.middleware' has no attribute 'SessionAuthenticationMiddleware'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/repo/.nox/sys-3-6/lib/python3.6/site-packages/django/utils/autoreload.py", line 225, in wrapper
    fn(*args, **kwargs)
  File "/root/repo/.nox/sys-3-6/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 137, in inner_run
    handler = self.get_handler(*args, **options)
  File "/root/repo/.nox/sys-3-6/lib/python3.6/site-packages/django/contrib/staticfiles/management/commands/runserver.py", line 27, in get_handler
    handler = super().get_handler(*args, **options)
  File "/root/repo/.nox/sys-3-6/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 64, in get_handler
    return get_internal_wsgi_application()
  File "/root/repo/.nox/sys-3-6/lib/python3.6/site-packages/django/core/servers/basehttp.py", line 42, in get_internal_wsgi_application
    return get_wsgi_application()
  File "/root/repo/.nox/sys-3-6/lib/python3.6/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
    return WSGIHandler()
  File "/root/repo/.nox/sys-3-6/lib/python3.6/site-packages/django/core/handlers/wsgi.py", line 136, in __init__
    self.load_middleware()
  File "/root/repo/.nox/sys-3-6/lib/python3.6/site-packages/django/core/handlers/base.py", line 34, in load_middleware
    middleware = import_string(middleware_path)
  File "/root/repo/.nox/sys-3-6/lib/python3.6/site-packages/django/utils/module_loading.py", line 24, in import_string
    ) from err
ImportError: Module "django.contrib.auth.middleware" does not define a "SessionAuthenticationMiddleware" attribute/class

@reyang
Copy link
Copy Markdown
Contributor

reyang commented Mar 26, 2019

@zoidbergwill thanks for working on this! Regarding supporting Python3.7 in CI, and testing against both Django v1 and v2, let's address them separately. I think we need a major improvement on the CI process:

  1. Move to Travis CI.
  2. Speed up the CI by run virtual environments / tests in parallel.
  3. Provide ability to run different framework versions / combinations.
  4. Probably separate the core library test and extension test (for example 1: today some of the core lib coverage is relying on test cases in extensions, 2: we cannot detect missing dependencies in the core lib as extensions might install them during test run).

@c24t and I are working on GA release, we would definitely love to make improvements once we get out of the fire.

For this PR, you can focus on v2 test. I think the downside is that we're lacking CI protection for Django v1 for a while.

@zoidyzoidzoid
Copy link
Copy Markdown
Author

Sorry, I've been lazily pushing to test stuff on CircleCI because I was lazy to get a GOOGLE_APPLICATION_CREDENTIALS

@zoidyzoidzoid
Copy link
Copy Markdown
Author

I'm working on fixing the tests for Django 2.0+ and 1.11. I'll take a look at code coverage today or tomorrow too.

With the other work towards GA and other improvements, do we have issues for all of them? I'd be keen to help.

Problem 1 and 2 would be awesome to improve, since running tests faster is always useful, and having more thorough feedback about what's failing and passing in an overview would be great.

Re: problem 3

Provide ability to run different framework versions / combinations.

I saw #202 covers similar stuff, which would be great for the Django support too. Using a Travis CI matrix would help too.

For problem 4, that sounds useful to improve and fix, also making it easier to run certain system tests with less configuration would be great.

@reyang
Copy link
Copy Markdown
Contributor

reyang commented Mar 26, 2019

@zoidbergwill you can find all the GA related issues using the GA-Effort tag.

@reyang
Copy link
Copy Markdown
Contributor

reyang commented Apr 11, 2019

@zoidbergwill would you fix the coverage test failures? Thanks.

@zoidyzoidzoid
Copy link
Copy Markdown
Author

@reyang Sorry about the delay. I'll work on fixing the coverage test failures this week.

@zoidyzoidzoid
Copy link
Copy Markdown
Author

I'm struggling to get my tests to pass locally, and/or improving the coverage. From my code and tests it should be 100%. I think for some reason my tracer isn't initialising, but I'll have another look later.

@odeke-em
Copy link
Copy Markdown
Member

Kindly paging @timgraham to take a look too.

@timgraham
Copy link
Copy Markdown
Contributor

What's the policy on Django version support? setup.py has install_requires=[ 'Django >= 1.11.0'] but there's a recent commit (b1075ce) which mentions Django < 1.8!

At this point, Django < 1.11 is unsupported (no security updates) so install_requires is consistent with what I'd recommend. If that's the policy then we can simplify things and remove shims for older Django version support..

I'll do a detailed review after that decision and perhaps send a PR with cleanups that can precede this.

@reyang
Copy link
Copy Markdown
Contributor

reyang commented Jun 25, 2019

What's the policy on Django version support? setup.py has install_requires=[ 'Django >= 1.11.0'] but there's a recent commit (b1075ce) which mentions Django < 1.8!

We intend to follow https://www.djangoproject.com/download/#supported-versions - the Django version we support in OpenCensus should align with the supported (including LTS) Django versions.

@timgraham
Copy link
Copy Markdown
Contributor

Great. I created #694 with clean ups to remove support for Django < 1.11.

As far as I can tell, the remaining change is this PR don't really "add support for Django 2.0+" but merely add a new feature that requires Django 2.0+. I haven't seen anything obvious in the current code that won't work with Django 2.0, 2.1, or 2.2, but I'll check the tests with those versions.

@reyang
Copy link
Copy Markdown
Contributor

reyang commented Aug 6, 2019

This PR has been superseded by #697.

@reyang reyang closed this Aug 6, 2019
@timgraham
Copy link
Copy Markdown
Contributor

Not entirely. There's a a new feature in this PR (see the changes to the middleware) that hasn't been implemented in #697.

@reyang
Copy link
Copy Markdown
Contributor

reyang commented Aug 6, 2019

@timgraham you're right, there is database part. @zoidbergwill please rebase and reopen the PR if you plan to continue. Thanks!

@zoidyzoidzoid zoidyzoidzoid deleted the django-db-instrumentation branch August 27, 2019 12:11
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Django DB Instrumentation

8 participants