Skip to content

chore(appsec): don't load appsec modules (iast) [backport 2.19]#12350

Merged
avara1986 merged 14 commits into
2.19from
backport-11931-to-2.19
Feb 18, 2025
Merged

chore(appsec): don't load appsec modules (iast) [backport 2.19]#12350
avara1986 merged 14 commits into
2.19from
backport-11931-to-2.19

Conversation

@avara1986

@avara1986 avara1986 commented Feb 14, 2025

Copy link
Copy Markdown
Member

backports
#11931
#12184
#12212
#12320

to 2.19

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

Partial migration of the tests in `appsec/integrations/`

- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

- [x] Reviewer has checked that all the criteria below are met
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

(cherry picked from commit c46c302)
@avara1986 avara1986 added changelog/no-changelog A changelog entry is not required for this PR. ASM Application Security Monitoring labels Feb 14, 2025
@github-actions

github-actions Bot commented Feb 14, 2025

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

.riot/requirements/5e31227.txt                                          @DataDog/apm-python
.riot/requirements/628e8fe.txt                                          @DataDog/apm-python
.riot/requirements/8dd53b1.txt                                          @DataDog/apm-python
.riot/requirements/968fdc9.txt                                          @DataDog/apm-python
releasenotes/notes/ensure_no_appsec_loading-8ce46c58d6ecf81f.yaml       @DataDog/apm-python
tests/appsec/integrations/django_tests/__init__.py                      @DataDog/asm-python
tests/appsec/integrations/flask_tests/__init__.py                       @DataDog/asm-python
tests/appsec/integrations/flask_tests/mini.py                           @DataDog/asm-python
tests/appsec/integrations/flask_tests/test_appsec_loading_modules.py    @DataDog/asm-python
.circleci/config.templ.yml                                              @DataDog/python-guild @DataDog/apm-core-python
ddtrace/_monkey.py                                                      @DataDog/apm-core-python
ddtrace/appsec/__init__.py                                              @DataDog/asm-python
ddtrace/appsec/_asm_request_context.py                                  @DataDog/asm-python
ddtrace/appsec/_common_module_patches.py                                @DataDog/asm-python
ddtrace/appsec/_constants.py                                            @DataDog/asm-python
ddtrace/appsec/_ddwaf/ddwaf_types.py                                    @DataDog/asm-python
ddtrace/appsec/_iast/__init__.py                                        @DataDog/asm-python
ddtrace/appsec/_iast/_handlers.py                                       @DataDog/asm-python
ddtrace/appsec/_iast/_iast_request_context.py                           @DataDog/asm-python
ddtrace/appsec/_iast/_pytest_plugin.py                                  @DataDog/asm-python
ddtrace/appsec/_iast/taint_sinks/insecure_cookie.py                     @DataDog/asm-python
ddtrace/appsec/_processor.py                                            @DataDog/asm-python
ddtrace/appsec/_utils.py                                                @DataDog/asm-python
ddtrace/bootstrap/preload.py                                            @DataDog/apm-core-python
ddtrace/contrib/dbapi/__init__.py                                       @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/dbapi_async/__init__.py                                 @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/fastapi/patch.py                               @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/httplib/patch.py                               @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/langchain/patch.py                             @DataDog/ml-observability
ddtrace/contrib/internal/mysql/patch.py                                 @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/mysqldb/patch.py                               @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/requests/patch.py                              @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/sqlalchemy/patch.py                            @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/sqlite3/patch.py                               @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/starlette/patch.py                             @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/subprocess/patch.py                            @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/urllib/patch.py                                @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/urllib3/patch.py                               @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/webbrowser/patch.py                            @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/pytest/_plugin_v2.py                                    @DataDog/ci-app-libraries
ddtrace/contrib/pytest/plugin.py                                        @DataDog/ci-app-libraries
ddtrace/settings/asm.py                                                 @DataDog/asm-python
hatch.toml                                                              @DataDog/python-guild
riotfile.py                                                             @DataDog/apm-python
tests/appsec/app.py                                                     @DataDog/asm-python
tests/appsec/iast/fixtures/integration/main_configure.py                @DataDog/asm-python
tests/appsec/suitespec.yml                                              @DataDog/asm-python
tests/contrib/dbapi/test_dbapi_appsec.py                                @DataDog/asm-python
tests/contrib/django/test_django_appsec_iast.py                         @DataDog/asm-python
tests/contrib/flask/test_flask_appsec_iast.py                           @DataDog/asm-python
tests/contrib/subprocess/test_subprocess.py                             @DataDog/asm-python
tests/contrib/subprocess/test_subprocess_patch.py                       @DataDog/asm-python
tests/utils.py                                                          @DataDog/python-guild
tests/appsec/integrations/flask_tests/module_with_import_errors.py      @DataDog/asm-python
tests/appsec/integrations/flask_tests/test_flask_remoteconfig.py        @DataDog/asm-python
tests/appsec/integrations/flask_tests/test_gunicorn_handlers.py         @DataDog/asm-python
tests/appsec/integrations/flask_tests/test_iast_flask_entrypoint_iast_patches.py  @DataDog/asm-python
tests/appsec/integrations/flask_tests/test_iast_flask_patching.py       @DataDog/asm-python
tests/appsec/integrations/flask_tests/test_iast_flask_telemetry.py      @DataDog/asm-python
tests/appsec/integrations/flask_tests/test_iast_langchain.py            @DataDog/asm-python
tests/appsec/integrations/flask_tests/test_iast_psycopg2.py             @DataDog/asm-python
tests/appsec/integrations/flask_tests/utils.py                          @DataDog/asm-python

@pr-commenter

pr-commenter Bot commented Feb 14, 2025

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2025-02-17 15:36:17

Comparing candidate commit eca3d0a in PR branch backport-11931-to-2.19 with baseline commit 2662f8e in branch 2.19.

Found 1 performance improvements and 2 performance regressions! Performance is the same for 391 metrics, 2 unstable metrics.

scenario:iast_aspects-ospathdirname_aspect

  • 🟥 execution_time [+428.042ns; +495.490ns] or [+11.497%; +13.308%]

scenario:iast_aspects-ospathsplit_aspect

  • 🟥 execution_time [+404.786ns; +467.757ns] or [+10.510%; +12.146%]

scenario:iast_aspects-ospathsplitext_aspect

  • 🟩 execution_time [-347.507ns; -302.303ns] or [-8.782%; -7.640%]

@avara1986 avara1986 marked this pull request as ready for review February 17, 2025 07:23
@avara1986 avara1986 requested review from a team as code owners February 17, 2025 07:23
github-actions Bot and others added 2 commits February 17, 2025 08:33
Backport 4f0bcb5 from #12102 to 2.20.

Depending of the timing, libddwaf loading process could create triggers
that would create loops in our instrumentation.
From what I investigated:
- if loaded too early, it could have bad interactions with gevent.
- if loaded too late, it could be self instrumented by the tracer,
creating a loop, as ctypes is using Popen and subprocess.

while keeping the late loading introduced by 2 previous PRs
- #11987
- #12013
this PR introduced a mechanism to bypass tracer instrumentation during
ctypes loading, to avoid a possible loop that would prevent the WAF to
be loaded.

- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

- [x] Reviewer has checked that all the criteria below are met
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Co-authored-by: Christophe Papazian <114495376+christophe-papazian@users.noreply.github.com>
Comment thread ddtrace/contrib/internal/requests/patch.py
@avara1986 avara1986 requested a review from a team as a code owner February 17, 2025 09:22
Comment thread ddtrace/appsec/_iast/_iast_request_context.py
Comment thread ddtrace/appsec/_iast/taint_sinks/insecure_cookie.py
Comment thread ddtrace/contrib/internal/requests/patch.py
Comment thread ddtrace/contrib/internal/subprocess/patch.py
@avara1986 avara1986 changed the title chore(appsec): migrate test integrations to gitlab [backport 2.19] chore(appsec): don't load appsec modules (iast) [backport 2.19] Feb 17, 2025
@avara1986 avara1986 merged commit 90df752 into 2.19 Feb 18, 2025
@avara1986 avara1986 deleted the backport-11931-to-2.19 branch February 18, 2025 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ASM Application Security Monitoring changelog/no-changelog A changelog entry is not required for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants