Skip to content

Recursive replace#2864

Merged
gaborbernat merged 4 commits intotox-dev:mainfrom
masenf:recursive-replace
Jan 16, 2023
Merged

Recursive replace#2864
gaborbernat merged 4 commits intotox-dev:mainfrom
masenf:recursive-replace

Conversation

@masenf
Copy link
Copy Markdown
Contributor

@masenf masenf commented Jan 16, 2023

Recursive Replace

Expand substitution expressions that result from a previous substitution expression replacement value (up to 100 times).

Add a new MatchRecursionError and stack depth tracking to avoid infinite recursion or python throwing a RecursionError and blowing up tox. Instead, log an error and take the last value as-is, which is what tox did prior to 4.3.0.

The behavior with this patch is more similar to pre-4.3.0 parser changes with the large caveat that replacement values are themselves expanded within their own context only, previously the value was replaced into the original string and the entire result was re-expanded, which led to some strange edge cases when expansions contained un-escaped special characters.

Fix #2863

Thanks for contribution

Please, make sure you address all the checklists (for details on how see
development documentation)!

  • ran the linter to address style issues (tox -e fix)
  • wrote descriptive pull request text
  • ensured there are test(s) validating the fix
  • added news fragment in docs/changelog folder
  • updated/extended the documentation

When a replacement references a replacement
in a non-testenv section it should also be expanded
Expand substitution expressions that result from a previous subsitution
expression replacement value (up to 100 times).

Fix tox-dev#2863
REPLACE_START = "{"
REPLACE_END = "}"
BACKSLASH_ESCAPE_CHARS = ["\\", ARG_DELIMITER, REPLACE_START, REPLACE_END, "[", "]"]
MAX_REPLACE_DEPTH = 100
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test for this with a circular reference of length 5, and one with 101?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a test that checks different recursion levels and asserts that the log message is triggered and that expansion stops at the correct level.

Create a long chain of substitution values and assert that
they stop being processed after some time.
@gaborbernat gaborbernat merged commit 99b849b into tox-dev:main Jan 16, 2023
@masenf masenf deleted the recursive-replace branch January 16, 2023 04:46
descope bot referenced this pull request in descope/django-descope Jan 30, 2023
This PR contains the following updates:

| Package | Type | Update | Change | Pending |
|---|---|---|---|---|
| [tox](https://togithub.com/tox-dev/tox)
([changelog](https://tox.wiki/en/latest/changelog.html)) | dev | patch |
`4.3.0` -> `4.3.1` | `4.4.2` (+6) |

---

### Release Notes

<details>
<summary>tox-dev/tox</summary>

### [`v4.3.1`](https://togithub.com/tox-dev/tox/releases/tag/4.3.1)

[Compare Source](https://togithub.com/tox-dev/tox/compare/4.3.0...4.3.1)

#### What's Changed

- Recursive replace by [@&#8203;masenf](https://togithub.com/masenf) in
[https://github.com/tox-dev/tox/pull/2864](https://togithub.com/tox-dev/tox/pull/2864)

**Full Changelog**: tox-dev/tox@4.3.0...4.3.1

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC4xMDEuMCIsInVwZGF0ZWRJblZlciI6IjM0LjEwMS4wIn0=-->

Co-authored-by: descope[bot] <descope[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tox 4.3.0 breaks substitution for values from other sections

2 participants