fix(internal): crash in ContextVar [backport 4.7]#17607
Conversation
Codeowners resolved as |
This comment has been minimized.
This comment has been minimized.
Performance SLOsComparing candidate kowalski/backport-17407-to-4.7 (55b40d4) with baseline 4.7 (9a070ef) 📈 Performance Regressions (2 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 103.274µs (SLO: <130.000µs 📉 -20.6%) vs baseline: +2.6% Memory: ✅ 43.913MB (SLO: <46.000MB -4.5%) vs baseline: +5.2% ✅ add_inplace_aspectTime: ✅ 103.020µs (SLO: <130.000µs 📉 -20.8%) vs baseline: ~same Memory: ✅ 43.814MB (SLO: <46.000MB -4.8%) vs baseline: +4.7% ✅ add_inplace_noaspectTime: ✅ 28.329µs (SLO: <40.000µs 📉 -29.2%) vs baseline: +0.5% Memory: ✅ 43.724MB (SLO: <46.000MB -4.9%) vs baseline: +4.5% ✅ add_noaspectTime: ✅ 48.995µs (SLO: <70.000µs 📉 -30.0%) vs baseline: -1.2% Memory: ✅ 43.840MB (SLO: <46.000MB -4.7%) vs baseline: +4.6% ✅ bytearray_aspectTime: ✅ 253.433µs (SLO: <400.000µs 📉 -36.6%) vs baseline: -0.4% Memory: ✅ 43.591MB (SLO: <46.000MB -5.2%) vs baseline: +4.5% ✅ bytearray_extend_aspectTime: ✅ 662.058µs (SLO: <800.000µs 📉 -17.2%) vs baseline: +0.5% Memory: ✅ 43.815MB (SLO: <46.000MB -4.8%) vs baseline: +4.9% ✅ bytearray_extend_noaspectTime: ✅ 276.056µs (SLO: <400.000µs 📉 -31.0%) vs baseline: -1.1% Memory: ✅ 43.863MB (SLO: <46.000MB -4.6%) vs baseline: +4.9% ✅ bytearray_noaspectTime: ✅ 144.709µs (SLO: <300.000µs 📉 -51.8%) vs baseline: +1.4% Memory: ✅ 43.837MB (SLO: <46.000MB -4.7%) vs baseline: +4.9% ✅ bytes_aspectTime: ✅ 222.283µs (SLO: <300.000µs 📉 -25.9%) vs baseline: +0.4% Memory: ✅ 43.902MB (SLO: <46.000MB -4.6%) vs baseline: +5.0% ✅ bytes_noaspectTime: ✅ 137.285µs (SLO: <200.000µs 📉 -31.4%) vs baseline: -0.2% Memory: ✅ 43.901MB (SLO: <46.000MB -4.6%) vs baseline: +4.9% ✅ bytesio_aspectTime: ✅ 3.874ms (SLO: <5.000ms 📉 -22.5%) vs baseline: +0.1% Memory: ✅ 43.891MB (SLO: <46.000MB -4.6%) vs baseline: +4.9% ✅ bytesio_noaspectTime: ✅ 321.642µs (SLO: <420.000µs 📉 -23.4%) vs baseline: +0.7% Memory: ✅ 43.765MB (SLO: <46.000MB -4.9%) vs baseline: +4.5% ✅ capitalize_aspectTime: ✅ 89.431µs (SLO: <300.000µs 📉 -70.2%) vs baseline: -0.8% Memory: ✅ 43.903MB (SLO: <46.000MB -4.6%) vs baseline: +5.0% ✅ capitalize_noaspectTime: ✅ 273.078µs (SLO: <300.000µs -9.0%) vs baseline: +5.9% Memory: ✅ 43.802MB (SLO: <46.000MB -4.8%) vs baseline: +4.9% ✅ casefold_aspectTime: ✅ 89.841µs (SLO: <500.000µs 📉 -82.0%) vs baseline: ~same Memory: ✅ 43.771MB (SLO: <46.000MB -4.8%) vs baseline: +4.6% ✅ casefold_noaspectTime: ✅ 315.582µs (SLO: <500.000µs 📉 -36.9%) vs baseline: -0.1% Memory: ✅ 43.857MB (SLO: <46.000MB -4.7%) vs baseline: +5.1% ✅ decode_aspectTime: ✅ 87.300µs (SLO: <100.000µs 📉 -12.7%) vs baseline: +0.5% Memory: ✅ 43.850MB (SLO: <46.000MB -4.7%) vs baseline: +4.9% ✅ decode_noaspectTime: ✅ 155.205µs (SLO: <210.000µs 📉 -26.1%) vs baseline: -2.1% Memory: ✅ 43.845MB (SLO: <46.000MB -4.7%) vs baseline: +5.1% ✅ encode_aspectTime: ✅ 84.682µs (SLO: <200.000µs 📉 -57.7%) vs baseline: +0.2% Memory: ✅ 43.783MB (SLO: <46.000MB -4.8%) vs baseline: +4.7% ✅ encode_noaspectTime: ✅ 143.994µs (SLO: <200.000µs 📉 -28.0%) vs baseline: -0.7% Memory: ✅ 43.857MB (SLO: <46.000MB -4.7%) vs baseline: +5.1% ✅ format_aspectTime: ✅ 14.670ms (SLO: <19.200ms 📉 -23.6%) vs baseline: +0.1% Memory: ✅ 43.986MB (SLO: <46.000MB -4.4%) vs baseline: +4.9% ✅ format_map_aspectTime: ✅ 16.419ms (SLO: <21.500ms 📉 -23.6%) vs baseline: -0.4% Memory: ✅ 44.054MB (SLO: <46.000MB -4.2%) vs baseline: +4.9% ✅ format_map_noaspectTime: ✅ 378.346µs (SLO: <500.000µs 📉 -24.3%) vs baseline: -1.0% Memory: ✅ 43.810MB (SLO: <46.000MB -4.8%) vs baseline: +4.8% ✅ format_noaspectTime: ✅ 316.216µs (SLO: <500.000µs 📉 -36.8%) vs baseline: -0.2% Memory: ✅ 43.829MB (SLO: <46.000MB -4.7%) vs baseline: +4.5% ✅ index_aspectTime: ✅ 136.233µs (SLO: <300.000µs 📉 -54.6%) vs baseline: 📈 +12.2% Memory: ✅ 43.873MB (SLO: <46.000MB -4.6%) vs baseline: +5.0% ✅ index_noaspectTime: ✅ 40.467µs (SLO: <300.000µs 📉 -86.5%) vs baseline: -0.6% Memory: ✅ 43.885MB (SLO: <46.000MB -4.6%) vs baseline: +4.9% ✅ join_aspectTime: ✅ 221.034µs (SLO: <300.000µs 📉 -26.3%) vs baseline: +1.6% Memory: ✅ 43.796MB (SLO: <46.000MB -4.8%) vs baseline: +4.7% ✅ join_noaspectTime: ✅ 144.889µs (SLO: <300.000µs 📉 -51.7%) vs baseline: +0.5% Memory: ✅ 43.808MB (SLO: <46.000MB -4.8%) vs baseline: +4.8% ✅ ljust_aspectTime: ✅ 589.714µs (SLO: <700.000µs 📉 -15.8%) vs baseline: 📈 +14.9% Memory: ✅ 43.942MB (SLO: <46.000MB -4.5%) vs baseline: +4.9% ✅ ljust_noaspectTime: ✅ 265.737µs (SLO: <300.000µs 📉 -11.4%) vs baseline: +1.4% Memory: ✅ 43.919MB (SLO: <46.000MB -4.5%) vs baseline: +5.1% ✅ lower_aspectTime: ✅ 311.613µs (SLO: <500.000µs 📉 -37.7%) vs baseline: -0.8% Memory: ✅ 43.804MB (SLO: <46.000MB -4.8%) vs baseline: +4.6% ✅ lower_noaspectTime: ✅ 246.041µs (SLO: <300.000µs 📉 -18.0%) vs baseline: +1.8% Memory: ✅ 43.825MB (SLO: <46.000MB -4.7%) vs baseline: +5.0% ✅ lstrip_aspectTime: ✅ 0.283ms (SLO: <3.000ms 📉 -90.6%) vs baseline: +0.8% Memory: ✅ 43.944MB (SLO: <46.000MB -4.5%) vs baseline: +5.3% ✅ lstrip_noaspectTime: ✅ 0.178ms (SLO: <3.000ms 📉 -94.1%) vs baseline: -0.2% Memory: ✅ 43.809MB (SLO: <46.000MB -4.8%) vs baseline: +4.8% ✅ modulo_aspectTime: ✅ 14.315ms (SLO: <18.750ms 📉 -23.7%) vs baseline: ~same Memory: ✅ 44.107MB (SLO: <46.000MB -4.1%) vs baseline: +5.1% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.903ms (SLO: <19.350ms 📉 -23.0%) vs baseline: +0.4% Memory: ✅ 43.960MB (SLO: <46.000MB -4.4%) vs baseline: +4.7% ✅ modulo_aspect_for_bytesTime: ✅ 14.468ms (SLO: <18.900ms 📉 -23.4%) vs baseline: -0.2% Memory: ✅ 44.054MB (SLO: <46.000MB -4.2%) vs baseline: +4.8% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.622ms (SLO: <19.150ms 📉 -23.6%) vs baseline: -0.3% Memory: ✅ 43.951MB (SLO: <46.000MB -4.5%) vs baseline: +4.6% ✅ modulo_noaspectTime: ✅ 0.369ms (SLO: <3.000ms 📉 -87.7%) vs baseline: +0.2% Memory: ✅ 43.865MB (SLO: <46.000MB -4.6%) vs baseline: +5.1% ✅ replace_aspectTime: ✅ 18.408ms (SLO: <24.000ms 📉 -23.3%) vs baseline: -0.2% Memory: ✅ 44.027MB (SLO: <46.000MB -4.3%) vs baseline: +4.8% ✅ replace_noaspectTime: ✅ 286.580µs (SLO: <400.000µs 📉 -28.4%) vs baseline: +0.2% Memory: ✅ 43.785MB (SLO: <46.000MB -4.8%) vs baseline: +4.8% ✅ repr_aspectTime: ✅ 329.997µs (SLO: <420.000µs 📉 -21.4%) vs baseline: +0.8% Memory: ✅ 43.913MB (SLO: <46.000MB -4.5%) vs baseline: +4.9% ✅ repr_noaspectTime: ✅ 46.889µs (SLO: <90.000µs 📉 -47.9%) vs baseline: +0.3% Memory: ✅ 43.810MB (SLO: <46.000MB -4.8%) vs baseline: +4.4% ✅ rstrip_aspectTime: ✅ 395.221µs (SLO: <500.000µs 📉 -21.0%) vs baseline: +1.3% Memory: ✅ 43.938MB (SLO: <46.000MB -4.5%) vs baseline: +4.9% ✅ rstrip_noaspectTime: ✅ 188.007µs (SLO: <300.000µs 📉 -37.3%) vs baseline: +0.8% Memory: ✅ 43.898MB (SLO: <46.000MB -4.6%) vs baseline: +4.9% ✅ slice_aspectTime: ✅ 184.963µs (SLO: <300.000µs 📉 -38.3%) vs baseline: +0.6% Memory: ✅ 43.859MB (SLO: <46.000MB -4.7%) vs baseline: +5.0% ✅ slice_noaspectTime: ✅ 54.117µs (SLO: <90.000µs 📉 -39.9%) vs baseline: -0.3% Memory: ✅ 43.828MB (SLO: <46.000MB -4.7%) vs baseline: +4.4% ✅ stringio_aspectTime: ✅ 3.913ms (SLO: <5.000ms 📉 -21.7%) vs baseline: ~same Memory: ✅ 43.787MB (SLO: <46.000MB -4.8%) vs baseline: +5.0% ✅ stringio_noaspectTime: ✅ 396.800µs (SLO: <500.000µs 📉 -20.6%) vs baseline: 📈 +11.1% Memory: ✅ 43.849MB (SLO: <46.000MB -4.7%) vs baseline: +4.8% ✅ strip_aspectTime: ✅ 279.800µs (SLO: <350.000µs 📉 -20.1%) vs baseline: -0.8% Memory: ✅ 43.869MB (SLO: <46.000MB -4.6%) vs baseline: +4.8% ✅ strip_noaspectTime: ✅ 179.597µs (SLO: <240.000µs 📉 -25.2%) vs baseline: -0.5% Memory: ✅ 43.871MB (SLO: <46.000MB -4.6%) vs baseline: +5.1% ✅ swapcase_aspectTime: ✅ 350.415µs (SLO: <500.000µs 📉 -29.9%) vs baseline: +0.7% Memory: ✅ 43.691MB (SLO: <46.000MB -5.0%) vs baseline: +4.7% ✅ swapcase_noaspectTime: ✅ 278.845µs (SLO: <400.000µs 📉 -30.3%) vs baseline: +0.6% Memory: ✅ 43.945MB (SLO: <46.000MB -4.5%) vs baseline: +5.4% ✅ title_aspectTime: ✅ 337.434µs (SLO: <500.000µs 📉 -32.5%) vs baseline: +1.7% Memory: ✅ 43.764MB (SLO: <46.000MB -4.9%) vs baseline: +4.7% ✅ title_noaspectTime: ✅ 267.165µs (SLO: <400.000µs 📉 -33.2%) vs baseline: -1.5% Memory: ✅ 43.731MB (SLO: <46.000MB -4.9%) vs baseline: +4.6% ✅ translate_aspectTime: ✅ 501.684µs (SLO: <700.000µs 📉 -28.3%) vs baseline: -0.5% Memory: ✅ 43.833MB (SLO: <46.000MB -4.7%) vs baseline: +4.9% ✅ translate_noaspectTime: ✅ 431.971µs (SLO: <500.000µs 📉 -13.6%) vs baseline: -2.4% Memory: ✅ 43.831MB (SLO: <46.000MB -4.7%) vs baseline: +4.8% ✅ upper_aspectTime: ✅ 309.729µs (SLO: <500.000µs 📉 -38.1%) vs baseline: -0.2% Memory: ✅ 43.723MB (SLO: <46.000MB -5.0%) vs baseline: +4.6% ✅ upper_noaspectTime: ✅ 243.843µs (SLO: <400.000µs 📉 -39.0%) vs baseline: +2.0% Memory: ✅ 43.853MB (SLO: <46.000MB -4.7%) vs baseline: +4.8% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 508.405µs (SLO: <700.000µs 📉 -27.4%) vs baseline: 📈 +19.7% Memory: ✅ 43.731MB (SLO: <46.000MB -4.9%) vs baseline: +5.1% ✅ ospathbasename_noaspectTime: ✅ 435.127µs (SLO: <700.000µs 📉 -37.8%) vs baseline: -0.2% Memory: ✅ 43.568MB (SLO: <46.000MB -5.3%) vs baseline: +4.2% ✅ ospathjoin_aspectTime: ✅ 625.247µs (SLO: <700.000µs 📉 -10.7%) vs baseline: +0.4% Memory: ✅ 43.606MB (SLO: <46.000MB -5.2%) vs baseline: +4.9% ✅ ospathjoin_noaspectTime: ✅ 634.360µs (SLO: <700.000µs -9.4%) vs baseline: +0.8% Memory: ✅ 43.555MB (SLO: <46.000MB -5.3%) vs baseline: +4.9% ✅ ospathnormcase_aspectTime: ✅ 353.204µs (SLO: <700.000µs 📉 -49.5%) vs baseline: +0.1% Memory: ✅ 43.681MB (SLO: <46.000MB -5.0%) vs baseline: +5.3% ✅ ospathnormcase_noaspectTime: ✅ 365.025µs (SLO: <700.000µs 📉 -47.9%) vs baseline: +0.5% Memory: ✅ 43.913MB (SLO: <46.000MB -4.5%) vs baseline: +4.8% ✅ ospathsplit_aspectTime: ✅ 495.130µs (SLO: <700.000µs 📉 -29.3%) vs baseline: +0.9% Memory: ✅ 43.608MB (SLO: <46.000MB -5.2%) vs baseline: +4.8% ✅ ospathsplit_noaspectTime: ✅ 499.564µs (SLO: <700.000µs 📉 -28.6%) vs baseline: +0.5% Memory: ✅ 43.584MB (SLO: <46.000MB -5.3%) vs baseline: +5.0% ✅ ospathsplitdrive_aspectTime: ✅ 371.787µs (SLO: <700.000µs 📉 -46.9%) vs baseline: -0.1% Memory: ✅ 43.869MB (SLO: <46.000MB -4.6%) vs baseline: +5.8% ✅ ospathsplitdrive_noaspectTime: ✅ 73.117µs (SLO: <700.000µs 📉 -89.6%) vs baseline: -0.5% Memory: ✅ 43.586MB (SLO: <46.000MB -5.2%) vs baseline: +4.8% ✅ ospathsplitext_aspectTime: ✅ 461.820µs (SLO: <700.000µs 📉 -34.0%) vs baseline: -0.6% Memory: ✅ 43.625MB (SLO: <46.000MB -5.2%) vs baseline: +5.1% ✅ ospathsplitext_noaspectTime: ✅ 469.553µs (SLO: <700.000µs 📉 -32.9%) vs baseline: +0.4% Memory: ✅ 43.530MB (SLO: <46.000MB -5.4%) vs baseline: +3.8% 🟡 Near SLO Breach (3 suites)🟡 djangosimple - 30/30✅ appsecTime: ✅ 19.637ms (SLO: <22.300ms 📉 -11.9%) vs baseline: +0.3% Memory: ✅ 69.521MB (SLO: <73.500MB -5.4%) vs baseline: +4.8% ✅ exception-replay-enabledTime: ✅ 1.323ms (SLO: <1.450ms -8.8%) vs baseline: -0.6% Memory: ✅ 67.731MB (SLO: <71.500MB -5.3%) vs baseline: +4.7% ✅ iastTime: ✅ 19.677ms (SLO: <22.250ms 📉 -11.6%) vs baseline: -0.1% Memory: ✅ 69.658MB (SLO: <75.000MB -7.1%) vs baseline: +5.0% ✅ profilerTime: ✅ 15.203ms (SLO: <16.550ms -8.1%) vs baseline: -0.2% Memory: ✅ 60.183MB (SLO: <61.000MB 🟡 -1.3%) vs baseline: +4.8% ✅ resource-renamingTime: ✅ 19.689ms (SLO: <21.750ms -9.5%) vs baseline: +0.7% Memory: ✅ 69.540MB (SLO: <73.500MB -5.4%) vs baseline: +5.0% ✅ span-code-originTime: ✅ 20.307ms (SLO: <28.200ms 📉 -28.0%) vs baseline: +1.9% Memory: ✅ 69.595MB (SLO: <75.000MB -7.2%) vs baseline: +5.0% ✅ tracerTime: ✅ 19.621ms (SLO: <21.750ms -9.8%) vs baseline: -0.3% Memory: ✅ 69.481MB (SLO: <75.000MB -7.4%) vs baseline: +4.7% ✅ tracer-and-profilerTime: ✅ 21.045ms (SLO: <23.500ms 📉 -10.4%) vs baseline: -0.2% Memory: ✅ 71.388MB (SLO: <75.000MB -4.8%) vs baseline: +4.7% ✅ tracer-dont-create-db-spansTime: ✅ 19.692ms (SLO: <21.500ms -8.4%) vs baseline: -0.2% Memory: ✅ 69.540MB (SLO: <75.000MB -7.3%) vs baseline: +4.8% ✅ tracer-minimalTime: ✅ 16.878ms (SLO: <17.500ms -3.6%) vs baseline: ~same Memory: ✅ 69.560MB (SLO: <75.000MB -7.3%) vs baseline: +4.8% ✅ tracer-nativeTime: ✅ 19.671ms (SLO: <21.750ms -9.6%) vs baseline: -0.1% Memory: ✅ 69.442MB (SLO: <72.500MB -4.2%) vs baseline: +4.7% ✅ tracer-no-cachesTime: ✅ 17.644ms (SLO: <19.650ms 📉 -10.2%) vs baseline: ~same Memory: ✅ 69.560MB (SLO: <75.000MB -7.3%) vs baseline: +4.8% ✅ tracer-no-databasesTime: ✅ 19.517ms (SLO: <20.100ms -2.9%) vs baseline: +0.4% Memory: ✅ 69.442MB (SLO: <75.000MB -7.4%) vs baseline: +4.7% ✅ tracer-no-middlewareTime: ✅ 19.459ms (SLO: <21.500ms -9.5%) vs baseline: +0.4% Memory: ✅ 69.403MB (SLO: <75.000MB -7.5%) vs baseline: +4.7% ✅ tracer-no-templatesTime: ✅ 19.683ms (SLO: <22.000ms 📉 -10.5%) vs baseline: +0.5% Memory: ✅ 69.502MB (SLO: <73.500MB -5.4%) vs baseline: +4.8% 🟡 flasksimple - 18/18✅ appsec-getTime: ✅ 3.374ms (SLO: <4.750ms 📉 -29.0%) vs baseline: -0.1% Memory: ✅ 56.521MB (SLO: <66.500MB 📉 -15.0%) vs baseline: +4.9% ✅ appsec-postTime: ✅ 2.867ms (SLO: <6.750ms 📉 -57.5%) vs baseline: +0.3% Memory: ✅ 56.595MB (SLO: <66.500MB 📉 -14.9%) vs baseline: +4.9% ✅ appsec-telemetryTime: ✅ 3.380ms (SLO: <4.750ms 📉 -28.8%) vs baseline: +0.7% Memory: ✅ 56.473MB (SLO: <66.500MB 📉 -15.1%) vs baseline: +4.6% ✅ debuggerTime: ✅ 1.879ms (SLO: <2.000ms -6.0%) vs baseline: ~same Memory: ✅ 49.298MB (SLO: <51.500MB -4.3%) vs baseline: +4.7% ✅ iast-getTime: ✅ 1.874ms (SLO: <2.000ms -6.3%) vs baseline: ~same Memory: ✅ 45.946MB (SLO: <49.000MB -6.2%) vs baseline: +4.7% ✅ profilerTime: ✅ 1.912ms (SLO: <2.100ms -8.9%) vs baseline: -0.4% Memory: ✅ 52.461MB (SLO: <53.500MB 🟡 -1.9%) vs baseline: +5.2% ✅ resource-renamingTime: ✅ 3.334ms (SLO: <3.650ms -8.7%) vs baseline: -0.3% Memory: ✅ 56.618MB (SLO: <60.000MB -5.6%) vs baseline: +4.9% ✅ tracerTime: ✅ 3.378ms (SLO: <3.650ms -7.5%) vs baseline: +0.6% Memory: ✅ 56.543MB (SLO: <60.000MB -5.8%) vs baseline: +4.8% ✅ tracer-nativeTime: ✅ 3.355ms (SLO: <3.650ms -8.1%) vs baseline: ~same Memory: ✅ 56.528MB (SLO: <60.000MB -5.8%) vs baseline: +4.6% 🟡 recursivecomputation - 8/8✅ deepTime: ✅ 311.385ms (SLO: <320.950ms -3.0%) vs baseline: ~same Memory: ✅ 37.513MB (SLO: <38.750MB -3.2%) vs baseline: +5.0% ✅ deep-profiledTime: ✅ 328.583ms (SLO: <359.150ms -8.5%) vs baseline: -0.2% Memory: ✅ 43.549MB (SLO: <46.000MB -5.3%) vs baseline: +4.9% ✅ mediumTime: ✅ 7.295ms (SLO: <7.400ms 🟡 -1.4%) vs baseline: -0.2% Memory: ✅ 36.412MB (SLO: <38.000MB -4.2%) vs baseline: +5.2% ✅ shallowTime: ✅ 1.020ms (SLO: <1.050ms -2.9%) vs baseline: +1.6% Memory: ✅ 36.372MB (SLO: <38.000MB -4.3%) vs baseline: +5.1%
|
This PR fixes a crash in `ddtrace` discovered by Crash Tracking where we crash in garbage collection. While this definitely does sound like a Python bug and not a `ddtrace` one, we need to fix this for existing versions of Python and in the meantime until Python fixes it itself.
The crash is in `PyObject_GC_UnTrack`, called from the dealloc path of a HAMT node that `PyContextVar_Set` is trying to free via `Py_DECREF`. If that HAMT node's memory is already unmapped, the `PyObject_GC_UnTrack` write faults.
```
Error UnixSignal: Process terminated with SEGV_MAPERR (SIGSEGV)
```
The crash happens in `PyObject_GC_UnTrack`, called from the dealloc path of a HAMT node that `PyContextVar_Set` is trying to free via `Py_DECREF`. If that HAMT node's memory is already unmapped, the `PyObject_GC_UnTrack` write faults.
`BaseWrappingContext.__enter__` does this...
```python
def __enter__(self) -> "BaseWrappingContext":
token: Token = self._storage.set({}) # 1. create new HAMT entry
self._storage.get()["__dd_wrapping_context_token__"] = token # 2. store Token in dict
return self
```
- Step 1 stores a fresh `{}` dict in the context's HAMT under `self._storage`
- Step 2 puts the `Token` returned by `set()` into that same dict.
A `Token` object holds `tok_ctx`, which is a strong reference to the `PyContext` that was active when `set` was called (i.e. `ts->context`).
This creates the following reference cycle:
```
ts->context
└─▶ ctx_vars (HAMT root)
└─▶ ... HAMT nodes ...
└─▶ dict {"__dd_wrapping_context_token__": token}
└─▶ token (PyContextToken)
└─▶ tok_ctx ────────────────▶ ts->context ⟲
```
Now, `PyContextVar_Set` is **not atomic**. Its simplified CPython implementation is:
```c
static PyObject *
contextvar_set(PyContextVar *var, PyObject *val) {
PyContext *ts_ctx = ts->context;
PyObject *new_hamt = _PyHamt_Assoc(ts_ctx->ctx_vars, var, val);
Py_DECREF(ts_ctx->ctx_vars); // ← old HAMT root refcount decremented here
ts_ctx->ctx_vars = new_hamt;
...
}
```
At `Py_DECREF(ts_ctx->ctx_vars)`:
1. The old HAMT root's refcount drops. If it reaches 0, `_Py_Dealloc` is called, which calls `hamt_tp_dealloc` which calls `PyObject_GC_UnTrack(old_hamt)`.
2. Inside `_Py_Dealloc` the old HAMT decrefs its children (intermediate nodes).
3. Those node decrefs cascade. A leaf node that is now only referenced by the *old* HAMT (not the new one) reaches refcount 0 and is also freed, its memory potentially returned to the OS.
4. Meanwhile `ts_ctx->ctx_vars` still points to the (now freed) old root until line 3 of the snippet above executes.
5. If the Python cyclic GC fires at the wrong moment -- triggered by another allocation during the HAMT rebuild -- it may traverse the cycle above, see the Token's `tok_ctx` pointing at `ts->context`, and interfere with refcounts, causing a node to be collected prematurely.
6. `PyObject_GC_UnTrack` is then called on a node whose memory is already unmapped which results in **SEGV_MAPERR**.
Store the **previous value** of `_storage` in the dict instead of the `Token`. Restore it with `set` rather than `reset`. In other words:
```python
def __enter__(self) -> "BaseWrappingContext":
prev = self._storage.get()
self._storage.set({"__dd_wrapping_context_prev__": prev})
# Token returned by set() is discarded immediately → refcount 0 → freed
return self
def _pop_storage(self) -> dict[str, t.Any]:
storage = self._storage.get()
self._storage.set(storage.pop("__dd_wrapping_context_prev__"))
return storage
```
The `Token` object is now ephemeral: it is returned by `set`, never stored anywhere, and freed immediately when the local expression is evaluated. Nothing inside the HAMT ever holds a `Token`, so the cycle is gone.
`reset(token)` and `set(prev_value)` are equivalent for this use-case:
- Both restore `_storage` to its value before `__enter__` was called.
- `reset()` additionally validates that the context hasn't changed since the token was created, raising `ValueError` if it has. That guard is not needed here because `BaseWrappingContext` fully controls the enter/exit lifecycle.
- Nested and recursive calls continue to work correctly: each `__enter__` pushes a new dict that records the prior one; each `_pop_storage` pops it.
Co-authored-by: thomas.kowalski <thomas.kowalski@datadoghq.com>
a406380 to
55b40d4
Compare
Backport of #17407