Switch the destructors implementation for thread locals on Windows to use FLS #148799

ohadravid · 2025-11-10T17:19:14Z

Summary

Switch the thread local destructors implementation on Windows to use the Fiber Local Storage APIs, which provide native support for setting a callback to be called on thread termination, replacing the current tls_callback symbol-based implementation.

Except for some spellchecking, no LLMs were used to produce code / comments / text in this PR.

Current Implementation

On Windows, in order to support thread locals with destructors,
the standard library uses a special tls_callback symbol that is used to call the destructors::run() hook on thread termination.

This has two downsides:

It is not well documented, and seems to cause some problems 1 2 3.
It disallows some synchronization operations, as mentioned in LocalKey's documentation.

as an example of point 2, this code, which uses JoinHandle::join in a thread local Drop impl, will deadlock on stable:

Join-on-Drop Deadlock Example

struct JoinOnDrop(Option<JoinHandle<()>>);

impl Drop for JoinOnDrop {
    fn drop(&mut self) {
        self.0.take().unwrap().join().unwrap();
    }
}

thread_local! {
    static HANDLE: JoinOnDrop = {
        let thread = std::thread::spawn(|| {   
            println!("Starting...");
            // std::thread::sleep(Duration::from_secs(3));
            println!("Done");
        });

        JoinOnDrop(Some(thread))
    };
}


fn main() {
    let thread = std::thread::spawn(|| {
        HANDLE.with(|_| {
            println!("Some other thread");
        })
    });

    thread.join().unwrap();

    println!("Done");
}

Proposed Change

We can use the Fls{Alloc,Set,Get,Free} functions (see https://devblogs.microsoft.com/oldnewthing/20191011-00/?p=102989)
to implement the dtor callback needed for thread locals that have a Drop implementation.

We allocate a single key, and use its destructor callback to run all the registered destructors when a thread is shutting down.

With this implementation, the above code sample will not deadlock (but it still might not be a good idea to do this!).

Safety and Compatibility

Destructors will only run once: we use the common thread_local + atomic pattern to only set the Fls maker value once. The destructor callback is only called when that value is non-zero, so we are guaranteed that it will only be called once.

Destructors will only run at thread exit: we verify that we are not running in a fiber during the destructors callback. This means that using fibers (which is very rare) will result in thread local being leaked, unless the fiber is converted back to a thread using ConvertFiberToThread before thread termination. This is not ideal, but should be OK as destructors are not guaranteed to run, but it needs to be documented.

To be documented (replaces the current note in the docs about synchronization, and should also be noted in the rt module).

It might be possible for the user to use something like the current tls_callback to observe an already-freed thread locals, which is something that can also happen in the current implementation.

~~Destructors will only run on the correct thread: Fibers cannot be moved between threads.~~
Destructors will only run on the correct thread: they are registered to a thread_local list, so fiber movement between threads does not matter.

Users cannot observe different locals because they are using fibers: because we only use an Fls local marker to trigger the destructors callback, we don't change anything about how users interact with "normal" thread locals and fiber locals.

Other Notes

The implementation is based on the key::racy and guard::apple code, because we need a LazyKey-like racey static and an enable function.

While TLS slots are limited to 1088,
FLS slots are currently limited to 4000
per process.

Miri

Because miri is aware to the thread local implementation, I also implemented these functions and support for them in the interpreter here:

https://github.com/rust-lang/miri/compare/master...ohadravid:miri:windows-fls-support?expand=1

I guess that this will need to be merged before this PR (if this is accepted) - let me know and I'll open that PR as well.

Targets without `target_thread_local`

In *-gnu Windows targets, the target_thread_local feature is unavailable.

We could also change the "key" (non-target_thread_local) Windows impl at
library\std\src\sys\thread_local\key\windows.rs
to be based on the Fls functions. I can add it to this PR, or as a separate PR, if you think this is preferable.

Also, I used a Cell in a #[thread_local] to store the resulting key, like the other implementations.
This works, but I'm not sure if this is 100% OK given that we have these targets as well.

rustbot · 2025-11-10T17:19:20Z

r? @ChrisDenton

rustbot has assigned @ChrisDenton.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

ohadravid · 2025-12-12T05:11:20Z

Hi @ChrisDenton, any chance you can take a look at this? Tests are only failing because of missing support in Miri (which I have implemented in a branch)

Thanks!

bjorn3 · 2025-12-25T11:05:54Z

Wouldn't this conflict with the use of fibers by user code? If you switch to a fiber, access a tls variable for the first time on a thread, switch back and destroy the fiber, the tls variable would get incorrectly deinitialized. And if you move a fiber to another thread and exit the original thread, the tls variable would get deallocated while the fiber still has a reference to it that will cause a use-after-free when destroying the fiber.

ohadravid · 2025-12-25T13:26:15Z

Wouldn’t this conflict with the use of fibers by user code?

No - The way the new code works is that std will, at runtime initialization, call enable, which sets a single fiber-local to invoke the special std destructor when that thread exits. Because enable is called at that point (and not when the first TLS variable is accessed), there should be no UAF: the user can start a fiber (== convert the current thread into a fiber), but by then enable has already been called and the TLS variables will be alive for the lifetime of the thread.

Edit: fibers can't be moved between threads, so the text below is not really relevant. I also update the code to match my next comment about order of fiber/thread exit.

However, I now think you are technically right @bjorn3 , but only if the user starts the executable outside of Rust and performs runtime initialization (or otherwise triggers enable()) while already running in a non-main-fiber. Following your example, they would need to: (1) create a non-Rust thread, (2) convert that thread to a fiber, (3) create a new fiber, (4) cause enable() to be called in that fiber, and (5) move that fiber to a new thread and exit the original thread, leading to a UAF.

I couldn’t find whether there is even a documented way to do this (since rt::init is private). Maybe init_current via std::thread::current(), but that isn’t exposed via FFI anyway. Is this documented anywhere else?

In theory, I could add a check when setting the destructor that GetCurrentFiber returns null (though it’s a macro, not a function, so not easy), and avoid registering the callback in that case.

However, it seems better to document that this usage is unsupported if that's something that's not currently guaranteed to work.

bjorn3 · 2025-12-25T14:03:21Z

Initially, I thought not. The way the new code works is that std will, at runtime initialization, call enable, which sets a single fiber-local to invoke the special std destructor when that thread exits. Because enable is called at that point (and not when the first TLS variable is accessed), there should be no UAF: the user can start a fiber (== convert the current thread into a fiber), but by then enable has already been called and the TLS variables will be alive for the lifetime of the thread.

That still doesn't account for multiple fibers running on the same thread or fibers migrating between threads, right? The TLS variables have unique storage per thread, while the fiber-local destructor runs once per fiber on whichever thread destroys the fiber in the end as I understand it.

However, I now think you are technically right @bjorn3 , but only if the user starts the executable outside of Rust and performs runtime initialization (or otherwise triggers enable()) while already running in a non-main-fiber. Following your example, they would need to: (1) create a non-Rust thread, (2) convert that thread to a fiber, (3) create a new fiber, (4) cause enable() to be called in that fiber, and (5) move that fiber to a new thread and exit the original thread, leading to a UAF.

enable would have to be called as soon as the first Rust TLS variable is accessed, right? Otherwise cdylibs would never run the destructor for TLS variables.

ohadravid · 2025-12-25T14:50:27Z

the fiber-local destructor runs once per fiber

I will add a comment in the code so it's more clear, but no - because we only execute the unsafe { set(key, ptr::without_provenance(1)) }; line once, in some fiber (== usually just the thread), and the dtor callback is only called when the key's value is not zero, the dtors will only run when the thread exists, no matter what fibers are executed later, which matches the existing behvaior.

My understanding is that there isn't a safe way to exit a fiber without terminating the thread anyway (to use fibers, a thread must always start by calling ConvertThreadToFiber and DeleteFiber says "If the currently running fiber calls DeleteFiber, its thread calls ExitThread and terminates. However, if a currently running fiber is deleted by another fiber, the thread running the deleted fiber is likely to terminate abnormally because the fiber stack has been freed.")

enable would have to be called as soon as the first Rust TLS variable is accessed, right? Otherwise cdylibs would never run the destructor for TLS variables.

Hm, seems like library/std/src/sys/thread_local/destructors/list.rs::register does that on first access, yes. There's also __rust_std_internal_init in library/std/src/sys/thread_local/native/mod.rs which I think suggests that this happens post-rt::init anyway? I'm not sure.

…miri

rustbot · 2025-12-26T12:34:18Z

The Miri subtree was changed

cc @rust-lang/miri

rustbot · 2025-12-26T12:34:20Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

ChrisDenton · 2026-01-08T19:57:34Z

I did see this before the holidays but didn't have time to investigate. Last time I considered this I had concerns because rust does not manage threads, except those it spawns itself (and even then only to a degree). Which means an FLS destructor may run before the OS thread finishes whereas TLS is expected to be valid for the duration of the OS thread. Maybe those concerns are unfounded or mitigate but I'd want to be very sure before switching to it.

I'll ping the windows group in case anyone has reasons we should/shouldn't do this.

@rustbot ping windows

rustbot · 2026-01-08T19:57:59Z

Hey Windows Group! This bug has been identified as a good "Windows candidate".
In case it's useful, here are some instructions for tackling these sorts of
bugs. Maybe take a look?
Thanks! <3

cc @albertlarsan68 @arlosi @ChrisDenton @danielframpton @dpaoliello @gdr-at-ms @kennykerr @luqmana @nico-abram @retep998 @sivadeilra @wesleywiser

dpaoliello

Overall, I'm in favor of this approach: it replaces undocumented features that have caused issues with a documented feature.

View changes since this review

dpaoliello · 2026-01-09T22:23:53Z

library/std/src/sys/thread_local/guard/windows.rs

-            super::super::destructors::run();
-            #[cfg(not(target_thread_local))]
-            super::super::key::run_dtors();
+    if !REGISTERED.replace(true) {


Is it possible for a fiber to be suspended part way through this function? If so, the next fiber coming in will see that REGISTERED is true and skip setting up FLS, correct?

I think it's not possible, because REGISTERED is a thread-local, and fibers are not preemptively scheduled, so the only way this can happen is if the thread was suspended, which means that when the thread is resumed the same fiber will continue executing.

From https://learn.microsoft.com/en-us/windows/win32/procthread/fibers :

Fibers are not preemptively scheduled. You schedule a fiber by switching to it from another fiber. The system still schedules threads to run. When a thread running fibers is preempted, its currently running fiber is preempted but remains selected. The selected fiber runs when its thread runs.

Perfect, thanks!

dpaoliello · 2026-01-09T22:40:41Z

My understanding is that there isn't a safe way to exit a fiber without terminating the thread anyway (to use fibers, a thread must always start by calling ConvertThreadToFiber and DeleteFiber says "If the currently running fiber calls DeleteFiber, its thread calls ExitThread and terminates. However, if a currently running fiber is deleted by another fiber, the thread running the deleted fiber is likely to terminate abnormally because the fiber stack has been freed.")

Is it possible to delete a fiber that isn't running and will never run again?

dpaoliello · 2026-01-09T22:42:48Z

Last time I considered this I had concerns because rust does not manage threads, except those it spawns itself (and even then only to a degree). Which means an FLS destructor may run before the OS thread finishes whereas TLS is expected to be valid for the duration of the OS thread.

Assuming that Fiber destruction is linked to thread destruction (or that we can somehow only run this code when the last fiber is destroyed), then is the gap between the fibers being destroyed and the thread being destroyed observable? I think it only matters to code running in DLL Detach, which shouldn't be messing with TLS stuff in Rust anyway...

ChrisDenton · 2026-01-10T04:25:01Z

We have no way of ensuring that FlsSetValue is called on the main fiber. So if DeleteFiber is called on the fiber that used FlsSetValue then the FLS callback will be run but the OS thread could continue indefinitely (running other fibers or switching back to a non-fiber thread).

A very quick sketch

use std::ffi::c_void;
use windows::Win32::System::Threading::{
    ConvertThreadToFiber, CreateFiber, DeleteFiber, FlsAlloc, FlsSetValue, SwitchToFiber, ConvertFiberToThread,
};

fn main() {
    unsafe {
        let main = ConvertThreadToFiber(None);
        let fiber = CreateFiber(0, Some(fiber_start), Some(main));
        println!("switching to another fiber");
        SwitchToFiber(fiber);
        DeleteFiber(fiber); // Invokes the FLS callback.
        println!("end of main fiber");
    }
}

unsafe extern "system" fn fls_callback(_param: *const c_void) {
    println!("fls dealloc");
}

extern "system" fn fiber_start(main: *mut c_void) {
    println!("fiber started");
    unsafe {
        let index = FlsAlloc(Some(fls_callback));
        let _ = FlsSetValue(index, Some(1234 as _));
        SwitchToFiber(main);
    };
}

ohadravid · 2026-01-10T09:07:11Z

Is it possible to delete a fiber that isn't running and will never run again?

I totally missed that! Thanks @ChrisDenton's for the sketch 🙏

We have no way of ensuring that FlsSetValue is called on the main fiber.

I think ~~it should be possible to call GetCurrentFiber~~ we can use IsThreadAFiber to check if we are running in a fiber or a thread, and not ~~register~~ run the callback when running in a fiber (leading to a leak).

That's a bit of a hack (which might defeat the purpose of replacing the current tls_callback hack), but I think it solves this problem ~~because you cannot DeleteFiber(main) from a different fiber~~.

dpaoliello · 2026-01-12T17:02:44Z

I think using IsThreadAFiber and leaking is fine: use of fibers is extremely rare, and replacing the use of an undocumented, hacky feature with a documented, supported feature that leaks if the user is using a rare OS feature seems like a reasonable trade-off to me.

Just to confirm, we don't get called back twice if the thread is converted to a fiber and not back again (i.e., once for the implicit fiber deletion and once for the thread ending)? Just trying to make sense of this documentation:

If the FLS slot is in use, FlsCallback is called on fiber deletion, thread exit, and when an FLS index is freed.

Also, it is possible to execute a fiber on a different thread than the one that created it, per this doc:

To execute any fiber created with CreateFiber, call the SwitchToFiber function. You can call SwitchToFiber with the address of a fiber created by a different thread. To do this, you must have the address returned to the other thread when it called CreateFiber and you must use proper synchronization.

But given the current design, I think that's a moot point.

dpaoliello · 2026-01-12T17:05:00Z

library/std/src/sys/thread_local/guard/windows.rs

+    //
+    // By checking that we are not running in a fiber here, we are guaranteed that the hook is only running at the thread teardown.
+    // See also the `fiber_does_not_trigger_dtor` test.
+    if is_thread_a_fiber() {


If we are leaking in this case, please add a note to the comment here.

We should also put a note in the docs... somewhere. What ends up getting leaked? Is it thread-local stuff? If so, then the docs for that should mention that exiting a Windows thread in fiber-mode without first converting back to non-fiber-mode will skip drops.

ChrisDenton · 2026-01-12T18:13:59Z

replacing the use of an undocumented, hacky feature

To be clear, the tech the existing implementation is built on is mostly documented: https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#the-tls-section. The part that isn't documented (the special C runtime section for CRT compatibility) is not the bit that causes issues. The "hacky" part I agree with; the current lack of rustc support for TLS callbacks is more problematic and means we've ended up hacking stuff into the library.

Using fiber callbacks does address the problems but potentially leaking TLS destructors does give me pause. Yes, it's rarely used but a quick github search suggests it isn't exactly unused either https://github.com/search?q=ConvertThreadToFiber+language%3ARust&type=code&l=Rust

If we go ahead with this change we'd probably want a formal FCP and release notes to at least try to ensure awareness of the change (even if it won't affect the vast majority of users).

dpaoliello · 2026-01-12T18:45:01Z

If we go ahead with this change we'd probably want a formal FCP and release notes to at least try to ensure awareness of the change (even if it won't affect the vast majority of users).

+1 for the FCP and release notes. We should also add docs to APIs that may get leaked.

rustbot assigned ChrisDenton Nov 10, 2025

rustbot added O-windows Operating system: Windows S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Nov 10, 2025

This comment has been minimized.

Sign in to view

ohadravid mentioned this pull request Dec 25, 2025

Add supprot for FLS functions on Windows rust-lang/miri#4788

Open

ohadravid added 5 commits December 26, 2025 03:42

Added Windows bindings for FLS-related functions

bacf105

Replace thread_local::guard on Windows with an FLS-based impl

d5b86c0

Use a static atomic to store a common key

62ffa11

Document safety of using the fls callback to run dtors

176b2f5

Add support for the Fls{Alloc,GetValue,SetValue,Free} functions in …

0327bec

…miri

ohadravid force-pushed the windows-thread-local-dtors-using-fls branch from 6502684 to 0327bec Compare December 26, 2025 12:34

This comment was marked as outdated.

Sign in to view

dpaoliello reviewed Jan 9, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

ohadravid force-pushed the windows-thread-local-dtors-using-fls branch from 57cd474 to c9d4789 Compare January 11, 2026 09:29

ohadravid force-pushed the windows-thread-local-dtors-using-fls branch 2 times, most recently from ab9eddd to 456fa3b Compare January 11, 2026 10:15

This comment has been minimized.

Sign in to view

ohadravid force-pushed the windows-thread-local-dtors-using-fls branch from 456fa3b to 25cf05e Compare January 11, 2026 12:08

Only run the destructors hook when not running in a fiber

e559e78

ohadravid force-pushed the windows-thread-local-dtors-using-fls branch from 25cf05e to e559e78 Compare January 11, 2026 12:44

dpaoliello approved these changes Jan 12, 2026

View reviewed changes

Uh oh!

Switch the destructors implementation for thread locals on Windows to use FLS #148799

Are you sure you want to change the base?

Switch the destructors implementation for thread locals on Windows to use FLS #148799

Uh oh!

Conversation

ohadravid commented Nov 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Current Implementation

Proposed Change

Safety and Compatibility

Other Notes

Miri

Targets without target_thread_local

Uh oh!

rustbot commented Nov 10, 2025

Uh oh!

This comment has been minimized.

ohadravid commented Dec 12, 2025

Uh oh!

bjorn3 commented Dec 25, 2025

Uh oh!

ohadravid commented Dec 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bjorn3 commented Dec 25, 2025

Uh oh!

ohadravid commented Dec 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Dec 26, 2025

Uh oh!

rustbot commented Dec 26, 2025

Uh oh!

ChrisDenton commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

rustbot commented Jan 8, 2026

Uh oh!

dpaoliello left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dpaoliello Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

ohadravid Jan 11, 2026

Choose a reason for hiding this comment

Uh oh!

dpaoliello Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

dpaoliello commented Jan 9, 2026

Uh oh!

dpaoliello commented Jan 9, 2026

Uh oh!

ChrisDenton commented Jan 10, 2026

Uh oh!

ohadravid commented Jan 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

dpaoliello commented Jan 12, 2026

Uh oh!

dpaoliello Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

ChrisDenton commented Jan 12, 2026

Uh oh!

dpaoliello commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

ohadravid commented Nov 10, 2025 •

edited

Loading

Targets without `target_thread_local`

ohadravid commented Dec 25, 2025 •

edited

Loading

ohadravid commented Dec 25, 2025 •

edited

Loading

ChrisDenton commented Jan 8, 2026 •

edited

Loading

dpaoliello left a comment •

edited by rustbot

Loading

ohadravid commented Jan 10, 2026 •

edited

Loading