nit: can we add "_" prefix for these internal consts?

It is expected for users configure these variables, so technically these variables are not "internal", correct? Then shall we remove "_" from the env var name?

Do we want to invalidate all cached records every time? What is the downside of per-key invalidation?

Mostly complexity. Handling per-record TTL and invalidation only for the mutated key just simply adds complexity for an action that is (hopefully) so infrequent that it doesn't really make sense to add that complexity.

Make sense, let's start with the complete invalidation and then revisit if we hit performance issues

[sorry if I misunderstand the time-bucketed ephemeral encryption flow]
With the current secret derivation I wonder if it's not ensured that secrets in time-bucketed cache are "permanently unrecoverable after a short time window even if an attacker gains access to the base key via a memory dump".
Let's say a secret is stored in the cache at 00:00:00 and ttl_seconds=60, and then attackers dump the memory at 00:05:00. Attackers figure out the base key and encrypted secret strings at 00:10:00. They can compute bucket key by trying the time bucket number one by one (10 -> 9 -> ... -> 0) and find the time bucket key that can decrypt the secrets relatively efficiently:

enc = EphemeralCacheEncryption()
enc._base_key='extracted_base_key'
time_bucket = enc._get_time_bucket()
while (True):
  try:
    secret = enc.decrypt('extracted_encrypted_bytes', time_bucket)
    return secret
  catch Exception:
     time_bucket -= 1

If my statement above is true, probably we should generate a time bucket key in a way that cannot be computed from base_key and time bucket, and then have a separate thread to remove the time bucket secret from memory if it's expired.

This is a good point on the ease of cracking. While we definitely can't make this completely iron-clad (the only way to do this is through hardware-based encryption where the key is generated and decrypted external to memory such that the CPU never receives or processes the decryption), we can make this an order of magnitude harder to crack by using a random value that is purged in a separate process when the TTL triggers.
I'm going to go with that approach and also list out the limitations of what we're doing here (for what it's worth, other OSS implementations use even lower security grades to the time window encryption; going with the random encryption value upgrades this to well beyond anything else that I can find).
The case now is, at worst, with someone who has root access to the tracking server, they could theoretically get cached keys that are present in the cache within the time window iff they find them in the memory dump AND they can figure out which bytearray represents the random decryption key. This now effectively becomes finding the right two needles in a stack of needles (not impossible, but highly implausible that anyone would want to spend weeks going through bytearray permutations to find 2 bytearrays in a sea of bytearrays to satisfy decryption).
This satisfies the defense in depth requirements for Mitre's posting (reference included in the docstrings).

Shouldn't we have a separate thread that continuously clean up the expired bucket keys? Otherwise expired keys stay on memory until encrypt or decrypt is called again (probably not a huge blocker, but just wondered).

Totally - sorry for not including the separate thread on the last push (I forgot to push the stack early last night which had this change included - it's critical to have the timed daemon thread in case the cache isn't hit after n time period (otherwise the cache could stay warm far past the TTL).

nit: If we store up to one bucket key, we can have active_bucket: int and active_bucket_key: bytes separately instead of dict[int, bytes]

great call - I'll create the typed entries since we only have 4 values by design.