Proposed commit message

Improves S3 polling mode state registry copy by considering the bucket_list_prefix usage. Before this change, input stored all registry entries loaded from the underlying registry (ex:- when storing, restarting beats or upgrading while pointing to the same registry). With this improvement, when bucket_list_prefix is used, the state registry copy only holds entries matching the given prefix.

This improvement benefits the state registry clean-up planned through #41694. Further, when beats restart, this change improves the input-specific state copy memory usage by only storing relevant entries, matching the configured prefix.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

None - this change only concerns the input-specific copy of the registry entries

How to test this PR locally

Requires a build and S3 bucket entries with prefixes.

• Generate a mix of S3 bucket entries with prefixes and without. You may use data gen tool ¹
• Build filebeat from this branch & configure multiple inputs with prefixes
• Observe state registry filling at startup. You may first run without a prefix to store all entries and later restart beats to observe the loading behavior

Related issues

#39116