Skip to content

Opening plugin with RTLD_DEEPBIND causes issues when not using the default allocator #3704

@irozzo-1A

Description

@irozzo-1A

Describe the bug

The falco libraries recently introduced a change to use RTLD_DEEPBIND option in dlopen when loading plugins i.e. chore(userspace/plugin): use RTLD_DEEPBIND for loading plugins.
This is causing problem described here:

The inconsistency that RTLD_DEEPBIND causes with jemalloc is that dynamic libraries opened with RTLD_DEEPBIND will use libc's malloc while libc is still using jemalloc. A libc function may return a pointer to something that should be passed to free, and the dynamic library will call libc's free, but libc used jemalloc to allocate the memory.

This can be easily reproduced by loading the k8smeta plugin with the 0.42.0-rc1 version of Falco.

Mon Oct 20 09:48:13 2025: [libs]: Trying to open the right engine!
free(): invalid pointer

Thread 28 "falco" received signal SIGABRT, Aborted.
[Switching to Thread 0xffff6c9f3140 (LWP 2727780)]
__pthread_kill_implementation (threadid=281472504115520, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
warning: 44	./nptl/pthread_kill.c: No such file or directory
(gdb) bt
#0  __pthread_kill_implementation (threadid=281472504115520, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
#1  0x0000fffff7da7670 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
#2  0x0000fffff7d5cb3c in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x0000fffff7d47e00 in __GI_abort () at ./stdlib/abort.c:79
#4  0x0000fffff7d9abf4 in __libc_message_impl (fmt=fmt@entry=0xfffff7e83bc0 "%s\n") at ../sysdeps/posix/libc_fatal.c:134
#5  0x0000fffff7db210c in malloc_printerr (str=str@entry=0xfffff7e7eff8 "free(): invalid pointer") at ./malloc/malloc.c:5772
#6  0x0000fffff7db44cc in _int_free (av=0xfffff7ed0a50 <main_arena>, p=p@entry=0xffff6b41bff0, have_lock=have_lock@entry=0) at ./malloc/malloc.c:4507
#7  0x0000fffff7db6f88 in __GI___libc_free (mem=<optimized out>) at ./malloc/malloc.c:3398
#8  0x0000fffff71df22c in ares_strsplit (in=in@entry=0xffff6401d887 "localdomain", delms=delms@entry=0xfffff7350a38 ", ", make_set=make_set@entry=1, num_elm=num_elm@entry=0xffff6c9f1560) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/third_party/cares/cares/ares_strsplit.c:171
#9  0x0000fffff71d8448 in set_search (channel=channel@entry=0xffff64022110, str=0xffff6401d887 "localdomain") at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/third_party/cares/cares/ares_init.c:2295
#10 0x0000fffff71d9d44 in init_by_resolv_conf (channel=0xffff64022110) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/third_party/cares/cares/ares_init.c:1693
#11 ares_init_options (channelptr=0xffff64022010, options=<optimized out>, optmask=<optimized out>) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/third_party/cares/cares/ares_init.c:208
#12 0x0000fffff70b3d88 in grpc_ares_ev_driver_create_locked (ev_driver=ev_driver@entry=0xffff64021ff0, pollset_set=pollset_set@entry=0xffff6401e1c0, query_timeout_ms=query_timeout_ms@entry=120000, request=request@entry=0xffff64021fa0)
    at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc:503
#13 0x0000fffff70b4ec0 in grpc_dns_lookup_ares_continue_after_check_localhost_and_ip_literals_locked (r=r@entry=0xffff64021fa0, dns_server=dns_server@entry=0xffff640210f0 "", name=name@entry=0xffff64021110 "localhost:32609", default_port=default_port@entry=0xfffff7331b40 "https", interested_parties=interested_parties@entry=0xffff6401e1c0,
    query_timeout_ms=query_timeout_ms@entry=120000) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc:833
#14 0x0000fffff70b6b18 in grpc_dns_lookup_ares_impl (dns_server=0xffff640210f0 "", name=0xffff64021110 "localhost:32609", default_port=0xfffff7331b40 "https", interested_parties=0xffff6401e1c0, on_done=<optimized out>, addrs=<optimized out>, balancer_addrs=<optimized out>, service_config_json=<optimized out>, query_timeout_ms=120000)
    at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/ext/filters/client_channel/resolver/dns/c_ares/grpc_ares_wrapper.cc:1069
#15 0x0000fffff70af82c in grpc_core::(anonymous namespace)::AresClientChannelDNSResolver::StartResolvingLocked (this=0xffff640210d0) at /usr/include/c++/13/bits/basic_string.h:222
#16 0x0000fffff6f80984 in grpc_core::ClientChannel::CreateResolverLocked (this=0xffff6401dfa8) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/ext/filters/client_channel/client_channel.cc:1534
#17 0x0000fffff6f80c0c in grpc_core::ClientChannel::TryToConnectLocked (this=0xffff6401dfa8) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/ext/filters/client_channel/client_channel.cc:1803
#18 0x0000fffff6fd9794 in std::function<void ()>::operator()() const (this=0xffff6401b1c8) at /usr/include/c++/13/bits/std_function.h:591
#19 grpc_core::WorkSerializer::WorkSerializerImpl::DrainQueueOwned (this=0xffff6401b380) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/lib/iomgr/work_serializer.cc:200
#20 0x0000fffff6fd9a04 in grpc_core::WorkSerializer::WorkSerializerImpl::Run(std::function<void ()>, grpc_core::DebugLocation const&) (this=<optimized out>, callback=..., location=...) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/lib/iomgr/work_serializer.cc:93
#21 0x0000fffff6fd9be8 in grpc_core::WorkSerializer::Run(std::function<void ()>, grpc_core::DebugLocation const&) (this=<optimized out>, callback=..., location=...) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/lib/iomgr/work_serializer.cc:216
#22 0x0000fffff6f7ad64 in operator() (__closure=0x0, arg=<optimized out>) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/ext/filters/client_channel/client_channel.cc:2319
#23 _FUN () at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/ext/filters/client_channel/client_channel.cc:2319
#24 0x0000fffff6fca93c in exec_ctx_run (closure=<optimized out>) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/lib/iomgr/exec_ctx.cc:49
#25 grpc_core::ExecCtx::Flush (this=this@entry=0xffff6c9f1de0) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/lib/iomgr/exec_ctx.cc:164
#26 0x0000fffff7029100 in grpc_core::ExecCtx::~ExecCtx (this=0xffff6c9f1de0, __in_chrg=<optimized out>) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/lib/iomgr/exec_ctx.h:126
#27 grpc_call_start_batch (call=0xffff6401f3f0, ops=0xffff6c9f1f08, nops=4, tag=<optimized out>, reserved=<optimized out>) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/src/core/lib/surface/call.cc:1762
#28 0x0000fffff6df37dc in grpc::internal::CallOpSet<grpc::internal::CallOpSendInitialMetadata, grpc::internal::CallOpSendMessage, grpc::internal::CallOpClientSendClose, grpc::internal::CallOpRecvInitialMetadata, grpc::internal::CallNoOp<5>, grpc::internal::CallNoOp<6> >::ContinueFillOpsAfterInterception (this=0xffff640201f8)
    at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/include/grpcpp/impl/codegen/call_op_set.h:971
#29 0x0000fffff6df0a6c in grpc::internal::Call::PerformOps (ops=0xffff640201f8, this=0xffff640201c0) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/include/grpcpp/impl/codegen/call.h:68
#30 grpc::internal::ClientCallbackReaderImpl<metadata::Event>::StartCall (this=0xffff640201b0) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/include/grpcpp/impl/codegen/client_callback.h:762
#31 0x0000fffff6d9b95c in grpc::ClientReadReactor<metadata::Event>::StartCall (this=0xffff6c9f2518) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/build/_deps/grpc-src/include/grpcpp/impl/codegen/client_callback.h:378
#32 K8sMetaClient::K8sMetaClient (this=this@entry=0xffff6c9f2518, node_name="", ip_port="localhost:32609", ca_PEM_encoding="", mu=..., cv=..., thread_quit=std::atomic<bool> = { false }, handler=...) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/src/grpc_client.cpp:71
#33 0x0000fffff6dccb94 in my_plugin::async_thread_loop (this=0x24d2470, h=std::unique_ptr<falcosecurity::async_event_handler> = {...}) at /Users/iacopo.rozzo/git_clones/plugins-main/plugins/k8smeta/src/plugin.cpp:399
#34 0x0000fffff6dcd3c8 in std::__invoke_impl<void, void (my_plugin::*)(std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> >) noexcept, my_plugin*, std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> > >(std::__invoke_memfun_deref, void (my_plugin::*&&)(std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> >) noexcept, my_plugin*&&, std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> >&&) (__f=<optimized out>, __t=<optimized out>) at /usr/include/c++/13/bits/invoke.h:74
#35 std::__invoke<void (my_plugin::*)(std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> >) noexcept, my_plugin*, std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> > >(void (my_plugin::*&&)(std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> >) noexcept, my_plugin*&&, std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> >&&) (__fn=<optimized out>) at /usr/include/c++/13/bits/invoke.h:96
#36 std::thread::_Invoker<std::tuple<void (my_plugin::*)(std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> >) noexcept, my_plugin*, std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> > > >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) (
    this=<optimized out>) at /usr/include/c++/13/bits/std_thread.h:292
#37 std::thread::_Invoker<std::tuple<void (my_plugin::*)(std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> >) noexcept, my_plugin*, std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> > > >::operator()() (this=<optimized out>)
    at /usr/include/c++/13/bits/std_thread.h:299
#38 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (my_plugin::*)(std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> >) noexcept, my_plugin*, std::unique_ptr<falcosecurity::async_event_handler, std::default_delete<falcosecurity::async_event_handler> > > > >::_M_run() (this=<optimized out>)
    at /usr/include/c++/13/bits/std_thread.h:244
#39 0x0000fffff6991ae0 in ?? () from /lib/aarch64-linux-gnu/libstdc++.so.6
#40 0x0000fffff7da595c [PAC] in start_thread (arg=0xfffff7fe6780) at ./nptl/pthread_create.c:

How to reproduce it

falco.k8smeta.yaml

sudo falcoctl artifact install k8smeta:0.4.0-rc1
sudo ~/falco-0.42.0-rc1-aarch64/usr/bin/falco -c falco.k8smeta.yaml -r rules/falco_rules.yaml

Expected behaviour

Falco should load and run the k8smeta plugin without problems.

Screenshots

Environment

  • Falco version:
$ ~/falco-0.42.0-rc1-aarch64/usr/bin/falco --version
Falco version: 0.42.0-rc1
Libs version:  1de61cd2b7abcfbb492b5da7fbeaef5b0a5c0f20
Plugin API:    3.12.0
Engine:        0.57.0
Driver:
  API version:    8.0.0
  Schema version: 4.0.0
  Default driver: 1de61cd2b7abcfbb492b5da7fbeaef5b0a5c0f20

Additional context

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions