Candidates are:

• Set optimization level to "s"
• Enable LTO
• Set debug to true (not an optimization, but this leaves metadata around that are really useful)
• -Zbuild-std=core, applying all the above to core
• -Zbuild-std-features=panic_immediate_abort (depending on that), completely sidestepping panic handling in favor of an abort (HCF, or whatever that is on the platform). That makes debugging a lot harder but should help code size a lot.
• Disable the panic-handler_format feature of riot-wrappers by default. That'd be like the little brother of panic_immediate_abort, and would still print which thread panicked but not where. May be a middle ground, but let's test aborting panics too: It could well be that when a debugger is attached, the stack recovery prints that RIOT does actually give better information on the error cause than the line-number-less simple panic handler.