Skip to content

[Code] Remove Regex Compile Bloat#4947

Merged
Akkadius merged 1 commit intomasterfrom
akkadius/remove-regex
Jun 22, 2025
Merged

[Code] Remove Regex Compile Bloat#4947
Akkadius merged 1 commit intomasterfrom
akkadius/remove-regex

Conversation

@Akkadius
Copy link
Copy Markdown
Contributor

@Akkadius Akkadius commented Jun 22, 2025

Description

This is based off of efforts to cleanup the codebase, modernize and make it more efficient.

I have a branch called build-analyze that runs a trace of what takes up all of the build time (example here http://drone.akkadius.com/EQEmu/Server/18259/1/2)

This shows that std::regex is insanely expensive to compile and we had an include of a library that did tons of regex instantiations in a header through transitive includes. By removing this header it drops compile times like a rock for this specific issue.

A ~10x to ~100x reduction in regex-related template instantiations, which cut compile times in that area from over 72s to under 8s.

🧠 Compile-Time Performance Impact

This change significantly reduced std::regex instantiation overhead by removing transitive includes and consolidating usage.

Metric Before After Improvement
std::basic_regex<char>::basic_regex 72,151 ms (139x) 7,246 ms (18x) ~10x faster
std::regex_search & related ~160,000 ms total < 6,000 ms total ~25x faster
Total regex-related template instantiation time ~500,000 ms ~60,000 ms ~8.3x reduction
Max avg per instantiation ~519 ms ~402 ms ~22% faster
Total instantiations removed 120+ — ~90% fewer calls

🔬 Powered by ClangBuildAnalyzer.

Before

**** Templates that took longest to instantiate:
 72151 ms: std::basic_regex<char>::basic_regex (139 times, avg 519 ms)
 72081 ms: std::basic_regex<char>::_M_compile (139 times, avg 518 ms)
 68605 ms: std::__detail::_Compiler<std::regex_traits<char>>::_Compiler (139 times, avg 493 ms)
 58578 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_disjunction (139 times, avg 421 ms)
 58085 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_alternative (139 times, avg 417 ms)
 56689 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_term (139 times, avg 407 ms)
 48316 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_atom (139 times, avg 347 ms)
 16054 ms: std::regex_search<std::char_traits<char>, std::allocator<char>, std:... (139 times, avg 115 ms)
 16026 ms: std::regex_search<__gnu_cxx::__normal_iterator<const char *, std::ba... (139 times, avg 115 ms)
 15985 ms: std::__detail::__regex_algo_impl<__gnu_cxx::__normal_iterator<const ... (139 times, avg 115 ms)
 15100 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_insert_charact... (139 times, avg 108 ms)
 13857 ms: std::regex_match<const char *, std::allocator<std::sub_match<const c... (139 times, avg 99 ms)
 13836 ms: std::__detail::__regex_algo_impl<const char *, std::allocator<std::s... (139 times, avg 99 ms)
 13251 ms: httplib::detail::write_content_chunked<(lambda at /drone/src/zone/..... (232 times, avg 57 ms)
  8234 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_quantifier (139 times, avg 59 ms)
  7888 ms: std::reverse_iterator<std::_Bit_iterator> (336 times, avg 23 ms)
  7489 ms: std::__detail::_Executor<__gnu_cxx::__normal_iterator<const char *, ... (139 times, avg 53 ms)
  7474 ms: std::__detail::_Executor<__gnu_cxx::__normal_iterator<const char *, ... (139 times, avg 53 ms)
  7462 ms: std::__detail::_Executor<__gnu_cxx::__normal_iterator<const char *, ... (139 times, avg 53 ms)
  7043 ms: std::__detail::_Executor<const char *, std::allocator<std::sub_match... (139 times, avg 50 ms)
  7032 ms: std::__detail::_Executor<const char *, std::allocator<std::sub_match... (139 times, avg 50 ms)
  7020 ms: std::__detail::_Executor<const char *, std::allocator<std::sub_match... (139 times, avg 50 ms)
  6735 ms: httplib::detail::write_content<(lambda at /drone/src/zone/../common/... (232 times, avg 29 ms)
  6450 ms: httplib::detail::write_content_chunked<(lambda at /drone/src/zone/..... (116 times, avg 55 ms)
  6333 ms: std::reverse_iterator<std::_Bit_const_iterator> (336 times, avg 18 ms)
  6301 ms: std::unordered_map<unsigned long, unsigned int>::emplace<unsigned lo... (261 times, avg 24 ms)
  6233 ms: std::_Hashtable<unsigned long, std::pair<const unsigned long, unsign... (261 times, avg 23 ms)
  6167 ms: std::_Hashtable<unsigned long, std::pair<const unsigned long, unsign... (261 times, avg 23 ms)
  6131 ms: std::_Rb_tree<int, std::pair<const int, std::map<int, BotSpellTypesB... (268 times, avg 22 ms)
  6036 ms: std::__detail::_BracketMatcher<std::regex_traits<char>, false, false... (139 times, avg 43 ms)

After

**** Templates that took longest to instantiate:
  7624 ms: std::reverse_iterator<std::_Bit_iterator> (336 times, avg 22 ms)
  7246 ms: std::basic_regex<char>::basic_regex (18 times, avg 402 ms)
  7240 ms: std::basic_regex<char>::_M_compile (18 times, avg 402 ms)
  6914 ms: std::__detail::_Compiler<std::regex_traits<char>>::_Compiler (18 times, avg 384 ms)
  6323 ms: std::_Rb_tree<int, std::pair<const int, std::map<int, BotSpellTypesB... (268 times, avg 23 ms)
  6129 ms: std::reverse_iterator<std::_Bit_const_iterator> (336 times, avg 18 ms)
  5913 ms: std::unordered_map<unsigned long, unsigned int>::emplace<unsigned lo... (261 times, avg 22 ms)
  5842 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_disjunction (18 times, avg 324 ms)
  5840 ms: std::_Hashtable<unsigned long, std::pair<const unsigned long, unsign... (261 times, avg 22 ms)
  5788 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_alternative (18 times, avg 321 ms)
  5778 ms: std::_Hashtable<unsigned long, std::pair<const unsigned long, unsign... (261 times, avg 22 ms)
  5707 ms: rapidjson::GenericDocument<rapidjson::UTF8<>>::ParseStream<rapidjson... (162 times, avg 35 ms)
  5659 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_term (18 times, avg 314 ms)
  5650 ms: std::map<std::basic_string<char>, std::basic_string<char>>::operator[] (237 times, avg 23 ms)
  5450 ms: rapidjson::GenericDocument<rapidjson::UTF8<>>::ParseStream<0U, rapid... (156 times, avg 34 ms)
  5404 ms: std::_Rb_tree<int, std::pair<const int, std::map<int, BotSpellTypesB... (134 times, avg 40 ms)
  5086 ms: rapidjson::GenericReader<rapidjson::UTF8<>, rapidjson::UTF8<>>::Pars... (156 times, avg 32 ms)
  5083 ms: std::vector<cereal::JSONInputArchiveSingleLine::Iterator>::rbegin (162 times, avg 31 ms)
  5005 ms: std::reverse_iterator<__gnu_cxx::__normal_iterator<cereal::JSONInput... (162 times, avg 30 ms)
  4820 ms: std::__detail::_Compiler<std::regex_traits<char>>::_M_atom (18 times, avg 267 ms)
  4640 ms: std::map<unsigned char, std::basic_string<char>>::map (263 times, avg 17 ms)
  4594 ms: std::map<int, std::basic_string<char>>::map (260 times, avg 17 ms)
  4531 ms: std::vector<char>::resize (240 times, avg 18 ms)
  4437 ms: std::vector<char>::_M_fill_insert (225 times, avg 19 ms)
  4289 ms: std::map<unsigned int, std::basic_string<char>>::map (261 times, avg 16 ms)
  4142 ms: std::_Rb_tree<unsigned char, std::pair<const unsigned char, std::bas... (263 times, avg 15 ms)
  4077 ms: std::_Rb_tree<int, std::pair<const int, std::basic_string<char>>, st... (260 times, avg 15 ms)
  3988 ms: std::optional<std::tuple<uv_udp_send_s *, char *, EmbeddedContext *>> (139 times, avg 28 ms)
  3797 ms: std::_Rb_tree<unsigned int, std::pair<const unsigned int, std::basic... (261 times, avg 14 ms)
  3783 ms: std::map<unsigned int, std::pair<std::basic_string<char>, unsigned i... (260 times, avg 14 ms)

Type of change

  • Code cleanup

Testing

CI

Checklist

  • I have tested my changes
  • I have performed a self-review of my code. Ensuring variables, functions and methods are named in a human-readable way, comments are added only where naming of variables, functions and methods can't give enough context.
  • I own the changes of my code and take responsibility for the potential issues that occur

@Akkadius Akkadius merged commit e846bb8 into master Jun 22, 2025
2 checks passed
@Akkadius Akkadius deleted the akkadius/remove-regex branch June 22, 2025 07:08
@Akkadius Akkadius mentioned this pull request Jun 25, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@KimLS KimLS KimLS approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Akkadius @KimLS