Enable JITLink via the CLING_JITLINK environment variable.#12
Conversation
| const clang::CompilerInstance& CI, | ||
| bool Verbose); | ||
| const clang::CompilerInstance& CI, bool Verbose, | ||
| bool Debug); |
There was a problem hiding this comment.
Can we use DebugInfoKind in the CompilerInstance's CodeGenOptions instead of this flag?
There was a problem hiding this comment.
Done. I was not aware of this possibility. I kept the bool in the IncrementalJIT, because otherwise I needed to include the header for the CodeGenOptions.
6bebe36 to
1941a03
Compare
| // if the flag was set, add GDBRegistrationListener to JIT instance | ||
| for (auto const& s : m_Opts.CompilerOpts.Remaining) { | ||
| if (std::strcmp(s, "-g") == 0) { | ||
| getCI()->getCodeGenOpts().setDebugInfo( |
There was a problem hiding this comment.
Isn’t this automatically set by the CIFactory?
There was a problem hiding this comment.
Sorry, I was pretty busy this month. Let's see, if I can find it out.
There was a problem hiding this comment.
Intermediate result of today: Without my change, the maximum debug level is set to clang::codegenoptions::DebugInfoConstructor (unsigned int) 4. I tested several flags: -g, -g3, -gfull, -ggdb -gused and --debug. But clang::codegenoptions::FullDebugInfo (unsigned int) 6 is required. Otherwise it does not emit the necessary debug information.
I believe something is wrong, in because if I set the flag -gfull, I expect, that the debug level is set to FullDebugInfo. I will check if the cling code overwrites the debug level next time.
There was a problem hiding this comment.
I see -debug-info-kind=standalone which might be able to set it?
There was a problem hiding this comment.
I found out, where the concrete debug level is set and how to enable full debug information.
The concrete debug level is set via clang -cc1 -debug-info-kind=constructor ... argument. This means, the decision happens in the CI factory, when the variable CC1Args is created:
root/interpreter/cling/lib/Interpreter/CIFactory.cpp
Lines 1387 to 1399 in 1941a03
This means the clang code is responsible to choose the internal debug level and our cling "user code" does not affect it. This line of code chooses the debug level, if I set -g or -gfull:
After a text search I found also the only line of code, which can set FullDebugInfo:
Thanks to comments I know, that I have to run cling -g -fstandalone-debug to enable full debug information.
Okay, how we want to solve it? I have three ideas:
- Document, that the user needs to pass the arguments
cling -g -fstandalone-debugand don't manipulate the debug level inside cling. - Keep the current solution
- Add the argument
-fstandalone-debugbefore parsing the cc1 arguments if-gor other debug flags are set.
There was a problem hiding this comment.
I'd prefer option 1.) because when we go to re-use clang-repl interfaces we will just use it as it is. I was thinking that when we call cling -g we can add also -fstandalone-debug, however that would require parsing arguments which can be a bit ugly. Another option might be if we detect CLING_DEBUG env variable to be set we add both flags.
There was a problem hiding this comment.
Okay, sounds reasonable. Is the llvm 13 port already rebased on the cling documentation PR root-project#11474 ? I like to document there.
There was a problem hiding this comment.
@vgvassilev I didn't introduced an environment variable to add the flags. This requires at least changes in two different places in the code, where I have to check for the env variable. So the direct connection of both checks is little bit hidden. I think the extra work to maintain is bigger than the benefit.
But I introduced a second environment variable.
CLING_JITLINK: enables thejitlinklinkerCLING_JITLINK_DEBUG: enables thejitlinklinker and register the gdb listener
3148f4f to
b8540ce
Compare
1e6cba8 to
82dbd2b
Compare
3270913 to
0ab0776
Compare
JITLink works only with ELF and MachO and not with COFF. Therefore it cannot be used as a standard object linker. But JITLink offers some advantages, like debugger support for jittered code. Therefore, the possibility to enable JITLink on supported platforms has been implemented. By default, debugger support is not enabled. Run `CLING_JITLINK_DEBUG=ON ./cling -g -fstandalone-debug` on supported platforms to get debugger support for jitted code.
1941a03 to
0b93462
Compare
6/1344 Test #12: pyunittests-pyroot-import-load-libs .......................................***Failed 1.35 sec test_import (import_load_libs.ImportLoadLibs.test_import) Test libraries loaded after importing ROOT ... ERROR ====================================================================== ERROR: test_import (import_load_libs.ImportLoadLibs.test_import) Test libraries loaded after importing ROOT ---------------------------------------------------------------------- Exception: Found not whitelisted libraries after importing ROOT: - libtbbmalloc If the test fails with a library that is loaded on purpose, please add it to the whitelist. ---------------------------------------------------------------------- Ran 1 test in 1.186s FAILED (errors=1) This failure is seen in Fedora 41 after tbb was updated from version 2021.11.0 to version 2021.13.0 in the distribution.
The test was dynamically allocating the array data members of the `Data` struct, but never deallocating them. This commit polishes the `Data` struct definition and ensures proper management of the data members. The previous way of writing data to the TTree was leading to a bad memory access in the ReadBasicPointer inlined function in TStreamerInfoReadBuffer.cxx while reading the `double*` array. In particular, the issue arises when accessing and then deallocating the array at the current index provided by the `TCompInfo` object. ``` Target 0: (repro.out) stopped. (lldb) Process 13498 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = step in frame #0: 0x00000001044cf140 libRIO.so`int TStreamerInfo::ReadBuffer<char**>(this=<unavailable>, b=<unavailable>, arr=<unavailable>, compinfo=<unavailable>, first=<unavailable>, last=<unavailable>, narr=<unavailable>, eoffset=<unavailable>, arrayMode=0) at TStreamerInfoReadBuffer.cxx:923:65 [opt] 920 case TStreamerInfo::kOffsetP + TStreamerInfo::kLong: ReadBasicPointer(Long_t); continue; 921 case TStreamerInfo::kOffsetP + TStreamerInfo::kLong64: ReadBasicPointer(Long64_t); continue; 922 case TStreamerInfo::kOffsetP + TStreamerInfo::kFloat: ReadBasicPointer(Float_t); continue; -> 923 case TStreamerInfo::kOffsetP + TStreamerInfo::kDouble: ReadBasicPointer(Double_t); continue; 924 case TStreamerInfo::kOffsetP + TStreamerInfo::kUChar: ReadBasicPointer(UChar_t); continue; 925 case TStreamerInfo::kOffsetP + TStreamerInfo::kUShort: ReadBasicPointer(UShort_t); continue; 926 case TStreamerInfo::kOffsetP + TStreamerInfo::kUInt: ReadBasicPointer(UInt_t); continue; Target 0: (repro.out) stopped. (lldb) Process 13498 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = step in frame #0: 0x00000001044cf184 libRIO.so`int TStreamerInfo::ReadBuffer<char**>(TBuffer&, char** const&, TStreamerInfo::TCompInfo* const*, int, int, int, int, int) [inlined] TBuffer::BufferSize(this=0x000060e00010ef00) const at TBuffer.h:98:41 [opt] 95 TObject *GetParent() const; 96 char *Buffer() const { return fBuffer; } 97 char *GetCurrent() const { return fBufCur; } -> 98 Int_t BufferSize() const { return fBufSize; } 99 void DetachBuffer() { fBuffer = nullptr; } 100 Int_t Length() const { return (Int_t)(fBufCur - fBuffer); } 101 void Expand(Int_t newsize, Bool_t copy = kTRUE); // expand buffer to newsize Target 0: (repro.out) stopped. (lldb) p fBufSize (Int_t) 32008 (lldb) s Process 13498 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = step in frame #0: 0x00000001044cf194 libRIO.so`int TStreamerInfo::ReadBuffer<char**>(this=<unavailable>, b=<unavailable>, arr=<unavailable>, compinfo=<unavailable>, first=<unavailable>, last=<unavailable>, narr=<unavailable>, eoffset=<unavailable>, arrayMode=0) at TStreamerInfoReadBuffer.cxx:923:65 [opt] 920 case TStreamerInfo::kOffsetP + TStreamerInfo::kLong: ReadBasicPointer(Long_t); continue; 921 case TStreamerInfo::kOffsetP + TStreamerInfo::kLong64: ReadBasicPointer(Long64_t); continue; 922 case TStreamerInfo::kOffsetP + TStreamerInfo::kFloat: ReadBasicPointer(Float_t); continue; -> 923 case TStreamerInfo::kOffsetP + TStreamerInfo::kDouble: ReadBasicPointer(Double_t); continue; 924 case TStreamerInfo::kOffsetP + TStreamerInfo::kUChar: ReadBasicPointer(UChar_t); continue; 925 case TStreamerInfo::kOffsetP + TStreamerInfo::kUShort: ReadBasicPointer(UShort_t); continue; 926 case TStreamerInfo::kOffsetP + TStreamerInfo::kUInt: ReadBasicPointer(UInt_t); continue; Target 0: (repro.out) stopped. (lldb) s Process 13498 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xbebebebebebebeae) frame #0: 0x0000000107bac674 libclang_rt.asan_osx_dynamic.dylib`__asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) + 76 libclang_rt.asan_osx_dynamic.dylib`__asan::Allocator::Deallocate: -> 0x107bac674 <+76>: casalb w8, w9, [x22] 0x107bac678 <+80>: cmp w8, #0x2 0x107bac67c <+84>: b.ne 0x107bac6f4 ; <+204> 0x107bac680 <+88>: mov x8, #-0x100000000 ; =-4294967296 Target 0: (repro.out) stopped. (lldb) bt * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xbebebebebebebeae) * frame #0: 0x0000000107bac674 libclang_rt.asan_osx_dynamic.dylib`__asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) + 76 frame #1: 0x0000000107c0c444 libclang_rt.asan_osx_dynamic.dylib`wrap__ZdaPv + 232 frame #2: 0x00000001044d4a60 libRIO.so`int TStreamerInfo::ReadBuffer<char**>(this=<unavailable>, b=<unavailable>, arr=<unavailable>, compinfo=<unavailable>, first=<unavailable>, last=<unavailable>, narr=<unavailable>, eoffset=<unavailable>, arrayMode=0) at TStreamerInfoReadBuffer.cxx:923:65 [opt] frame #3: 0x0000000103ffc888 libRIO.so`TStreamerInfoActions::GenericReadAction(buf=0x000060e00010ef00, addr=0x0000602000056bd0, config=0x0000604000149910) at TStreamerInfoActions.cxx:195:45 frame #4: 0x0000000103caa5ec libRIO.so`TStreamerInfoActions::TConfiguredAction::operator()(this=0x00006030001693f0, buffer=0x000060e00010ef00, object=0x0000602000056bd0) const at TStreamerInfoActions.h:123:17 frame #5: 0x0000000103ca9ef8 libRIO.so`TBufferFile::ApplySequence(this=0x000060e00010ef00, sequence=0x000060600011ac20, obj=0x0000602000056bd0) at TBufferFile.cxx:3702:10 frame #6: 0x00000001064bc570 libTree.so`TBranchElement::ReadLeavesMemberBranchCount(this=0x0000619000566380, b=0x000060e00010ef00) at TBranchElement.cxx:4603:6 frame #7: 0x0000000106455ce4 libTree.so`TBranch::GetEntry(this=0x0000619000566380, entry=0, getall=0) at TBranch.cxx:1753:4 frame #8: 0x00000001064a1764 libTree.so`TBranchElement::GetEntry(this=0x0000619000566380, entry=0, getall=0) at TBranchElement.cxx:2783:27 frame #9: 0x000000010739915c libTreePlayer.so`ROOT::Detail::TBranchProxy::Read(this=0x00006110000c9580) at TBranchProxy.h:163:42 frame #10: 0x0000000107649ba8 libTreePlayer.so`(anonymous namespace)::TObjectArrayReader::At(this=0x0000603000169900, proxy=0x00006110000c9580, idx=1) at TTreeReaderArray.cxx:176:22 frame #11: 0x000000010000c2e4 repro.out`ROOT::Internal::TTreeReaderArrayBase::UntypedAt(this=0x000000016fdfe740, idx=1) const at TTreeReaderArray.h:41:62 frame #12: 0x000000010000c200 repro.out`TTreeReaderArray<double>::At(this=0x000000016fdfe740, idx=1) at TTreeReaderArray.h:205:54 frame root-project#13: 0x00000001000065e0 repro.out`TTreeReaderArray<double>::operator[](this=0x000000016fdfe740, idx=1) at TTreeReaderArray.h:207:44 frame root-project#14: 0x0000000100007b48 repro.out`simpleTest() at repro.cpp:123:26 frame root-project#15: 0x0000000100007e10 repro.out`main at repro.cpp:128:5 frame root-project#16: 0x000000018c718274 dyld`start + 2840 ```
526/1416 Test root-project#63: pyunittests-bindings-pyroot-pythonizations-pyroot-pyz-tf-pycallables ..........................***Failed 75.51 sec test_callable (tf_pycallables.TF1.test_callable) Test function provided as callable ... /usr/include/c++/15/bits/stl_bvector.h:1134: std::vector<bool, _Alloc>::reference std::vector<bool, _Alloc>::operator[](size_type) [with _Alloc = std::allocator<bool>; reference = std::vector<bool>::reference; size_type = long unsigned int]: Assertion '__n < this->size()' failed. *** Break *** abort =========================================================== The lines below might hint at the cause of the crash. If you see question marks as part of the stack trace, try to recompile with debugging information enabled and export CLING_DEBUG=1 environment variable before running. You may get help by asking at the ROOT forum https://root.cern/forum preferably using the command (.forum bug) in the ROOT prompt. Only if you are really convinced it is a bug in ROOT then please submit a report at https://root.cern/bugs or (preferably) using the command (.gh bug) in the ROOT prompt. Please post the ENTIRE stack trace from above as an attachment in addition to anything else that might help us fixing this issue. =========================================================== #10 0x00007f6bcac7fea4 in __pthread_kill_implementation () from /lib64/libc.so.6 #11 0x00007f6bcac264de in raise () from /lib64/libc.so.6 #12 0x00007f6bcac0e350 in abort () from /lib64/libc.so.6 root-project#13 0x00007f6bc9a0be02 in std::__glibcxx_assert_fail(char const*, int, char const*, char const*) () from /lib64/libstdc++.so.6 root-project#14 0x00007f6bb9b15257 in std::vector<bool, std::allocator<bool> >::operator[](unsigned long) [clone .part.0] [clone .lto_priv.0] (__n=<optimized out>, this=<optimized out>) at /usr/include/c++/15/bits/stl_bvector.h:1134 root-project#15 0x00007f6bb9bab976 in std::vector<bool, std::allocator<bool> >::operator[] (this=<synthetic pointer>, __n=0) at /usr/include/c++/15/bits/stl_bvector.h:201 root-project#16 CPyCppyy::Utility::ConstructCallbackPreamble (retType="Double_t", argtypes=..., code=Python Exception <class 'IndexError'>: list index out of range root-project#17 0x00007f6bb9b389e0 in PyFunction_AsCPointer (pyobject=<optimized out>, pyobject entry=0x7f6bb8f838c0, rettype="Double_t", signature="(Double_t*,Double_t*)") at /builddir/build/BUILD/root-6.34.02-build/root-6.34.02/bindings/pyroot/cppyy/CPyCppyy/src/Converters.cxx:2713 root-project#18 0x00007f6bb9b39851 in CPyCppyy::(anonymous namespace)::FunctionPointerConverter::SetArg (this=0x55eb25d61d40, pyobject=0x7f6bb8f838c0, para=..., ctxt=0x7ffc484855c0) at /builddir/build/BUILD/root-6.34.02-build/root-6.34.02/bindings/pyroot/cppyy/CPyCppyy/src/Converters.cxx:2768
This fixes root-project#18236 As seen in cms-sw/cmssw#47763 (comment) there are cases where the ROOT global lock is taken too late: In the stack trace below the lock is requested on frame #7 where as it should really be (also) requested on frame #12 as soon as the transaction is being generated. ``` #6 0x00001501421f55ea in ROOT::TVirtualRWMutex::Lock() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCore.so #7 0x0000150133a680ca in TCling::HandleNewTransaction(cling::Transaction const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCling.so #8 0x0000150133a85778 in TCling::UpdateListsOnCommitted(cling::Transaction const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCling.so #9 0x0000150133b60018 in cling::MultiplexInterpreterCallbacks::TransactionCommitted(cling::Transaction const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCling.so #10 0x0000150133bec069 in cling::IncrementalParser::commitTransaction(llvm::PointerIntPair<cling::Transaction*, 2u, cling::IncrementalParser::EParseResult, llvm::PointerLikeTypeTraits<cling::Transaction*>, llvm::PointerIntPairInfo<cling::Transaction*, 2u, llvm::PointerLikeTypeTraits<cling::Transaction*> > >&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCling.so #11 0x0000150133b5663a in cling::Interpreter::PushTransactionRAII::~PushTransactionRAII() () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCling.so #12 0x0000150133b6b883 in cling::LookupHelper::findType(llvm::StringRef, cling::LookupHelper::DiagSetting) const () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCling.so root-project#13 0x00001501339b6b06 in ROOT::TMetaUtils::TClingLookupHelper::GetPartiallyDesugaredNameWithScopeHandling(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCling.so root-project#14 0x00001501422f1458 in ResolveTypedefProcessType(char const*, unsigned int, unsigned int, bool, unsigned int, unsigned int, unsigned int, bool&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) [clone .constprop.0] () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCore.so root-project#15 0x00001501422f1a0f in ResolveTypedefImpl(char const*, unsigned int, unsigned int&, bool&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCore.so root-project#16 0x00001501422f2ef3 in TClassEdit::ResolveTypedef[abi:cxx11](char const*, bool) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCore.so root-project#17 0x00001501422f5c20 in TClassEdit::TSplitType::ShortType(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCore.so root-project#18 0x00001501422f6888 in TClassEdit::GetNormalizedName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::basic_string_view<char, std::char_traits<char> >) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCore.so root-project#19 0x000015014232e795 in ROOT::Internal::GetDemangledTypeName[abi:cxx11](std::type_info const&) () from /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02883/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_15_1_RNTUPLE_X_2025-03-31-2300/external/el8_amd64_gcc12/lib/libCore.so ```
5/1406 Test #12: pyunittests-bindings-pyroot-pythonizations-pyroot-import-load-libs ............................***Failed 1.35 sec test_import (import_load_libs.ImportLoadLibs.test_import) Test libraries loaded after importing ROOT ... ERROR ====================================================================== ERROR: test_import (import_load_libs.ImportLoadLibs.test_import) Test libraries loaded after importing ROOT ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/root-6.34.06/bindings/pyroot/pythonizations/test/import_load_libs.py", line 124, in test_import raise Exception('Found not whitelisted libraries after importing ROOT:' \ Exception: Found not whitelisted libraries after importing ROOT: - oqsprovider - liboqs If the test fails with a library that is loaded on purpose, please add it to the whitelist. ---------------------------------------------------------------------- Ran 1 test in 1.188s FAILED (errors=1) CMake Error at /builddir/build/BUILD/root-6.34.06/cmake/modules/RootTestDriver.cmake:232 (message): error code: 1
JITLink works only with ELF and MachO and not with COFF. Therefore it cannot be used as a standard object linker. But JITLink offers some advantages, like debugger support for jittered code. Therefore, the possibility to enable JITLink on supported platforms has been implemented.
By default, debugger support is not enabled. Run
CLING_JITLINK=ON ./cling -gon supported platforms to get debugger support for jitted code.Source code debugging is not working at the moment.