Skip to content

Merge from dev#1076

Merged
ffulin merged 155 commits intomainfrom
dev
May 1, 2025
Merged

Merge from dev#1076
ffulin merged 155 commits intomainfrom
dev

Conversation

@ffulin
Copy link
Contributor

@ffulin ffulin commented May 1, 2025

Merge from dev

ffulin added 30 commits February 7, 2024 16:49
 - process can be "running" after ReadAllData is called if aborted, in which case cleanup happens in WaitForExit
 - removing assert here makes process handling consistent with other invocations
…builds

[Note] Failed/aborted multi-target builds will now behave the same as single target builds, stopping as soon as possible. -nostoponerror can be used to build as much as possible instead, as was already the case for single target builds.
[Improvement] Build stop on failure/abort behavior for multiple targets matches one target builds
[Improvement] Build output now differentiates "Failure" from "Incomplete" for multiple targets
 - ensure tests reliably check desired behaviors
 - make remaining work to enable some tests for OSX/Linux more explicit
 - the test and the processes we create try to coordinate with the same mutex
 - the launched executable now handles the test acquiring the mutex before it has started which would occur under heavy load
 - there are still some issue with this test:
  -- build can finish as "Incomplete" depending on timing) but this needs deeper fixes to build cancellation to fix
  -- test is Windows only
 - fix implcit conversion of NODE_RESULT_FAILED to boolean false
 - remove NODE_RESULT_OK_CACHE state. The state of the node is OK whether retrieved from the cache or not, and the cache state can already be gleaned from the stat flags
 - remove NODE_RESULT_OK_CACHE state. The state of the node is OK whether retrieved from the cache or not, and the cache state can already be gleaned from the stat flags
 - make enum a class enum and modernize naming
 - fix issues with executable path preventing test from working and enable test
 - remove disabling of -Wc++17-extensions (we use c++17 everywhere now)
 - remove old references to Clang 3/pre-Clang 6 which is no longer supported on any platform
FASTBuild can still invoke older versions of clang - but it requires C++17 and newer versions to be built from source.
 - Make cancelled tasks an explicit concept
 - Extend tests to cover various scenarios
 - set in FBuildTestOptions so it doesn't need to be set per-test
 - FBuildTestOptions already configures the port
[Fix] Fix typo in Error 1035 regarding exessive depth complexity
[Fix] Fix typo in build-time errors regarding VCXProjExternal config problems
[Improvement] Optimize startup time by avoiding unnecessary work during DB loading
 - avoid setting the build pass tag during loading as it's not used (it's only needed when saving)
[Improvement] Optimize startup time after bff edits by avoiding unnecessary work during DB migration
 - utilize the already calculated node name hash in various find and create calls when migrating nodes
 - avoid unnecesdsarily sanitizing file names when migrating the DB by using a different CreateNode call
[Note] DB version changed
 - in "cl mode" Clang on Windows emulates MSVC and creates an additional object for the pch which needs to be linked in
 - make DeleteDeletor the default (using new/delete rather than alloc/free)
 - rename Release() to ReleaseOwnership() for clarity
… used

 - LightCache was incorrecly using the "final" args, which is just one arg referencing the repsonse file, which meant all the normal args were unavailable for things like include path resolution
 - It now uses the "raw" args, which are the real args and a test is added to cover this behavior
 - de-duplicate code into helper function
 - extend test to cover quoted arg where quotes don't surround entire arg
[Note] Arg processing has been overhauled or correctness and consistency and fixes may invalidate some previous work-around
[Fix] Escaped quotes are correctly handled in process arguments
[Fix] Quotes are properly tokenized regardless of quote position
[Fix] Args with quotes and spaces are correctly tokenized
[Fix][OSX][Linux] Args with quotes are handled correctly when spawning processes
 - arg tokenization was recently fixed which means compdb got some fixes automatically, but the code was using some custom quote handlign logic which was incorect. That now uses the standardized logic in AString.
 - tests are extended slightly to cover strings with spaces in them
 - avoid node lookup for alias resoluation when we have the node available already
 - tidy up optional arguments by wrapping in a helper structure
…ths (regression in v1.12)

 - search params were not correctly encoding excludePaths into the directory listing, so different results were incorectly collapsed
 - fix the encoding error and add test coverage for directory listing node name generation
ffulin and others added 29 commits April 6, 2025 13:08
 - Remove initial capacity constructor. Is rarely needed, obfuscates things and is potentially error prone (esp as it differs in behavior from std::vector)
 - Leverage StackArray in several places to additionally avoid some memory allocations
Remove unnecessary initial capacity setting in a few places when code later already sets the capacity (such as Tokenize() calls)
…the fdb file

 - when loading the DB we already have the nodeNameHash available which we can take better advantage of
…ed significantly

 - define constant arrays used for arg lookups globally instead of reconstructing them for every config in every project
 - leverage StackArray to avoid many allocations and reallocations for every config in every project
 - not an actual leak, but rather asymetrical alloc/free
…le paths in various places

 - most callers to GetRelativePath have AStackString destinations, but several cases in VCXProject generation deal with Arrays of AString
 - presizing the output string prevents numerous resizes operations when relative paths have ..\ prefixes
 - avoid most memory allocations when tokenizing args
 - obtain a list of token boundaries and process tokens individually on the stack instead of allocating many copies on the heap unnecessarily
 - as well as avoiding the cost of the allocations, this greatly reduces allocator lock contention on systems with many CPU cores
 - avoid unecessary memcopies by writing to multiple chained buffers instead of a single large buffer
 - this can eliminate hundreds of MiB of mem copies for large databases
 - this also reduced peak memory use at shutdown for most large databases
…city is already >= that which is desired

 - in particular this can avoid an AStackString overflowing to the heap unnecessarily
…able and DLL

 - use stack based args processing to avoid dynamic memory allocations
* Add new UseDeterministicPaths_Experimental flag to CompilerNode

* Fix VS2022 Analyze build
 - make all tests stress test this new behavior which should eventually be the default
 - github has deprecated 20.04 :(
 - pass a single array of Start/End pairs
 - use indices instead of pointers to halve stack space use
 - increase stack allocation size in default tokenizer and in project generation
 - greatly reduce memory allocations during args processing. As well as avoiding memory allocation overhead, this reduces allocator lock contention which can be significant if spawning many short-lived processes simultaneously and in rapid succession (which can be common in some phases of some builds)
…itives

 - needs further investigation, but seems like a false positive
 - avoid per command line arg memory allocations
[Improvement] Further reduce bff parsing cost for every ObjectList slightly for non-MSVC compilers
 - skip some MSVC (or ClangCL) specific work if not using those compilers
…eopt with msvc

 - regression in recent update for /dynamicdeopt support, but was not yet released
@ffulin ffulin merged commit 08dd432 into main May 1, 2025
16 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants