Skip to content

Log and include response files in binlog#8146

Merged
JaynieBai merged 36 commits intodotnet:mainfrom
edvilme:edvilme-response-files
Dec 14, 2022
Merged

Log and include response files in binlog#8146
JaynieBai merged 36 commits intodotnet:mainfrom
edvilme:edvilme-response-files

Conversation

@edvilme
Copy link
Copy Markdown
Contributor

@edvilme edvilme commented Nov 9, 2022

Fixes #7214

Context

When using response files, it can be easy to lose track of all switches used. For that reason, it would be helpful to log a message at the beginning of each build and also include them in the binary logger.

Changes Made

  • A message with normal verbosity is logged for each file at the beginning of every build
  • A new event type ResponseFileUsedEventArgs was implemented
  • The BinaryLogger responds to such event by including the response file it returns

Testing

Testing was done with several resp files

Notes

Changes are only applicable to standalone msbuild, as with dotnet, dotnet itself analyzes and expands the response files

edvilme and others added 6 commits November 3, 2022 14:24
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It probably doesn't matter too much just because there probably aren't too many response files, but this is probably faster if you add to lists. (I'm happy trying to verify or falsify that with you if you'd like to see how Benchmark.NET works!)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely agree here. Collect the initial list with .ToList(), then add to it in this loop. I think it can continue to be a local variable, too.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping on this 🙂

@edvilme edvilme force-pushed the edvilme-response-files branch from 35f6f0e to 7ac66f7 Compare November 10, 2022 20:18
Copy link
Copy Markdown
Member

@rainersigwald rainersigwald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In offline discussion with @Forgind, an idea came up: is it possible to pass in a list of BuildEventArgs, rather than DeferredBuildMessage, so that if we decide we need another type in the future it's already there?

I'd also like to see some end-to-end tests of this functionality. Probably in src\Build.UnitTests\BinaryLogger_Tests.cs?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely agree here. Collect the initial list with .ToList(), then add to it in this loop. I think it can continue to be a local variable, too.

edvilme and others added 11 commits November 21, 2022 11:20
Addresses dotnet#7214. A message is logged at the beginning og a build whenever a response file(s) is used.
Addresses dotnet#7214. Using custom events, the binlogger includes the response files used when running MSBuild.
Code formatting. Fixes identation, spacing and adds missing comments
For greater flexibility and avoiding massive changes in the codebase, DeferredResponseFile has been removed. Instead, DeferredBuildMessage now has a FilePath member that when not blank, will inlcude the specified file in the binlogger
ResponseFilePath member is initialized as an empty string by default.
Co-authored-by: Rainer Sigwald <raines@microsoft.com>
Co-authored-by: Rainer Sigwald <raines@microsoft.com>
@edvilme edvilme force-pushed the edvilme-response-files branch from 4a48c5d to 933cbd4 Compare November 21, 2022 19:22
Copy link
Copy Markdown
Member

@rainersigwald rainersigwald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of style nits, but I think the big thing is to add a test, which should make the changes fairly clear.

Added RoadtripResponseFileUsedEventArgs test and specific methods at BuildEventArgsReader.cs and BuildEventArgsWriter.cs for ResponseFileUsedEventArgs
Added trailing comma to enum BinaryLogRecordKind, and replaced Message ith ResponseFilePath in tests
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping on this 🙂

Copy link
Copy Markdown
Contributor

@Forgind Forgind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

/// <summary>
/// Messages to be logged into loggrers
/// </summary>
private static IEnumerable<BuildManager.DeferredBuildMessage> messagesToLogInBuildLoggers = Enumerable.Empty<BuildManager.DeferredBuildMessage>();
Copy link
Copy Markdown
Contributor

@Forgind Forgind Dec 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unclear on why this has to exist at all, but ¯\(ツ)

@rainersigwald rainersigwald added the merge-when-branch-open PRs that are approved, except that there is a problem that means we are not merging stuff right now. label Dec 13, 2022
@rainersigwald
Copy link
Copy Markdown
Member

Lot of commits here. @dotnet/kitten please make sure to squash :)

@JaynieBai JaynieBai merged commit d642d53 into dotnet:main Dec 14, 2022
@KirillOsenkov
Copy link
Copy Markdown
Member

This looks good, the only thing I'd change is rename to just FileUsedEventArgs and FilePath

@KirillOsenkov
Copy link
Copy Markdown
Member

Binlog viewer version 2.1.758 has support for binlog format version 15

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-when-branch-open PRs that are approved, except that there is a problem that means we are not merging stuff right now.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Log a message for every response file used and embed in binlog

5 participants