Skip to content

[Feature Request] Download xrefmap.json file on docfx build is significantly slower #9558

@filzrev

Description

@filzrev

Is your feature request related to a problem? Please describe.
I'm trying to use xref: links for .NET types inside markdown document.
And add xref settings with following instruction.
https://dotnet.github.io/docfx/docs/links-and-cross-references.html?q=xref#cross-reference-to-net-basic-class-library

But when running docfx build command.
It takes too long time to download xrefmap file. (about 10 minutes on my environment. And It's disconnected from server sometimes)
When downloading same file with browser. It's completed about a few seconds.

Describe the solution you'd like
It seems there are some problems in XRefMapDownloader

1. File size of xrefmap.json
xrefmap.json file size is about 316MB. And this file is downloaded everytime when running dotnet build command.

  • I thought. it's better to set AutomaticDecompression = System.Net.DecompressionMethods.All for HttpClient handler to enable HTTP response compression. (When compressed with gzip. File size is shrieked to about16MB)
  • It might be better to cache downloaded xrefmap file locally with ETag.
    And use ETag to skip download when server-side document is not updated.

2. StreamReader's default buffer size(1KB) is too small to synchronously processing stream
Yaml deserialization is executed synchronously. So it frequently blocking file download process.
As far as I've confirmed. It's significantly faster when increasing buffer size (1MB).

Describe alternatives you've considered

Download xrefmap.jsonfile manually (or docfx download) before running docfx build.
And specify local xrefmap.zip file in docfx.json file.

Additional context
There is docfx download command exists.
But it's not documented and It seems not workings (There is no way to pass download URL from command line parameter?)

docfx download -x https://learn.microsoft.com/en-us/dotnet/.xrefmap.json xrefmap.zip

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceMakes the pull request appear in "Performance" section of the next release note

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions