Add `swift.emit_symbol_graph` feature by jpsim · Pull Request #838 · bazelbuild/rules_swift

jpsim · 2022-06-01T03:32:17Z

That emits the symbol graph for the Swift target to bazel-bin.

Symbol graphs can then be fed to tools such as DocC or jazzy to generate documentation for a target.

For example, for Envoy Mobile:

$ ./bazelw build //library/swift:ios_lib --config=ios
$ "$(xcrun --find docc)" convert \
  --index \
  --fallback-display-name Envoy \
  --fallback-bundle-identifier io.envoyproxy.EnvoyMobile \
  --fallback-bundle-version 0.4.6 \
  --output-dir Envoy.doccarchive \
  --transform-for-static-hosting \
  --additional-symbol-graph-dir bazel-bin/library/swift/ios_lib.symbolgraph

That emits the symbol graph for the Swift target to `bazel-bin`. Symbol graphs can then be fed to tools such as DocC or jazzy to generate documentation for a target. For example, for [Envoy Mobile](https://github.com/envoyproxy/envoy-mobile): ``` $ ./bazelw build //library/swift:ios_lib --config=ios $ "$(xcrun --find docc)" convert \ --index \ --fallback-display-name \ Envoy \ --fallback-bundle-identifier \ io.envoyproxy.EnvoyMobile \ --fallback-bundle-version \ 0.4.6 \ --output-dir \ Envoy.doccarchive \ --transform-for-static-hosting \ --additional-symbol-graph-dir \ bazel-bin/library/swift/ios_lib.symbolgraph ```

thii · 2022-06-01T12:53:57Z

Do you think the symbol graph should be part of the direct outputs? (that I think would probably make it easier for other rules to consume)

jpsim · 2022-06-01T13:16:51Z

Do you think the symbol graph should be part of the direct outputs?

I'm not strongly opposed, but in my mind the symbol graph files are more of a byproduct of the build than an output, similar to the index store files. Is there a formal definition of what a "direct output" is? That could help determine if this fits or not.

The other aspect is that we have control over the output directory for symbol graph files but the files themselves are picked by the compiler. The other entries in the direct outputs all appear to be files.

swift/internal/compiling.bzl

brentleyjones

Look good! I'll wait for another approval before merging though.

Augustyniak · 2022-06-01T14:05:36Z

unrelated observation: looking at the documentation page you generated I realized that our mocks are public 🤔

jpsim · 2022-06-01T15:04:55Z

Yes the Envoy Mobile mocks are in the same framework for ease of distribution, but that’s something we should fix when we redesign the API.

thii · 2022-06-02T01:31:04Z

Is there a formal definition of what a "direct output" is? That could help determine if this fits or not.

In my understanding, it's an output returned by the build by default (that is, what gets printed out at the end of the build).

If the symbol graph is just an intermediate output, Bazel might not produce the output if the build is a cached build or if you build with --remote_download_toplevel.

The other aspect is that we have control over the output directory for symbol graph files but the files themselves are picked by the compiler. The other entries in the direct outputs all appear to be files.

An output can also be a directory (a tree artifact).

I see that you just added the symbol graph output to the OutputGroupInfo provider that swift_library returns, so I think this is good for now. (The only downside is that these kinds of outputs have to be requested via the --output_groups command-line option, see the example below.)

bazel build --features=swift.emit_symbol_graph --output_groups=+swift_symbol_graph //examples/apple/objc_interop:Printer

INFO: Invocation ID: d11be84f-b35d-45b9-b421-95b72ccc399d
INFO: Analyzed target //examples/apple/objc_interop:Printer (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //examples/apple/objc_interop:Printer up-to-date:
  bazel-bin/examples/apple/objc_interop/Printer-Swift.h
  bazel-bin/examples/apple/objc_interop/examples_apple_objc_interop_Printer.swiftdoc
  bazel-bin/examples/apple/objc_interop/examples_apple_objc_interop_Printer.swiftmodule
  bazel-bin/examples/apple/objc_interop/examples_apple_objc_interop_Printer.swiftsourceinfo
  bazel-bin/examples/apple/objc_interop/libPrinter.a
  bazel-bin/examples/apple/objc_interop/Printer.symbolgraph
INFO: Elapsed time: 0.202s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action

BalestraPatrick · 2022-06-02T08:38:14Z

swift/internal/compiling.bzl

+    )
+    args.add(
+        "-emit-symbol-graph-dir",
+        prerequisites.symbol_graph_directory.path,


I assume the build time penalty to generate this is pretty irrelevant, right?

Regardless it's behind a feature flag, similar to indexing while building.

Yeah just out of curiosity. If it has a big impact, it might be worth documenting it.

Not sure if my methodology is correct, but I see a 2% hit to overall build times with the feature enabled:

$ hyperfine \ --warmup 5 \ --prepare "bazel clean && sleep 0.5" \ "bazel build --features=swift.emit_symbol_graph --output_groups=+swift_symbol_graph //examples/apple/objc_interop:Printer" \ "bazel build //examples/apple/objc_interop:Printer" Benchmark 1: bazel build --features=swift.emit_symbol_graph --output_groups=+swift_symbol_graph //examples/apple/objc_interop:Printer Time (mean ± σ): 12.694 s ± 0.390 s [User: 0.063 s, System: 0.055 s] Range (min … max): 12.379 s … 13.643 s 10 runs Benchmark 2: bazel build //examples/apple/objc_interop:Printer Time (mean ± σ): 12.504 s ± 0.258 s [User: 0.062 s, System: 0.051 s] Range (min … max): 12.105 s … 12.907 s 10 runs Summary 'bazel build //examples/apple/objc_interop:Printer' ran 1.02 ± 0.04 times faster than 'bazel build --features=swift.emit_symbol_graph --output_groups=+swift_symbol_graph //examples/apple/objc_interop:Printer'

Since it's opt-in I don't see this needing a perf warning.

Mind you, I edited Printer.swift to artificially make its compile time longer by adding a bunch of 2 * 2 * 2 * 2.0 / 2 + 2 statements to make the compiler work harder, taking ~6s in the Swift Compile step.

Without that artificial bloat, I can't measure a difference on a file that small:

Benchmark 1: bazel build --features=swift.emit_symbol_graph --output_groups=+swift_symbol_graph //examples/apple/objc_interop:Printer Time (mean ± σ): 6.222 s ± 0.189 s [User: 0.076 s, System: 0.065 s] Range (min … max): 6.013 s … 6.563 s 10 runs Benchmark 2: bazel build //examples/apple/objc_interop:Printer Time (mean ± σ): 6.228 s ± 0.217 s [User: 0.074 s, System: 0.062 s] Range (min … max): 5.911 s … 6.587 s 10 runs Summary 'bazel build --features=swift.emit_symbol_graph --output_groups=+swift_symbol_graph //examples/apple/objc_interop:Printer' ran 1.00 ± 0.05 times faster than 'bazel build //examples/apple/objc_interop:Printer'

jpsim · 2022-06-02T12:50:07Z

Thanks for the reviews, everyone! I may revisit @thii's comment (#838 (comment)) after we use this some more but for now at least I can generate docs for my project.

And include in GitHub release artifacts Updates rules_swift to pull in bazelbuild/rules_swift#838 When users open the archive, they'll have rich docs they can explore using Xcode's documentation navigator. When we update to Xcode 13.4, we'll be able to generate HTML docs with this approach and publish to the documentation website. Signed-off-by: JP Simard <jp@jpsim.com>

And include in GitHub release artifacts Updates rules_swift to pull in bazelbuild/rules_swift#838 When users open the archive, they'll have rich docs they can explore using Xcode's documentation navigator. When we update to Xcode 13.4, we'll be able to generate HTML docs with this approach and publish to the documentation website. Here's a demo of what that static site looks like: https://envoy-mobile-docc-demo.netlify.app/documentation/envoy/ https://user-images.githubusercontent.com/474794/171704142-993072a3-fbc2-4db6-9c70-65e9e1a47aac.mp4 Risk Level: Low Testing: Local, CI Docs Changes: Added Release Notes: Added Signed-off-by: JP Simard <jp@jpsim.com>

That emits the symbol graph for the Swift target to `bazel-bin`. Symbol graphs can then be fed to tools such as DocC or jazzy to generate documentation for a target. For example, for [Envoy Mobile](https://github.com/envoyproxy/envoy-mobile): ``` $ ./bazelw build //library/swift:ios_lib --config=ios $ "$(xcrun --find docc)" convert \ --index \ --fallback-display-name \ Envoy \ --fallback-bundle-identifier \ io.envoyproxy.EnvoyMobile \ --fallback-bundle-version \ 0.4.6 \ --output-dir \ Envoy.doccarchive \ --transform-for-static-hosting \ --additional-symbol-graph-dir \ bazel-bin/library/swift/ios_lib.symbolgraph ```

# Summary Closes #1420 This adds support for building `.doccarchive` for rules_apple targets. It uses the [symbol_graph](bazelbuild/rules_swift#838) feature from `rules_swift` to collect the symbol graph. It then invokes `xcrun` to find `docc` and create the `.doccarchive`. Example: ```python docc_archive( name = "HelloWorldSwift.doccarchive", dep = ":HelloWorldSwift", fallback_bundle_identifier = "com.example.hello-world-swift", fallback_bundle_version = "1.0.0", fallback_display_name = "HelloWorldSwift", ) ``` Build it: ```sh bazel build //examples/ios/HelloWorldSwift:HelloWorldSwift.doccarchive xed bazel-bin/examples/ios/HelloWorldSwift/HelloWorldSwift.doccarchive ``` Preview the docs: ```sh bazel run //examples/ios/HelloWorldSwift:HelloWorldSwift.doccarchive ```

jpsim requested review from BalestraPatrick, brentleyjones, keith, segiddins and thii as code owners June 1, 2022 03:32

jpsim mentioned this pull request Jun 1, 2022

DocC Rule bazelbuild/rules_apple#1420

Closed

brentleyjones reviewed Jun 1, 2022

View reviewed changes

swift/internal/compiling.bzl Outdated Show resolved Hide resolved

swift/internal/compiling.bzl Outdated Show resolved Hide resolved

swift/internal/compiling.bzl Show resolved Hide resolved

Address feedback from code review

c372032

brentleyjones approved these changes Jun 1, 2022

View reviewed changes

thii approved these changes Jun 2, 2022

View reviewed changes

BalestraPatrick reviewed Jun 2, 2022

View reviewed changes

BalestraPatrick approved these changes Jun 2, 2022

View reviewed changes

brentleyjones merged commit 3bc7bc1 into bazelbuild:master Jun 2, 2022

jpsim deleted the add-swift.emit_symbol_graph-feature branch June 2, 2022 12:48

jpsim mentioned this pull request Jun 2, 2022

swift: generate API docs as a DocC archive envoyproxy/envoy-mobile#2335

Merged

This was referenced Jul 4, 2023

Add support for Apple .doccarchive bazelbuild/rules_apple#2007

Merged

Add support for forwarding symbol graph bazel-ios/rules_ios#734

Merged

Conversation

jpsim commented Jun 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thii commented Jun 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jpsim commented Jun 1, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

brentleyjones left a comment

Choose a reason for hiding this comment

Uh oh!

Augustyniak commented Jun 1, 2022

Uh oh!

jpsim commented Jun 1, 2022

Uh oh!

thii commented Jun 2, 2022

Uh oh!

BalestraPatrick Jun 2, 2022

Choose a reason for hiding this comment

Uh oh!

brentleyjones Jun 2, 2022

Choose a reason for hiding this comment

Uh oh!

BalestraPatrick Jun 2, 2022

Choose a reason for hiding this comment

Uh oh!

jpsim Jun 2, 2022

Choose a reason for hiding this comment

Uh oh!

jpsim Jun 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jpsim commented Jun 2, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jpsim commented Jun 1, 2022 •

edited

Loading

thii commented Jun 1, 2022 •

edited

Loading

jpsim Jun 2, 2022 •

edited

Loading