Skip to content

docs: implement Starlark domain plugin for Sphinx#1909

Merged
rickeylev merged 2 commits intobazel-contrib:mainfrom
rickeylev:sphinx.stardoc.plugin
May 23, 2024
Merged

docs: implement Starlark domain plugin for Sphinx#1909
rickeylev merged 2 commits intobazel-contrib:mainfrom
rickeylev:sphinx.stardoc.plugin

Conversation

@rickeylev
Copy link
Copy Markdown
Collaborator

@rickeylev rickeylev commented May 20, 2024

This implements a Sphinx plugin to support Starlark as a first-class domain in Sphinx.
A Starlark domain allows writing object descriptions directly using Sphinx's object
markup language, which allows better integration with cross references, understanding
and distinguishing types, rendering information about things, and referencing types
from other projects.

Note that this doesn't affect the docs today because the proto_to_markdown tool is
still generating regular markdown; updating that to generate Sphinx domain markdown
will be done separately.

Summary of features:

  • Types and arguments can be documented using Python syntax, e.g. list[str] This includes
    unions, generics, and custom types. Each component of the type expression is linked and
    cross referenced appropriately. Each object can have its approprate pieces documented and
    defined (e.g. a rule can have attributes, attributes can have their defaults etc
    documented, etc)
  • An index of Starlark objects is generated automatically. This makes it easy, for example,
    to find the rules that have a particular attribute.
  • The following objects can be documented: functions, methods, rules, repository rules,
    providers, aspects, bzlmod extensions, tag classes, targets, flags, and
    attributes/fields of the aforementioned objects.
  • Arbitary docs can cross reference to Starlark types. e.g., a manually written "Getting
    Started" doc can write {bzl:obj}PyInfo.some_field and it will automatically link to
    the appropriate API docs.
  • Resolution of cross-references is much smarter and customizable. Instead of relying
    Markdown's link resolution rules, the Sphinx's crossreference resolution hooks are
    used. This allows more concise references (e.g., just a rule's base name), distinguishing
    a particular object type (e.g. a function vs rule), or referring to an absolute object.

@rickeylev rickeylev requested a review from aignas May 20, 2024 07:11
@rickeylev rickeylev force-pushed the sphinx.stardoc.plugin branch from 52c352b to 460331a Compare May 20, 2024 23:13
This implements a Sphinx plugin to support Starlark as a first-class
domain in Sphinx. A Starlark domain allows writing object descriptions
directly using Sphinx's object markup language, which allows better
integration with cross references, understanding and distinguishing
types, rendering information about things, and referencing types
from other projects.

Note that this doesn't affect the docs today because the
proto_to_markdown tool is still generating regular markdown; updating
that to generate Sphinx domain markdown will be done separately.

Summary of features:
* Types and arguments can be documented using Python syntax, e.g.
  `list[str]` This includes unions, generics, and custom types. Each
  component of the type expression is linked and cross referenced
  appropriately. Each object can have its approprate pieces documented
  and defined (e.g. a rule can have attributes, attributes can have
  their defaults etc documented, etc)
* An index of Starlark objects is generated automatically. This makes
  it easy, for example, to find the rules that have a particular
  attribute.
* The following objects can be documented: functions, methods, rules,
  repository rules, providers, aspects, bzlmod extensions, tag classes,
  targets, flags, and attributes/fields of the aforementioned objects.
* Arbitary docs can cross reference to Starlark types. e.g., a manually
  written "Getting Started" doc can write `{bzl:obj}PyInfo.some_field` and
  it will automatically link to the appropriate API docs.
@rickeylev rickeylev force-pushed the sphinx.stardoc.plugin branch from 460331a to 271be5f Compare May 20, 2024 23:21
Copy link
Copy Markdown
Collaborator

@aignas aignas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stardocs.py extension being a single file make the thing a little unwieldy imho. Having it across multiple could improve the readability, but it also has a lot of similar logic, so I am in favour of continuing with a single big file for now.

mnemonic = "SphinxBuildDocs",
progress_message = "Sphinx building {} for %{{label}}".format(format),
env = env,
execution_requirements = {"no-sandbox": ""},
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why no-sandbox?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, debugging stuff left over. Removed.


# Make a //foo:bar.bzl convert to foo.bar, not .foo.bar
if label.startswith("//"):
label = label.lstrip("/")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically you can have dots in the repo name, so this may break or have degenerate cases.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh, good point. Directories and files could have dots in them, too.

I've added a TODO for this for now. I think I'll change the internal representation to use @repo//file%symbol notation instead. The dotted notation grew out of originally only having within-same-file references, but it didn't evolve well once targets became cross-referencable, and I found it more natural to write the @repo//foo:bar%baz in docs rather than repo.foo.bar.baz.

@rickeylev rickeylev enabled auto-merge May 23, 2024 01:38
@rickeylev rickeylev added this pull request to the merge queue May 23, 2024
Merged via the queue into bazel-contrib:main with commit c7defbc May 23, 2024
@rickeylev rickeylev deleted the sphinx.stardoc.plugin branch May 23, 2024 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants