Skip to content

Generate YAML doc#747

Merged
crazy-max merged 1 commit intodocker:masterfrom
crazy-max:yaml-docs
Sep 2, 2021
Merged

Generate YAML doc#747
crazy-max merged 1 commit intodocker:masterfrom
crazy-max:yaml-docs

Conversation

@crazy-max
Copy link
Member

Use cli-docs-tool to generate Markdown and now YAML docs.

@thaJeztah I have not used a sub gomod to avoid copy-pasting "require(...)" from the main one. github.com/docker/cli-docs-tool will not be used by ./cmd/buildx as shown with the command below:

$ cd ./cmd/buildx/
$ go list -deps -f '{{define "M"}}{{.Path}}@{{.Version}}{{end}}{{with .Module}}{{if not .Main}}{{if .Replace}}{{template "M" .Replace}}{{else}}{{template "M" .}}{{end}}{{end}}{{end}}' | sort -u
cloud.google.com/go@v0.81.0
github.com/agext/levenshtein@v1.2.1
github.com/agl/ed25519@v0.0.0-20170116200512-5312a6153412
github.com/apparentlymart/go-cidr@v1.0.1
github.com/apparentlymart/go-textseg/v12@v12.0.0
github.com/beorn7/perks@v1.0.1
github.com/cenkalti/backoff/v4@v4.1.1
github.com/cespare/xxhash/v2@v2.1.1
github.com/compose-spec/compose-go@v0.0.0-20210729195839-de56f4f0cb3c
github.com/containerd/console@v1.0.2
github.com/containerd/containerd@v1.5.5
github.com/containerd/continuity@v0.1.0
github.com/containerd/typeurl@v1.0.2
github.com/davecgh/go-spew@v1.1.1
github.com/distribution/distribution/v3@v3.0.0-20210316161203-a01c71e2477e
github.com/docker/cli@v20.10.3-0.20210702143511-f782d1355eff+incompatible
github.com/docker/compose-on-kubernetes@v0.4.19-0.20190128150448-356b2919c496
github.com/docker/distribution@v2.7.1+incompatible
github.com/docker/docker-credential-helpers@v0.6.4-0.20210125172408-38bea2ce277a
github.com/docker/docker@v20.10.3-0.20210609100121-ef4d47340142+incompatible
github.com/docker/go-connections@v0.4.0
github.com/docker/go-metrics@v0.0.1
github.com/docker/go-units@v0.4.0
github.com/docker/go@v1.5.1-1.0.20160303222718-d30aec9fd63c
github.com/docker/spdystream@v0.0.0-20181023171402-6480d4af844c
github.com/fvbommel/sortorder@v1.0.1
github.com/go-logr/logr@v0.2.0
github.com/gofrs/flock@v0.7.3
github.com/gogo/googleapis@v1.4.0
github.com/gogo/protobuf@v1.3.2
github.com/golang/protobuf@v1.5.2
github.com/golang/snappy@v0.0.4-0.20210608040537-544b4180ac70
github.com/google/go-cmp@v0.5.6
github.com/google/gofuzz@v1.1.0
github.com/google/shlex@v0.0.0-20191202100458-e7afc7fbc510
github.com/google/uuid@v1.2.0
github.com/googleapis/gnostic@v0.4.1
github.com/gorilla/mux@v1.8.0
github.com/grpc-ecosystem/go-grpc-middleware@v1.2.0
github.com/grpc-ecosystem/grpc-gateway@v1.16.0
github.com/hashicorp/go-cty-funcs@v0.0.0-20200930094925-2721b1e36840
github.com/hashicorp/hcl/v2@v2.8.2
github.com/imdario/mergo@v0.3.12
github.com/joho/godotenv@v1.3.0
github.com/json-iterator/go@v1.1.11
github.com/klauspost/compress@v1.12.3
github.com/mattn/go-shellwords@v1.0.12
github.com/matttproud/golang_protobuf_extensions@v1.0.2-0.20181231171920-c182affec369
github.com/mitchellh/go-wordwrap@v0.0.0-20150314170334-ad45545899c7
github.com/mitchellh/mapstructure@v1.4.1
github.com/moby/buildkit@v0.8.2-0.20210702160134-1a7543a10527
github.com/moby/locker@v1.0.1
github.com/moby/sys/mount@v0.2.0
github.com/moby/sys/mountinfo@v0.4.1
github.com/moby/term@v0.0.0-20201110203204-bea5bbe245bf
github.com/modern-go/concurrent@v0.0.0-20180306012644-bacd9c7ef1dd
github.com/modern-go/reflect2@v1.0.1
github.com/morikuni/aec@v1.0.0
github.com/opencontainers/go-digest@v1.0.0
github.com/opencontainers/image-spec@v1.0.1
github.com/pkg/errors@v0.9.1
github.com/prometheus/client_golang@v1.7.1
github.com/prometheus/client_model@v0.2.0
github.com/prometheus/common@v0.10.0
github.com/prometheus/procfs@v0.6.0
github.com/serialx/hashring@v0.0.0-20190422032157-8b2912629002
github.com/sirupsen/logrus@v1.8.1
github.com/spf13/cobra@v1.2.1
github.com/spf13/pflag@v1.0.5
github.com/theupdateframework/notary@v0.6.1
github.com/tonistiigi/fsutil@v0.0.0-20210609172227-d72af97c0eaf
github.com/tonistiigi/units@v0.0.0-20180711220420-6950e57a87ea
github.com/tonistiigi/vt100@v0.0.0-20210615222946-8066bb97264f
github.com/xeipuuv/gojsonpointer@v0.0.0-20180127040702-4e3ac2762d5f
github.com/xeipuuv/gojsonreference@v0.0.0-20180127040603-bd5ef7bd5415
github.com/xeipuuv/gojsonschema@v1.2.0
github.com/zclconf/go-cty@v1.7.1
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc@v0.21.0
go.opentelemetry.io/contrib@v0.21.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc@v1.0.0-RC1
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp@v1.0.0-RC1
go.opentelemetry.io/otel/exporters/otlp/otlptrace@v1.0.0-RC1
go.opentelemetry.io/otel/sdk@v1.0.0-RC1
go.opentelemetry.io/otel/trace@v1.0.0-RC1
go.opentelemetry.io/otel@v1.0.0-RC1
go.opentelemetry.io/proto/otlp@v0.9.0
golang.org/x/crypto@v0.0.0-20210322153248-0c34fe9e7dc2
golang.org/x/net@v0.0.0-20210405180319-a5a99cb37ef4
golang.org/x/oauth2@v0.0.0-20210402161424-2e8d93401602
golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c
golang.org/x/sys@v0.0.0-20210510120138-977fb7262007
golang.org/x/term@v0.0.0-20201126162022-7de9c90e9dd1
golang.org/x/text@v0.3.5
golang.org/x/time@v0.0.0-20200630173020-3af7569d3a1e
google.golang.org/genproto@v0.0.0-20210602131652-f16073e35f0c
google.golang.org/grpc@v1.38.0
google.golang.org/protobuf@v1.26.0
gopkg.in/inf.v0@v0.9.1
gopkg.in/yaml.v2@v2.4.0
k8s.io/api@v0.20.6
k8s.io/apimachinery@v0.20.6
k8s.io/client-go@v0.20.6
k8s.io/klog/v2@v2.4.0
k8s.io/utils@v0.0.0-20201110183641-67b214c5f920
sigs.k8s.io/structured-merge-diff/v4@v4.0.3
sigs.k8s.io/yaml@v1.2.0

@crazy-max crazy-max requested review from thaJeztah and tonistiigi and removed request for tonistiigi August 24, 2021 22:58
Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks good.

The only thing we should discuss / look at / brainstorm is what the most practical location is for the generated yaml files. Does it work best if they're together in the same directory as the markdown files (what it currently does, i.e. /docs/reference/XXXX.yaml), or would it be more convenient to have them in their own directory (/docs/yaml/XXX.yaml (for example)).

Mostly thinking if the script at https://github.com/docker/docker.github.io/blob/master/_scripts/fetch-upstream-resources.sh would possibly need the markdown files to be put somewhere else, etc.

github.com/serialx/hashring v0.0.0-20190422032157-8b2912629002
github.com/sirupsen/logrus v1.8.1
github.com/spf13/cobra v1.1.1
github.com/spf13/cobra v1.2.1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the (a) bit of the concern I had with not making it a submodule. (As it may unintentionally be "pushing" dependencies to newer versions).

The replace rules are a bit of a pain though, so I agree (at least for now) it's probably ok to keep it as-is.

@thaJeztah
Copy link
Member

Mostly thinking if the script at https://github.com/docker/docker.github.io/blob/master/_scripts/fetch-upstream-resources.sh would possibly need the markdown files to be put somewhere else, etc.

Of course, we could (should) decide if we want these files to be fetched when building the docs, or to store a copy there (as we currently do); in both cases, we will need to look what's more practical (separate directory, or together)

@crazy-max
Copy link
Member Author

crazy-max commented Aug 25, 2021

@thaJeztah

The only thing we should discuss / look at / brainstorm is what the most practical location is for the generated yaml files. Does it work best if they're together in the same directory as the markdown files (what it currently does, i.e. /docs/reference/XXXX.yaml), or would it be more convenient to have them in their own directory (/docs/yaml/XXX.yaml (for example)).

I think a flatten dir is ok to avoid maintenance cost if paths change in the future.

Of course, we could (should) decide if we want these files to be fetched when building the docs, or to store a copy there (as we currently do); in both cases, we will need to look what's more practical (separate directory, or together)

I think storing a copy on docker.github.io is better to track them as any other source code for reviews and so on. Also we should discuss about an auto repo sync like GitHub does on https://github.com/github/docs to be able to automatically create PR on changes. I think it would be great :)

Mostly thinking if the script at https://github.com/docker/docker.github.io/blob/master/_scripts/fetch-upstream-resources.sh would possibly need the markdown files to be put somewhere else, etc.

About this line, as of Git 2.19, this is possible to clone a subdir with git clone --filter.

@thaJeztah
Copy link
Member

I think a flatten dir is ok to avoid maintenance cost if paths change in the future.

My train of thought was that "separate" would;

  • keep generated and hand-written files separate
  • possibly, the reference dir could contain markdown that's not directly associated with a command (? not sure, perhaps those should be stored outside of reference)
  • will it be harder to navigate through GitHub if both are in the same directory?

Again, just thoughts, so not strongly attached to one or the other, but thought I'd "brain dump" them here

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
@crazy-max
Copy link
Member Author

PTAL @thaJeztah @tonistiigi

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@crazy-max crazy-max merged commit c1c414e into docker:master Sep 2, 2021
@crazy-max crazy-max deleted the yaml-docs branch September 2, 2021 18:16
@tonistiigi
Copy link
Member

Why do we need to commit the generated yaml docs into the repo?

@crazy-max
Copy link
Member Author

crazy-max commented Sep 2, 2021

@tonistiigi To be able to fetch them from the docs repo (git clone ... v0.6.3). I'm not sure if we could put them somewhere else. Maybe another folder like @thaJeztah said?

@tonistiigi
Copy link
Member

Can't the fetcher just generate them. I'm fine that we have a dockerfile target etc. here to run the generator and we can test it in this CI. But if the yaml itself is just an input to some external tool then I don't think we need to commit in copies for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants