Yea this doesn't handle the roundtrip scenario, that will matter for ie buf export. I think it's a little more complicated than this, can talk about it

Wait, might not matter for buf export, but yea I think I thought about this before and it was more complicated - @doriable might have insight as well (might involve docs)

We could go about this in a few different ways:

1. modulev1alpha1.Module could gain a new field, string fallback_documentation
2. modulev1alpha1.Module could gain a new field, string documentation_filename
3. modulev1alpha1.Module could gain a new field, DocumentationFile documentation_file which has filename and content.

All three of these scenarios have the BSR storing the filename, we could default to buf.md if empty.

Are there alternative approaches?

buf export to my knowledge doesn't output documentation? It just writes an Image, which I don't think has documentation?

buf export to my knowledge doesn't output documentation? It just writes an Image, which I don't think has documentation?

Yes this is correct -- buf export will not affect documentation, so that is less of a concern.

You likely need to store the file path in the Module yea. It should be documented to be normalized, similar to other paths in Protobuf definitions

This may affect the digest as well

I guess I'm questioning a little bit the user experience problem we are solving here. In our original discussions, we were concerned with just picking up README.md because some users may be storing internal-facing information in a README.md file alongside their proto files in a private repository. In the situation where the source code is closed, this may be crossing a boundary. It was a conscious decision to make a file that is both Buf branded and explicit so there is no leaking of information.

Agreed with @doriable, I think at the least we could warn to the user that we're falling back to their README.md and this may not be desired.

This may affect the digest as well

This will but that's OK. A new release will include this file when sending to/receiving from the BSR. Older clients will ignore it.

It does mean using an old and new client together will cause a ping-pong of commits, but that's expected behavior given each one is sending different content to the BSR.

Putting this on hold until we find out how to proceed due to the concerns of its side effects and the user experience problem.

Had an offline discussion with @bufdev. We came to the conclusion that private information in a README alongside a Buf module is likely an edge case, and that doing this fallback behaviour early is better than later. This can help ease of use when it comes to publishing documentation and emulates the same behaviour as Docker (getting your README from your source code repository).

With that being said, a couple of notes for this change:

• There should be a very clear CHANGELOG entry for this
• We should be storing the file name on the BSR side, and default to buf.md for all the existing data without a file name
• The fallback path should include README.markdown if we are going to go down this route, since both md and markdown are both valid extensions per RFC 7763. We would be following the same conventions as Go documentation (https://github.com/golang/pkgsite/blob/6862513e1b973c06468b98ed333cf9b80498b2a6/internal/postgres/searchdoc.go#L175-L179)

* We should be storing the file name on the BSR side, and default to `buf.md` for all the existing data without a file name

* The fallback path should include `README.markdown` if we are going to go down this route, since both `md` and `markdown` are both valid extensions per RFC 7763. We would be following the same conventions as Go documentation (https://github.com/golang/pkgsite/blob/6862513e1b973c06468b98ed333cf9b80498b2a6/internal/postgres/searchdoc.go#L175-L179)

Something to keep in mind: the more paths that we can consume for the "same" documentation, the more we have to support when trying to parse documentation. Case in point, the consideration of what to do when a user has buf.md, README.md, and README.markdown in the same module.