Skip to content

Include original source dirs in dune describe#3834

Closed
sigwinch28 wants to merge 2 commits intoocaml:mainfrom
sigwinch28:include-original-source-dirs-in-dune-describe
Closed

Include original source dirs in dune describe#3834
sigwinch28 wants to merge 2 commits intoocaml:mainfrom
sigwinch28:include-original-source-dirs-in-dune-describe

Conversation

@sigwinch28
Copy link
Copy Markdown

For ROTOR we'd like to have access to the original source files in the output of dune describe, not the locations of these artifacts in dune's build directory.

In this PR we just drop the build dir from the paths, but this doesn't always work, e.g. when the .ml/.mli file has been created by a rule.

In addition, we add a new top-level root key which tells us the absolute path to the current workspace root because we don't want to have to walk the filesystem to work it out, like dune already does.

This is a work in progress and is definitely not ready for merging yet, especially since the recent reorganisation of dune's code (rules split?)

Whenever possible, drop the build context from source paths returned for ml and
mli files when describing the current workspace.
Additionally, introduce a new top-level variant "workspace" which provides the
root of the current workspace as an absolute path, and information about
libraries required in the current context.
@bobot
Copy link
Copy Markdown
Collaborator

bobot commented Sep 30, 2020

It seems nice to have. But we should start to make dune describe backward compatible, so just to remind that the final version should trigger the changes only for a new version of dune.

Remarks:

  • Why changing (library ...)(library ...) to (libraries (...)(...))?
  • source_dir is changed to the source directory: Is the build directory not useful for someone? A new build_directory field? However since the build directory has no backward compatibility perhaps we should not advertise people to look into _build.

For "how to find the real source file", I don't know but the .cmt should have this information, no?

@ghost
Copy link
Copy Markdown

ghost commented Sep 30, 2020

@bobot, about backward compatibility: currently dune describe is still unstable. The only version that is accepted by the --lang argument is 0.1 to reflect that. Given that ROTOR is currently the only consumer and there is not yet a stable release of ROTOR using dune describe, we just need to synchronise between us. If someome else wants to consume the output of dune describe pro-grammatically, or ROTOR is ready for a stable release, then we should stabilise the output of dune describe and switch to the same version number as the one we use with (lang dune x.y).

The intent is to keep the development as light as possible until a production software needs it.

@nojb
Copy link
Copy Markdown
Collaborator

nojb commented Oct 25, 2020

Given that ROTOR is currently the only consumer and there is not yet a stable release of ROTOR using dune describe, we just need to synchronise between us. If someome else wants to consume the output of dune describe pro-grammatically, or ROTOR is ready for a stable release, then we should stabilise the output of dune describe and switch to the same version number as the one we use with (lang dune x.y).

Just for info, at LexiFi we have a number of .cmt-based tools that we use to handle refactoring, large-scale migrations, etc and we are currently experimenting with using dune describe to drive them (previously, under omake we just looked for the .cmt file next to the source file).

@nojb
Copy link
Copy Markdown
Collaborator

nojb commented Oct 25, 2020

  • source_dir is changed to the source directory: Is the build directory not useful for someone? A new build_directory field?

Yes, having both build_dir and source_dir seems a good idea (especially because the build directory can be separate from the source directory in general).

@nojb
Copy link
Copy Markdown
Collaborator

nojb commented Oct 25, 2020

Is it possible to know if a file is generated or not when doing dune describe? If yes, it may be better to include an optional field with the file in the source directory in addition to the filename in the build directory.

@nojb nojb self-assigned this Oct 25, 2020
@nojb
Copy link
Copy Markdown
Collaborator

nojb commented Oct 25, 2020

Another possibility instead of blindly dropping the build prefix from the filenames in dune describe (which can result in non-existent filenames in general), is to include the build prefix (typically _build/<context>) in the output of dune describe and leave it to the consumer to apply the heuristic of just dropping that prefix from the filenames reported in the output to get the "original" source.

@sigwinch28
Copy link
Copy Markdown
Author

Is it possible to know if a file is generated or not when doing dune describe? If yes, it may be better to include an optional field with the file in the source directory in addition to the filename in the build directory.

Yes, I was having trouble with this.

We want access to the original source files wherever possible because we want to apply changes to the original source files instead of the copied/generated sources in the build directory.

I don't know enough about Dune's internal architecture to retrieve this information: isn't there some internal mapping or reverse map that we can use to determine which files came from where, even including rules?

include the build prefix (typically _build/) in the output of dune describe and leave it to the consumer to apply the heuristic of just dropping that prefix from the filenames reported in the output to get the "original" source.

In order to properly apply this heuristic we also need _build/<context> in the output of dune describe because otherwise, just like omitting the workspace root, we would have to guess.

Base automatically changed from master to main January 14, 2021 17:08
@rgrinberg
Copy link
Copy Markdown
Member

I don't think ROTOR is on the cards for dune anymore.

@rgrinberg rgrinberg closed this Nov 4, 2021
@ghost
Copy link
Copy Markdown

ghost commented Nov 4, 2021

@rgrinberg from what I heard recently, it's still in development.

@rgrinberg
Copy link
Copy Markdown
Member

Ah well they're welcome to re-open with a fresh PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants