Skip to content

Initial implementation of dune fmt#1130

Merged
emillon merged 1 commit intomasterfrom
dune-fmt
Aug 20, 2018
Merged

Initial implementation of dune fmt#1130
emillon merged 1 commit intomasterfrom
dune-fmt

Conversation

@emillon
Copy link
Copy Markdown
Collaborator

@emillon emillon commented Aug 14, 2018

This is a first draft of dune fmt (see #940) with three main limitations:

  • it is language agnostic, so it does not know about field names
  • it is not able to parse comments
  • it does not break long lines

The formatting rules are pretty simple:

  • lists composed only of atoms, quoted strings, templates, and singletons are displayed on a single line
  • other lists are displayed with a line break after each element
  • an empty line is inserted between toplevel stanzas

The CLI is minimal: it can either read a file or standard input, but will always output on the standard input. In addition, it is necessary to pass --unstable to enable this feature, until some guarantees are given.

It is possible to test the output on all the dune files in the repository using the following command (zsh; possibly bash supports that glob syntax as well):

for x in test/**/dune ; do
    if [ ! -x $x ] ; then
        echo "; $x" >> /tmp/output
        ./_build/default/bin/main_dune.exe fmt < $x >> /tmp/output 2>&1
    fi
done

I am pretty satisfied with the output:

; test/blackbox-tests/test-cases/js_of_ocaml/lib/dune
(library
 (name x)
 (public_name x)
 (js_of_ocaml
  (flags (--pretty))
  (javascript_files runtime.js)
 )
 (c_names stubs)
 (preprocess
  (pps js_of_ocaml-ppx)
 )
)

; test/blackbox-tests/test-cases/loop/dune
(rule
 (copy %{read:x} a)
)

(rule
 (copy %{read:y} b)
)

(rule
 (progn
  (run true)
  (with-stdout-to
   x
   (echo b)
  )
 )
)

Indenting with only one space is pretty narrow, but this works well because it aligns with the left parens.

Let me know what you think.

@emillon emillon requested review from a user and rgrinberg August 14, 2018 16:01
@emillon
Copy link
Copy Markdown
Collaborator Author

emillon commented Aug 14, 2018

NB: this triggers some errors in some of the dune files, I intend to fix these.

@rgrinberg
Copy link
Copy Markdown
Member

rgrinberg commented Aug 14, 2018

Can we maintain the standard lisp formatting? Where trailing parens are on the same line. I don't see a reason to break with tradition here.

@ghost
Copy link
Copy Markdown

ghost commented Aug 14, 2018

I like the output, but I agree with @rgrinberg about trailing parentheses. If we don't group them they occupy a lot of lines.

It would be nice if we could also preserve the original formatting of strings. Could you also add a mode where it just reformat all the dune files in the source tree in-place?

@emillon
Copy link
Copy Markdown
Collaborator Author

emillon commented Aug 14, 2018

I don't have a lot of lisp experience, so I tried to keep this as simple as possible for now. Is something like this more idiomatic?

(rule
 (progn
  (run true)
  (with-stdout-to
   x
   (echo b))))

It would be nice if we could also preserve the original formatting of strings.

I believe that this will require an alternative parser, so I suggest that I do this in a separate PR (with comments as well).

Could you also add a mode where it just reformat all the dune files in the source tree in-place?

Sure!

@emillon
Copy link
Copy Markdown
Collaborator Author

emillon commented Aug 14, 2018

I don't see a reason to break with tradition here.

We can probably make two (related) arguments: it diffs poorly, and it's difficult to edit the end of a s-expression, for example to add a field to (library) or the like.

@ghost
Copy link
Copy Markdown

ghost commented Aug 15, 2018

Personally, i don't mind about breaking the tradition, all that matters is that it looks good. It's true that it might be annoying if it's hard to add a field at the end.

Do you know how we can integrate this with emacs? so that for example the file is automatically reformatted when saving the file to disk. I'd like to try to see how it feels.

@rgrinberg
Copy link
Copy Markdown
Member

We can probably make two (related) arguments: it diffs poorly,

What's the issue with diffing? It works quite well for me. Even better is the structural diff that is available in Emacs.

and it's difficult to edit the end of a s-expression, for example to add a field to (library) or the like.

With what editor? In emacs this is 1 key combination. In vim it shouldn't be very hard either. Just go to the last field, hit %i<return> and write.

Here's another point for the classical formatting: almost everyone uses it. The vast majority of jbuild files use it. You can check it yourself with cat $(find . -name jbuild) | less if you don't believe me. That signals preference towards the classical style.

src/dune_fmt.ml Outdated
)
| None ->
let lines = Io.input_lines stdin in
let contents = String.concat ~sep:"" lines in
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sep should be "\n", otherwise 2 lines that contain a single atom would end up concatenated as a single one.

@emillon
Copy link
Copy Markdown
Collaborator Author

emillon commented Aug 16, 2018

I don't want this PR to turn into a debate on what should be the definitive syntax. The point is to start from something easy to implement and then we can improve based on feedback.

Even better is the structural diff that is available in Emacs.

Emacs is very powerful to edit s-expressions, but not all dune users use it.

IMO having ) characters more visible will make dune easier to use for people less used to s-expressions.

Here's another point for the classical formatting: almost everyone uses it. The vast majority of jbuild files use it. You can check it yourself with cat $(find . -name jbuild) | less if you don't believe me. That signals preference towards the classical style.

I believe you, but this is a bit biased. For example, when contributing my first dune files I checked in this repo to see how this was already done here, and adjusted. But for fresh projects, I used the the more vertical style like this one (see also this one).

Do you know how we can integrate this with emacs? so that for example the file is automatically reformatted when saving the file to disk. I'd like to try to see how it feels.

I'm not familiar enough with elisp to provide a working example, but something like this should work:

; warning: untested
(defun dunefmt-before-save ()
  (interactive)
  (shell-command-on-region
   (point-min) (point-max)
   "dune fmt --unstable"))

(add-hook 'before-save-hook #'dunefmt-before-save))

@ghost
Copy link
Copy Markdown

ghost commented Aug 16, 2018

I agree, let's not discuss this eagerly. We need to try this to see what's best in practice, so merging this as an unstable feature that we can play with seems good.

@rgrinberg
Copy link
Copy Markdown
Member

Okay, we can leave the syntax aside. I still think its dubious to include the formatting tool if we can't even use it on our dune files.

Btw, do we really need this --unstable flag? I'd like to see some motivation for it as I think it's just bad UI to have a command that only works with a flag. We might as well rename the command to $ dune unstable-fmt.

@emillon
Copy link
Copy Markdown
Collaborator Author

emillon commented Aug 17, 2018

Btw, do we really need this --unstable flag?

The idea was to just remove that flag at the end rather than rename the command, but it's the same. I followed your suggestion and renamed the command.

@emillon emillon changed the title WIP: Initial implementation of dune fmt Initial implementation of dune fmt Aug 17, 2018
This is a first draft with three main limitations:

- it is language agnostic, so it does not know about field names
- it is not able to parse comments
- it does not break long lines

The formatting rules are pretty simple:

- lists composed only of atoms, quoted strings, templates, and
  singletons are displayed on a single line
- other lists are displayed with a line break after each element
- an empty line is inserted between toplevel stanzas

The CLI is pretty light: it can either read a file or standard input,
and fix a file in place. In addition, the command is named
`unstable-fmt` for now, until some guarantees are given.

Closes #940

Signed-off-by: Etienne Millon <me@emillon.org>
@emillon emillon merged commit 99d82c2 into master Aug 20, 2018
@emillon emillon deleted the dune-fmt branch August 20, 2018 08:41
@bobot
Copy link
Copy Markdown
Collaborator

bobot commented Aug 20, 2018

It is a nice addition. A syntax related comment (I don't know where you want to keep them). The following seems not right. Instead of the hard rules you used for when line break should appear, can't you use a more dynamic box-like approach (e.g. Format? ) so that this is on one line?

  (with-stdout-to
   x
   (echo b)
  )

@emillon
Copy link
Copy Markdown
Collaborator Author

emillon commented Aug 20, 2018

Thanks for the feedback! Can you open a separate issue for this?

@bobot
Copy link
Copy Markdown
Collaborator

bobot commented Aug 20, 2018

Done in #1153.

rgrinberg added a commit to rgrinberg/opam-repository that referenced this pull request Sep 14, 2018
CHANGES:

- Ignore stderr output when trying to find out the number of jobs
  available (ocaml/dune#1118, fix ocaml/dune#1116, @diml)

- Fix error message when the source directory of `copy_files` does not exist.
  (ocaml/dune#1120, fix ocaml/dune#1099, @emillon)

- Highlight error locations in error messages (ocaml/dune#1121, @emillon)

- Display actual stanza when package is ambiguous (ocaml/dune#1126, fix ocaml/dune#1123, @emillon)

- Add `dune unstable-fmt` to format `dune` files. The interface and syntax are
  still subject to change, so use with caution. (ocaml/dune#1130, fix ocaml/dune#940, @emillon)

- Improve error message for `dune utop` without a library name (ocaml/dune#1154, fix
  ocaml/dune#1149, @emillon)

- Fix parsing `ocamllex` stanza in jbuild files (ocaml/dune#1150, @rgrinberg)

- Highlight multi-line errors (ocaml/dune#1131, @anuragsoni)

- Do no try to generate shared libraries when this is not supported by
  the OS (ocaml/dune#1165, fix ocaml/dune#1051, @diml)

- Fix `Flags.write_{sexp,lines}` in configurator by avoiding the use of
  `Stdune.Path` (ocaml/dune#1175, fix ocaml/dune#1161, @rgrinberg)

- Add support for `findlib.dynload`: when linking an executable using
  `findlib.dynload`, automatically record linked in libraries and
  findlib predicates (ocaml/dune#1172, @bobot)

- Add support for promoting a selected list of files (ocaml/dune#1192, @diml)

- Add an emacs mode providing helpers to promote correction files
  (ocaml/dune#1192, @diml)

- Improve message suggesting to remove parentheses (ocaml/dune#1196, fix ocaml/dune#1173, @emillon)

- Add `(wrapped (transition "..message.."))` as an option that will generate
  wrapped modules but keep unwrapped modules with a deprecation message to
  preserve compatibility. (ocaml/dune#1188, fix ocaml/dune#985, @rgrinberg)

- Fix the flags passed to the ppx rewriter when using `staged_pps` (ocaml/dune#1218, @diml)

- Add `(env var)` to add a dependency to an environment variable.
  (ocaml/dune#1186, @emillon)

- Add a simple version of a polling mode: `dune build -w` keeps
  running and restarts the build when something change on the
  filesystem (ocaml/dune#1140, @kodek16)

- Cleanup the way we detect the library search path. We no longer call
  `opam config var lib` in the default build context (ocaml/dune#1226, @diml)

- Make test stanzas honor the -p flag. (ocaml/dune#1236, fix ocaml/dune#1231, @emillon)

- Test stanzas take an optional (action) field to customize how they run (ocaml/dune#1248,
  ocaml/dune#1195, @emillon)

- Add support for private modules via the `private_modules` field (ocaml/dune#1241, fix
  ocaml/dune#427, @rgrinberg)

- Add support for passing arguments to the OCaml compiler via a
  response file when the list of arguments is too long (ocaml/dune#1256, @diml)

- Do not print diffs by default when running inside dune (ocaml/dune#1260, @diml)

- Interpret `$ dune build dir` as building the default alias in `dir`. (ocaml/dune#1259,
  @rgrinberg)

- Make the `dynlink` library available without findlib installed (ocaml/dune#1270, fix
  ocaml/dune#1264, @rgrinberg)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants