Summary

Make it easier for new contributors to understand the compiler, so that they can make better contributions sooner, and for current maintainers, so that they have a better idea for what's safe to merge.

Motivation

In my opinion, compiler development has slowed down again. Harry often gave feedback to various issues and PRs; now that he's stepped down, we have one less person who can do that. While I have contributed in the past and have write access, my understanding of this repo is lacking and I am hesitant to merge things for fear of breaking something. Others that I've talked to either aren't as interested in contributing to the compiler or have similar reservations as me.

One action that can help address this is adding more documentation to this repo. Once written, it'll make it easier for the next person to contribute something and for current maintainers to have more confidence as to whether they can merge something or not.

In the process, I believe a number of "clean up" / "tech debt" tasks will be discovered. For example, changing a data to a newtype, identifying where to do #4115, etc.

Proposal

There's a few pieces of documentation that are worth adding:

• docs on various types: what they are / represent and how they are used
• docs on functions: what their goal is and how they achieve that
• docs on support tools (e.g. linking to Happy's guide and clarifying the settings we use, clarifying why we don't use alex for lexing, etc.)
• an explanation for how the type checker works
• an ARCHITECTURE.md file that overviews the entire compiler from a very high level and discusses some of the inherit tradeoffs with its design and alternatives as well as references to things useful for understanding it

I think if we started at the top of the list and worked our way down, the writer of the harder parts of this (i.e. type checking and Architecture file) would be better informed before writing it.

Since this issue is quite large, I propose addressing it via many mini-PRs. If all of this was done in a single PR, it would be massive and likely bitrot as other changes get merged. However, if a few parts were documented in a single PR, it would be easier to review and merge. Moreover, there might be multiple passes. In the first pass, docs were added to some type that simply indicated what it represented. Later, when more parts of the compiler are documented and more understanding made, another pass could be added that provides insight into how the type is used.
Although the mini-PR approach might lead to a huge changelog that feels spammy, I feel like the alternatives are less likely to get merged.

Regardless, all such PRs should backlink to this issue so that we can see all that was done towards this effort.

Useful Links:

• Haddocks - Haskell Docs:
  • Haddocks User Guide
  • haddock-cheatsheet package - Haddocks by Example
• Typecheck links:
  • Simon Peyton Jones' "how GHC type inference engine actually works" (YouTube)
  • Academic / theoretical basis of the PureScript type system Discourse Post
  • Type Inference From Scratch
  • Flub: an example compiler for a functional language
  • Bidirectional Typing