[ new ] explicit polarity annotations#6385
Conversation
4b5b41c to
63fad8e
Compare
63fad8e to
58907f1
Compare
|
One thing I'm still unsure is whether |
58907f1 to
1df02df
Compare
|
Re. the TODO items:
My two cents: Something like "Introduced explicit polarity annotations (user guide link), enabled with the Yes add a flag, yes add this to the user guide, and please more tests 🙂 I have a question about the actual feature, though, even before looking at the code changes — though I'm not sure my input there will be of any value 😅 Suppose you had something like... {-# OPTIONS --without-K #-}
data Id {l} {A : Set l} (x : A) : A → Set l where
refl : Id x x
-- for example:
precast : ∀ {l l'} {A B : Set l} {C : Set l'} → @++ Id A B → (A → C) → (B → C)
precast refl f = f
-- or even generically:
subst : ∀ {l l'} {A : Set l} {B : A → Set l'} {x y : A} → @++ Id x y → B x → B y
subst refl x = xIs this code modality-correct? No, right? Is that enforced right now? |
Right now it is forbidden, precisely because we don't know yet what semantics to give to pattern-matching on arguments annotated with polarity != mixed. The error is the following: We really only know the semantics for annotated variables of type I'll let the modal theorists of the team decide whether this is modality-correct, to my naive eye this is... fine? As for tests, documentation and changelog, very good, on it, thanks! |
So two things:
|
|
Right, I'm okay with having all matching on non- precast p f = transp (λ i → p i → C) i0 fand the occurrence of |
|
@plt-amy I think the occurrence of Strict positivity escapes me but for all the other modalities I would be ok to allow pattern matching even for multiple constructors, even for the An example where positive pattern matching is useful, is when creating covariant functions from |
Is it necessary to check the full type again, or would it suffice to check the polarity annotations? Or could the polarity annotations be checked the first time?
Given how much trouble subtyping seems to cause for meta-variable unification I am skeptical towards adding more subtyping to Agda.
I think there should be a flag called something like
Have you thought about how you would handle induction-recursion?
agda/src/full/Agda/Syntax/Common.hs Lines 619 to 624 in b5105f4 |
It's not strictly necessary to check the full type, we could also get away with a specialized function that traverses types in the telescope and only checked polarity annotations, however that would probably end up duplicating a lot of code down the line. This approach seems more robust to me, although the internal checker doesn't completely re-check everything, for example the other modalities. Checking the polarity annotations the first time wouldn't really be possible: what we actually want to do is check that the fields of each constructor use the parameters with the proper polarities, but the types of the constructors themselves don't have to respect the polarities of the parameters. Here's an example, and an explanation why: data Ex (@++ A : Set) : Set where
cons : A → Ex A
However, All of that wall of text to say that no, we unfortunately need to extract the telescope before being able to type-check.
Right. We'll try to follow what you implemented for erasure on that front.
No, not yet. The first roadblock we encountered was for indexed datatypes, for which we don't have a satisfiying answer yet. Induction-recursion is still off the table for now.
Maybe this could be refactored into a generic solution for all modalities, instead of being specific to quantity. To display better error messages for polarities, we already do something similar where the polarity modality actually carries the original polarity the user specified as well as the currently used polarity. That could also be done for other modalities. Apart from that, did anyone have a chance at trying out the interface? |
What are the performance implications of your choice? If it currently takes a lot of time to type-check a given constructor declaration, does it take twice as much time if your commits are applied? I think constructor declarations rarely take a lot of time to type-check, so perhaps this is not a major problem. However, I think this choice and the performance implications should at least be documented clearly in the source code.
The Agda source code contains other |
I definitely agree and we will document it. I also think the impact on performance should be insignificant, but is there any documented way to test this out? Is the Agda CI benchmarking the test suite (or any suite) and tracking performance degradation between versions? How should we go about this, in a more principled manner than just using
Right, I think Josselin was specifically talking about modalities here. It can become fairly annoying to wrap this information inside every modality constructor, and perhaps if each modality currently implemented has to keep track of where it comes from (inferred or explicitely annotated, and where), maybe it would be good to think about moving this out of constructors for each modality and into a bundle carried along with modality information, or something.
Definitely, which is why we stayed out of it. Although precisely because Agda doesn't have subtyping for function types with modalities, users really don't rip the benefits that using them more often could provide. When you start using them, you have to redefine everything, which can grow tiresome. Perhaps a reliable solution doesn't exist, but it looks like subtyping is the key requirement to make their use practical, for some meaning of practical. |
1df02df to
6cbeca4
Compare
|
I just rebased on top of your recent erasure changes @nad. I also added some documentation, although it is pretty terse for now, and fixed some failing tests (among them, As for a possible Regarding tracking whether the user had explicitely annotated a variable or not, it could be doable, but I haven't found in practice that polarities show up unless actually used. |
|
So I'm investigating the test failure of From my limited understanding, that new check introduces new size constraints. If we comment out the rechecking, we get a cluster with and with the rechecking, we get the additional In the first case, after converting the size constraints to a common context, everything seems fine, but in the second case, |
|
Could someone summarize the current state of this PR and make a todo list of what still needs to be done to bring it into a mergable state? |
Sure, let me try: we should have the basic functionality ready, with the omission of some that we don't have a good justification for, highlighted in the top post. Here is the state of the todo list, ordered by rough priority:
|
f89d377 to
4eeb2db
Compare
|
So I think I've completed the above todo list, but some questions remain: by adding the In other news, I've added an entry to the changelog and updated the manual a bit to clarify and expand on what was there before, as well as mention the new option. I chose not to explain what polarity is in general and assume that the reader already knows, so that might make it a bit unintelligible, feedback welcome. WDYT? |
|
Please add the flag agda/doc/user-manual/tools/command-line-options.rst Lines 1106 to 1108 in 1c5a4e9 agda/doc/user-manual/tools/command-line-options.rst Lines 1145 to 1147 in 1c5a4e9 |
|
If the problem with sized types is only that some meta-variables get eagerly instantiated to |
|
Thanks @jespercockx for the review! I realised this morning with @anuyts that we actually have some problems with polarities and modules. The following type-checks: module _ where
data ⊤ : Set where
tt : ⊤
module _ (@- A : Set) where
@- test : Set
test = A → ⊤
test2 : Set
test2 = test → ⊤We know exactly why this happens, but it'll need some work to fix it. Basically, I wanted to get a minimal version working without changing too much and thus was following the quantity/relevance modalities implementation closely, but some particularities of these modalities let them get away with some lazy checking of definition annotations that we can't mirror. So, the plan is to rather follow the more morally correct cohesion modality. This will require some pretty annoying changes (which I had already done before but stashed somewhere else), such as having different default annotations for binders and for definitions... Sorry to have wasted your time, although the documentation part will surely not change too much and so the comments will definitely still be applicable! |
Co-authored-by: Joris Ceulemans <joris.ceulemans@kuleuven.be>
Co-authored-by: Joris Ceulemans <joris.ceulemans@kuleuven.be>
Co-authored-by: flupe <lucas@escot.me>
We also need to remove a part of a test that relied on a non-normalizable type in a postulate, since we normalize postulate types to get their argument polarities. Co-authored-by: Malin Altenmüller <malin.altenmueller@gmail.com>
This mitigates unwanted performance issues when re-checking constructors of datatypes when polarities are not used at all.
Co-authored-by: flupe <lucas@escot.me>
0a3ba43 to
f99227c
Compare
|
I did some interactive rebasing so now the resolution of the import cycle comes before all the other commits in this PR. That way the individual commits should remain buildable (as an early Christmas gift for @andreasabel). I also added the new warning to the user manual. |
|
I added a short explanation on the likely performance impact of running the internal checker when checking constructors with the |
|
This (finally) looks like it's in a good enough state to be merged into master. Thank you @flupe and all other people who contributed to this PR! |
…Occurrences The deep-strictness of argOccurrences computed by the positivity checker was lost in PR #6385. (See 32e8c29#r173041607.) On the std-lib-test, the restored strictness is not significant in the workload sharing between Positivity and Polarity, but @nad has an example with a very large mutual block (and thus positivity graph) where it matters.
Joint work from @JorisCeulemans, Malin Altenmüller, @anuyts, @flupe and @jpoiret,
started during the last AIMXXXI. (This draft co-authored by @jpoiret).
This PR introduces explicit polarity annotations, as requested in #570 (and #2411).
It is now possible to specify in function types that a specific variable must be
used strictly-positively (or any other polarity), and this will be enforced during typechecking.
The following (usual) definitions of
MuandFreeare now valid:For a more complete example, see
test/Succeed/PolarityAnnotations.agda.This PR is purposefully kept minimal, in order to leave the chunk of code to review manageable. The first part of the PR is mostly some bug fixing related to modalities in Agda.
All tests still pass, albeit slightly modified!
Because some modality machinery has been slightly changed, some further testing with actual codebases that use modalities (esp. Cohesion) a lot would ensure that this doesn't break them: we've mostly used tests and simple examples to test until now.
What has been done
A new modal system has been added: polarity. It reuses the
Modalityabstractions already in place in the code base to handle erasure, cohesion and irrelevance. The possible (endo)modalities for this modal system are@++(strictly positive),@+(positive),@-(negative),@mixed(mixed polarity),@unused(parameter is unused). The default polarity for either declarations or binders is@mixed, so that you're able to use them in any way you want.The typechecker has been modified to account for this new modal system.
In particular, the rule for checking dependent function types divides polarities in the context by
@-when checking the domain.The positivity checker is now aware of the polarity annotations that appear in the types of (datatype and record) parameters (hence why
MuandFreecan now be defined).The (explicit) polarity annotations get priority over what the positivity checker is able to infer.
In particular, some things that the positivity checker could not see as strictly-positive can now be fully annotated, and as long as it typechecks, it can be used as a strictly-positive function later on. (See
test/Succeed/PolarityProduct.agda. Without any annotations, the positivity checker alone cannot figure out thatn-aryis strictly-positive overX)The polarity information for a given variable does not contain simply one polarity, but actually 3 polarities:
This was implemented by Joris in order to have more understandable error messages.
Maybe we want to be even more informative about polarity changes, alike the positivity checker's reports, but this is pretty nice already.
Using
{-# POLARITY ... #-}on a postulate whose type has already been explicitely annotated with polarities (other than@mixed) will throw an error. We're a bit hesitant to deprecate thePOLARITYpragma yet, because it doesn't change the type of the postulate, it merely informs the positivity checker. Until the work on subtyping (see below) has been implemented, it is therefore more flexible than polarity annotations.Limitations
While it's clear that having
@++ List : @++ Set -> Setin the context when checking the types of the arguments ofListconstructors should be enough to enforce strict-positivity, the situation for mutual datatypes is less clear, see future work below.Because the semantics of using a polarity modality are still being investigated, we are unsure of the following things and have chosen to either not implement them or specifically forbid them for now:
Splitting on an argument that has a non-mixed polarity: this should be okay if the given argument only has one applicable constructor, with the constructor fields inheriting the annotation. This is probably okay for all modalities as well.
Having non-mixed fields in record types: this should be fine in general for modalities if we're following Modal Type Theory, but we would lose eta-equality or even projections depending on which annotations we choose! In the worst case, they would just because data declarations with one constructor and nothing else.
Annotating indices of datatype declarations with non-mixed polarities: here, we just don't know how that should behave.
So currently we prevent them altogether and throw an error if an annotation was written.
(Therefore indices always get assigned polarity
@mixed, as already does the positivity checker)Subtyping:
Currently we decided to stay away from subtyping to keep things simple. Note that this is also the case for the other modal systems. Figuring out the typing rules
with subtyping is hard. Because of this limitation:
F : @++ Set -> Setwhere something of typeSet -> Setis expected, you'd have to eta-expand manually, as you would for erasure.F : Set -> Setcan never use it in place of something of type@++ Set -> Set, even if the positivity checker does detect thatFis strictly-positive.To do
Things that may have to be done before merging this PR.
--erased-matchesand--erasure#6349 suggests preventing expliciterasure annotations without
--erasure. We may want to do the same for our new annotation. One issue is that polarities can already appear in error messages about modalities, even when they aren't explicitely used.Future work
Refactoring modalities
This will probably be the subject of another issue/PR, but the modal type theorists of the team
seem to think that a refactor of Agda's current handling of modalities is in order,
and would be beneficial in the long term.
Things that could be explored that way would be:
Builtin
fmapWe are now able to define
Mu, the least fixed-point of any strictly-positive functor.However, as of now it is not particularily useful, because you cannot define a generic
foldoverMu.Indeed, to make this possible, you would need
fmapreadily available for anyF : @++ Set -> Set.We've already discussed this, and think it should be sound to add a new primitive to Agda:
that internally will have the correct computational behavior and be known to be size-preserving by the
termination checker. It could even be enough to require
Fpositive rather than strictly positive. This would essentially be a directed type theory version of cong combined with directed univalence, so adding that with the proper computational behavior would be quite a feat already.Typechecking (only) as an alternative to the positivity checker
An interesting direction would be to try and get rid of the positivity checker, and rely solely
on this new polarity modal system and its associated typing rules.
When modifying the positivity-checker to account for our annotations, it became clear that it is
a very intricate beast. There are many edge cases to take care of in its implementation, and any modification can lead to weird bugs. Replacing it with a principled set of typing rules seems like a worthy investigation.
Of course, if we want to stay backwards-compatible (and we do), this means we:
would have to infer the most precise type for every definition i.e have:
be inferred to have actual type
@++ Set -> Set.Side note: For this to work, we would need to distinguish between unannotated variables and variables annotated with
@mixedso as to run the analysis only for un-annotated variables. Currently there is no distinction.must have subtyping so that
Id : @++ Set -> Setcan still be used in place ofSet -> Setwithout requiring manual eta-expansion.(Users that don't care about annotations shouldn't have to do extra work)
have to figure out the rules when checking a block of mutually-defined datatypes, as discussed above (and the ton of weird datatypes allowed by Agda).
must find a way to not clutter pretty-printing with annotations that the user dit not write but were inferred.
Comments welcome!