Skip to content

Ensure that C allocation functions do not trigger callbacks#8897

Merged
dra27 merged 2 commits intoocaml:trunkfrom
stedolan:fix-alloc-async
Aug 29, 2019
Merged

Ensure that C allocation functions do not trigger callbacks#8897
dra27 merged 2 commits intoocaml:trunkfrom
stedolan:fix-alloc-async

Conversation

@stedolan
Copy link
Copy Markdown
Contributor

Despite reviewing and merging #8691, I didn't notice a serious bug in that patch: it fails to prevent callbacks during allocation functions in the codepaths that allocate directly to the major heap (for large allocations).

The fix is trivial. Most of the code in this PR is to add some testing for this invariant. The test works by setting up a finaliser, then doing lots of allocation from C and verifying that the finaliser doesn't actually run until control returns to OCaml.


/* Called by [Alloc_small] when [caml_young_ptr] reaches [caml_young_limit].
We have to either call memprof or the gc. */
We may have to either call memprof or the gc. */
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a finaliser (or other async callback) is pending, then in native code this function will be called on each minor allocation even when the heap is not exhausted. The code of the function will correctly do nothing in this case, but the comment suggested otherwise.

Copy link
Copy Markdown
Member

@dra27 dra27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary (or possible) to consider doing anything clever to try to enforce the correct function being called? i.e. does the interface of caml_check_urgent_gc want to change instead?

@stedolan
Copy link
Copy Markdown
Contributor Author

I think the thing that enforces the correct function being called is testing! (The unit test in this patch fails if any of the check_gc_without_async_callbacks are changed back to check_urgent_gc)

@stedolan stedolan force-pushed the fix-alloc-async branch 3 times, most recently from 3c4d9d4 to 711db11 Compare August 27, 2019 13:43
@jhjourdan
Copy link
Copy Markdown
Contributor

Well spotted. The fix seems good to me.

Sorry for having forgotten these calls when submitting #8691.

@jhjourdan
Copy link
Copy Markdown
Contributor

Seems like MSVC is not happy with your tests, @stedolan !

@dra27
Copy link
Copy Markdown
Member

dra27 commented Aug 27, 2019

Hmm, it’s not happy with that tweak either... I have another test running locally...

@dra27
Copy link
Copy Markdown
Member

dra27 commented Aug 28, 2019

OK, my tweak should stay as that restores compatibility with older C compilers (really must update configure to add -Wdeclaration-after-statement for gcc...)

The problem is that you have a C identifier called test which can't be used as an identifier in MASM. While this is clearly a bug in ocamlopt, I haven't managed to work out the required MASM fu to make this happen, so for this PR the quickest thing will be to rename the C stub to something else, preferably not mov.

@jhjourdan
Copy link
Copy Markdown
Contributor

Thanks, @dra27, for investigating. I pushed a fix.

Once CI passes, we will be able to squash+merge.

@dra27
Copy link
Copy Markdown
Member

dra27 commented Aug 28, 2019

As it happens, on the OCaml side you can call the stub whatever you like - the issue was only with the C naming. Further investigation reveals even cl generates invalid assembler (using cl /Fa) for this case. Apparently ml64 assumes all identifiers will begin with an underscore...

Copy link
Copy Markdown
Member

@dra27 dra27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've squashed my commit into Stephen's original commit and simplified the Windows fix as outlined above (and tested it...!). I'll let CI pass again for the sake of it, but I'd like merge rather than squash to keep those two commits as if/when #8901 gets fixed I'll revert the second commit as a simple regression test.

@dra27 dra27 merged commit 2195dba into ocaml:trunk Aug 29, 2019
@jhjourdan
Copy link
Copy Markdown
Contributor

Many Thanks, @dra27!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants