Skip to content

Make CPU fuser tests less flaky (and report better errors)#11897

Closed
zou3519 wants to merge 4 commits intopytorch:masterfrom
zou3519:jit-mem-27
Closed

Make CPU fuser tests less flaky (and report better errors)#11897
zou3519 wants to merge 4 commits intopytorch:masterfrom
zou3519:jit-mem-27

Conversation

@zou3519
Copy link
Contributor

@zou3519 zou3519 commented Sep 20, 2018

This pr wraps the flaky system() call in a new function that does the following:

  • call malloc_trim() before each system() call used by the CPU fuser.
    This is to get rid of excess memory that may prevent a fork()

I also improved the error message for when system() fails so one knows from
the logs what is wrong ("Could not allocate memory, etc")

Test Plan: code reading (I don't really have better ideas). I've verified through a debugger that the posix_spawn + vfork path is taken on two of my machines.

cc @apaszke @ezyang

@pytorchbot pytorchbot added the oncall: jit Add this issue/PR to JIT oncall triage queue label Sep 20, 2018
@zou3519 zou3519 changed the title Make CPU fuser tests less flaky Make CPU fuser tests less flaky (and report better errors) Sep 20, 2018
@zou3519 zou3519 force-pushed the jit-mem-27 branch 7 times, most recently from b8256fb to 8d755f3 Compare September 21, 2018 20:50
@zou3519
Copy link
Contributor Author

zou3519 commented Sep 24, 2018

Mac build is flaky; this should be ready for review

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zou3519 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

This pr wraps the flaky system() call in a new function that does the following:
- call malloc_trim() before each system() call used by the CPU fuser.
  This is to get rid of excess memory that may prevent a fork()

I also improved the error message for when system() fails so one knows from
the logs what is wrong ("Could not allocate memory, etc")

Test Plan: code reading
@zdevito zdevito removed their request for review February 13, 2019 01:22
@gchanan gchanan removed their request for review February 28, 2019 16:38
@zou3519
Copy link
Contributor Author

zou3519 commented Mar 12, 2019

Not needed anymore

@zou3519 zou3519 closed this Mar 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

oncall: jit Add this issue/PR to JIT oncall triage queue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants