I think this may be a feature request/issue report rather than a question, so I am posting here rather than on the discourse.
I am basically wondering: do Distributed.addprocs and the CLI flag --heap-size-hint interact with eachother, and, if so, how? If I specify --heap-size-hint to the head Julia process, and then dynamically create Julia processes with addprocs, how (if?) does it get split up among each of the processes created?
My current workaround is to pass --heap-size-hint to exeflags in each addprocs call, and divvy up the memory into smaller chunks, but this seems a bit hacky.
Just wondering about this because I've ran into various segfaults which I've never been able to track down (#47957). I realized that memory usage seems to explode when using multiple processes which are allocated dynamically, but the memory can be controlled by setting, e.g., --heap-size-hint when dynamically creating the workers.
I think this may be a feature request/issue report rather than a question, so I am posting here rather than on the discourse.
I am basically wondering: do
Distributed.addprocsand the CLI flag--heap-size-hintinteract with eachother, and, if so, how? If I specify--heap-size-hintto the head Julia process, and then dynamically create Julia processes withaddprocs, how (if?) does it get split up among each of the processes created?My current workaround is to pass
--heap-size-hinttoexeflagsin eachaddprocscall, and divvy up the memory into smaller chunks, but this seems a bit hacky.Just wondering about this because I've ran into various segfaults which I've never been able to track down (#47957). I realized that memory usage seems to explode when using multiple processes which are allocated dynamically, but the memory can be controlled by setting, e.g.,
--heap-size-hintwhen dynamically creating the workers.