Skip to content

sbApp --plot: limit num_cores based on max memory#646

Merged
yunjunz merged 1 commit intoinsarlab:mainfrom
yunjunz:plot
Aug 25, 2021
Merged

sbApp --plot: limit num_cores based on max memory#646
yunjunz merged 1 commit intoinsarlab:mainfrom
yunjunz:plot

Conversation

@yunjunz
Copy link
Member

@yunjunz yunjunz commented Aug 25, 2021

Description of proposed changes

  • smallbaselineApp.py: limit the number of parallel processes for plotting based on the max memory from the template file, to avoid exceeding the memory. Circle CI kills processes because of this. Note that matplotlib reserved much larger memory than it actually uses.

For users who want to use a lot of parallel processes for plotting, please increase your mintpy.compute.maxMemory setting. @falkamelung.

  • view.py: add --memory option. Note this is NOT used in the smallbaselineApp.py

  • plot.auto_multilook_num():

    • more detailed multilook num list, for more detailed memory management.
    • add max_memory arg to adjust according. Details are in the code comments.
  • add quotes to the version control for conda/mamba install in the command line in docs/installation.md and .circleci/config.yml files

  • circleci: use pip from conda to avoid warning msg

  • docs/api/attributes.md: add EPSG (save_gdal.py does not work with hyp3 data #644)

Reminders

  • Pass Codacy code review (green)
  • Pass Circle CI / local test (green)
  • Make sure that your code follows our style. Use the other functions/files as a basis.
  • If modifying functionality, describe changes to function behavior and arguments in a comment below the function declaration.
  • If adding new functionality, add a detailed description to the documentation and/or an example.

+ smallbaselineApp.py: limit the number of parallel processes for plotting based on the max memory from the template file, to avoid exceeding the memory. Circle CI kills processes because of this. Note that matplotlib reserved much larger memory than it actually uses.

For users who want to use a lot of parallel processes for plotting, please increase your `mintpy.compute.maxMemory` setting.

+ view.py: add --memory option. Note this is NOT used in the smallbaselineApp.py

+ plot.auto_multilook_num():
   - more detailed multilook num list, for a more detailed memory manegement.
   - add max_memory arg to adjust according. Details are in the code comments.

+ add quotes to the version control for conda/mamba install in command line in docs/installation.md and .circleci/config.yml files
+ circleci: use pip from conda to avoid warning msg
+ docs/api/attributes.md: add EPSG
@yunjunz yunjunz merged commit 08ae0e0 into insarlab:main Aug 25, 2021
@yunjunz yunjunz deleted the plot branch August 25, 2021 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant