Mem/disk mib fix and unit test#1
Mem/disk mib fix and unit test#1BEFH merged 2 commits intoBEFH:mem_disk_fixfrom hermidalc:mem_disk_fix
Conversation
|
maybe we should also test the calculation of For |
Sorry if I'm not up-to-date on standards, but for |
|
The issue is that the code is currently fixed but extremely literal. If you specify G or GB (technically an SI unit), it's GB. If you specify GiB, it's the binary unit. I don't know why you would ever would want to use SI units for memory, so what you're saying is potentially a good idea, but it also potentially breaks the distinction between |
Definitely should keep with the existing impl to not break people's code. I just tested it on 8.16 and |
|
4 GB yielding |
Yep I tested it with local and on the slurm cluster and 4G is 3815 MB in Snakemake 8.16, odd. I guess your fix will correct that too? |
|
I fixed that in my first commit, or at least should have. They were doing
the mib calculation for mb.
…On Thu, Aug 22, 2024 at 12:35 PM Leandro Hermida ***@***.***> wrote:
4 GB yielding mem_mb=3815 is incorrect behavior. It should be either 4000
or 4295 depending on implementation. The current value is a huge bug.
Yep I tested it with local and with on the slurm cluster and 4G is 3815 MB
in Snakemake 8.16, odd.
—
Reply to this email directly, view it on GitHub
<#1 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAZ2Z2BYI6AY36OQ3J2PAZ3ZSYHMVAVCNFSM6AAAAABM6HVCOGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBVGE4DSMBZGE>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
I added gb and tb to mib tests |
|
Thanks! is there any way you could change the base branch for the pull request from main to mem_disk_fix? |
Damn I don't know why it did that I was working locally on the mem_disk_fix branch I just double checked it. Have to figure out how to change it |
|
Thank you @BEFH it didn't look like I had the permissions to do it because I didn't get a dropdown menu to change the base branch I can only see a link there |
|
I figured out how to change it myself. I will be adding the MB to the outputs as well. |
Pixi install github action is failing with "failed to parse pypi name mapping" errors likely due to rate limiting when 30+ jobs are kicked off nearly simultaneously I tested this fix on snakemake#3820 since the tests kept failing due to the `pixi` install action failing. After committing this change, [the actions ran successfully](https://github.com/snakemake/snakemake/actions/runs/20684893321). In [this failing run's](https://github.com/snakemake/snakemake/actions/runs/20682879051/job/59383597583#step:3:3261) debug logs we see: ``` pixi install -e py311 [...] WARN resolve_conda{group=py313 platform=win-64}: reqwest_retry::middleware: Retry attempt #1. Sleeping 1.225245051s before the next attempt Error: × failed to parse pypi name mapping ├─▶ error decoding response body ╰─▶ expected value at line 1 column 1 ``` This warning is repeated many times until finally pixi stops retrying - this is what suggested to me that some sort of rate limit was the issue. One downside is that this does make the CI take a bit longer to run. We could consider using the `cache` feature of the pixi action. And turning up the max-parallel, or reducing the number of test-groups ### QC <!-- Make sure that you can tick the boxes below. --> * [ ] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [ ] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Chores** * Updated development toolchain dependencies for improved build and test infrastructure. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai -->
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).