Enhancement to scheduler logic and code refactoring.#1739
Merged
shahzebsiddiqui merged 10 commits intodevelfrom Apr 1, 2024
Merged
Enhancement to scheduler logic and code refactoring.#1739shahzebsiddiqui merged 10 commits intodevelfrom
shahzebsiddiqui merged 10 commits intodevelfrom
Conversation
…output/error file and exitcode that can be used by subclasses add logic for extracting output and error file in slurm using 'scontrol show job' at job submission time which will be useful for extracting the output. Add method get_output_and_error_files in base class Job that will be used by subclass to implement how each scheduler will extract output and error files
for more information, see https://pre-commit.ci
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## devel #1739 +/- ##
===========================================
- Coverage 80.85% 33.88% -46.97%
===========================================
Files 57 58 +1
Lines 6453 6467 +14
===========================================
- Hits 5217 2191 -3026
- Misses 1236 4276 +3040 ☔ View full report in Codecov by Sentry. |
…was some typos in variable names
…e job data that is implemented by subclass used for retrieving job record upon completion. The method jobdata will return job record which will be stored in internal variable self._jobdata. We renamed the methods in each subclass to use retrieve_jobdata to get the job records. Finally in each executor class we will invoke builder.job.jobdata() method to get the job data instead of having a return value from method which helps clean up code
for more information, see https://pre-commit.ci
executor names in buildspec
for more information, see https://pre-commit.ci
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR will make several enhancement and code refactoring to scheduling class.
Slurm job using
scontrol show jobThis PR will use
scontrol show jobto get output and error file instead of usingsacctsince we can use this information at job submission time to get the data. This will address an issue where we can't rely on extracting output from workdir if one specifies an alternative path such assacct -o /tmp/job.out -e /tmp/job.err test.shShown below is a sample build run with log enabled which shows
scontrol show jobis runFix bugs with Cobalt Scheduler
This PR will fix job submission issues related to Cobalt scheduler after code refactoring, we were able to submit job