grades_cog: update to pydantic 2.0.0 and add support for P/F classes#885
grades_cog: update to pydantic 2.0.0 and add support for P/F classes#885sctigercat1 wants to merge 6 commits intoClemBotProject:masterfrom
Conversation
✅ Deploy Preview for clembotdocs canceled.
|
|
Black linting failed: Please run |
|
Black linting failed: Please run |
|
@sctigercat1 we do all the normalize/fix logic in the normalize scripts here https://github.com/ClemBotProject/ClemBot/blob/master/ClemBot.Bot/scripts/normalize_grade_data.py should be just a copy paste then run it against the existing assets |
|
Hmm actually that script is not idempotent thats very annoying what was i thinking 🤦 |
|
Ok, @sctigercat1 updated the script in 9eb295e to make it idempotent and also ended up pulling in the #### fix as well. But the other changes and the has_numeric fix we can merge yours after a rebase |
| # clean some p/f data data | ||
| for col in ("P", "F(P)"): | ||
| self.grades_df[col] = ( | ||
| self.grades_df[col] | ||
| .astype(str) | ||
| .str.replace("####", "100%", regex=False) # #### means 100% pass | ||
| .str.replace("%", "", regex=False) | ||
| .pipe(pd.to_numeric, errors="coerce") | ||
| .fillna(0) | ||
| ) | ||
|
|
||
| self.grades_df.loc[self.grades_df[col] > 1, col] = ( | ||
| self.grades_df.loc[self.grades_df[col] > 1, col] / 100 | ||
| ) |
There was a problem hiding this comment.
This can be removed now that its in the script itself
Encountered this error in the discord earlier when running the grades_cog:
Looked into it and seems it broke after commit 9b64210, which I suspect was due to a change in the pydantic dependency. I suppose one of the columns for which it is trying to calculate mean has some non-numeric data which maybe it was previously ignoring but no longer is. I figured the easiest solution would be to add the
numeric_onlyflag to the mean function which seems to solve the issue and will also protect against future non-numeric data in the CSV files.Separately, I have also added some support for classes with pass/fail grades. Most classes are either universally A-F or universally P/F and I allow one random semester with a different grade type to be ignored in the event that something just weird happened that semester. Each class will only show either A-F or P/F depending on its type. There is also a provision for a very small number of classes that have had multiple semesters with both A-F and P/F grading and in those cases it shows A-F and P/F percentages.
These could potentially be two separate PRs which I can separate if y'all would like!
Thanks