Conversation
- Add default species selection (hsapiens, ecoli, scerevisiae) when none specified - Improve error handling with logger.error instead of logger.debug - Add detailed logging for task processing and completion - Update docstring to reflect default species behavior
…ings match/case, logging; LinearProbe: configurable classification_threshold, LR max_iter, simplified multilabel proba; Dataset: clean prints/len, logging; Hydra: expose options incl. classification_threshold; lint + callback tests pass
| from lobster.datasets import CalmPropertyDataset | ||
|
|
||
| from ._linear_probe_callback import LinearProbeCallback | ||
| from ._peer_utils import convert_numpy_to_python |
There was a problem hiding this comment.
nit: some of these utils seem more general than just for PEER so perhaps they can live elsewhere
There was a problem hiding this comment.
It's a good nit. I'll do some reorganizing in a separate PR to avoid this one getting too much bigger
| try: | ||
| train_embeddings, train_targets = self._get_embeddings(module, train_loader, modality="nucleotide") | ||
| test_embeddings, test_targets = self._get_embeddings(module, test_loader, modality="nucleotide") | ||
| if self.use_cross_validation: |
There was a problem hiding this comment.
nit: would self.use_cross_validation=False be the same as 1-fold CV?
There was a problem hiding this comment.
They would only differ in the test split, which would be an issue if we have dedicated test splits. 1-fold CV would merge the train + test and then do an iid random split. Also, linear_probe_callback uses sklearn's KFold, which requires 2+ folds
| return probe | ||
|
|
||
| def _evaluate_probe(self, probe, embeddings: Tensor, targets: Tensor) -> dict[str, float]: | ||
| def _evaluate_probe( |
There was a problem hiding this comment.
this is a long function 😅 might be out of scope of this MR but I wonder whether we should turn callbacks into Metrics which would implement the core logic and then the callback just calls metrics
There was a problem hiding this comment.
for now, we could probably just refactor the parts where we get metrics into a helper func
There was a problem hiding this comment.
That's a great idea! Just going to make the metric calls into helpers for this MR. We can carve this out into a separate module for metrics in another
|
do we need more tests for the new parts of the code? |
- Applied ruff check --fix to resolve 21 linting errors - Applied ruff format to ensure consistent code formatting - Updated callbacks, constants, datasets, and test files
Description
Type of Change