nn doesnt work on TOYv2.
bug
works on TOYv1 but v2 is too small, early stopping set is empty. we might catch this/fail gracefully or allow empty ES sets (fall back to no ES if ES is too small)
it's an edge case bug for tiny datasets so maybe a wontfix
```python
from drevalpy.experiment import drug_response_experiment
from drevalpy.models import MODEL_FACTORY
from drevalpy.datasets import AVAILABLE_DATASETS
naive_mean = MODEL_FACTORY["NaivePredictor"] # a naive model that just predicts the training mean
enet = MODEL_FACTORY["ElasticNet"] # An Elastic Net based on drug fingerprints and gene expression of 1000 landmark genes
simple_nn = MODEL_FACTORY["SimpleNeuralNetwork"] # A neural network based on drug fingerprints and gene expression of 1000 landmark genes
toyv2 = AVAILABLE_DATASETS["TOYv2"](path_data="data")
drug_response_experiment(
models=[enet, simple_nn],
baselines=[naive_mean], # If they are queried, ablation studies and robustness tests are not done for baselines.
response_data=toyv2,
n_cv_splits=2, # the number of cross validation splits. Should be higher in practice :)
test_mode="LCO", # LCO means Leave-Cell-Line out. This means that the test and validation splits only contain unseed cell lines.
run_id="my_first_run",
path_data="data", # where the downloaded drug response and feature data is stored
path_out="results", # results are stored here :)
hyperparameter_tuning=False) # if True (default), hyperparameters of the models and baselines are tuned.
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
[/tmp/ipython-input-4-1050526458.py](https://localhost:8080/#) in <cell line: 0>()
8 toyv2 = AVAILABLE_DATASETS["TOYv2"](path_data="data")
9
---> 10 drug_response_experiment(
11 models=[enet, simple_nn],
12 baselines=[naive_mean], # If they are queried, ablation studies and robustness tests are not done for baselines.
14 frames
[/usr/local/lib/python3.11/dist-packages/pytorch_lightning/callbacks/early_stopping.py](https://localhost:8080/#) in _validate_condition_metric(self, logs)
151 if monitor_val is None:
152 if self.strict:
--> 153 raise RuntimeError(error_msg)
154 if self.verbose > 0:
155 rank_zero_warn(error_msg, category=RuntimeWarning)
RuntimeError: Early stopping conditioned on metric `val_loss` which is not available. Pass in or modify your `EarlyStopping` callback to use any of the following: `train_loss`, `train_loss_step`, `train_loss_epoch`
```
0 条评论