Many time passes between epochs

### Bug description I am training the model using pytorch-lightning. The code is below One epoch passes smoothly in 1-2 seconds, however the interval between epochs is around 6-8 seconds. Could you recommed any guidelines where to search and how to fix it? ``` train_dataloader = torch.utils.data.DataLoader( train_dataset, batch_size=config.training.batch_size, shuffle=True, num_workers=2, #pin_memory=True, persistent_workers=True ) model = instantiate(config.model_cfg.model_obj) pl_model = PlWaveGrad(model, hydra_config=config) if config.model_cfg.conditional: print('using conditional model') else: print('uncondtional model') print('using dataset: ', config.dataset.name) loss_cb = LossCurveCallback() #profiler = AdvancedProfiler(dirparth='./outputs', filename='perf_logs') checkpoint_callback = pl.callbacks.ModelCheckpoint( filename="{epoch:02d}-{train_loss:.4f}", every_n_epochs=1000, # Save after every epoch save_last=True, # This ensures the last checkpoint is kept and overwritten each time ) trainer = pl.Trainer( max_epochs=config.training.n_epochs, log_every_n_steps=4, devices=[config.training.device], # FIXME callbacks=[checkpoint_callback, loss_cb], # here is no loss_cb precision='16-mixed', #enable_checkpointing=False, #profiler="advanced", ) print(f'\n\n\n using device: {config.training.device}') if config.ckpt_path is None: trainer.fit(pl_model, train_dataloaders=train_dataloader) else: ckpt_path = Path(config.ckpt_path).resolve() trainer.fit(pl_model, train_dataloaders=train_dataloader, ckpt_path=ckpt_path) ``` ### What version are you seeing the problem on? v2.2, v2.4 ### Reproduced in studio _No response_ ### How to reproduce the bug ```python ``` ### Error messages and logs ``` # Error messages and logs here please ``` ### Environment <details> <summary>Current environment</summary> ``` #- PyTorch Lightning Version (e.g., 2.6.0): #- PyTorch Version (e.g., 2.5): #- Python version (e.g., 3.12): #- OS (e.g., Linux): #- CUDA/cuDNN version: #- GPU models and configuration: #- How you installed Lightning(`conda`, `pip`, source): ``` </details> ### More info _No response_ cc @ethanwharris