apply_mask in HuBERT does not apply expected percentage of masks if the input shapes differ. which is true in the case of fintuning.

mask_indices = compute_mask_indices( (B, T), padding_mask, self.mask_prob, self.mask_length, self.mask_selection, self.mask_other, min_masks=2, no_overlap=self.no_mask_overlap, min_space=self.mask_min_space, **require_same_masks=False,** ) require_same_masks need to set to False, which is not the case default.