ITADN

Memory issue due to A and B matrix computation

#25Openanupsingh15 创建于 2024-03-11
A
anupsingh15commented
Hi, Thanks for providing the Mamba implementation. I would like to know if there is any workaround in the efficient computation of `deltaA` and `deltaB_u` that can avoid the GPU memory running out issue. The following are the parameters I used to create the Mamba instance: ``` d_model: 1024 n_layer: 4 d_state: int = 1024 expand: int = 2 ``` The other parameters are set to their default values. It results in a model of ~60M parameters. However, I run out of memory (max GPU memory= 24 GB) when I train with a batch size of 256 or even as low as 64 and this probably happens due to large matrix computations for `deltaA` and `deltaB_u`.
3 条评论