ITADN

NAF reported log-likelihood values are excesively high

#55Closedadrianjav 创建于 2024-07-23
bug
A
adrianjavcommented
### Description Now that I am revisiting my experience with Zuko, I have remembered that we did not use NAF in [our work](https://arxiv.org/abs/2306.05415) since, during testing, we highly suspected that there was a bug in NAF and did not feel confident using it. This is how we discovered it (back in Zuko 0.2.0, maybe it is fixed by now): we had different synthetic experiments for which we knew the ground-truth data-generating process, and we could compute the real log-likelihood of the data. When we fitted a NAF model using data from this model, the reported log-likelihoods reported by NAF were orders of magnitude larger than the ground-truth (which cannot happen since the KL is always non-negative). This behaviour was not observed with MAF or NSF, where they reported values slightly lower than the true log-likelihoods. (I will update the issue when I find time, but I guessed it is useful to raise the issue despite being incomplete.) ### Reproduce TBD, but it can be tested by substituting MAF by NAF in any of the synthetic experiments of [this repository](https://github.com/psanch21/causal-flows). ### Expected behavior The log-likelihoods computed by the flow should be lower (given enough data) than that of the model that generated the data. ### Causes and solution I am uncertain of what could be the reason, but it should be somewhere in the log-likelihood computation. ### Environment * Zuko version: 0.2.0 * PyTorch version: 1.12.0 * Python version: 3.8 * OS: Ubuntu 22.04
关闭于 2024-08-07 4 条评论