NAF reported log-likelihood values are excesively high
bug
### Description
Now that I am revisiting my experience with Zuko, I have remembered that we did not use NAF in [our work](https://arxiv.org/abs/2306.05415) since, during testing, we highly suspected that there was a bug in NAF and did not feel confident using it.
This is how we discovered it (back in Zuko 0.2.0, maybe it is fixed by now): we had different synthetic experiments for which we knew the ground-truth data-generating process, and we could compute the real log-likelihood of the data. When we fitted a NAF model using data from this model, the reported log-likelihoods reported by NAF were orders of magnitude larger than the ground-truth (which cannot happen since the KL is always non-negative). This behaviour was not observed with MAF or NSF, where they reported values slightly lower than the true log-likelihoods.
(I will update the issue when I find time, but I guessed it is useful to raise the issue despite being incomplete.)
### Reproduce
TBD, but it can be tested by substituting MAF by NAF in any of the synthetic experiments of [this repository](https://github.com/psanch21/causal-flows).
### Expected behavior
The log-likelihoods computed by the flow should be lower (given enough data) than that of the model that generated the data.
### Causes and solution
I am uncertain of what could be the reason, but it should be somewhere in the log-likelihood computation.
### Environment
* Zuko version: 0.2.0
* PyTorch version: 1.12.0
* Python version: 3.8
* OS: Ubuntu 22.04
关闭于 2024-08-07 4 条评论