Implement Inception Score
Hi,
Recently, there is some research still evaluating their model by Inception Score, and issue #20 also mentions it. I create a pull request to add the implementation of computing the inception score.
### Change
+ implement the `calculate_inception_score` . In this implementation, I used the weight of Inception-V3 from the Pytorch team to predict the probs of images (to validate my implementation in the test phase).
+ add --inception-score as `store_true` argument and --splits with default is 10 (just like the original code).
### Test
I tested it with CIFAR-10 (`pytorch-fid/src/pytorch-fid/test.py`), in [this paper](https://arxiv.org/abs/1801.01973), the mean and std of CIFAR-10 are 9.737 and 0.1489. In this PR, the test result return inception score 9.6728±0.1499. (the difference may come from Torch or NumPy version).
### Problems
+ While testing, I found out that IS is really hard to use, not only because of the issues from the paper above. Because the predicted results then have to be split into groups, the arrangement of data is also a problem (if `shuffle=True` in `dataloader`, the result will be different).
+ Different from FID, IS is computed individually for each dataset, so the input only needs one path. The function is taking the first path as input and ignoring the second, I wonder is there any more flexible way to fix this.
合并状态:未合并 关闭于 2023-11-24 0 条评论