The symmetric problem about the CLUB MI estimator?

[XiXiRuPan]

Thanks for your sharing. Your work is very interesting. The standard MI is symmetric but CLUB MI estimator is not. Have you tried building two variational estimators between x,y, which estimate x->y and y->x mi info? Thanks again.

[Linear95]

Hi, sorry for replying late. Yes, our CLUB is not symmetric for x and y. But the asymmetry of CLUB is symmetric to x and y:) You can choose by yourself which variable to be x and which to be y. As you said, we can build two estimators x->y and y-x, to estimate MI(x;y). But here are some of my concerns:

1. With the two variational estimators, we introduced more parameters to the model.
2. The performance of the combination of x->y and y->x will be bounded by x->y or y->x (depends on which one is better). So why not only use the better one between x->y and y->x?

Empirically, I suggest using the variable with a lower dimension as y, because it is easier for the network q(y|x) to fit the lower dimension outputs. We also have a discussion about this problem (known as network expressiveness) in our paper Section 3.2. Thank you.