Experiments

The Effect of VAE+HGMM and GMM Feedback :

In our VDGEC method, we jointly optimize the reconstruction loss and clustering loss in the objective function by VDGEC.

The loss function :

The first term is the clustering loss, the second term is reconstruction loss.

With epoch increase, $p(x|z) \rightarrow 1.0$, which means $\log p(x|z) \rightarrow 0$, the reconstruction loss goes down. And the $q(z, c|x) \rightarrow p(z,c)$, so the clustering loss goes down.

About clear reduction in early epochs:

• Total loss: Clear reduction can be found in picture.
• Reconstruction Loss: There are clear reduction in the early epochs, which show the Encoder and Decoder are learning features of concept.
• Clustering Loss: After every epoch, we cluster concepts base on their emebddings (i.e. the latent $z$ generated by encoder.). After every 100 epochs, we fuse the $k$ compontents GMM to $m (m < k)$ compontents GMM, with the information loss threshold $\tau$, which change the $p(z, c)$ and increase the loss of the first term. When τ goes small, HGMM will hard to fuse, the learning process will end earilier. When τ goes big, HGMM will merge more components in an iteration, the total level will be lower. In practice, we raise τ after every fusion, as the margin of cluster goes wide, as shown in the peaks of picture.
  

About no clear reduction in latter epochs:

• Reconstruction Loss: The reconstruction loss is stable after enough epoch. Although the fusion of GMM changes the prior of VGDEC, there is only small flush can be found on 100,200,300,400 epoch, as the $p(z, c)$ changed.
• Clustering Loss: We do clustering every epoch, however the embedding of concept are stable and the parameters of GMM never change until the fusion of GMM, so the undulate can only be found around fusion epoch.

Effect of Model Parameters.

The evaluation is conducted by changing one parameter (e.g., K) while fixing the other (e.g., τ). The above picture shows the performance of VDGEC for different values of K and τ. When τ goes small, HGMM will hard to fuse, the learning process will end earilier. When τ goes big, HGMM will merge more components in an iteration, the total level will be lower. In practice, we raise τ after every iteration, as the margin of cluster goes wide. When K goes small, the initial cluster number goes down, HGMM contributes less as the inital cluster could be the final result. The bigger the K goes, the edge number of taxonomy raises and the fine-grained relation will output.