Taxonomy learning is an important problem and facilitates various applications such as semantic understanding and information retrieval. Previous work for building semantic taxonomies has primarily relied on labor-intensive human contributions or focused on text-based extraction. In this paper, we investigate the problem of automatically learning multimodal taxonomies from the multimedia data on the Web. A systematic framework called Variational Deep Graph Embedding and Clustering (VDGEC) is proposed consisting of two stages as concept graph construction and taxonomy induction via variational deep graph embedding and clustering. VDGEC discovers hierarchical concept relationships by exploiting the semantic textual-visual correspondences and contextual co-occurrences in an unsupervised manner. The unstructured semantics and noisy issues of multimedia documents are carefully addressed by VDGEC for high quality taxonomy induction.We conduct extensive experiments on the real-world datasets. Experimental results demonstrate the effectiveness of the proposed framework, where VDGEC outperforms previous unsupervised approaches by a large gap.
We investigate the problem of automatically learning multimodal taxonomies from the multimedia data on the Web.
Dataset 7.23GB
Every line is a document in json format as: x
1{2 images:[3 'http://zhuanzhiimg.b0.upaiyun.com/images/wx/9cc43e5bd67854194860257410f94634',4 'http://zhuanzhiimg.b0.upaiyun.com/images/wx/8032a0a94cf2f197132490f3b40b5e2c'5 ],6 videos:[],7 content:''8}