Tensor Space Model based 2-Channel Text Classification with Contextual Embedding and the Transformer Model

Recently, the explosive increase in the amount of text data and the rapid spread of text generative models has caused automatic text classification for information (e.g., social data, review data, and fake news) to become increasingly important. This paper proposes a 2-channel deep learning architecture that effectively combines contextual word embedding and the Transformer model under the tensor space representation model to achieve more reliable text classification. The tensor space representation model represents a single document as a term-by-concept matrix that contains the semantic information of words; however, it does not accommodate the polysemy problem or word sequence information. To achieve near-perfect text classification, we propose a 2-channel deep learning architecture that can learn both word context information and word sequence information under a tensor space model. In our approach, the Transformer model is utilized to learn word sequence information; as a result, our proposed architecture produces a 2-channel learning model for text classification. Using six textual datasets, we demonstrate the performance improvement of our proposed multimodal text classification architecture.