Transfer Learning Framework for Warping From Representations to Point of View (TWARP)
Abstract
With the advancement in deep learning, neural network architectures like recurrent neural networks (RNN and LSTM) and convolutional neural networks (CNN) have shown decent improvement in solving several Natual Language Processing (NLP) tasks like text classification,language modeling, machine translation, etc. However, the performance of deep learning models in NLP was not so pronounced, possibly due to the lack of large labeled text datasets. Most of the labeled text datasets are not big enough to train deep neural networks because these networks have a huge number of parameters and training such networks on small datasets will cause overfitting. This was the case with NLP until 2018 when the transformer-based model was introduced by Google. Ever since transfer learning in NLP is helping in solving many tasks with the state of the art performance and has been instrumental in the success of deep learning in computer vision. Contemporarily, The Bidirectional Encoder Representations from Transfomrers (BERT) from Google and OpenAI’s Generatrive Pretrained Transformer GPT are the two state of the art models available for several NLP-based tasks, we primarily used BERT for our study. We propose a dynamic-projections based framework to leverage transfer learning by using fine-tuned BERT word embeddings for text classification in order to understand data pertaining to violent political protests on social media and enhance the interpretability of text classification networks. The proposed framework uses an attention mechanism that includes multiple settings as per the number of labels. The individual attention layers provide a better understanding of what causes a black-box model to make a certain prediction. The novel architecture is based on siamese networks that can automatically select relevant data for supervised and unsupervised domain adaptation.
Subject Area
Computer science|Artificial intelligence|Information science
Recommended Citation
Ohlan, Rahul, "Transfer Learning Framework for Warping From Representations to Point of View (TWARP)" (2023). ETD Collection for Fordham University. AAI30420917.
https://research.library.fordham.edu/dissertations/AAI30420917
