Using Transfer Learning Techniques for Analysing Text Documents

Abstract

Transfer learning is becoming a widely used technique in the field of deep learning. In this thesis, it is used for detecting offensive text which is becoming a prevailing phenomenon on online social media. While the technique showed promising results in the task of offensive text classification, the results also showed how these models aren’t robust to simple text substitution adversarial attacks. Moreover, hate speech is a specific type of offensive text that needs to be properly represented in the used data sets such that the model can correctly classify them as offensive text. The thesis is divided into five chapters as listed below: Chapter 1 is an introductory chapter demonstrating the motivation for using transfer learning for Arabic offensive text classification.

Chapter 2 gives an overview of the different transfer learning techniques that have emerged for building deep learning models especially in the field of natural language processing.

Chapter 3 first describes the offensive and hate speech data sets that are used in the various experiments done throughout the thesis. Then, the different transfer learning paradigms and the adversarial attacking algorithms are described.

Chapter 4 presents the results for the transfer learning experiments on OffensEval2020 data set. Additionally, it reports the results of attacking the best performing model using the new adversarial attacking method. At the end of the chapter, the zero-shot learning performance of the best performing model is reported using three different Arabic hate speech data sets.

Chapter 5 summarises the conclusions of the thesis and provides recommendations regarding future work that can be done to improve the fine-tuned models and to combat the new adversarial attacking methods. Keywords: Natural Language Processing, Transfer Learning, Transformer models, Adversarial attacks, Arabic NLP