Automatic Text Summarization using Natural Language Processing and Artificial Intelligence Techniques

Abstract

The Internet has an exponentially increasing amount of textual data. Searching for a certain topic can become a daunting task because users cannot read and comprehend all potentially long documents in the search results. As a result, it becomes urgent to help users by summarizing textual content. Manual text summarization consumes a lot of time, effort, cost, and even becomes impractical with the gigantic amount of textual content. Therefore, Automatic Text Summarization (ATS) in this case is clearly beneficial. Researchers have been trying to improve ATS techniques since the 1950s. ATS approaches are either extractive, abstractive, or hybrid. The extractive approach selects the most important sentences in the input document(s) then concatenates them to form the summary. The abstractive approach represents the input document(s) in an intermediate representation then generates the summary with sentences that are different than the original sentences. The hybrid approach merges between both the extractive and abstractive approaches. This thesis provides a comprehensive survey for the researchers by presenting the different aspects of ATS: approaches, building blocks, techniques, evaluation methods, and future research directions. Despite all the proposed methods in the literature, the generated summaries are still far away from the human-generated summaries. To enhance ATS for single documents, this thesis also proposes a novel extractive graph-based framework “EdgeSumm” that relies on four proposed algorithms. The first algorithm constructs a new text graph representation model from the input document. The second and third algorithms search the constructed text graph for sentences to be included in the candidate summary. When the number of words of the resulting candidate summary still exceeds a user-required length