SPAM Detection

Binary Classification

Binary classification is the task of classifying the elements of a set into two groups (each called class) on the basis of a classification rule.

For this application one message can either be spam or ham.

Text Mining

Text mining is the process of deriving high-quality information from text.

Combines concepts from Machine Learning, Linguistic, and statistical analysis.

In this area, we will explore the methods used to rank words/tokens and the BoW model.

Natural Language Processing (NLP)

NLP gives computers the ability to understand text.

Combines Syntax and Semantics into the analysis.

One famous example is the Large Language Models (LLMs) that power OpenAI ChatGPT.

Bag of Words (Bow)

The "Bag of Words" (BoW) model is a fundamental technique in natural language processing (NLP) and text analysis. It represents a text document as a simple and unordered collection of words, ignoring grammar and word order but focusing on the frequency of word occurrences. The BoW model is used for various NLP tasks, including text classification, sentiment analysis, document retrieval, and more.

The Bag of Words model is a simple and effective way to represent text data for various NLP tasks.

Limitations

• Loss of Word Order: BoW completely ignores the order of words in a document. This means that documents with the same words but in a different order will have the same BoW representation. As a result, it fails to capture the sequential or contextual information in a text.

• Loss of Semantic Information: BoW treats each word as an independent entity and does not consider the semantic meaning or relationships between words. For instance, it can't distinguish between synonyms or understand that "good" and "excellent" have similar meanings.

• Fixed-Length Vector: BoW results in fixed-length vectors, where the length is determined by the vocabulary size. This can be inefficient for documents with a large vocabulary or those with varying lengths. Padding or truncation may be necessary.

• Sparse Representations: BoW representations are often very sparse, with most elements in the document vectors being zero. This can lead to computational inefficiency and storage problems, especially in the case of large vocabularies.