Sentiment Analysis for Mental Health Using NLP & ML

Project Overview

We commenced the project by collecting and normalizing a set of text statements containing descriptions of mental health status. Our primary objective was to categorize the statements into known classes. We implemented TF-IDF feature extraction to collect necessary features, trained various classification models using machine learning algorithms, and finally employed visual techniques for better comprehension. In this process, we faced issues such as class imbalance, which we handled through resampling, and maximized the performance of our models by optimizing hyperparameters. Lastly, we persistently stored the trained model in anticipation of predicting the future and put it through a real-world scenario test!

Prerequisites

Learners must develop some skills before undertaking this project. Here’s what you should ideally know:

• A fundamental understanding of Python programming language, machine learning concepts and ideas, and text preprocessing methods and techniques.
• Knowledge and understanding of libraries including Pandas, NumPy, Scikit-learn, Matplotlib, and Seaborn for data analysis and data visualization.
• Knowledge of feature extraction methods, such as TF-IDF backed by implementations, and the handling of class imbalance through resampling.
• Familiarity with the end-to-end cycle of designing a classifier including training, evaluation, persistence, and prediction on machine learning models.
• Jupyter Notebook, VScode, or a Python-compatible IDE.

Approach

Initially, the dataset was pre-processed by eliminating irrelevant elements such as URLs, and special symbols, and tokenizing the content. Stemming was included to sanitize the words used in their base forms, and feature extraction was done with TF-IDF and other numeric measures like character and sentence counts. To achieve balance in the dataset, Random Over-Sampling was performed while making sure all categories were fairly represented. Models of machine learning composed of Logistic Regression, Decision Tree, Naive Bayes, and XGBoost were developed, improved, and performed in training and testing for the use of various metrics including accuracy and confusion matrices. Lastly, the model with the best performance was optimized and applied to new data for effective prediction to take place.

Workflow and Methodology

Workflow

• Data Collection and Cleaning: The dataset was assembled and refined by eliminating various distractions and organizing the textual data.
• Text Tokenization and Stemming: In preparation for feature engineering processes, textual data was tokenized and stemmed.
• Feature Extraction: The TF-IDF was used to extract features while other numerical features like the number of characters, and sentences were incorporated as well.
• Handling Class Imbalance: The Random Over-Sampling technique was opted to create a balanced representation of the dataset in regards to class distribution.
• Data Splitting: Data was divided into two portions for evaluation purposes, namely the training set and the testing set.
• Model Training: Machine learning models were trained on the classification tasks after optimizing hyperparameters.
• Model Evaluation: Models were assessed in terms of performance using accuracy and confusion matrices with visual representation.
• Model Saving and Testing: The model that had the best performance was kept and applied to new sets of data to test its validity.

Methodology

• Data preprocessing involved cleaning, tokenizing, stemming, and feature extraction with TF-IDF.
• Combined text features with numerical data like character and sentence counts.
• Balanced the dataset using Random Over-Sampling for equal class representation.
• Used machine learning models like Logistic Regression, Decision Tree, Naive Bayes, and XGBoost.
• Evaluated models with performance metrics and visualizations for deeper insights.
• Saved the trained model for future use and ensured reliability by testing on unseen data.

Data Collection and Preparation

Data Collection:

In this project, we collected the dataset from a public repository. If you are looking to work on a real-world problem, you can get these kinds of datasets from publicly available repositories such as Kaggle, UCI Machine Learning Repository, or company-specific data. We will provide the dataset in this project so that you can work on the same dataset.

Data Preparation Workflow:

• Import dataset and exclude distractions such as URLs, special characters, and user handles.
• Converted it to lowercase for consistency across the whole dataset.
• Tokenized the text data, into individual words to support a detailed analysis.
• Applied stemming to reduce words to their root forms so they are uniform.
• TF-IDF was used to extract features to capture the importance of words.
• Added numerical features like character and sentence counts to enrich the feature set.
• Handled missing values by removing rows
• Used Random Over-Sampling to balance the dataset.

Code Explanation

STEP 1:

Mounting Google Drive

First, mount Google Drive to access the dataset that is stored in the cloud.

Import Library

This section of the code imports the relevant libraries required for data analysis, data visualization, text processing, and even machine learning. This contains libraries such as Scikit-learn, NLTK, and XGBoost for carrying out tasks such as tokenizing, feature extraction, training a model, and then evaluating the performance of that said model. In addition, this also addresses the issue of class imbalance using RandomOverSampler and allows for some visualizations allowing for WordCloud and Seaborn.

STEP 2:

Loading Data and Checking Shape

This code loads the CSV file. After loading the dataset it prints the dataset’s shape to check the number of rows and columns. The %time magic command in the notebook records the time taken to perform the task.

Previewing Data

This code displays the dataset's first few rows for a quick overview.

Visualizing Status Distribution

The following code generates a horizontal bar graph illustrating how the different values in the status column of the data set are distributed. It applies a ‘Dark2’ color palette from Seaborn and eliminates the top and right borders of the chart for a more aesthetic appearance.

Statistical Description of Dataset

This code delivers a wide-ranging statistical description of all the number columns in the Aionlinecourse_df DataFrame. It includes information such as average values, dispersion, and the lowest and the highest values of each of the presented columns.

Dataset Overview

It provides the schema of the data present in the dataset and gives details about column names, their data types, and the number of non-null entries.

Checking Missing Values

This code calculates the overall number of null values present in every column of the Aionlinecourse_df Data Frame. This helps in identifying null values for further processing of the data.

Handling Missing Values

This code eliminates the Aionlinecourse_df DataFrame of all rows that have any missing values and then checks the missing values to ensure that no missing values are present.

Counting the Unique Values in the Status

This code counts the status’s unique values present in Aionlinecourse_df DataFrame and provides help in understanding the distribution of the different categories present.

Visualizing Mental Health Conditions Distribution

The code creates a visual representation of the distribution of status categories, in a pie chart form.

STEP 3:

Extracting Random Statements by Status

The provided code implements the method of selecting a randomly generated statement from every status group present in the data set. It also displays the status and the corresponding statement. The purpose of this is to offer representative samples for each of the categories.

Analyzing the Characters of Text

The code counts every character and the number of sentences for each statement in the dataset. It then further creates descriptive statistics such as means, minimums, and maximums to outline the text shape.

Filtering Statements with High Character Count

This script filters the DataFrame where the statement column consists of over 10000 characters. It recognizes those statements, which are extremely long, for further purposes of analysis or inquiry.

Renaming Names of Columns and Lowering the Case of Statement

The code renames the name of the column ‘statement’ to ‘original_statement’ and generates a column called statement filled with all the alphabets in lowercase.

Visualization of Various Text Properties with Status Distribution

This code outlines a set of visualizations arranged in a 2*2 matrix. It contains line graphs depicting the trends in the number of characters and the number of sentences, as well as violin graphs displaying how these metrics are distributed among the various status groups in the dataset. The visualizations help us understand how long the text is. It also shows how different the sentences are in the dataset.

Text Data Cleaning

In this code, a function remove_patterns is defined to clean the text by removing URLs, markdown links, handles, and special characters. It also applies to the statement column to facilitate a cleaner and more uniform text for analysis purposes.

Tokenize into Individual Words.

This code executes the word_tokenize function to break down every sentence present in the data set into smaller tokens or lexemes. The tokens are stored in a new column, tokens, for subsequent analysis of the text.

STEP 4:

Stemming Tokens

This code applies Porter Stemmer for reducing every token to its root form in the tokens column. The stemmed tokens are converted back into strings and saved into a new column tokens_stemmed for additional processing.

WordCloud Visualization by Status

The provided code creates a 3x3 matrix of WordClouds for every distinct status in the data, indicating the most relevant words for every given status. It employs distinct colors for the WordClouds, excludes one subplot to leave it empty, and makes sure that everything is neat with headings and proper adjustments of spaces in between

Preparing Features and Target Variable

This segment of code t uses tokens_stemmed, num_of_characters, and num_of_sentences as features (X) while designating status as the target variable (y). Furthermore, it counts the number of instances for each class in y to better understand the class distribution.

Transforming the target categorical variable

This implementation makes use of LabelEncoder to convert the status categorical values present in y into numeric labels. This process is very important in carrying out machine learning model training.

.

STEP 5:

Dividing Data into the Training and Testing Sets

This code divides the Features (X) and target variable (y) available into training and testing sets, with 80% of the data allocated for training and 20% for testing. A random state is provided to ensure that the output can be reproduced.

Feature Engineering with TF-IDF and Numerical Features

This code implements TfidfVectorizer on the tokens_stemmed column and makes use of numerical features (num_of_characters and num_of_sentences). TF-IDF features are augmented with additional numerical features using the sparse matrix stacking technique to form a complete set of features to train and evaluate the model. The total number of unique TF-IDF features is also printed.

Checking Training Feature Dimensions

This snippet of code provides the dimensions of the combined training feature matrix (X_train_combined). It specifies the amount of samples and the number of training features for the model.

Balancing Classes with Random Over-Sampling

This code implements RandomOverSampler to achieve a class balance in the training dataset. Specifically, it is applied to the training features (X_train_combined) and the associated classes (y_train) so that all classes are equal.

Checking Resampled Training Feature Dimensions

This snippet of code provides the dimensions of the resampled training feature matrix ( X_train_resampled). It specifies the amount of samples and the number of training features for the model after balancing the dataset.

Optimized Classifiers

This section provides code that implements several classifiers whose hyperparameters have been already optimized using GridSearchCV. These classifiers are Bernoulli Naive Bayes, Decision Tree, Logistic Regression, and XGBoost. All of these classifiers are ready for training and evaluation.

STEP 6:

Training, Evaluating, and Visualizing Classifier Performance

The code in this section trains each classifier on the resampled training data and tests it on the test data, computing accuracy, a classification report, and a heatmap that shows the confusion matrix for each classifier. The accuracy scores are saved for further comparison.

Visualizing Classifier Accuracy

This section of the code demonstrates how the Accuracy scores of some classifiers are stored in a DataFrame, arranged in descending order. The section also presents a bar graph aimed at visually comparing the different classifiers of which accuracy level is higher, employing a distinct color scheme and ensuring the plot is orderly and comprehensible.

Saves the trained model

The code saves the trained XGBoost classifier (clf) as a .pkl file using joblib. The model is also saved in the mentioned Google Drive location for future reference.

Loading Model and Making Predictions

This code snippet consists of functions for loading the previously trained XGBoost model, performing predictions on new data after transforming it into TF-IDF and number-based encoding, and loads a model. Lastly, the output is returned by converting the predicted label after doing the inference using the label encoder back to the original class.

This code utilizes the loaded model to make predictions after transforming the new input text data into TF-IDF and feature vectors. The class label predicted by the model is encoded back to the original class label format and presented on the display.

Conclusion

In this project, we have been able to build and deploy a machine-learning pipeline that can classify textual statements related to mental health. Several steps were undertaken in transforming raw text into useful insights such as data preprocessing, TF-IDF feature extraction, and stemming. Solutions to problems such as a class imbalance where the Random Over-Sampling was used to make sure that extreme categories were not underrepresented were addressed. Different models such as Logistic Regression, Decision Trees, Naive Bias, and XGboost were trained and evaluated with metrics such as accuracy and confusion matrices. The best model was saved and tested on new data proving it can be in classification tasks in real life.

The challenges identified in this project are those caused by the vast amount of information regarding mental health which can be dealt with by machine learning and text analysis in an efficient way.

Challenges New Coders Might Face

• Challenge: Handling noisy or unstructured text data.
  Solution: Utilize text cleaning methods, which may include the exclusion of special symbols, figures, and extra spaces.

• Challenge: Preprocessing Large Text Data
  Solution: Enhance text cleaning processes by employing better libraries such as NLTK and adopting batch processing for the data.

• Challenge: Curse of Dimensionality in the high dimensional text datasets affecting clustering and classification results.
  Solution: Use TF-IDF vectorization and reduction techniques like (PCA) to control dimensionality.

• Challenge: Inaccessibility of GPU
  Solution: For debugging or initial testing, consider using smaller datasets or incorporating GPU-based cloud platforms for quick turnaround times.

Frequently Asked Questions (FAQs)

Question 1: Which NLP model is best for sentiment analysis?
Answer: Statistical machine learning models like Naive Bayes Classifier, Support Vector Machine (SVM), Logistic Regression, Random Forest, and Gradient Boosting Machines (GBM) are all valuable for sentiment analysis, each with their strengths.

Question 2: How does mental health impact daily life?
Answer: The impact of mental illness is not limited to emotional and social aspects; it also permeates the professional realm. Mental illness can hinder an individual's ability to perform well at work, resulting in absenteeism, decreased job satisfaction, and even economic hardships.

Question 3: How can you validate data collected from sentiment analysis?
Answer: Assess the quality of the sentiment analysis results against human-annotated data, provide metrics, test on new data not previously seen, and employ standard language across similar sentences.

Question 4: What is the purpose of sentiment analysis?
Answer: Sentiment analysis distinguishes three types of emotions — negative, neutral, and positive. It can be applied to a separate sentence or its part as well as being used for document classification, where the term document covers a broad range of textual items like emails, reviews, comments, articles, and more.

Question 5: How does sentiment analysis work in NLP?
Answer: Sentiment analysis uses ML models and NLP to perform text analysis of human language. The metrics used are designed to detect whether the overall sentiment of a piece of text is positive, negative, or neutral.