IMDb Sentiment Analysis Tutorial Uses DistilBERT LoRA

This tutorial presents a comprehensive sentiment analysis workflow utilizing the Stanford NLP IMDb Large Movie Review Dataset, contrasting traditional machine learning approaches with parameter-efficient transformer fine-tuning. The process begins with establishing a reproducible computational environment and conducting a thorough audit of the dataset. This audit scrutinizes aspects such as class ordering, potential biases in review length, the presence of duplicate data that could lead to leakage, and artifacts introduced during preprocessing. Following this foundational step, a robust baseline model is trained using TF-IDF features and a Logistic Regression classifier. The tutorial then proceeds to fine-tune the DistilBERT transformer model, employing the LoRA (Low-Rank Adaptation) technique through the PEFT (Parameter-Efficient Fine-Tuning) library. Model performance is evaluated using a suite of metrics including accuracy, macro-F1 score, ROC-AUC, confusion matrices, and ROC curves. Further analysis delves into threshold selection and probability calibration, assessed via Expected Calibration Error and reliability analysis. Beyond headline performance indicators, the tutorial investigates the model's behavior with confident errors, its performance across varying review lengths, and employs word-level occlusion saliency and head-versus-tail truncation to elucidate how the model arrives at its predictions and to identify limitations affecting performance with long contexts. Finally, the unlabeled portion of the IMDb dataset is leveraged for confidence-based pseudo-labeling. The resulting semi-supervised model is then compared against the established baseline, and the merged transformer model is saved for subsequent, reusable sentiment inference tasks. The tutorial emphasizes reproducibility and provides code for installing necessary libraries such as transformers, datasets, peft, and accelerate, along with environment variable configurations to manage parallel processing and disable weight and biases logging. The libraries used include numpy, pandas, torch, and matplotlib for data manipulation, numerical operations, deep learning, and visualization, respectively. The datasets library is used for loading the IMDb dataset, and the PEFT library facilitates efficient fine-tuning of large language models. The analysis also incorporates scikit-learn for classical machine learning components and evaluation metrics. The process aims to provide a transparent and detailed understanding of modern NLP sentiment analysis techniques, from data preparation to advanced model interpretation and semi-supervised learning.