Deep Learning Approaches for Context-Aware Sentiment Analysis in Social Media Text

Abstract

Context-aware sentiment analysis aims to infer polarity in social media while accounting for signals that are frequently absent when a post is modeled in isolation [2][5], such as reply history, topic drift, emoji pragmatics, and user-specific language. Strong transformer models such as BERT [2], RoBERTa [3], and DeBERTa [4] still struggle with sarcasm and conversational ellipsis [19]. We first synthesize findings from seminal and recent work, highlighting why strong text-only transformers still fail under sarcasm, stance reversal, and conversational ellipsis. We then introduce CAST (Context-Aware Social Transformer), a hierarchical architecture that pairs a domain-appropriate transformer encoder with a context fusion layer that attends over a bounded set of conversational and topical context items, and a lightweight metadata module for user and topic embeddings. CAST is trained end-to-end with AdamW and evaluated using macro-F1 and accuracy on two public benchmarks: TweetEval sentiment (Twitter; 3-way polarity) and a sentiment-collapsed variant of GoEmotions (Reddit) derived by grouping fine-grained emotions into positive/neutral/negative. Using simulated but representative experiments reflecting typical benchmark conditions, CAST improves macro-F1 by +1.7 points on TweetEval and +1.0 point on GoEmotions over strong transformer baselines. Ablation suggests that conversational context contributes most on Reddit, whereas topical cues (hashtags/subreddits) are especially beneficial on Twitter. Error analysis indicates remaining challenges in irony, implicit negation, and domain-specific slang. We further propose a calibration check (expected calibration error) and an out-of-topic stress-test; both suggest that context reduces overconfidence on ambiguous posts. Although results are simulated, we provide concrete preprocessing, hyperparameters, and evaluation recipes reproducible with public data.