CLARITY - Unmasking Political Question Evasions

Engineered a complete NLP pipeline and fine-tuned Large Language Models to classify evasive political responses.

TL;DR: Engineered an accelerated NLP pipeline and fine-tuned Large Language Models (including Llama 3.1) to classify direct, partial, and evasive political responses, outperforming state-of-the-art baselines on the SemEval CLARITY dataset.

SemEval Clarity Classification Pipeline

Figure source: SemEval-2026 CLARITY Task

Technologies & Methodology

  • Software & Frameworks: Python, PyTorch, Hugging Face Transformers, spaCy, NLTK, ELFEN, scikit-learn, Tinker.
  • Machine Learning Models: Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct, RoBERTa-base, Neural Networks, Support Vector Machines (SVM).
  • Dataset & Tools: SemEval CLARITY dataset, Optuna (hyperparameter optimization).

Key Contributions & Results

  • Developed an accelerated NLP preprocessing and feature extraction pipeline utilizing parallelization, spaCy for negation detection, and ELFEN for hedge detection.
  • Fine-tuned a Llama-3.1-8B-Instruct decoder model via the Tinker platform, achieving a 0.71 Weighted F1 score that significantly outperformed existing dataset baselines.
  • Evaluated encoder-only (RoBERTa) and classical baseline architectures (SVM, Random Forest, Neural Networks) to benchmark semantic alignment and contextual performance against raw linguistic features.
  • Addressed severe class imbalances (Clear Reply vs. Ambivalent vs. Clear Non-Reply) by optimizing macro-averaged metrics through Optuna hyperparameter tuning.