My research focuses on parameter-efficient fine-tuning of transformer models — making state-of-the-art NLP accessible in resource-constrained environments.
Computers · MDPI
Open Access · Scopus Indexed
Published March 2026
A Resource-Efficient Approach to Fine-Tuning a BERT-Base Model for Sentiment Analysis
Abdullah M. Basahel, Shreyanth H. Giriyappa, Furqan Alam, Tahani Saleh Mohammed Alnazzawi, Saqib Qamar, Adnan Ahmed Abi Sen
Computers 2026, 15(3), 159 · DOI: 10.3390/computers15030159 · Section: AI-Driven Innovations
This paper introduces SCALE (Selective Critical Adapter Layer Efficiency), a novel parameter-efficient fine-tuning method. Rather than adapting every transformer layer uniformly, SCALE dynamically profiles layers using activation magnitude and attention entropy, then injects lightweight adapter modules only into the most influential layers. Benchmarked against LoRA, AdaLoRA, and BitFit across five sentiment analysis datasets (IMDB, SST-2, Yelp, TweetEval, XLive), SCALE consistently delivers superior accuracy, F1-score, and AUC while substantially reducing training cost.
0% reduction in training time vs. full fine-tuning
0% of full-model accuracy retained
0% of parameters used (61.4% fewer trainable)
+0% accuracy over LoRA on IMDB
01
Dynamic Layer Profiling
A novel profiling technique that scores each transformer layer via activation magnitude and attention entropy, identifying the layers most relevant to the downstream task before any training begins.
02
Selective Adapter Injection
Lightweight residual adapter modules are inserted only into the top-k highest-scoring layers, preserving pretrained representations while enabling targeted, interpretable adaptation.
03
Validated Efficiency
Rigorous 5-fold evaluation across binary and multi-class benchmarks, with a full ablation study against uniform and adaptive layer-selection strategies under identical training conditions.