ARTICLE TYPE : RESEARCH ARTICLE
Published on : 05 Nov 2026,
Volume - 2
Journal Title :
WebLog Journal of Allergy
| WebLog J Allergy
| WJA
Source URL:
https://weblogoa.com/articles/wja.2026.k0502
Permanent Identifier (DOI) :
Sequence Similarity, Deduplication and Feature Choice in the Evaluation of Allergen Protein Classifiers
2Lord Buddha Education Foundation, Kathmandu, Nepal
Abstract
Background: Allergen protein classifiers are often evaluated on benchmarks built from overlapping sources, where duplicates and homologues can inflate performance.
Methods: We conducted a computational robustness and reproducibility study of the public AlgPred 2 dataset (repository commit 533225b). Exact-sequence auditing reduced 20,150 records to 13,294 unique sequences; deduplicating the training files left 10,514 development sequences (2,454 allergens and 8,060 putative non-allergens). Logistic regression and random forest classifiers using amino-acid composition and log length (feature set A) or five added physicochemical descriptors (feature set B) were evaluated under random splitting and MMseqs2 sequence groups at 30%, 40% and 60% identity with 80% bidirectional coverage, with three seeds and nested tuning.
Results: Random forest Matthews Correlation Coefficient (MCC) was 0.84-0.85 under random splitting and 0.67-0.72 (set A) under audited grouped splitting; logistic regression stayed near 0.52 0.55 in every strategy. Initial groups left detected cross-partition matches, which were merged before fitting; a post hoc higher-sensitivity search still found residual related allergens in 30% splits. Added descriptors changed mean MCC by -0.010 to +0.021 with inconsistent direction. On the reserved authors-defined holdout (2,780 unique sequences), random forest MCC was 0.617 (A) and 0.596 (B); 225 sequences had a detected match of at least 40% identity to development data, and MCC for the other 2,555 was 0.568 and 0.535.
Conclusions: Deduplication, grouped splitting with similarity auditing and simple baselines materially changed the interpretation of performance; the added descriptors gave no reliable benefit. These findings concern benchmark evaluation, not patient diagnosis or food safety.
Keywords: Allergen Protein Classification; Sequence Similarity; Data Leakage; Deduplication; Matthews Correlation Coefficient; Reproducibility; MMseqs2
Citation
Bhattarai B, Gurung D, Thakur RN. Sequence Similarity, Deduplication and Feature Choice in the Evaluation of Allergen Protein Classifiers. WebLog J Allergy. wja.2026.k0502. https://doi.org/10.5281/zenodo.24223971