Biomedical Entity-Aware Semantic Role La-belling via Model Merging
Abstract
Semantic Role Labeling (SRL) is essential for extracting structured knowledge from biomedical texts, yet it remains challenging due to the complexity of biomedical terminology. This paper introduces a novel method that enhances biomedical SRL by directly integrating Named Entity Recognition (NER) knowledge using model merging. NER plays a crucial role in anchoring domain-specific ar- guments, allowing the model to better distinguish seman- tic roles in biomedical contexts. Unlike prior approaches that rely on multitask learning or syntax-aware features, our method merges pretrained SRL and NER models in a way that preserves entity knowledge while aligning with semantic role structures. Experimental results demonstrate that our approach surpasses state-of-the-art multitask and syntax-aware SRL models in both accuracy and training speed. The integration of biomedical entity knowledge proves especially effective for predicates with high proportions of entity-based arguments, highlighting the value of direct NER- to-SRL knowledge transfer in advancing biomedical semantic understanding.
References
W.-C. Chou, R. T.-H. Tsai, Y.-S. Su, W. K., T.-Y. Sung, and W.-L. Hsu, “A semi-automatic method for annotating a biomedical proposition bank,” in Proceedings of the Work- shop on Frontiers in Linguistically Annotated Corpora 2006, 2006, pp. 5–12.
M. Miwa, P. Thompson, J. McNaught, D. B. Kell, and S. Ananiadou, “Extracting semantically enriched events from biomedical literature,” BMC Bioinformatics, vol. 13, no. 1, p. 108, 2012.
P. Thompson, P. Cotter, S. Ananiadou, J. Mcnaught, S. Mon- temagni, A. Trabucco, and G. Venturi, “Building a bio-event annotated corpus for the acquisition of semantic frames from biomedical corpora,” in Proceedings of the Sixth Interna- tional Conference on Language Resources and Evaluation (LREC’08), 2008.
L. He, K. Lee, M. Lewis, and L. Zettlemoyer, “Deep semantic role labeling: What works and what’s next,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 473–483.
T. Wattarujeekrit, P. K. Shah, and N. Collier, “Pasbio: Predicate-argument structures for event extraction in molec- ular biology,” BMC Bioinformatics, vol. 5, no. 1, p. 155, 2004.
H.-D. Tuan-Nguyen, H.-S. Pham, and V.-T. Hoang, “A semi-automatic approach to biomedical semantic role corpus construction,” Science and Technology Development Journal - Natural Sciences, vol. 6, no. 2, pp. 2083–2094, 2022.
J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, and J. Kang, “Biobert: A pre-trained biomedical language repre- sentation model for biomedical text mining,” Bioinformatics, vol. 36, no. 4, pp. 1234–1240, 2020.
H.-D. Tuan-Nguyen, L.-T. Tran-Tien-Loi, and V.-H. Le- Dinh, “A deep-learning model for semantic role labelling in medical documents,” Science and Technology Development Journal - Natural Sciences, vol. 5, no. 2, pp. 1032–1039, 2021.
C. F. Baker, C. J. Fillmore, and J. B. Lowe, “The berkeley framenet project,” in Proceedings of the 36th Annual Meet- ing on Association for Computational Linguistics, 1998, pp. 86–90.
M. S. Palmer and K. K. Schuler, “Verbnet: A broad- coverage, comprehensive verb lexicon,” University of Penn- sylvania, Computer and Information Science Dept., 2000 South 33rd St., Philadelphia, PA, United States, Tech. Rep., 2005.
S. Pradhan, J. Bonn, S. Myers, K. Conger, T. O’gorman, J. Gung, K. Wright-Bettner, and M. Palmer, “Propbank comes of age—larger, smarter, and more diverse,” in Pro- ceedings of the 11th Joint Conference on Lexical and Com- putational Semantics, 2022, pp. 278–288.
P. Kingsbury and M. Palmer, “From treebank to propbank,” in Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02), M. G. Rodr´ıguez and C. P. S. Araujo, Eds. European Language Resources Association (ELRA), 2002.
O. Majewska, C. Collins, S. Baker, J. Bjo¨rne, S. W. Brown, A. Korhonen, and M. Palmer, “Bioverbnet: A large semantic- syntactic classification of verbs in biomedicine,” Journal of Biomedical Semantics, vol. 12, no. 1, 2021.
R. T.-H. Tsai, W.-C. Chou, Y.-C. Lin, C.-L. Sung, W. Ku, T.- Y. Sung, and W.-L. Hsu, “Biosmile: Adapting semantic role labeling for biomedical verbs,” in Proceedings of the HLT- NAACL BioNLP Workshop on Linking Natural Language and Biology, 2006, pp. 57–64.
R. T.-H. Tsai, W.-C. Chou, Y.-S. Su, Y.-C. Lin, C.-L. Sung, H.-J. Dai, I. T.-H. Yeh, W. Ku, T.-Y. Sung, and W.-L. Hsu, “Biosmile: A semantic role labeling system for biomedical verbs using a maximum-entropy model with automatically generated template features,” BMC Bioinformatics, vol. 8, no. 1, p. 325, 2007.
P.-T. Lai, Y.-Y. Lo, M.-S. Huang, Y.-C. Hsiao, and R. T.- H. Tsai, “Belsmile: A biomedical semantic role labeling approach for extracting biological expression language from text,” Database, 2016.
R. T.-H. Tsai and P.-T. Lai, “A resource-saving collective approach to biomedical semantic role labeling,” BMC Bioin- formatics, vol. 15, no. 1, pp. 160–171, 2014.
F. Eckert and M. Neves, “Semantic role labeling tools for biomedical question answering: a study of selected tools on the bioasq datasets,” in Proceedings of the 6th BioASQ Workshop A Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering, 2018, pp. 11–21.
A. D. P. Ariyanto, D. Purwitasari, and C. Fatichah, “A systematic review on semantic role labeling for information extraction in low-resource data,” IEEE Access, vol. 12, pp. 57 917–57 946, 2024.
F. Ikhwantri, S. Louvan, K. Kurniawan, B. Abisena, V. Rach- man, A. F. Wicaksono, and R. Mahendra, “Multi-task active learning for neural semantic role labeling on low resource conversational corpus,” in Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP, 2018, pp. 43–50.
H.-D. Tuan-Nguyen, T. Duong Luu, and Q. Duy Huynh, “A syntax-aware deep-learning model for biomedical semantic role labelling,” Science and Technology Development Jour- nal - Natural Sciences, vol. 7, no. 4, pp. 2750–2762, 2023.
W. Liu, S. Yang, and K. Tu, “Structured mean-field varia- tional inference for higher-order span-based semantic role,” in Findings of the Association for Computational Linguis- tics: ACL 2023, 2023, pp. 918–931.
S. S. S. Das, H. Zhang, P. Shi, W. Yin, and R. Zhang, “Unified low-resource sequence labeling by sample-aware dynamic sparse finetuning,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 6998–7010.
J. G. Moreno, J. Lovon-Melgarejo, M. Rick, Robin- Charlet, C. Damase-Michel, and L. Tamine, “Patientdx: Merging large language models for protecting data-privacy in healthcare,” in Proceedings of the Second Workshop on Patient-Oriented Language Processing (CL4Health). Albuquerque, New Mexico: Association for Computational Linguistics, May 2025, pp. 1–11, retrieved June 1, 2025. [Online]. Available: https://aclanthology.org/2025.cl4health-1.1/
T. Rousset, T. Kakibuchi, Y. Sasaki, and Y. Nomura, “Merg- ing language and domain specific models: The impact on technical vocabulary acquisition,” February 2025, preprint on arXiv.
J.-P. Corbeil, A. Dada, J.-M. Attendu, A. B. Abacha, A. Sordoni, L. Caccia, F. Beaulieu, T. Lin, J. Kleesiek, and P. Vozila, “A modular approach for clinical slms driven by synthetic data with pre-instruction tuning, model merging, and clinical-tasks alignment,” May 2025, preprint on arXiv.
Y.-A. Lee, C.-Y. Ko, T. Pedapati, I.-H. Chung, M.- Y. Yeh, and P.-Y. Chen, “Star: Spectral truncation and rescale for model merging,” in Proceedings of the 2025 Conference of the North, Central and South American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers). Albuquerque, New Mexico: Association for Computational Linguistics, April 2025, pp. 496– 505, retrieved June 1, 2025. [Online]. Available: https://aclanthology.org/2025.naacl-short.42/.
T. Parekh, I.-H. Hsu, K.-H. Huang, K.-W. Chang, and N. Peng, “Geneva: Benchmarking generalizability for event argument extraction with hundreds of event types and ar- gument roles,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, pp. 3664–3686.
Y. Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon, “Domain- specific language model pretraining for biomedical natural language processing,” ACM Transactions on Computing for Healthcare, vol. 3, no. 1, pp. 1–23, January 2022. [Online]. Available: https://doi.org/10.1145/3458754.
K. Huang, J. Altosaar, and R. Ranganath, “Clinicalbert: Modeling clinical notes and predicting hospital readmis- sion,” April 2019, preprint.
G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi, “Editing models with task arithmetic,” in Proceedings of the Eleventh Inter- national Conference on Learning Representations, February 2023.
E. Yang, Z. Wang, L. Shen, S. Liu, G. Guo, X. Wang, and D. Tao, “Adamerging: Adaptive model merging for multi-task learning,” in Proceedings of the International Conference on Learning Representations (ICLR 2024), October 2024. [Online]. Available: https://arxiv.org/abs/2310.02575.
M.-S. Huang, P.-T. Lai, R. T.-H. Tsai, and W.-L. Hsu, “Revised jnlpba corpus: A revised version of biomedical ner corpus for relation extraction task,” in Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications, 2024, pp. 73–78.
