TY - GEN
T1 - EMS-BERT
T2 - 8th IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies, CHASE 2023
AU - Rahman, M. Arif
AU - Preum, Sarah Masud
AU - Williams, Ronald D.
AU - Alemzadeh, Homa
AU - Stankovic, John
N1 - Publisher Copyright:
© 2023 ACM.
PY - 2023/6/21
Y1 - 2023/6/21
N2 - Emergency Medical Services (EMS) is an important domain of healthcare. First responders save millions of lives per year. Machine learning and sensing technologies are actively being developed to support first responders in their EMS activities. However, there are significant challenges to overcome in developing these new solutions. One of the main challenges is the limitations of existing methods for EMS text mining, and developing a highly accurate language model for the EMS domain. Several important Bidirectional Encoder Representations from Transformer (BERT) models for medical domains, i.e., BioBERT and ClinicalBERT, have significantly influenced biomedical text mining tasks. But extracting information from the EMS domain is a separate challenge due to the uniqueness of the EMS domain, and the significant scarcity of a high-quality EMS corpus. In this research, we propose EMS-BERT - a BERT model specifically developed for EMS text-mining tasks. For data augmentation on our small, classified EMS corpus which consists of nearly 2.4M words, we use a simultaneous pre-training method for transfer-learning relevant information from medical, bio-medical, and clinical domains; and train a high-performance BERT model. Our thorough evaluation shows at least 2% to as much as 11% improvement of F-1 scores for EMS-BERT on different classification tasks, i.e., entity recognition, relation extraction, and inferring missing information when compared both with existing state-of-the-art clinical entity recognition tools, and with various medical BERT models.
AB - Emergency Medical Services (EMS) is an important domain of healthcare. First responders save millions of lives per year. Machine learning and sensing technologies are actively being developed to support first responders in their EMS activities. However, there are significant challenges to overcome in developing these new solutions. One of the main challenges is the limitations of existing methods for EMS text mining, and developing a highly accurate language model for the EMS domain. Several important Bidirectional Encoder Representations from Transformer (BERT) models for medical domains, i.e., BioBERT and ClinicalBERT, have significantly influenced biomedical text mining tasks. But extracting information from the EMS domain is a separate challenge due to the uniqueness of the EMS domain, and the significant scarcity of a high-quality EMS corpus. In this research, we propose EMS-BERT - a BERT model specifically developed for EMS text-mining tasks. For data augmentation on our small, classified EMS corpus which consists of nearly 2.4M words, we use a simultaneous pre-training method for transfer-learning relevant information from medical, bio-medical, and clinical domains; and train a high-performance BERT model. Our thorough evaluation shows at least 2% to as much as 11% improvement of F-1 scores for EMS-BERT on different classification tasks, i.e., entity recognition, relation extraction, and inferring missing information when compared both with existing state-of-the-art clinical entity recognition tools, and with various medical BERT models.
KW - EMS Entity Recognition
KW - Emergency Medical Services (EMS) Data Processing and Analysis
KW - Language Model
KW - Medicine and Health
UR - https://www.scopus.com/pages/publications/85167430213
U2 - 10.1145/3580252.3586978
DO - 10.1145/3580252.3586978
M3 - Conference article
AN - SCOPUS:85167430213
SN - 9798400701023
T3 - Proceedings of the 8th ACM/IEEE International Conference on Connected Health: Applications, Systems and Engineering Technologies
SP - 34
EP - 43
BT - Proceedings - 2023 IEEE/ACM International Conference on Connected Health
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 21 June 2023 through 23 June 2023
ER -