TY - GEN
T1 - Influence of Language Proficiency on the Readability of Review Text and Transformer-based Models for Determining Language Proficiency
AU - Sazzed, Salim
N1 - Publisher Copyright:
© 2022 Owner/Author.
PY - 2022/8/16
Y1 - 2022/8/16
N2 - In this study, we analyze the influence of the English language proficiency of non-native speakers on the readability of the text written by them. In addition, we present multiple approaches for automatically determining the language proficiency levels of non-native English speakers from the review data. To accomplish the above-mentioned tasks, we first introduce an annotated social media corpus of around 1000 reviews written by non-native English speakers of the following five English language proficiency (ELP) groups: very high proficiency (VHP), high proficiency (HP), moderate proficiency (MP), low proficiency (LP), and very low proficiency (VLP). We employ the Flesch Reading Ease (FLE) and Flesch-Kincaid Grade (FKG) tests to compute the readability scores of the reviews written by various ELP groups. We leverage both the classical machine learning (ML) classifiers and transformer-based language models for deciding the language proficiency groups of the reviewers. We observe that distinct ELP groups do not exhibit any noticeable differences in the mean FRE scores, although slight differences are observed in the FKG test. The results imply that the readability measures do not possess high discriminating capabilities to distinguish various ELP groups. In the language proficiency determination task, we notice fine-tuned transformer-based approaches yield slightly better efficacy than the traditional ML classifiers.
AB - In this study, we analyze the influence of the English language proficiency of non-native speakers on the readability of the text written by them. In addition, we present multiple approaches for automatically determining the language proficiency levels of non-native English speakers from the review data. To accomplish the above-mentioned tasks, we first introduce an annotated social media corpus of around 1000 reviews written by non-native English speakers of the following five English language proficiency (ELP) groups: very high proficiency (VHP), high proficiency (HP), moderate proficiency (MP), low proficiency (LP), and very low proficiency (VLP). We employ the Flesch Reading Ease (FLE) and Flesch-Kincaid Grade (FKG) tests to compute the readability scores of the reviews written by various ELP groups. We leverage both the classical machine learning (ML) classifiers and transformer-based language models for deciding the language proficiency groups of the reviewers. We observe that distinct ELP groups do not exhibit any noticeable differences in the mean FRE scores, although slight differences are observed in the FKG test. The results imply that the readability measures do not possess high discriminating capabilities to distinguish various ELP groups. In the language proficiency determination task, we notice fine-tuned transformer-based approaches yield slightly better efficacy than the traditional ML classifiers.
KW - BERT
KW - language proficiency
KW - native language classification
KW - readability
KW - user demographics
UR - https://www.scopus.com/pages/publications/85137424778
U2 - 10.1145/3487553.3524666
DO - 10.1145/3487553.3524666
M3 - Conference article
AN - SCOPUS:85137424778
T3 - WWW 2022 - Companion Proceedings of the Web Conference 2022
SP - 881
EP - 886
BT - WWW 2022 - Companion Proceedings of the Web Conference 2022
PB - Association for Computing Machinery, Inc
T2 - 31st Companion of the World Wide Web Conference, WWW 2022
Y2 - 25 April 2022 through 25 April 2022
ER -