Skip to main navigation Skip to search Skip to main content

Influence of Language Proficiency on the Readability of Review Text and Transformer-based Models for Determining Language Proficiency

  • Old Dominion University

Research output: Contribution to book or proceeding › Conference article › peer-review

5 Scopus citations

Abstract

In this study, we analyze the influence of the English language proficiency of non-native speakers on the readability of the text written by them. In addition, we present multiple approaches for automatically determining the language proficiency levels of non-native English speakers from the review data. To accomplish the above-mentioned tasks, we first introduce an annotated social media corpus of around 1000 reviews written by non-native English speakers of the following five English language proficiency (ELP) groups: very high proficiency (VHP), high proficiency (HP), moderate proficiency (MP), low proficiency (LP), and very low proficiency (VLP). We employ the Flesch Reading Ease (FLE) and Flesch-Kincaid Grade (FKG) tests to compute the readability scores of the reviews written by various ELP groups. We leverage both the classical machine learning (ML) classifiers and transformer-based language models for deciding the language proficiency groups of the reviewers. We observe that distinct ELP groups do not exhibit any noticeable differences in the mean FRE scores, although slight differences are observed in the FKG test. The results imply that the readability measures do not possess high discriminating capabilities to distinguish various ELP groups. In the language proficiency determination task, we notice fine-tuned transformer-based approaches yield slightly better efficacy than the traditional ML classifiers.

Original languageEnglish
Title of host publicationWWW 2022 - Companion Proceedings of the Web Conference 2022
PublisherAssociation for Computing Machinery, Inc
Pages881-886
Number of pages6
ISBN (Electronic)9781450391306
DOIs
StatePublished - Aug 16 2022
Externally publishedYes
Event31st Companion of the World Wide Web Conference, WWW 2022 - Virtual, Lyon, France
Duration: Apr 25 2022 → Apr 25 2022

Publication series

NameWWW 2022 - Companion Proceedings of the Web Conference 2022

Conference

Conference31st Companion of the World Wide Web Conference, WWW 2022
Country/TerritoryFrance
CityVirtual, Lyon
Period04/25/22 → 04/25/22

Scopus Subject Areas

  • Computer Networks and Communications
  • Software

Keywords

  • BERT
  • language proficiency
  • native language classification
  • readability
  • user demographics

Fingerprint

Dive into the research topics of 'Influence of Language Proficiency on the Readability of Review Text and Transformer-based Models for Determining Language Proficiency'. Together they form a unique fingerprint.

Cite this