Measuring L2 Proficiency: Perspectives from SLA (Second Language Acquisition): 78 - Softcover

Pascale Leclercq

Softcover

ISBN 10: 1783092270 ISBN 13: 9781783092277

Publisher: Multilingual Matters, 2014

Synopsis

The creation of the Common European Framework of Reference for Languages (CEFR) has given rise to interest and debate among policy makers, testers, teachers and researchers alike in the reliability and feasibility of the assessment of second language (L2) proficiency. This volume brings together concrete ideas on identifying and measuring L2 proficiency from different branches of SLA research (psycholinguistic, sociolinguistic, corpus-based, applied linguistics) to contribute to a deeper understanding of what it means to be proficient in an L2. The chapters introduce a wide range of tools that are innovative, reliable, and easy-to-use for the evaluation of learners' language level with respect to both productive and receptive skills and provide a variety of answers to the question of how to assess L2 proficiency in a valid, reliable and practical manner. The collection will therefore inspire language teachers, teacher trainers and language testing specialists and help them adapt their assessment practices when necessary, and will also be a valuable resource for to postgraduate students and researchers.

About this title

About the Author

Pascale Leclercq is a Lecturer in the English Studies Department of Universite Paul Valery Montpellier 3, France, where she teaches EFL, SLA and L2 pedagogy to undergraduate and postgraduate students, and trains future language teachers. Amanda Edmonds is a Lecturer in French as a Foreign Language at the Universite de Pau et des Pays de l'Adour, France. She is currently director of the Master's program in French as a Foreign Language, and her teaching includes courses on FFL, second language acquisition, and language pedagogy. Heather Hilton is a Professor in the Language Department at the Universite de Lyon 2, France. She has extensive experience of language teaching and now researches language acquisition and foreign language teaching methodology with a particular focus on young learners and learners with learning differences.

Excerpt. © Reprinted by permission. All rights reserved.

Measuring L2 Proficiency

Perspectives from SLA

By Pascale Leclercq, Amanda Edmonds, Heather Hilton

Multilingual Matters

Copyright © 2014 Pascale Leclercq, Amanda Edmonds, Heather Hilton and the authors of individual chapters
All rights reserved.

ISBN: 978-1-78309-227-7

Contents

Contributors, 
Acknowledgments, 
Preface, 
Introduction, 
1 How to Assess L2 Proficiency? An Overview of Proficiency Assessment Research Pascale Leclercq and Amanda Edmonds, 
Part 1: General Considerations for L2 Assessment, 
2 Oral Fluency and Spoken Proficiency: Considerations for Research and Testing Heather Hilton, 
3 Multiple Assessments of Oral Proficiency: Evidence from a Collaborative Platform John Osborne, 
4 Using Learner Corpora for Testing and Assessing L2 Proficiency Marcus Callies, María Belén Díez-Bedmar and Ekaterina Zaytseva, 
Part 2: Language Processing and L2 Proficiency, 
5 Listening Comprehension: Processing Demands and Assessment Issues Peter Prince, 
6 A Psycholinguistic Measurement of Second Language Proficiency: The Coefficient of Variation Carrie A. Ankerstein, 
7 Evaluating the Workings of Bilingual Memory with a Translation Recognition Task Dominique Bairstow, Jean-Marc Lavaur, Jannika Laxén and Xavier Aparicio, 
Part 3: Focused Assessment Instruments, 
8 'Repeat as Much as You Can': Elicited Imitation as a Measure of Oral Proficiency in L2 French Nicole Tracy-Ventura, Kevin McManus, John M. Norris and Lourdes Ortega, 
9 Exploring the Acquisition of the French Subjunctive: Local Syntactic Context or Oral Proficiency? Kevin McManus, Nicole Tracy-Ventura, Rosamond Mitchell, Laurence Richard and Patricia Romero de Mills, 
10 Testing L2 Listening Proficiency: Reviewing Standardized Tests Within a Competence-Based Framework Naouel Zoghlami, 
11 Assessing Language Dominance with the Bilingual Language Profile Libby M. Gertken, Mark Amengual and David Birdsong, 
Epilogue, 
Index, 

CHAPTER 1

How to Assess L2 Proficiency? An Overview of Proficiency Assessment Research

Pascale Leclercq and Amanda Edmonds

This volume addresses an area of great interest to both language teachers and researchers interested in pedagogy and second language acquisition (SLA), namely how to assess proficiency in a second language (L2). For teachers, it is important to be able to assess accurately and reliably the L2 proficiency of their learners, whether for formative evaluation, summative evaluation or within the context of language certification. For researchers, valid proficiency assessment measures are crucial, as without them meaningful interpretation of research results remains elusive (Norris & Ortega, 2003: 717; Pallotti, 2009). In her thorough review of the assessment practices in more than 150 articles published in four major SLA journals, Thomas forcefully argues this point, concluding that 'there is evidence that L2 proficiency is sometimes inadequately assessed in current second language acquisition research. In certain cases, this compromises empirical claims; in many cases, it limits the generalizability of research results' (Thomas, 1994: 330). According to Thomas, and many others, it is important that proficiency assessment measures be valid (i.e. testing what they purport to test), reliable (i.e. providing trustworthy information) and practical (i.e. fitting the needs of their users in terms of ease of implementation). However, two decades after Thomas's call for a better control of the proficiency variable in the field of SLA, authors continue to highlight the difficulties associated with proficiency assessment. Hulstijn et al., for example, point to the lack of reliable level assignment as a general problem in SLA research:

SLA [...] has frequently simply taken groups of learners at supposedly different levels of ability, conducted cross-sectional research and claimed that the results show development. Yet the levels have been woefully undefined, often crudely labelled 'intermediate' or 'advanced', or 'first and second year university students' – which means little if anything in developmental terms – and which cannot therefore be interpreted in any meaningful way. (Hulstijn et al., 2010: 16)

Carlsen (2012: 2) deplores the fact that proficiency level remains a 'fuzzy variable' in digital learner corpora: although she admits that most learner corpora developers report on the different levels of proficiency included in their material, she claims, like Hulstijn et al., that 'the levels of proficiency are not always carefully defined, and the claims about proficiency levels are seldom supported by empirical evidence.' (For discussions of proficiency levels, see the chapters in this volume by Tracy-Ventura, McManus, Norris & Ortega, and Callies, Díez-Bedmar & Zaytseva.) Carlsen moreover argues that the reliability of corpus-based research is jeopardized by the fuzziness of the proficiency variable. Thus, despite the clear need for valid, reliable and practical methods of assessing L2 proficiency, it seems that research agendas do not always grant sufficient attention to this issue.

Our volume attempts to respond to this need for additional attention with a presentation of recent research concerning L2 proficiency assessment in L2 English and L2 French. Our aim is to provide a variety of perspectives on L2 proficiency assessment, reflecting the multiple approaches that contribute to the field of SLA research. Some of the chapters present measures of receptive proficiency (those by Prince, Zoghlami, Ankerstein and Bairstow et al., this volume), while others focus on the assessment of productive proficiency (those by Hilton, Osborne, Callies et al., Tracy Ventura et al. & McManus et al., this volume). Although most of the studies presented in this volume deal with L2 learners, Gertken et al. and Bairstow et al. take a slightly different perspective and present assessment tools for bilingual populations. This volume also includes testing instruments designed to be used in an educative setting (Prince, Zoghlami, Osborne, Gertken et al. & Callies et al., this volume), and a range of assessment instruments more specifically designed for SLA researchers: coefficient of variation (Ankerstein, this volume), translation recognition task (Bairstow et al., this volume), elicited imitation (Tracy-Ventura et al., this volume), measurement of oral fluency (Hilton, this volume), and cluster analysis (Callies et al., this volume).

This first chapter is intended to serve as a general introduction to proficiency assessment research within the larger field of SLA and to situate the different contributions to the volume. Our presentation is necessarily brief, and we refer interested readers to Norris and Ortega (2003), Bachman and Palmer (1996), McNamara (2000), Housen and Kuiken (2009) and Hulstijn (2010a, 2010b, 2011) for more complete treatments of the topic. Our own presentation will begin with a contextualization of the issue of L2 proficiency, followed by a discussion of the key concepts used in the fields of SLA and L2 proficiency research. We will then provide a preview of the different contributions to the present volume.

L2 Proficiency: A Moving Target

One of the challenges facing teachers, researchers, language testers and students interested in L2 proficiency is the defining of what it actually means to be proficient in an L2. Many proposals have been put forth, each of which is tied to a particular theoretical stance on language acquisition. The relationship between SLA theory and proficiency is clearly highlighted by Norris and Ortega (2003: 723–729) in their review of L2 proficiency from a generative, an interactionist and an emergentist perspective. As noted by these authors, 'what counts as acquisition is so dependent on the theoretical premises of the research domain that the same measurement data may be interpreted as evidence of acquisition or the lack thereof, depending on the theoretical approach adopted' (2003: 728). In addition to theoretical stance, conceptions of L2 proficiency are dependent on what we believe knowing an L2 actually entails (i.e. model of L2 competence).

Conceptions of L2 proficiency

Models of L2 competence – that is, models of what constitutes L2 proficiency – have undergone numerous changes over the past several decades. In the realm of language testing and assessment, Lado's (1961: 25–29) model of language proficiency prepared the way for language teaching and testing for decades to come. In it, he describes the intersections between four language 'elements' (pronunciation, grammatical structure, lexicon and cultural meaning) and four language 'skills' (speaking, listening, writing and reading). He acknowledges that learners' language development may occur at a different pace in the four skills, and considers 'the degree of achievement' in each of these separate skills as a variable to be tested. He also mentions 'the ability to translate' as a fifth skill, but one that should be tested 'as an end in itself and not as a way to test the mastery of language' (1961: 26). Most language textbooks and language tests are still organized today around these concepts. The year 1961 also saw the first publication of the now classic article by Carroll (1961), in which he identified fundamental considerations for the testing of L2 proficiency. In that article, Carroll argued that L2 competence included both knowledge of the language system and facility in its use, and he championed the use of both discrete-point and integrative test items in order to test both aspects (see later for a definition of these concepts). The inclusion of facility in his conception of L2 proficiency is particularly noteworthy, as it foreshadows the focus on learners' language-processing capacities that has recently come back to the forefront in language-learning theory.

In the field of linguistics and, later, in the field of SLA research, numerous models of language competence have been proposed. If, in the middle of the 20th century, certain linguists focused quasi-exclusively on the knowledge of grammatical structures as an indication of the mastery of a language (influenced by Chomsky's 1965 term linguistic competence), the publication of Hymes' (1972) article on communicative competence called this narrow interpretation of the notion of 'competence' into question, insisting on the pragmatic aspects of language use. Canale and Swain (1980) attempted a typology of the skills involved in communicative competence in a second or foreign language, particularly emphasizing its pragmatic and strategic sub-components. For them, communicative L2 competence involves grammatical and sociolinguistic competence, as well as strategic competence ('the compensatory communication strategies to be used when there is a breakdown in one of the other competencies' [1980: 27]). The importance of social and communicative (in addition to purely linguistic) competence has been more recently stressed in the Common European Framework of Reference for Languages (CEFR, Council of Europe, 2001), a highly influential statement of European language policy, which is solidly grounded in these early sociolinguistic models of communicative competence. Indeed, it contains functional descriptors of 'general and communicative language competences' (2001: 9), including descriptors covering what Canale and Swain (1980) referred to as strategic competence (Council of Europe, 2001: Ch. 4), as well as numerous proficiency scales examining aspects of grammatical, sociolinguistic, pragmatic and cultural competence (Council of Europe, 2001: Ch. 3 and 5) to describe learners' ability to produce, comprehend, interact in, and mediate their L2. (See Hulstijn, 2010b, and the recent EUROSLA Monographs Series edited by Bartning et al., 2010, for numerous studies examining the CEFR.) This document has given rise to interest and debate among policy makers, testers, teachers and researchers alike, and in many European countries, the widespread application of the CEFR has constituted a turning point in teaching and assessment practices.

This brief review mentions but a sampling of some of the different conceptions of language competence. It is important to note that the changes described here have been felt far beyond the restricted circle of those researchers interested in modeling L2 competence: they have modified our conception of what deserves our attention in language classrooms and, more importantly for the current volume, they have required changes in our conception of what it means to be proficient in an L2 and how to assess such proficiency (see Richer, 2012: 38, for a presentation of the evolution of the concept of competence as it relates to language teaching).

Defining proficiency

Higgs (1984: 12, cited by Kramsch, 1987: 356) defined proficiency as 'the ability to function effectively in the language in real-life contexts,' whereas Thomas (1994: 330, note 1) considered proficiency to correspond to 'a person's overall competence and ability to perform in L2'. More recently, Hulstijn (2010b: 186) has taken issue with the definition proposed by Thomas, finding it ambiguous because it raises the question of how to define competence and ability. In his 2010 articles and in subsequent publications, Hulstijn (2011, 2012) has set about the task of providing a more comprehensive definition of the notion of language proficiency. His 2011 article, in particular, contains a detailed discussion of this concept. What is particularly interesting is that Hulstijn's definition covers both native speakers' and learners' language proficiency, including linguistic as well as cognitive competences. The definition begins as follows:

... language proficiency is the extent to which an individual possesses the linguistic cognition necessary to function in a given communicative situation, in a given modality (listening, speaking, reading, or writing). Linguistic cognition is the combination of the representation of linguistic information (knowledge of form-meaning mappings) and the ease with which linguistic information can be processed (skill). Form-meaning mappings pertain to both the literal and pragmatic meanings of forms (in decontextualized and socially-situated language use, respectively). (Italics in the original, Hulstijn, 2011: 242)

He goes on to specify that language proficiency comprises peripheral and core components. Peripheral components include metacognitive competences, such as metalinguistic knowledge and strategic competence, whereas core components refer to linguistic cognition (in the domains of phonetics, phonology, morphosyntax etc.). He moreover claims that linguistic cognition (for native and non-native speakers alike) can be divided into basic language cognition (BLC) and higher language cognition (HLC), which he defines as follows.

(1) Basic language cognition: '(a) the largely implicit, unconscious knowledge in the domains of phonetics, prosody, phonology, morphology and syntax; (b) the largely explicit, conscious knowledge in the lexical domain (form-meaning mappings), in combination with (c) the automaticity with which these types of knowledge can be processed. BLC is restricted to frequent lexical items and frequent grammatical structures, that is, to lexical items and morphosyntactic structures that may occur in any communicative situation, common to all adult L1-ers, regardless of age, literacy, or educational level.' (Italics in the original, 2011: 230)

(2) Higher language cognition: 'the complement or extension of BLC. HLC is identical to BLC, except that (a) in HLC, utterances that can be understood or produced contain low-frequency lexical items or uncommon morphosyntactic structures, and (b) HLC utterances pertain to written as well as spoken language. In other words, HLC utterances are lexically and grammatically more complex (and often longer) than BLC utterances and they need not be spoken.' (2011: 231)

Hulstijn's definition of language proficiency, encompassing peripheral and core (HLC + BLC) elements, and covering the competence of both L1 and L2 speakers, is the one we feel to be most relevant for the current volume. It should be noted that in the different contributions, the term proficiency will sometimes be used to refer to overall language proficiency, as in Hulstijn's definition, and at other times to proficiency with respect to a specific component (e.g. listening proficiency).

Evidence used to evaluate L2 proficiency: Different practices and approaches

Discussions of L2 proficiency also vary as a function of the types of evidence that teachers and researchers have used to evaluate it. One recent and influential SLA-oriented approach focuses on the components of complexity, accuracy and fluency (CAF) in L2 proficiency (Housen & Kuiken, 2009; Housen et al., 2012). These variables have been used to measure progress in language learning and have been under close scrutiny in applied linguistics research since the early 1980s. Skehan (1989, cited by Housen & Kuiken, 2009: 461) was the first to propose CAF as three key features of L2 proficiency. The following definitions are provided by Housen and Kuiken (2009: 461):

Complexity has thus been commonly characterized as 'the extent to which the language produced in performing a task is elaborate and varied' (Ellis, 2003: 340), accuracy as the ability to produce error-free speech, and fluency as the ability to process the L2 with 'native-like rapidity' (Lennon, 1990: 390) or 'the extent to which the language produced in performing a task manifests pausing, hesitation, or reformulation' (Ellis, 2003: 342).

Housen and Kuiken (2009: 463) note that the use of these notions to assess L2 proficiency is not uncontroversial. If accuracy is an easily comprehended concept, and a widely used one in educative contexts, its operationalization as a proficiency measure remains problematic, in so far as it implies a comparison with a native speaker norm (which is a fuzzy notion in itself, native speakers' speech being extremely variable and influenced by speakers' literacy, as pointed out by Lado, 1961: 26; Davies, 2003 and Hulstijn, 2011). Complexity is an ambiguous concept that can be applied variously to the task (task difficulty being more or less equated with task complexity), to the linguistic properties of the language being used (but it remains unclear to what extent complex linguistic structures entail greater processing complexity) or to language processing itself. Fluency, the third member of this trilogy, covers a variety of dimensions related to the ease with which a speaker uses a language. However, a consensus as to which of the many dimensions involved should be taken into consideration in judging L2 proficiency has yet to be reached (but see Hilton, this volume, for a promising proposal on this issue). Moreover, in interlanguage development, accuracy, complexity and fluency (linguistic and cognitive) may interfere with one another: the focus a learner places on fluency may be detrimental to accuracy, or vice versa. The implementation of CAF indicators for L2 proficiency assessment may therefore be difficult for teachers, researchers and learners, although Martin et al. (2010) propose a model of analysis ('DEMfad') that seeks to trace changes in developmental patterns by looking at the developmental route in a given domain [D] from emergence [E] to mastery [M] with respect to the frequency [f], accuracy [a] and distribution [d] of forms.

(Continues...)

Excerpted from Measuring L2 Proficiency by Pascale Leclercq, Amanda Edmonds, Heather Hilton. Copyright © 2014 Pascale Leclercq, Amanda Edmonds, Heather Hilton and the authors of individual chapters. Excerpted by permission of Multilingual Matters.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

Edition details

• Publisher: Multilingual Matters
• Publication date: 2014
• Language: English
• ISBN 10: 1783092270
• ISBN 13: 9781783092277
• Binding: Paperback
• Number of pages: 248
• Editor: Pascale Leclercq, Amanda Edmonds, Heather Hilton