SciELO - Scientific Electronic Library Online

 
vol.56 número2PRESENTACIÓNCONSTRUCCIONES POSESIVAS DEL MAPUZUGUN. UNA APROXIMACIÓN TIPOLÓGICA índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • No hay articulos similaresSimilares en SciELO
  • En proceso de indezaciónSimilares en Google

Compartir


RLA. Revista de lingüística teórica y aplicada

versión On-line ISSN 0718-4883

RLA vol.56 no.2 Concepción dic. 2018

http://dx.doi.org/10.4067/S0718-48832018000200013 

ARTICULOS

THE ACQUISITION OF L2 LISTENING COMPREHENSION SKILLS IN PRIMARY AND SECONDARY EDUCATION SETTINGS: A COMPARISON BETWEEN CLIL AND NON-CLIL STUDENT PERFORMANCE

LA ADQUISICIÓN DE DESTREZAS DE COMPRENSIÓN ORAL EN L2 EN CONTEXTOS DE EDUCACIÓN PRIMARIA Y SECUNDARIA: UNA COMPARACIÓN ENTRE EL DESEMPEÑO DE ESTUDIANTES DE AICLE Y NO-AICLE

ESTHER NIETO MORENO DE DIEZMAS 1  

1University of Castilla-La Mancha. España. Esther.Nieto@uclm.es

ABSTRACT

This study explores the impact of CLIL (Content and Language Integrated Learning) on the development of listening skills in English both in Primary and Secondary Education in Spain. To do this, CLIL (n=2,790) and non-CLIL learners (n=17,070) enrolled in the 4th grade of Primary Education (9-10 years old), and CLIL (n=2,680), and non-CLIL students (n=17,638) in the 2nd year of Secondary Education (13-14 years old) were compared regarding their oral comprehension skills. Results showed no differences between CLIL and non-CLIL groups in Primary Education, while in Secondary School, the CLIL group significantly outperformed the non-CLIL group in the overall results of listening and in every oral comprehension subskill evaluated, and differences were higher in the most complex listening tasks. These findings seem to suggest that CLIL programs are more effective to promote the acquisition of listening skills in the target language in secondary than in primary school settings. The age of students, along with other contextual differences between the CLIL programme in Primary and in Secondary School might explain these results.

Keywords: CLIL; listening; effectiveness; Primary Education; Secondary Education

RESUMEN

Este estudio explora el impacto del AICLE (Aprendizaje Integrado de Contenido y Lengua) en el desarrollo de la comprensión oral en inglés en Educación Primaria y Secundaria en España. Para ello se compararon las destrezas receptivas orales de alumnos de AICLE (n=2790) y no AICLE (n=17,070) de 4° de Educación Primaria (9-10 años), y las de estudiantes de AICLE (n=2680), y no AICLE (n=17,638) de 2° de Educación Secundaria Obligatoria (13-14 años). Los resultados mostraron que no existían diferencias entre el grupo AICLE y el grupo no AICLE en la Educación Primaria, mientras que en la Educación Secundaria, el grupo AICLE superó significativamente al grupo no AICLE, no solo en los resultados globales, sino también en cada una de las destrezas de la comprensión oral evaluadas, y las diferencias fueron mayores en las tareas más complejas. Estos hallazgos parecen sugerir que el AICLE es más efectivo para promover la adquisición de habilidades de comprensión oral en la lengua meta en la escuela secundaria que en la escuela primaria. La edad de los estudiantes junto con otras diferencias contextuales entre el programa CLIL en Educación Primaria y en Educación Secundaria podrían explicar estos resultados.

Palabras clave: AICLE; escuchar; efectividad; Educación Primaria; Educación Secundaria

1. INTRODUCTION

Content and Language Integrated Learning (CLIL) is described by Dalton-Puffer, Nikula & Smit (2010:1) as an: "educational approach where subjects such as geography or biology are taught through the medium of a foreign lan guage", and Coyle, Hood & Marsh (2010:1) define it as "a dual-focused edu cational approach in which an additional language is used for the learning and teaching of both content and language".

To respond to the demands of the European construction in the framework of a multilingual and globalized society, European Institutions have been promoting a number of strategies in order to enhance second language learning and particu larly, to attain the so-called "mother tongue + two European languages" objec tive. In search of the most effective methodologies for acquiring second languages (Scott & Beadle, 2014), CLIL has emerged as an ingenious solution, a way to kill two birds with one stone (Dallinger, Jonkmann, Hollm & Fiege, 2016), since it is deemed to enhance second language learning (Dalton-Puffer, 2011; Pérez Ca ñado, 2012; Ruiz de Zarobe, 2011) without being detrimental for the acquisition of the subject content (Madrid 2011; Stohler 2006).

The main advantage of CLIL as opposed to traditional classes of English as a foreign language (EFL) is that CLIL can arrange for increased exposure to second languages within school time providing learning scenarios that are similar to the acquisition of the mother tongue. Thus, CLIL students, while using the second language to learn the content of school subjects such as Social Science, Natural Science or Art, are learning to use the second language (Marsh & Langé, 2000; Mehisto, Marsh & Frigols, 2008). In addition, CLIL focuses on communication, provides a rich and varied input (different language functions, academic language, classroom language...), and fosters students' production of comprehensive out put. Additionally, CLIL facilitates the process of learning new content conveyed through a new language by promoting the students' development of learning and cognitive strategies (Nieto-Moreno de Diezmas, 2016a).

Hence, underpinned by EU policy with the hope for it to be "the potential lynchpin to counter Europe's deficient language standards" (Pérez Cañado & Ráez Padilla 2015:1), CLIL "has firmly embedded itself in the language teaching sce nario" (Pérez Cañado, 2016:10). Thus, CLIL programmes "are burgeoning in Eu ropean school contexts" (Lasagabaster & Doiz 2015:1), and their implementation "has become commonplace in most European educational systems" (Pladeval-Ballester & Vallbona, 2016:37).

As a consequence of the proliferation of CLIL programmes, "the body of research tapping into its effects has also grown considerably" (Pérez Cañado, 2018:52) and because of this, CLIL has become "a highly topical issue" (Lasaga-baster & López Beloqui, 2015:42). Due to the fact that most researchers inter ested in CLIL are linguists (Merino & Lasagabaster, 2018; Sierra, Gallardo del Puerto, Ruiz de Zarobe, 2011), and also because the main rationale for CLIL provision is precisely to improve second languages learning (Dalton-Puffer, 2011), the area that has brought about more studies and has drawn more attention is the effect of CLIL on the acquisition of second languages being used as a means of instruction in these programmes.

However, despite the growing body of research that addresses the acquisition of the target language, there are still areas that require further investigation to fully understand the impact of CLIL (Paran, 2013; Pérez Cañado, 2012). This is espe cially the case with oral comprehension. Dalton-Puffer (2008:143) underscores that "certain aspects of language competence are developed more than others" and establishes the well-known classification between "competencies favourably affect ed by CLIL" and "competencies unaffected by CLIL". Although Dalton-Puffer (2008) places receptive skills among the competences favorably affected by CLIL, "there is still little research on receptive skills", and "further studies will help us research more definite conclusions" (Ruiz de Zarobe, 2015:57). What is more, in the case of oral comprehension competence, existing research shows "contradicto ry results" and "no clear conclusions can be drawn" (Ruiz de Zarobe, 2015:56). In turn, Prieto-Arranz, Rallo-Fabra, Calafat-Ripoll, & Catrain-González (2015:216) conclude that "the lack of research is perhaps especially all the more evident in the field of listening comprehension", and more evidence is needed on the repercus sions of CLIL on the development of oral reception competence (Pérez Cañado & Lancaster, 2017).

Therefore, the aim of this paper is to provide more evidence on the impact of CLIL on one of the most under-researched areas of foreign language competence, oral comprehension, by studying its acquisition in a span of four years: in the 4th grade of Primary Education (9-10 year olds) and in the 2nd year of Secondary Education (13-14 year olds). The article is organized as follows: after providing an overview of prior investigations into listening acquisition in CLIL settings, it goes on to explain the method: participants, instruments and data analysis. Next, results are presented and discussed. Finally, the main conclusions to be drawn are expounded.

2. THEORETICAL FRAMEWORK: OVERVIEW OF THE RESEARCH INTO LISTENING ACQUISITION IN CLIL

As Lasagabaster and Sierra state (2009:373) "The CLIL approach stems from im mersion programmes", since the inception of CLIL was inspired by the "promis ing results obtained in the pioneering immersion programmes in Canada and the USA" (Nieto-Moreno de Diezmas, 2016a:21), and both bilingual education pro grammes consist of delivering school subjects in a second language. Nevertheless, as Lasagabaster and Sierra (2009) argue, it is important to note that there are some differences between immersion programmes and CLIL, mainly as regards to the language of instruction, language objectives and teachers. In CLIL programmes, the language of instruction is a foreign language, which is not usually present in the students' context outside school, whereas the vehicular language in immer sion programmes is usually spoken locally, so that students can have contact with it after school. Additionally, teachers in immersion programmes are usually na tive speakers, in contrast to CLIL teachers who are foreign speakers of the target language. Consequently, language objectives are more ambitious in immersion (native-like proficiency) than in CLIL programmes (students should acquire a B1 or B2 level of proficiency of the Common European framework of Reference for Languages (CEFR), depending on the programme, by the end of Compulsory Secondary School).

One of the most debated issues in the field of the contribution of bilingual education for the acquisition of language skills has been the so-called dichotomy between receptive and productive skills. This framework emerged in the context of Canadian immersion programmes, in which it was observed that Anglophone learners "perform as well as native French-speaking students on tests of reading and listening comprehension in French" (Genesee, 1991:186), but "they seldom achieve the same high levels of competence in speaking and writing as they achieve in comprehension" (Genesee, 1991:186). Therefore, it was assumed that bilingual education impacted the acquisition of receptive skills more than productive com petences.

These findings were somewhat imported into CLIL settings by authors such as Dalton-Puffer (2008:2011) who listed receptive skills as competences positively affected by CLIL, and placed writing, a productive skill, among the unaffect ed areas. However, there was evidence that CLIL had a positive effect on lexical richness in writing tasks (Jiménez Catalán, Ruiz de Zarobe & Cenoz, 2006) and in speaking (Hüttner & Rieder-Bünemann, 2007; Lasagabaster, 2008; Ruiz de Zarobe, 2008), which are both productive skills, while in listening, which is a re ceptive skill, the benefits are not as conclusive (Lasagabaster, 2008; Navés, 2011). Hence, in her overview of studies on the effectiveness of CLIL, Ruiz de Zarobe (2011, 2015) questioned the application of the Canadian dichotomy between re ceptive and productive skills to CLIL environments, thus including most aspects of writing and speaking among the areas in which clear gains were observed, and moreover, the scholar cast doubt on the potential of CLIL for the acquisition of the receptive oral skills, since studies in this area were contradictory and insuf ficient.

This lack of adequate research may be because, due to the influence of Ca nadian immersion, it was assumed that CLIL had a positive impact on receptive skills, and researchers drew more attention to look into the influence of CLIL on the development of productive skills. Consequently, there is still a scarcity of investigations on receptive skills development in CLIL settings, and "published research studies showing evidence of listening and reading development are few" (San Isidro & Lasagabaster, 2018:3). Furthermore, in addition to the shortage of research "in the case of listening skills, results are not as categorical" (San Isidro & Lasagabaster, 2018:3), and the general trend that can be observed in the inves tigations reviewed below is the coexistence of studies that show no impact or a limited effect of CLIL on oral comprehension and investigations that detect much more optimistic outcomes. The design of the programme -particularly in terms of the amount of CLIL exposure to the target language-, the age of participants, the methodology of the investigation, the sociolinguistic context, and whether the school is public or private seem to be the main factors behind contradictory outcomes in this area.

In the context of Primary Education, there are two studies set in Switzerland that ascertain a positive impact of CLIL on the development of listening compre hension. Stotz and Meuter (2003) studied the impact of a CLIL project imple mented in Zurich aimed at introducing English into the curriculum starting in the first years of Primary School. In a particularly complex sociolinguistic context, English was not taught as a separate subject, but used as a language of instruction in some subjects and for about 90-100 minutes a week, thereby avoiding "stealing time from other subjects" (Stotz & Meuter, 2003:85). Despite the limited expo sure to English, researchers observed gains in listening skills.

More conclusive findings were found in the longitudinal study conducted by Serra (2007) in Chur (Swizerland). The author presented evidence concerning the assessment of L2 oral production, oral and written comprehension, and mathe matics from pupils in grades 1 to 6 in a CLIL programme. In contrast to the study previously mentioned, in which there were no control groups, Serra (2007) com pared the results of the CLIL group to the outcomes obtained by a control group made up of bilingual pupils who were in contact at home with one of the two minority languages of instruction of this CLIL programme: Italian or Romansch. The CLIL programme assessed by Serra (2007) provided a higher exposure to the second languages (50% of the curriculum), and the CLIL and bilingual students were together in the same class, thereby increasing the classroom conversational dynamics. Consequently, the programme was very successful in terms of effec tiveness on language learning. As far as listening comprehension was concerned, CLIL students performed just as well as their bilingual counterparts. These find ings, unsurprisingly, tally with the typical benefits of Canadian immersion pro grammes, since some aspects of the sociolinguistic situation and the features of this CLIL programme were quite close to Canadian immersion experiences.

In contrast to these positive outcomes, it is worth mentioning another three investigations set in Primary School in which no benefits from CLIL were ob served. All of them were carried out in Spain, and in all cases the L2 was a foreign language (mainly English) not present in the sociolinguistic context of students, and therefore their contact with the target language outside school was scarce.

The first study was conducted in Catalonia by Pladevall-Ballester & Vallbona (2016) with a sample of287 primary learners in the 5th and 6th grades coming from four state-funded private schools. All students had been exposed to the subject of English three hours per week, and the CLIL group had received an additional hour per week of exposure to English in the subjects of Science or Art and Crafts. To keep the number of hours of exposure to English constant, CLIL 5th graders and non-CLIL 6th graders were compared, and whereas no significant differences were found in reading comprehension, the non-CLIL group significantly outper formed the CLIL group in listening. Hence, the authors conclude that "EFL-only exposure is more effective in the development of young learners' listening skills in the short-term" (Pladevall-Ballester & Vallbona, 2016). Interestingly, findings also revealed the fact that the choice of the CLIL subject had an impact on the development of listening skills, since no differences were found between the CLIL Science and the non-CLIL groups in their oral comprehension, whereas the CLIL Arts and Crafts group scored significantly lower than the non-CLIL group.

Likewise, in Castilla-La Mancha, a monolingual Spanish autonomous com munity, Nieto-Moreno de Diezmas (2016b), who studied a group a bit younger (4 th grade, 9-10 years old), did not find significant differences in listening com prehension between CLIL and non-CLIL primary students enrolled in public schools, even if the CLIL group had received 250 additional hours of exposure to English through different subjects. Similar to Pladevall-Ballester and Vallbona (2016), the author concluded that to observe positive effects of CLIL methodol ogy, more time and exposure would be necessary.

In turn, the investigation carried out by Pérez Cañado (2018) in three mono lingual Spanish autonomous communities (Andalusia, Extremadura, and the Canary Islands) with 828 students in the 6th grade of Primary Education (11 12 years-old), did not detect significant differences in listening when CLIL and non-CLIL students enrolled in public schools were compared, but the author did identify significant gains when the CLIL group came from private schools. In this case, the CLIL programme implemented in private schools turned out to be more effective for the development of oral comprehension, and whether the school was public or private accounted for the differences found in the acquisition of this skill.

In Secondary Education settings, most of the studies show positive findings regarding the impact of CLIL on foreign language learning, although evidence seems to suggest that oral comprehension is the skill that is affected by CLIL to a lesser extent.

Thus, in the Basque Country, Lasagabaster (2008) compared CLIL and non-CLIL students enrolled in the 4th year of Secondary Education and CLIL learners significantly outperformed the non-CLIL group in all skills assessed, including lis tening. However, in an attempt to control the time of exposure to English, CLIL students in the 3rd year of Secondary Education (14-15 years-old) were compared to non-CLIL students in the 4th year (15-16 years-old), and CLIL students scored higher but not significantly in speaking and writing, but, precisely, not in listen ing.

Similarly, Navés (2011) analyzed data from CLIL and non-CLIL learners from the 5th to 10th grades in Catalonia and observed that CLIL learners outscored their non-CLIL peers at each grade in listening, dictation and grammar. How ever, when compared to older non-CLIL students, "CLIL learners matched or outperformed learners two or three grades ahead of them in all proficiency tests except for listening comprehension" (Navés, 2011:179), showing that, although benefits from CLIL on listening are detected when students in the same grade are considered, when contrasted to older learners, the less positively affected skill is, comparatively, oral comprehension.

In turn, in the context of the Balearic Islands, the longitudinal study of Pri-eto-Arranz et al. (2015) compared oral receptive skills of 50 CLIL and 37 non-CLIL Secondary students from 13 to 16 years-old enrolled in public schools, and concluded that "the CLIL learner group did not score significantly higher than the foreign language group" (Prieto-Arranz et al., 2015:131), probably because of limited exposure to English provided in this programme, (CLIL participants got additional exposure to the target language only by means of a single subject, either Science or Social Science). However, CLIL students outstripped their coun terparts in one of the listening tests, particularly the one considered to be more cognitively demanding, and this fact suggests that CLIL specifically contributes to developing skills to perform in more complex listening activities.

Pérez-Vidal & Roquet (2015) also did not find significant differences in the improvement in oral comprehension experienced by CLIL and non-CLIL learn ers in Barcelona (Catalonia) in a two-year span (from 13 to 15 years old), even if other language competences, such as reading and writing did significantly benefit from CLIL. Similarly to the previous study (Prieto Arranz et al., 2015), the extra exposure by means of CLIL was received through a single subject (Science). Ad ditionally, listening competence was measured by means of a dictation and this fact could have affected poor results in listening, having in mind the difficulty that entails both the complexion of the task by students and the attribution of mistakes to spelling or to oral comprehension.

In contrast to Prieto-Arranz et al. (2015) and Pérez-Vidal & Roquet (2015), in the Andalusian context, Pérez Cañado and Lancaster (2017) did detect sig nificant benefits for oral comprehension skills after a year of following CLIL and non-CLIL Secondary Education students in the 4th year, but when the students were in Baccalaureate, the authors detected that CLIL positive effects on listening did not pervade six months after. However, the improved oral production skills of CLIL students consolidated in Baccalaureate, thereby pointing at listening as the most fragile skill.

In the Basque Country, Merino and Lasagabaster (2018) looked into the ef fect of CLIL on the acquisition of English, Basque and Spanish of 12-13 year-old students by means of a one-year longitudinal study. Results showed that CLIL students started Secondary Education with a significantly higher level in listening comprehension, but, after a year, no significant differences between CLIL and non-CLIL groups' progress in oral reception was observed. However, when the evolution of CLIL and non-CLIL students is witnessed in slightly older students (14-15) with a two-year span (San Isidro & Lasagabaster, 2018), a significant im provement of the CLIL group is detected. The conclusion drawn by the authors is that effectiveness of CLIL is determined by the number of years of implementa tion and the intensity of the programme.

On the whole, prior research into the effects of CLIL on oral comprehension development seems to support the hypothesis, which has been put forth by some previous authors, that oral receptive skills are not necessarily found to be sig nificantly affected when comparing CLIL students to language learners in more traditional foreign language classrooms (Ruiz de Zarobe, 2015; Pérez Cañado & Lancaster, 2017), in contrast to what has been found in Canadian immersion contexts, where immersion students often demonstrate native-like competency in their listening comprehension skills. However, findings in Secondary Education settings seem to be more optimistic than in Primary School.

3. THE STUDY

3.1. Research questions

The following questions were posed in this study:

RQ1. Are there any significant differences between CLIL and non-CLIL Primary Education students within the Castilla-La Mancha educational programme re garding their listening skills? And for Secondary School students?

RQ2. Are there any significant differences regarding the acquisition of particular subskills and areas of oral comprehension in Primary or Secondary Education between CLIL and non- CLIL students?

3.2. Participants

This cross-sectional study aims at answering the aforementioned research ques tions. To do that, data collected from CLIL and non-CLIL students in the 4th grade of Primary Education (9-10 year-olds) and in the 2nd year of Secondary Ed ucation (13-14 year-olds) enrolled in public schools in the Spanish Autonomous Community of Castilla-La Mancha were analyzed. CLIL students (n=2,790) in the 4th grade of Primary Education (henceforth 4PE) were compared to a control group made up of non-CLIL students (n=17,070) enrolled in the same grade. On the other hand, CLIL students (n= 2,680) in the 2nd year of Secondary Educa tion (henceforth 2SE) were compared to 17,638 non-CLIL students in the same grade. In both cases, the participants in this study were the census of students of Castilla-La Mancha, which offers quite a clear picture of the effectiveness of bilin gual programmes in this autonomous community in Primary and in Secondary Education.

The non-CLIL group was made up of mainstream students who were only learning the target language in traditional classes of English as a foreign language (EFL) three hours a week. The CLIL group was composed of students in the bilin gual programme launched in Castilla-La Mancha (European Sections) and were receiving bilingual education through at least 50% of the school time of at least 2 content subjects, in addition to the subject of EFL. Since schools -depending on their availability of content teachers with the required level of English- establish the subjects to be taught through the target language, bilingual subjects can vary from one school to another. However, in the European Sections of Primary and Secondary Education the typical bilingual subjects are Social Science and Natural Science.

Compared to their non-CLIL counterparts, students enrolled in the CLIL pro gramme in the 4th grade of Primary School had received an additional exposure to the foreign language through CLIL of around 250 hours on average during four school years, and CLIL students enrolled in the 2nd year of Secondary School had studied in English the content of non-linguistic subjects for around 250 hours as well. The most part of secondary students started the CLIL programme at second ary school, only around 8% of them received bilingual education from primary school.

Although the bilingual programme is common for Primary and Secondary Education, there is a number of differences in its implementation depending on the educational level. Thus, in Primary Education, the CLIL teacher is generally a specialist in English (EFL), whereas in Secondary Education, CLIL teachers are the specialists of the content subjects (Music, Biology or History, for example) with at least a B2 level of the CEFR. In addition, in bilingual primary schools, all the students, regardless their academic level or their proficiency in English, take part in the bilingual programme (i.e., the bilingual programme is mandatory for all students enrolled in a bilingual school), and this fact, along with the prohibi tion of selection of students makes the bilingual programme egalitarian, inclusive and non-selective (Nieto-Moreno de Diezmas & Ruiz-Cordero, 2018).

In Secondary Education, access of students to the bilingual programme also follows the general rules of admission applicable to all schools (proximity of home to school, number of siblings already enrolled in the center, low income, etc.) and selection of students is not allowed. However, in bilingual secondary schools, there are different branches for bilingual and non-bilingual students, and this fact can contribute to self-selection, since more gifted students could choose to enroll the bilingual programme in Secondary Education, whereas students who experi ence difficulty in English, could opt for the non-bilingual branch.

3.3. Instruments and procedure

The oral comprehension tests analyzed in this study were developed by the Evalu ation Office of Castilla-La Mancha and were connected to the objectives and contents included in the curricular decrees in force for the 4th grade of Primary Education (4PE, henceforth) and for the 2nd year of Secondary Education (2SE, henceforth). Both tests were structured around a video and six tasks which were connected to a set of oral comprehension subskills. The topics chosen were close to the motivations and knowledge of both groups of students: healthy habits for 4PE, and ecology (the impact of pollution in the environment) for 2SE. The video for 4PE contained 132 words and had a duration of 1.41 minutes. The video for 2SE consisted of 83 words and lasted 1.07 minutes. Although the video for 4PE was longer than the one addressed to 2SE, the tasks were much easier due to the basic content and visual support.

Both primary and secondary students had 30 minutes to watch the respective videos twice and perform the 6 tasks. Every task was related to the acquisition of one of the following subskills: global comprehension, understanding of the situ ation of communication, identification of details, understanding paralinguistic elements, vocabulary and identification of space-time relations. For the test ad dressed to 4PE, the response format in all tasks was multiple choice with four alternatives: one correct option and three distractors. The maximum score of ev ery task was 1 point. In the test for 2SE, the subskills "global comprehension", "understanding of the situation of communication", "identification of details" and "understanding of paralinguistic elements" were assessed by means of multiple choice questions with four alternatives awarded 1 point each. For the evaluation of the subskills "vocabulary" and "identification of space-time relations", the re sponse format was short answer questions, and the maximum score that could be obtained in these questions was 2 points.

3.4. Data analysis

Statistical analysis of data was performed with the aid of the SPSS (Statistical Pack age for Social Science), in its 21.0 version. The internal consistency and reliability of the tests was high for the test for 2SE (Cronbach's alpha=0.798) and acceptable for the test for 4PE (Cronbach's alpha=0.651). The Kolmogorov-Smirnov Test (K-S test) showed the sample to have a normal distribution, so it was possible to do a parametric test. Independent sample t-tests were run to compare the results of the CLIL and the non-CLIL groups of 4PE and 2SE, and to determine if their differences were significant.

To answer RQ1, the overall results of CLIL and non-CLIL groups in 4EP and in 2SE are presented as marks out of 10.

4. RESULTS AND DISCUSSION

4.1. Overall oral comprehension in 4PE and 2SE

Results showed that in Primary Education (4PE), no significant differences were detected between the CLIL and the non-CLIL group in the development of their oral comprehension in English, and that even non-CLIL students scored slightly and non-significantly higher. However, the bilingual programme deploys all its potential for the development of listening skills in secondary school, since CLIL students in Secondary Education (2SE) significantly outperformed their non-bilingual peers (Table I).

Table I Overall results in oral comprehension in 4PE and 2SE. 

On the other hand, one of the findings that comes first to the fore is the poor performance of students in oral comprehension. Mainstream students, whether in Primary or in Secondary Education, and also CLIL learners in Primary Education, scored below 5 points out of 10 (although the Primary Education groups were quite close). In school contexts, this would mean that none of these groups would pass. These low outcomes in the oral receptive skill are unfortunately in keeping with poor achievements shown in international assessments, such as the European Survey on Language Competences (INEE, 2012), which showed that Spanish students in the last year of Compulsory Education were second to last in the ranking regarding their oral comprehension proficiency in English compared to the rest of European countries assessed.

This panorama alone justifies the need to improve methodologies in language teaching, including the implementation of the integrated curriculum. Precisely, the most cited rationale for the provision of CLIL, from the European Union to the regional governments, including national regulations, is enhancing commu nication competences in foreign languages, and particularly acquiring oral skills for the students to be able to interact in different contexts. In this sense, although CLIL has been heralded as a receipt to foster oral skills, the results of this study seem to indicate that its effectiveness is not visible in Primary Education. However, in Secondary Education, CLIL appears to significantly contribute to improving oral comprehension skills.

4.2. Oral comprehension in Primary Education (4PE)

Results showed that in 4PE, findings were controversial, since, although CLIL positively impacted some subskills, it has not got any influence on others, and there were even skills in which CLIL affected negatively. Thus, CLIL students scored significantly higher in global comprehension (p= .000) and in identifying details (p= .001), since p<.05, but there were no differences between the CLIL and non-CLIL group in understanding the situation of communication and in identifying paralinguistic elements. Non-CLIL students significantly outperformed their bilingual counterparts in vocabulary (p = .000) and in understanding of space-time relations (p=.000), as in both cases p<.05 (Table II).

Table II Results in oral comprehension subskills assessed in 4PE. 

The main conclusion to be drawn regarding bilingual Programmes of Castilla-La Mancha (Spain) in Primary Education is that CLIL did not significantly improve the oral comprehension of students 4PE (9-10 years old) since there were no-significant differences between both groups in the overall results for this skill. This outcome tallies with previous studies conducted in Spain in Primary Educa tion settings (Nieto-Moreno de Diezmas, 2016b; Pladevall-Ballester & Vallbona, 2016; Pérez Cañado, 2018).

The research of Pérez Cañado (2018) can be included among the investiga tions that fall in line with this study, since although the author found significant differences in listening in favor of the CLIL students in Primary Education, when results were compared in terms of type of school, no significant differences were detected between CLIL and non-CLIL groups of public schools, and this is the outcome that has to be taken into account, knowing that the sample analyzed in this study came precisely from public schools.

Unsurprisingly, the results of this investigation run counter to the outcomes found by Serra (2007), who detected a significant progress in listening comprehension of CLIL students. These differences in effectivity of CLIL can be explained by contextual differences in the implementation of the Suisse programme, in terms of quantity of exposure and contact to native speakers of the target language inside and outside the school.

Nevertheless, it can be surprising that all the effort that CLIL entails for the educational administration, teachers, students and families had no impact in the acquisition of listening in the target language. There are different factors that could account for this fact. Firstly, in the bilingual programme examined in this study, CLIL teachers of Primary Education are not generally content specialists, but English specialists, and, as result, CLIL teachers and English teachers have received a similar training and come from a similar methodological background. This fact may have influenced the results, since as shown by Stotz and Meuter (2003), different methodological approaches, probably absent in our context, are directly responsible for different outcomes.

However, this explanation alone cannot account for the results of this study, since, even assuming CLIL and non-CLIL teachers used the same methodology for the teaching of oral receptive skills, CLIL students still received a higher amount of exposure to English through CLIL, and no positive repercussions were observed. This fact can be explained for the effect of the age of students, since their cognitive development and learning and transference strategies are in prog ress (Nieto-Moreno de Diezmas, 2016b) and therefore they are not ready to take full advantage of CLIL yet.

Additionally, as several authors suggest, the time factor is key for CLIL to be successful in attaining its objectives (Merino & Lasagabaster, 2018; Nieto-Moreno de Diezmas, 2016b; Pladevall-Ballester & Vallbona, 2016; San Isidro & Lasagabaster, 2018). The participants in CLIL, and particularly pupils, need time to adapt to bilingual education, and in addition, CLIL requires a sufficient amount of hours to produce significant advantages (Ruiz de Zarobe, 2007).

Results in particular oral receptive competences are even more contentious. It is easy to explain that both groups showed a similar command in understanding the situation of communication and in identifying paralinguistic elements, since these subskills refer to oral competences that can be connected to "cognitive skills that mature independently of increased L2 input" (Hüttner & Rieder-Bünemann, 2007:27). However, it is more surprising that non-CLIL students significantly outperformed their CLIL peers in vocabulary and in understanding space-time relations.

Regarding vocabulary, significantly lower results of the CLIL group run counter to previous research confirming gains of CLIL students in receptive vocabulary (Canga Alonso, 2015; Jiménez Catalán, Ruiz de Zarobe & Cenoz 2006; Jiménez Catalán & Ruiz de Zarobe 2009), although, in contrast, such investigations focus on written vocabulary and not on its oral comprehension.

As for the oral comprehension of space-time relations, it is important to note that the task used for its assessment was the most difficult activity in the test, since both groups obtained their lowest score. However it was the non-CLIL group that significantly outstripped the CLIL branch. This outcome contrasts to the observation of Prieto-Arranz et al. (2015) and Pérez Cañado and Lancaster (2017), who detected that CLIL students enrolled in Secondary Education were significantly more proficient in more demanding and complex listening activities. In this bilingual Primary Education setting, this finding shows just the opposite trend, since CLIL students lagged behind in the most difficult task of the test.

In contrast, CLIL students significantly outperformed their non-CLIL peers in global comprehension and in identification of details, which are typically considered to be essential when evaluating oral comprehension. This can be indicative that CLIL pupils are acquiring solid grounds regarding oral reception skills, although more time, more cognitive development and more exposure to CLIL would be needed to expand CLIL students' proficiency in oral comprehension.

4.3. Oral comprehension subskills in Secondary Education (2SE)

In 2SE, CLIL positively impacted all subskills related to the acquisition of oral comprehension, since bilingual students scored significantly higher than mainstream students in all of them (p=.000 in all the skills): global comprehension, understanding of the situation of communication, identification of details, understanding of paralinguistic elements, vocabulary and identification of space-time relations (Table III).

Table III Results in oral comprehension subskills assessed in 2SE. 

Therefore, in contrast to Primary Education, in Secondary Education the effectiveness of CLIL methodology compared to traditional classes of English as a foreign language was widely confirmed, since the CLIL-group outstripped their non-CLIL peers in all subskills of the oral receptive competence. In fact, the tables have turned, since wider benefits of CLIL were precisely recorded in the subskills in which the CLIL group in Primary Education had shown significant disadvantages, i.e. vocabulary and comprehension of space time-relations.

This time, the findings do correlate with the studies of Prieto-Arranz et al. (2015) and Pérez Cañado and Lancaster (2017), also set in Secondary Education, which established that CLIL students performed better in more complex listening tasks. In the test used in the present study, vocabulary and comprehension of space-time relations, just like in the tests used for Primary Education, were the most difficult subskills, since both were the only subskills assessed by means of open questions and both groups recorded their lowest scores in these. In these subskills, differences in favor of the CLIL group were wider than in the rest.

Significant advantages of Secondary Education CLIL students in the acquisition of oral receptive competences are in keeping with data collected in other monolingual Spanish autonomous communities such as Andalusia, Extremadura and The Canary Islands (Pérez Cañado, 2018), in which similar CLIL programmes are being implemented in the framework of a comparable sociolinguistic context.

5. CONCLUSIONS

In this study, the impact of CLIL on oral comprehensive competence has been examined, by analyzing data from Primary Education students in the 4th grade (9-10 years old) and from Secondary Education in the 2nd year (13-14 years old) enrolled in public educational establishments of Castilla-La Mancha (Spain). Two groups were compared: CLIL students, who received an additional exposure to English of around 250 hours, by means of at least 50% of two non-linguistic subjects, and non-CLIL students, who learned the target language only by means of the subject of English (EFL).

Results suggest that the impact of CLIL on the development of oral comprehension was more noticeable in Secondary School than in Primary Education, since CLIL students in Secondary School scored significantly higher in all subskills of listening comprehension (and differences were higher in the most demanding and complex listening tasks), while no significant differences were found in the overall oral comprehension between CLIL and non-CLIL learners in Primary Education.

However, CLIL learners in the 4th grade of Primary School significantly outperformed their non-bilingual peers in global comprehension and identification of details, which are core skills of oral reception and predictors of listening comprehension proficiency. This finding might indicate that the process of English acquisition is moving in the right direction, but in line with previous research (La-sagabaster, 2008; Navés, 2011), oral comprehension appears to be an arduous skill to significantly improve by means of CLIL, and particularly in Primary School (Nieto-Moreno de Diezmas, 2016b; Pérez Cañado, 2018; Pladevall-Ballester & Vallbona, 2016), mainly due to the age and level of cognitive development of students among other factors.

The indisputable success of the CLIL programme in Secondary School might suggest that secondary students were in a better situation for fully taking advantage of CLIL, since they presumably developed their cognitive, learning and transference strategies. In this vein, Reilly and Medrano (2009) underscored that although pupils in bilingual programmes make progress during Primary Education, it is, however, in Secondary Education when bilingual students attaint higher levels in foreign language acquisition and develop the socalled "by-products" of bilingual education, in terms of cognitive and social skills improvement.

However, it cannot be ignored that along with the level of cognitive development, there were further factors in connection with the context that might account for such good results in the CLIL programme in Secondary School. Thus, although regulations forbade selection of students to enroll the CLIL programme, in both Primary and Secondary School, the access was voluntary, and it is possible that students opted for entering the bilingual programme because they were more gifted or more motivated for learning English. This would affect primary CLIL programmes to a lesser extent, since pupils enroll the bilingual school at the age of three, and their attitude and aptitude for learning foreign languages were probably not yet so clearly defined than in secondary settings.

All in all, the influence of motivation and aptitude in language learning outcomes in CLIL settings deserves further scrutiny, along with other aspects such as family support and extramural exposure. These variables should be controlled and study in future research on the CLIL programme analyzed in this investigation. Additionally, as suggested by Lázaro Ibarrola (2016), to complete the information that a quantitative picture offers, it would be interesting to conduct classroom observations to better understand the listening instruction delivered in CLIL and non-CLIL settings in Primary and in Secondary Education

REFERENCES

Canga Alonso, Andrés. (2015). Receptive Vocabulary of CLIL and Non-CLIL Primary and Secondary School Learners. Complutense Journal of English Studies, 23, 59-77. [ Links ]

Coyle, Do; Philip Hood & David Marsh. (2010). CLIL: Content and Language Integrated Learning. Cambridge, United Kingdom: Cambridge University Press. [ Links ]

Dallinger, Sara; Jonkmann, Kathrin; Hollm, Jan & Fiege, Christiane. (2016). The effect of content and language integrated learning on students' English and history competences: Killing two birds with one stone? Learning and Instruction, 41, 23-31. [ Links ]

Dalton-Puffer, Christiane. (2008). Outcomes and processes in Content and Language Integrated Learning (CLIL): current research from Europe. In W. Delanoy & L. Volkmann (eds.) Future Perspectives for English Language Teaching (pp. 139-157). Heidelberg, Germany: Carl Winter. [ Links ]

Dalton-Puffer, Christiane. (2011). Content-and-Language Integrated Learning: From practice to principles? Annual Review of Applied Linguistics, 31, 182-204. [ Links ]

Dalton-Puffer, Christiane, Nikula, Tarja & Smit, Ute. (2010). Charting policies, premises and research on Content and Language Integrated Learning. In Ch. Dalton-Puffer, T. Nikula & U. Smit (eds.) Language use and language learning in CLIL Classrooms (pp. 1-22). Amsterdam, Netherlands: John Benjamin Publishing. [ Links ]

Genesee, Fred. (1991). Second language learning in schools settings: Lessons from immersion. In A. Reynolds (ed.) Bilingualism, multiculturalism, and second language learning (pp. 183-201). Hillsdale, NJ, United States: Lawrence Erlbaum. [ Links ]

Hüttner, Julia, & Rieder-Bünemann, Angelika. (2007). The effect of CLIL instruction on children's narrative competence. Vienna English Working Papers, 16(3), 20-27. [ Links ]

INEE. (2012). Estudio Europeo de Competencia Lingüística EECL Vol I. Informe Español. Madrid, Spain: Secretaría General Técnica del Ministerio de Educación, Cultura y Deporte. [ Links ]

Jiménez Catalán, Rosa María & Ruiz de Zarobe, Yolanda. (2009). The receptive vocabulary of EFL learners in two instructional contexts: CLIL versus non-CLIL. In Y. Ruiz de Zarobe & R. M. Jiménez Catalán (eds.). Content and Language Integrated Learning. Evidence from research in Europe (pp. 81-92). London, United Kingdom: Multilingual Matters. [ Links ]

Jiménez Catalán, Rosa María, Ruiz de Zarobe, Yolanda & Cenoz, Jasone. (2006). Vocabulary profiles in English as a subject and as a vehicular language. VIEWZ, 15(3), 23-27. [ Links ]

Lasagabaster, David. (2008). Foreign language competence in content and language integrated courses. The Open Applied Linguistics Journal, 1, 31-42. [ Links ]

Lasagabaster, David & Doiz, Aintzane. (2015). A Longitudinal Study on the Impact of CLIL on Affective Factors. Applied Linguistics, 38(5), 688-712. doi:10.1093/applin/amv059. [ Links ]

Lasagabaster, David & López Beloqui, Raquel. (2015). The Impact of Type of Approach (CLIL versus EFL) and Methodology (Book-Based Versus Project Work) on Motivation. Porta Linguarum, 23(1), 41-57. [ Links ]

Lasagabaster, David & Sierra, Juan Manuel. (2009). Immersion and CLIL in English: More differences than similarities. ELT Journal, 63(4), 67-375. DOI:10.1093/elt/ccp082. [ Links ]

Lázaro Ibarrola, Amparo. (2016). Are CLIL learners simply faster or also different? Evidence from L1 use in the repair sequences and discourse markers of CLIL and EFL learners. Vigo International Journal of Applied Linguistics (VIAL), 13, 127-145. [ Links ]

Madrid, Daniel. (2011). Monolingual and Bilingual Students' Competence in Social Studies. In D. Madrid & St. Hughes (eds.) Studies in Bilingual Education (pp. 195-222). Bern, Switzerland: Peter Lang. [ Links ]

Marsh, David & Langé, Giselle. (2000). Using languages to learn and learning to use languages. Jyváskylá, Finland: University of Jyváskylá. [ Links ]

Mehisto, Peeter, Marsh, David & Frigols, María Jesús. (2008). Uncovering CLIL: Content and Language Integrated Learning in Bilingual and Multilingual Education. Oxford, United Kingdom: Macmillan. [ Links ]

Merino, Jon Ander & Lasagabaster, David. (2018). CLIL as a way to multilingualism. International Journal of Bilingual Education and Bilingualism, 21, 79-92. [ Links ]

Navés, Teresa. (2011). How promising are the results of integrating content and language for EFL writing and overall EFL proficiency. In Y. Ruiz de Zarobe, J. M. Sierra & F. Gallardo del Puerto (eds.) Content and foreign language integrated learning. Contributions to multilingualism in European contexts (pp. 155- 186). Bern, Switzerland: Peter Lang. [ Links ]

Nieto-Moreno de Diezmas, Esther. (2016a). The Impact of CLIL on the Acquisi tion of the Learning to Learn Competence in Secondary School Education in the Bilingual Programmes of Castilla-La Mancha. Porta Linguarum, 25, 21-34. [ Links ]

Nieto-Moreno de Diezmas, Esther . (2016b). The impact of CLIL on the acquisi tion of L2 competences and skills in primary education. International Journal of English Studies (IJES), 16(2), 81-101. [ Links ]

Nieto-Moreno de Diezmas, Esther & Ruiz-Cordero, Begoña. (2018). Evaluación de los programas AICLE en Castilla-La Mancha. In J. L. Ortega-Martín, St. Hughes & D. Madrid (eds.) Influencia de la política educativa de centro en la enseñanza bilingüe en España (pp. 93-104). Madrid, Spain: MECD-British Council, Secretaría General Técnica, Ministerio de Educación, Cultura y Deporte. [ Links ]

Paran, Amos. (2013). Content and language integrated learning: Panacea or policy borrowing myth? Applied Linguistics Review, 4(2), 317-342. [ Links ]

Pérez Cañado, María Luisa. (2012). CLIL research in Europe: Past, present, and future. International Journal of Bilingual Education and Bilingualism, 15(3), 315-341. [ Links ]

Pérez Cañado, María Luisa. (2016). Stopping the pendulum effect in CLIL research: Finding the balance between Pollyanna and Scrooge. Applied Linguistics Review ,, DOI: 10.1515/applirev-2016-2001. [ Links ]

Pérez Cañado, María Luisa. (2018). CLIL and Educational Level: A Longitudinal Study on the Impact of CLIL on Language Outcomes. Porta Linguarum, 29, 51-70. [ Links ]

Pérez Cañado, María Luisa & Lancaster, Nina K. (2017). The effects of CLIL on oral comprehension and production: A longitudinal case study. Language, Culture, and Curriculum, 30(3), 300-316. [ Links ]

Pérez Cañado, María Luisa & Ráez Padilla, Juan. (2015). Introduction and overview. In D. Marsh, M. L. Pérez Cañado & J. Ráez Padilla (eds.) CLIL in action: Voices from the classroom (pp. 1-12). Newcastle upon Tyne, United Kingdom: Cambridge Scholars Publishing. [ Links ]

Pérez-Vidal, Carmen & Roquet, Helena. (2015). CLIL in context: Profiling lan guage abilities. In M. Juan-Garau & J. Salazar Noguera (eds.) Content-based language learning in multilingual educational environments (pp. 237-255). Amsterdam, Netherlands: Springer. [ Links ]

Pladevall-Ballester, Elisabet & Vallbona, Anna. (2016). CLIL in minimal input contexts: A longitudinal study of primary school learners' receptive skills. System, 58, 37-48. [ Links ]

Prieto-Arranz, José Igor; Rallo Fabra, Lucrecia; Calafat-Ripoll, Catalina & Catrain González, Magdalena. (2015). Testing progress on receptive skills in CLIL and non-CLIL contexts. In M. Juan-Garau, M. & J. Salazar Noguera (eds.) Content-based language learning in multilingual educational environments (pp. 123-137). Amsterdam, Netherlands: Springer. [ Links ]

Reilly, Teresa & Medrano, Pilar. (2009). MEC/British council bilingual project. Twelve years of bilingual education and a smooth transition into secondary. In E. Dafouz & M. C. Guerrini (eds.) CLIL across educational levels (pp. 59-70). London, United Kingdom: Richmond. [ Links ]

Ruiz de Zarobe, Yolanda. (2007). CLIL in a Bilingual Community: Similarities and Differences with the Learning of English as a Foreign Language. VIEWZ Vienna English Working Papers, 16(3), 47-52. [ Links ]

Ruiz de Zarobe, Yolanda. (2008). CLIL and foreign language learning: A longitudinal study in the Basque Country. International CLIL Research Journal, 1(1), 60-73. [ Links ]

Ruiz de Zarobe, Yolanda. (2011). Which language competencies benefit from CLIL? An insight into Applied Linguistics research. In Y. Ruiz de Zarobe, J. M. Sierra & F. Gallardo del Puerto (eds.) Content and foreign language inte grated learning. Contributions to multilingualism in European contexts (pp. 129 -153). Frankfurt-am-Main, Germany: Peter Lang. [ Links ]

Ruiz de Zarobe, Yolanda (2015). The effects of implementing CLIL in education. In M. Juan-Garau & J. Salazar Noguera (eds.). Content-based language learning in multilingual educational environments (pp. 51-67). Amsterdam, Netherlands: Springer. [ Links ]

San Isidro, Xavier & Lasagabaster, David. (2018). The impact of CLIL on pluriliteracy development and content learning in a rural multilingual setting: A longitudinal study. Language Teaching Research DOI: 10.1177/136216881775 4103journals.sagepub.com/home/ltr [ Links ]

Scott, David & Beadle, Shane. (2014). Improving the effectiveness of language learning: CLIL and computer assisted language learning. London, United Kingdom: European Commission. [ Links ]

Serra, Cecilia. (2007). Assessing CLIL at primary school: A longitudinal study. International Journal of Bilingual Education and Bilingualism, 10(5), 582-602. [ Links ]

Sierra, Juan Manuel; Gallardo del Puerto, Franscisco & Ruiz de Zarobe, Yolanda. (2011). Good practice and future actions on CLIL: Learning and Pedagogy. In Y. Ruiz de Zarobe, J. M. Sierra & F. Gallardo del Puerto (eds.) Content and Foreign Language Integrated Learning (pp. 317-338). Bern, Switzerland: Peter Lang. [ Links ]

Stohler, Ursula. (2006). The acquisition of knowledge in bilingual learning: an empirical study on the role of language in content learning. VIEWZ Vienna English Working Papers, 15(3), 41-46. [ Links ]

Stotz, Daniel & Meuter, Tessa. (2003). Embedded English: Integrating content and language learning in a Swiss primary school project. Le Bulletin Suisse de Linguistique Appliquée, 77, 83-101 [ Links ]

1This study has been funded by the Spanish Ministry of Economy and Competitiveness (project FFI2014-54179-C2-2-P).

Received: July 21, 2018; Accepted: November 14, 2018

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License