I. INTRODUCTION
Each student activity is presented in written form and numbered by day. For example seven sheets are required for Day 1, and these are numbered 1-1 to 1-7. The sheets for Day 2 are numbered 2-1 to 2-6, and so on. At the end of the student activity sheets is the Los Arboles Dictionary that contains all the vocabulary used in the unit. There is a total of 78 student sheets, arranged on 42 double-sided pages.
A master copy of the student materials is provided for photocopying, along with specific instructions for teachers on how to copy the materials back-to-back. Teachers are asked to copy the student materials on pre-drilled 3-hole punch paper (preferably using recycled paper), and to distribute all the materials on the first day of the unit. Each student is asked to supply a small 3 ring binder to store all materials for the unit. Students are asked to bring their binders with the materials with them to class every day, and no additional materials are needed.
The 3-ring binder for student materials allows students to take out and replace certain pages (for example homework sheets turned in for grading, or dialogue sheets when reading in class). It also allows students to look for information throughout the materials (for example when using the dictionary, or when referring to one of the maps). Finally, the binder keeps student work organized throughout the unit so they are able to review all their work to prepare for the final assessment. Students do all of their writing on the sheets that are provided, so there is no need for a notebook or loose papers.
The Teachers Guide that accompanies the video and student materials is designed to provide a comprehensive collection of background information, organizational guidance and teaching strategies for the unit. The guide includes the following 9 resources:
1) An introduction to the curriculum, with a basic rationale for and overview of the curriculum.
2) A description of the organization of the curriculum, so that teachers are oriented to how the materials are organized.
3) Directions for duplication of the student materials, which explain how materials should be copied and disseminated to students.
4) Suggestions for integration with other subject area teachers. The curriculum offers many opportunities for integration with science and social studies to teach about the scientific and environmental roles of rain forests, and the geography and culture of Latin America.
5) A discussion of utilizing the video dialogues to promote language learning. Teachers are encouraged to review dialogues in English before viewing each video segment, then to have students practice the dialogues after watching the video. The text of the narration for the three trips appears on the video with Spanish subtitles, so students hear the Spanish pronunciation and see the written Spanish simultaneously, while watching images that provide a context for the dialogue and enhance their understanding of the spoken language.
6) A fully classroom-tested set of student evaluation instruments and suggestions for grading. There are many opportunities for assessing student progress throughout the curriculum unit, including daily homework assignments, a quiz after day 5, a final test, and a poster project. In addition, teachers can grade student participation in a variety of individual, group and whole-class activities.
7) Detailed day-by-day lesson plans, teaching tips and background information for each lesson. The approximate time for each classroom activity is given, and lesson plans have been designed for a 45-minute class period. In addition, background information for teachers is provided in order to enhance discussion of environmental and science questions that may come up in class. Suggestions for optional follow-up activities are also provided.
8) A complete set of student materials with answers. All student materials are reproduced in the teacher's guide along with answers in italics. The materials are organized sequentially by day, so that the lesson plan for each day is followed by the corresponding student materials. The Los Arboles Dictionary is also included.
9) A teacher feedback form is provided at the end of the guide and teachers are solicited to send their assessment of the curriculum unit to INCRE, with suggestions for improvement and ideas for new activities that could be included in revisions of the materials. A student feedback form is also provided, so that students using the curriculum can also give their reactions to the unit and make suggestions for improvements.
III. FIELD TESTING AND EVALUATION
The Los Arboles Hablan materials have been classroom tested and evaluated in 17 middle school Spanish I and Spanish II classes in the spring of 1993. This section describes the nature of the field test sites, the evaluation methodology, the evaluation results and conclusions, and recommendations based on the field testing of the curriculum unit.
Two middle school sites were selected for field testing and evaluation of the curriculum unit. One site was the Fairgrounds Junior High School in Nashua NH, and the other was the Randolph Junior High School in Randolph MA. Fairgrounds Junior High has won awards as the best junior high school in New Hampshire and was nationally recognized in 1989 as an outstanding school by then-President Bush. It is located in a small town environment in southern New Hampshire. Its student population is predominantly white and from relatively affluent families. Spanish instruction at the school begins in 8th grade, and classes meet daily for 50 minutes. All 275 8th and 9th grade students in Spanish I and Spanish II participated in the field test.
Randolph Junior High School, the other field test site, is located in a predominantly blue-collar suburb of Boston. School enrollment includes 31% non-white and Hispanic students, and 17% are students whose first language is not English. Spanish instruction at the school begins in 7th grade and classes meet daily for 45 minutes. All 190 8th grade students in Spanish II participated in the field test.
The two goals of the field testing and evaluation were 1) to obtain feedback from teachers and students and classroom observers that would be useful in improving the content and organization of the curriculum unit, and 2) to assess the extent to which the curriculum unit engaged student interest and promoted learning of Spanish and content related to Latin American forests.
The first phase of field testing at each site was a day-long teacher training workshop. The objective of the workshop was to familiarize the participating teachers with the entire curriculum, and at the same time to obtain their feedback about the appropriateness of activities and assessment items for their students. The workshop also provided an opportunity for teachers to record the correct answers for all activities, and to point out typographical and grammatical errors that somehow managed to creep into the student materials.
Selected classes were pre-tested using the final assessment instrument. The instrument assesses content knowledge related to rain forests and Latin American geography, as well as testing Spanish reading comprehension, sentence construction and sentence completion.
Teachers were given photocopied sets of student materials for all participating students on 3-hole punch paper, which were distributed to students on the first day of the unit, and which students kept in 3-ring binders.
INCRE evaluators logged a total of 45 hours of classroom observation, and collected data on student reaction to the materials, amount of class time per activity, difficulties with organization or content of activities, effective teaching strategies, and errors and additions in the student materials. Each curriculum day was observed in at least 3 different Spanish I and Spanish II classes. The classroom observations also offered a lot of informal and immediate feedback about the materials from the participating teachers. Changes in the curriculum materials based on observations and teacher feedback were noted and materials were revised as appropriate.
Several evaluation activities were conducted as the unit came to a close at each site. All students took the post-test, which was the same instrument as the pre-test. Students also completed a feedback form that solicited their opinions of the curriculum unit, asked them which activities they enjoyed the most and the least, and solicited their suggestions on how to improve the curriculum.
At the close of the unit, INCRE evaluators organized a teacher focus group at each site to debrief. Teachers were able to assess the impact of the curriculum, make more general suggestions for improvements, and share the results of the student poster projects. The debriefing meeting was also an opportunity to thank the teachers for their time and effort in field testing the unit. Finally, teachers were provided with a feedback form in order to give individual written feedback about the unit.
All data indicate that the curriculum unit was very well received and achieved defined learning objectives. Student and teacher reaction was overwhelmingly positive, and results of pre-post testing showed large gains in Spanish language skills and in content knowledge related to Latin American geography and rain forests. This section presents the results obtained from the various data collection instruments.
1. Test Results
Of the 465 participating students, a total of 177 students in 8 classes were sampled to take the pre- and post-tests. All other students only took the post-test. The pre-post sample consisted of 64 Spanish I and 113 Spanish II students.
The test results showed consistent and highly significant gains in all sections of the test. The composite score on the written test increased from a mean of 65.4 (s = 10.8) on the pre-test to a mean of 81.8 (s = 11.3) on the post test, a mean gain of 16.4 points, or 25.1% (t=34.8, p<.001). There were no significant differences in the pre-test scores between students in Spanish I and Spanish II. The average gain of Spanish I students was actually somewhat higher than the average gain of Spanish II students: Spanish I students increased their scores by an average of 18.2 points, while Spanish II students increased their scores by an average of 15.5 points (F=7.6, p<.05).
The gains held consistently across the different sections of the test. The test has two main components. The first component (Sections I to IV of the test) assesses Spanish vocabulary and reading comprehension, as well as content knowledge related to the three types of rain forest. The first component counts for 70% of the total test score. The second component (Sections V to VII of the test) assesses content knowledge related to the three types of rain forest in English, and accounts for 30% of the total test score. Table 3 below shows the gains from pretest to post-test on both components of the instrument.
TABLE 3: PRE/POST TEST GAINS |
|
|
Percent Gain |
Component 1:
|
Spanish Vocabulary
Spanish Reading Comprehension
Content Knowlede |
27.3%
|
Component 2: |
Content Knowledge in English |
19.9%
|
|
|
25.1% |
It should be noted that the sample that was pre/post tested was representative of all students participating in the field test. There was no significant difference in the mean post-test scores of the pretested group and the students who only took the post-test.
2. Student Feedback
A total of 319 students at the two field test sites completed the feedback form. 50% said that they enjoyed the Los Arboles Hablan curriculum "a lot" or "quite a bit", and only 13% said they enjoyed the curriculum "a little" or "not at all". 70% thought that the curriculum unit was more interesting than their regular Spanish class, and only 8% thought that the unit was less interesting than their regular class. 87% thought that the curriculum unit should be used in Spanish class again next year.
The most popular activity by far was watching the videos, which was cited by 66% of students as the activity they liked the most. Other popular activities that were listed by students included (in order of popularity) the final poster project, working in groups, learning about rain forests, the crossword puzzle, the dialogues, the worksheets, and calling the travel agency.
By far the least popular activity named by students (not surprisingly, perhaps) were the final test, quiz and homework, which were cited by 40% of students. Only 5% of student respondents named watching the videos as one of their least favorite activities.
The most interesting result on the student feedback form involved how much students thought they learned during the unit. While students thought that they learned some Spanish (3.2 on the 5 point Leikert scale), they thought that they learned quite a bit about Latin American forests (4.1 on the scale). This difference was most notable for the Spanish II students in Nashua, who averaged 2.9 on the scale for learning Spanish, but 4.1 for learning about Latin American forests. 81% of the Nashua students thought they learned "a lot" or "quite a bit" about Latin American forests, but only 22% thought they learned "a lot" or "quite a bit" of Spanish. The results of these questions are particularly interesting when compared with the final test results (reported in section 1 above), which showed that students advanced considerably in their Spanish language skills as well as in their knowledge of Latin American forests.
Another interesting finding was that while 33% of Randolph students thought that the unit was harder than their regular Spanish class, only 2% of Nashua students thought that the unit was harder. However, many of the Randolph students who thought the unit was hard also thought it was more interesting than their regular Spanish class. Also, it is notable that the majority of Nashua students who found the unit easier than their regular Spanish class were 9th grade Spanish II students.
Students also had a variety of comments about the unit, including the following:
"It makes students more willing to learn when the learning is based on something like the rain forests."
"You not only learn about the forests, you learn more Spanish while your (sic) at it."
"It's interesting to learn about the problems of the world in a different language."
"I enjoyed it. I learned about rain forests, even different cultures. I also learned about some of the different species. It was interesting."
"It helps you learn more about the culture of Latin America."
"I think it is an easier and more efficient way to learn Spanish. I would like to see units about all the other natural features of the world."
"The dictionary was a good idea because now I can keep it and know how to say a few words that I otherwise would not know how to say."
"I had no idea about the rain forests and the danger they are in."
"I really enjoyed being able to try out your new way of teaching Spanish to the kids."
"It gave us a good break from the textbook and is a more interesting way to learn."
"It is a good subject to head out of school with. I know that during the summer I will think about the rain forests."
"We learned a lot of Spanish and about working together. Also about the rain forests and how we can save or conserve them."
"It teaches Spanish vocabulary and it teaches geography we didn't know."
"I think there should be a program for elementary school kids too (Not in Spanish though). My little sister enjoyed looking at the unit and I translated for her."
"Make it longer. It was fun."
3. Teacher Feedback
All 6 teachers who field tested Los Arboles Hablan were unanimous in their praise of the curriculum unit. On the feedback form, all six said that their students enjoyed the curriculum unit "A lot", and all six also concurred that they enjoyed teaching the unit "A lot". All six also said that they are planning to use the curriculum unit again next year.
Teacher comments included the following:
"I think the unit is great, well timed and informative. I think you have done a great job putting this together."
"The students really enjoyed the project and they learned a lot of Spanish too. I also learned a great deal using the materials."
"The message is important and the change of pace was great."
"The unit was very well organized. I didn't have to do lesson plans for two weeks. What a treat!"
"The videos were great. I just wish you could have made them longer."
"This unit is terrific. The only problem was that my students wanted more, they did not want to stop. Are you going to make a sequel?"
In addition to their positive comments about the curriculum, teachers had many excellent suggestions for improvements and additions to the materials. Their suggestions were incorporated in revising and expanding the curriculum whenever it was appropriate.
The teachers who teach the 9th grade advanced Spanish II classes in Nashua thought that some of the true/false exercises were too easy, and felt that overall the unit was somewhat easier than their regular curriculum. The teachers who worked with Spanish I and 8th grade Spanish II classes all thought that the unit was about the same or harder than their regular class work in Spanish. All teachers concurred that Los Arboles Hablan is suitable for use at the end of the year in middle school Spanish I and at any time during the year in Spanish II.
All teachers concurred that students learned "A lot" of Spanish in this unit, and also learned "A lot" about Latin American forests.
4. Classroom Observations
The extensive classroom observation by INCRE evaluators yielded the following results:
• Students were engaged and on-task throughout the unit. The videos served to generate a lot of student interest which translated to rich discussions and attention to classwork and homework. Some students complained on the one day that a video segment is not used (Day 3).
• The extensive cooperative learning activities in pairs and small groups were integrated very well into the flow of the unit. There was no time lost to classroom mechanics or disruptive behavior.
• The length of all activities was very close to the times suggested in the lesson plans, and teachers had no difficulty in following the daily schedule. The teachers in Nashua, where classes are 50 minutes long, felt very comfortable with the timing of the unit. The teachers in Randolph, where some classes are as short as 42 minutes, felt that the timing was at times a little tight.
• The teacher's guide provided sufficient materials and guidance for teachers, none of whom needed additional support at any time during the field tests. Teachers felt that the materials and lessons provided in the teacher's guide were fully adequate.
• Nearly all activities were at an appropriate level of difficulty for middle school Spanish I and Spanish II students. Advanced Spanish II students found a few activities too easy.
• Teachers had no difficulty in operating the VCR and using the videotape. Teachers use the audiovisual equipment almost daily, and need to make arrangements to have exclusive access to the equipment throughout the unit. If more than one teacher is using the unit simultaneously, each teacher must have their own copy of the videotape and teacher's guide.
• The distribution of student materials on 3-hole punched paper works very well as long as all students supply a 3-ring binder. Handing out all student materials at the beginning of the unit is far preferable to passing out the pages day by day.
• Students were very motivated by the final poster project. Students are asked to make a fully bilingual poster showing one type of Latin American rain forest. Their interest in the theme was evident in the products, which were displayed in school corridors and generated many positive comments from school administrators and other teachers.
The main conclusion is that the curriculum unit Los Arboles Hablan has proved to be very popular with teachers and students alike, and has been shown to be effective in promoting Spanish language skills and knowledge of geography, science and environmental content related to the rain forests of Latin America. The success of the unit demonstrates that video can effectively be used as an integral component of instruction, and that an interdisciplinary curriculum can be effective in promoting language learning while teaching substantive content knowledge. For additional information about Los Arboles Hablan, please contact INCRE at 366 Massachusetts Avenue, Arlington MA, Phone (781) 643-2142, Fax (781) 643 1315, or E-mail at mail@incre.org