The population of English Language Learners (ELLs)¹; in the United States is growing. Between 1990 and 1997 the number of US residents not born in this country increased by 30%, from 19.8 million to 25.8 million, the largest total in the nation's history (Hakuta & Beatty, 2000, p. 1). One out of seven children in the US speaks a language other than English at home (Garcia, 1998). In California, Florida, New Mexico, and Texas, ELLs exceed 10% of the student population in public schools.

¹The term "English language learner" (ELL) refers to students who are not native speakers of English and are not as proficient in English as the native speakers are. A subgroup of these students with lower level of English proficiency is referred to as "limited English proficient" (LEP). The term LEP is used primarily by government-funded programs to classify students, as well as by the National Assessment of Educational Progress (NAEP) for determining inclusion criteria. In this article we use "English language learner (ELL)" to refer to students who are not native English speakers and who are not reclassified as fluent in English.

As the ELL population has grown, so has the need to include these students in large-scale testing programs. Recent legislation, from the reauthorization of the Elementary and Secondary Education Act (ESEA), through the enactment of the Improving America's Schools Act (IASA) of 1994, has mandated the inclusion of English language learners in large-scale academic assessments administered in English.

However, language factors may influence the validity of assessments for ELLs. Literature on the assessment of students with limited English proficiency has found a substantial link between students' language background and their performance in content-based areas. For example, studies by CRESST researchers have clearly demonstrated that language factors have significant impact on students' performance in math and science (Abedi & Lord, 2001; Abedi, Lord, Hofstetter, & Baker, 2000; Abedi, Lord & Hofstetter, 1998, Abedi, Lord & Plummer, 1997). Following is a summary of findings of studies on the impact of language on the assessment of ELLs.

Results of the studies that were summarized above along with studies conducted by other researchers clearly indicate that the performance-gap between ELL and non-ELL students is mainly due to language factors (see for example, Adams, 1990; Cocking & Chipman, 1988; Cummins, 1984; De Corte, Verschaffel, & DeWin, 1985; Hudson, 1983; Noonan, 1990; Ramirez, Yuen, Ramey, & Billings, 1991; Riley, Greeno, & Heller, 1983). To present a more valid picture of ELL students' content-knowledge, innovative ways of assessing student performance are encouraged, including modifications to existing instruments (August & Hakuta, 1997).

Provision of accommodations has helped to increase the rate of inclusion for ELL students (Mazzeo, 1997). Based on the promising results from using accommodations in the 1996 National Assessment of Educational Progress main assessment, accommodations were provided in the 1997 assessment in art and in the 1998 assessment in reading, writing, and civics.

There are, however, some major concerns regarding the use of accommodations for ELL students. Among the most important issues is the concern on the effectiveness and validity of accommodation strategies. How effective are accommodations in reducing the performance-gap between ELL and non-ELL students? How valid are the accommodated assessments? Do accommodations provide unfair advantage to the recipients, i.e., do they alter the construct under measurement?

Effectiveness

Research has clearly demonstrated that the effectiveness of accommodations in reducing the performance-gap between ELL and non-ELL students varies considerably across different types of accommodations and due to the characteristics of recipients.

To be effective, the accommodation should reduce the performance gap between ELL and non-ELL students by helping ELL students to improve their performance. In other words, it should level the playing field. In a study on the impact of accommodation on eighth grade students in math, Abedi et al. (2000) applied four different types of accommodations (linguistically modified English version of the test, standard NAEP items with glossary only, extra time only, and glossary plus extra time). Students were also tested using standard NAEP items with no accommodation. Students were randomly assigned to one of the four accommodation conditions or to a control group with no accommodation. Table 1 presents mean math score under the four different accommodations and under the standard NAEP condition. As the data in Table 1 show, some of the accommodations were effective and increased students' performance in both ELL and English-proficient groups. Glossary plus extra time was the most effective form of accommodation in this study. This accommodation helped to increase performance of ELL students by 1.62 score points or 13%; it also increased performance of English-proficient students by 2.81 points or 16% (35 points possible).

Table 1

Mean, Number of Students Tested under Different Accommodations and t-test Comparing Means

ELL Status

Note: A t-test examines the difference between two mean scores to determine if the difference is statistically significant. The denominator for the t-ratios was the pooled within subject variance to control for impact on alpha (Type I error rate) due to multiple comparisons.

The results of another study (Abedi & Lord, 2001) of 1,031 eighth grade students in southern California showed small but significant differences in the scores of students in low and average-level math classes. Among the linguistic features that appeared to contribute to the differences were low-frequency vocabulary and passive voice verb constructions (see Abedi et al., 1997, for discussion of the nature of and rationale for the modifications). Another study (Abedi et al., 1998) of 1,394 eighth graders in schools with high enrollments of Spanish speakers showed that modification of the language of the items contributed to improved performance on 49% of the items; the students generally scored higher on shorter problem statements. Rivera and Stansfield (2001) modified the complex language of science test items and compared student performance on regular and simplified fourth and sixth grade items. Although the small sample size did not show significant differences in scores for English language learners, the study did demonstrate that linguistic simplification did not affect the scores of the English-proficient students, indicating that linguistic simplification is not a threat to score comparability.

Extra time is a popular and easily implemented accommodation strategy and may lead to higher scores for English learners (Hafner, 2001; Kopriva, 2000). The results of several studies indicated that extra time helps ELL students. For example, extra time helped eighth grade English learners on NAEP math tests, but it also aided students already proficient in English, thus limiting its potential as an assessment accommodation (Abedi et al., 1998; Abedi et al., 2000; Hafner, 2001).

Commercially published English dictionaries are also used as a form of accommodations for ELL students. In a study, English dictionaries were provided to urban middle school students in Minnesota during a reading test (Thurlow, 2001). The results indicated that some students benefited from this accommodation. This accommodation strategy has an advantage in that students are probably already familiar with using dictionaries. However, commercially available dictionaries differ widely; some have entries in "plain language" that English learners can understand (Kopriva, 2000). Abedi, Courtney, Leon, Mirocha & Goldberg (2001) used a published dictionary as a form of accommodation for students in grades four and eight in several locations nationwide. They found the published dictionary was not effective and was administratively difficult. A major concern in the use of published dictionaries is the possibility that they may provide information that the test is measuring. Thus, dictionary as a form of accommodation may jeopardize the validity of assessment.

Another study of 422 students in eighth grade science classes (Abedi, Lord, Kim, & Miyoshi, 2000) compared performance on NAEP science items that were randomly distributed among students in three test formats: one booklet in original format (no accommodation); one booklet with an English glossary and Spanish translations in the margins; and one booklet with a customized English dictionary at the end of the test booklet. The customized dictionary included only words that appeared in the test items. English learners scored highest on the customized dictionary accommodation (their mean scores for the three formats were 8.36, 8.51, and 10.18, respectively, on a 20-item test). Interestingly, although the accommodations helped the English learners score higher, for the English-proficient students there was no significant difference between their scores in the three test formats. This suggests that the accommodation strategies did not affect the construct.

Validity

What do research findings suggest about the validity of scores from accommodated assessments? Reducing the performance gap between ELL and English-proficient students is desirable, but only if the accommodations do not give an unfair advantage to the recipients (Shepard, Taylor, & Betebenner, 1998). The results of CRESST studies showed that some accommodation strategies helped both ELL and English-proficient students. For example, Abedi et al. (2000) showed that both ELL and English-proficient students benefited substantially from having a glossary plus extra time. However, students who were already proficient in English showed a greater improvement (16%) than the ELL students (13%).

Table 1 presents t-ratios comparing performance of ELLs and non-ELLs under different forms of accommodation. The results of analyses in this table suggest that some accommodation helped non-ELL students significantly and did not have much impact on the performance of ELL students. This is a cause for concern because it may invalidate the accommodated results. Unfortunately, many states use accommodations without evidence of their validity. If an accommodation improves the performance of all students, then the accommodated results may not be combined with the results of non-accommodated assessments.

There are also validity concerns over other forms of accommodations. One of the most commonly used accommodation strategy is translation of assessment tools into student's native language. Research has indicated that native-language assessments are useful only when students can demonstrate their content knowledge more effectively in their native language. Otherwise translated items may confuse students who have learned content and concepts in English. Abedi et al. (1998) found that eighth grade Hispanic ELL math students taught in English or sheltered English scored higher on NAEP math items in English than their peers who received the same math items administered in Spanish. In contrast, students who received their math instruction in Spanish performed higher on the Spanish-language math items than those in either modified or Standard English. This is a threat to the validity of assessment since lack of understanding of content related terms may add a source of construct irrelevant variance to the assessment (Messick, 1994).

Other findings from CRESST research include the following.

Recommendations

The results of CRESST studies along with the findings of other studies nationwide leads us to make the following recommendations to policy makers and educators involved in the assessment of ELL students.