Search results

In the years 2014 through 2019, three U.S. universities, Michigan State University, the University of Minnesota, Twin Cities, and The University of Utah, received Language Proficiency Flagship Initiative grants as part of the larger Language Flagship, which is a National Security Education Program (NSEP) and Defense Language and National Security Education Office (DLNSEO) initiative to improve language learning in the United States. The goal of the three universities' Language Proficiency Flagship Initiative grants was to document language proficiency in regular tertiary foreign language programs so that the programs, and ones like them at other universities, could use the proficiency-achievement data to set programmatic learning benchmarks and recommendations, as called for by the Modern Language Association in 2007. This call was reiterated by the National Standards Collaborative Board in 2015.

During the first three years of the three, university-specific five-year grants (Fall 2014 through Spring 2017), each university collected language proficiency data during academic years 2014-2015, 2015-2016, and 2016-2017, from language learners in selected, regular language programs to document the students' proficiency achievements.

University A tested Chinese, French, Russian, and Spanish with the NSEP grant funding, and German, Italian, Japanese, Korean, and Portuguese with additional (in-kind) financial support from within University A.

University B tested Arabic, French, Portuguese, Russian, and Spanish with the NSEP grant funding, and German and Korean with additional (in-kind) financial support from University B.

University C tested Arabic, Chinese, Portuguese, and Russian with the NSEP grant funding, and Korean with additional (in-kind) financial support from University C.

Each university additionally provided the students background questionnaires at the time of testing. As stipulated by the grant terms, at the universities, students were offered to take up to three proficiency tests each semester: speaking, listening, and reading. Writing was not assessed because the grants did not financially cover the costs of writing assessments. The universities were required by grant terms to use official, nationally recognized, and standardized language tests that reported scores out on one of two standardized proficiency test scales: either the American Councils of Teaching Foreign Languages (ACTFL, 2012) proficiency scale, or the Interagency Language Roundtable (ILR: Herzog, n.d.) proficiency scale. The three universities thus contracted mostly with Language Testing International, ACTFL's official testing subsidiary, to purchase and administer to students the Oral Proficiency Interview - computer (OPIc) for speaking, the Listening Proficiency Test (LPT) for listening, and the Reading Proficiency Test (RPT) for reading. However, earlier in the grant cycling, because ACTFL did not yet have tests in all of the languages to be tested, some of the earlier testing was contracted with American Councils and Avant STAMP, even though those tests are not specifically geared for the specific populations of learners present in the given project.

Students were able to opt out of testing in certain cases; those cases varied from university to university. The speaking tests occurred normally within intact classes that came into computer labs to take the tests. Students were often times requested to take the listening and reading tests outside of class time in proctored language labs on the campuses on walk-in bases, or they took the listening and reading tests in a language lab during a regular class setting. These decisions were often made by the language instructors and/or the language programs. The data are cross-sectional, but certain individuals took the tests repeatedly, thus, longitudinal data sets are nested within the cross-sectional data.

The three universities worked mostly independently during the initial year of data collection because the identities of the three universities receiving the grants was not announced until weeks before testing was to begin at all three campuses. Thus, each university independently designed its background questionnaire. However, because all three were guided by the same set of grant-rules to use nationally-recognized standardized tests for the assessments, combining all three universities' test data was rather simple. During year two of data collection, the three universities organized to produce a more unified background questionnaire that would pose many of the same questions to students during the final third (2017) year of testing. Thus, this data deposit project, beyond the test scores and simple background data from all three years of testing, also contains data from the students' 2017 background questionnaire questions that were common across all three university background questionnaires.

Acknowledgements: The projects benefited over the years from the help of the following individuals: Daniel R. Isbell, Xiaowan Zhang, Elizabeth Webster, Angelika Kraemer, Shinhye Lee, Jessica Fox, Melody Wenyue Ma, Amaresh Joshi, Bill VanPatten, Charlene Polio, Daniel Reed, Koen Van Gorp, Steven Ross, and Steven Pierce aided the project at Michigan State University. Elaine Tarone, Stephanie Treat, Monica Frahm, Kate Paesani, Carter Griffith, Ellen Wormwood, Anna Hubbard, Diane Rackowski, Gabriela Sweet, Anna Olivero-Agney, Adolfo Carrillo Cabello, Caroline Vang, Beth Dillard, Andrew Wilson, and Colin Delong aided the project at the University of Minnesota, Twin Cities. Catherine Scott, Elvis Ryan, Lissie Ah Yen, Paul Allen, and Jeanine Alesch contributed to The University of Utah project. Special thank yous from the three university PIs are extended to Erwin Tschirner, Margaret E. Malone, and Helen Hamlyn for their valuable assistance over the years with data collection, data information, and testing assistance, and to Judith E. Liskin-Gasparro for her assistance with the advanced speaking project that occurred during years 4 and 5 of the project. The PIs at the three universities extend their sincere appreciation to Samuel D. Eisen and Kaveri Advani at DLNSEO and Carrie Reynolds and Chelsea Sypher at IIE for their grant guidance and overall project support.

References:ACTFL. (2012). ACTFL Proficiency Guidelines 2012. http://www.actfl.org/publications/guidelines-and-manuals/actfl-proficiency-guidelines-2012

Herzog, M.(n.d.). An overview of the history of the ILR language proficiency skill level descriptions and scale. https://www.govtilr.org/Skills/

Modern Language Association. (2007). Foreign languages and higher education: New structures for a changed world. https://www.mla.org/Resources/Research/Surveys-Reports-and-Other-Documents/Teaching-Enrollments-and-Programs/Foreign-Languages-and-Higher-Education-New-Structures-for-a-Changed-World

National Standards Collaborative Board. (2015). World-readiness standards for learning languages (4th ed.). Alexandria, VA: ACTFL.

The National Assessment of Educational Progress (NAEP) is the largest nationally representative and continuing assessment of what students in elementary and secondary schools in the United States know and can do in various subject areas. Assessments are conducted periodically in mathematics, reading, science, writing, the arts, civics, economics, geography, United States history, and beginning in 2014, in Technology and Engineering Literacy (TEL). Since NAEP assessments are administered uniformly using the same sets of test booklets across the United States, NAEP results serve as a common metric for all states and selected urban districts. The assessment stays essentially the same from year to year, with only carefully documented changes. This permits NAEP to provide a clear picture of student academic progress over time and for teachers, principals, parents, policymakers, and researchers to use NAEP results to assess progress and develop ways to improve education in the United States. For more information, please read An Introduction to NAEP.

There are two types of assessments: main NAEP and long-term trend NAEP. Main NAEP is administered to fourth-, eighth-, and twelfth-graders across the United States in a variety of subjects. The Main NAEP is conducted between the last week of January and the first week in March every year. National results are available for all assessments and subjects. Results for states and select urban districts are available in some subjects for grades 4 and 8. The Trial Urban District Assessment (TUDA) is a special project developed to determine the feasibility of reporting district-level NAEP results for large urban districts. In 2009 a trial state assessment was administered at grade 12. Long-term trend NAEP is administered nationally every four years. During the same academic year, 13-year-olds are assessed in the fall, 9-year-olds in the winter, and 17-year-olds in the spring. Long-term trend assessments measure student performance in mathematics and reading, and allow the performance of students from recent time periods to be compared with students since the early 1970s.

For example, the 1997 and 2008 NAEP arts assessments were part of the Main NAEP Assessments. The NAEP 1997 Arts Assessment was conducted nationally at grade 8. For music and visual arts, representative samples of public and nonpublic school students were assessed. A special "targeted" sample of students took the theatre assessment. Schools offering at least 44 classroom hours of a theatre course per semester, and offering courses including more than the history or literature of theatre, were identified. Students attending those schools who had accumulated 30 hours of theatre classes by the end of the 1996-97 school year were selected to take the theatre assessment. The NAEP 2008 Arts Assessment was administered to a nationally representative sample of 7,900 eighth-grade public and private school students. Approximately one-half of these students were assessed in music, and the other half were assessed in visual arts. The music portion of the assessment measured students' ability to respond to music in various ways. Students were asked to analyze and describe aspects of music they heard, critique instrumental and vocal performances, and demonstrate their knowledge of standard musical notation and music's role in society. The visual arts portion of the assessment included questions that measured students' ability to respond to art as well as questions that measured their ability to create art. Responding questions asked students to analyze and describe works of art and design. For example, students were asked to describe specific differences in how certain parts of an artist's self-portrait were drawn. Creating questions required students to create works of art and design of their own. For example, students were asked to create a self-portrait that was scored for identifying detail, compositional elements, and use of materials.

Most recently, in 2016, a total of 8,800 eighth-graders in the nation's public and private schools responded to and critiqued existing works of music and visual art and created their own original artwork. NCES collected and analyzed the data and released the 2016 report highlighting key findings. Average music and visual arts responding scores are reported separately on a scale of 0 to 300 points. Average creating scores for visual arts are reported on a scale of 0 to 100 percent. Results are also reported by student groups, school type, and region, as well as in comparison to the 2008 assessment.

In addition, NAEP has a number of special studies that are conducted periodically. These include research and development efforts such as the High School Transcript Study and the National Indian Education Study. More information on these special studies is available on the NAEP Web site.