Pilot Implementations In Public School Classrooms
In our pilot implementations of River City, using three public school classrooms in Boston, MA and one in Arlington, VA, we have examined usability, student motivation, and classroom implementation issues.
Our initial pilot sites were Gunston Middle School in Arlington, Virginia, Timilty Middle School and Quincy Upper Middle School, both in Boston, Massachusetts. All three school had high populations of free and reduced lunch pupils and English-as-a-Second-Language learners.
We selected these pilot sites for several reasons: the diverse populations, both socioeconomically and ethnically; the convenient location near Harvard University, the Smithsonian and George Mason University, where most of the research and development teams were housed; and the good technology infrastructure to support the connectivity and bandwidth requirements of the project. Also, Gunston is a language immersion school, so representative populations of students who take science in either English or Spanish were available.
From the very beginning of the project, we included the sixth and seventh grade science teachers on our development team to ensure that learning experiences within the MUVE both met the needs of diverse students and were practical to implement given typical classroom and curriculum constraints. The teachers were happy to join us in the development process because they are seeking ways to motivate and aid underperforming students and also want to enhance bilingual learning through science materials.
During the Spring of 2002, sixty-three sixth and seventh-grade students participated in the River City Unit, with an additional thirty-six control students. Due to scheduling constraints and a computer lab availability each site had unique timing of the intervention.
Both qualitative and quantitative data were collected from students and teachers over the two to three-week implementation period. Both the Patterns for Adaptive Learning Survey (Midgley, 2000) [with subsections on science interest, thoughtfulness of inquiry, motivation, collaboration, academic efficacy, and technology interest] and a content test, (modified from Tobin, 1999) were administered to students, pre- and post-intervention. In addition, demographic data and teachers' expectations of students' successes were collected. Observational data was collected from the experimental intervention classrooms throughout the project and sporadically from the control classrooms. All teachers responded to a pre and post questionnaire regarding their methods and comfort with technology. The experimental intervention classroom teachers also wrote a narrative about their perceptions of the MUVE at the end of the project.
Usability and Motivation
All students mastered the interface readily upon their first exposure. Apart from a general familiarity with the concept of navigating through different levels of content to get information, which they get from using Web browsers, middle school students regularly use Instant Messaging and other chat-type applications. Almost all have experience playing computer and video games that involve navigating through a three-dimensional space. No problems particular to our MUVE interface were noted.
Students were highly engaged by the environment, its content, and the opportunity to collaborate. Preliminary results indicate the MUVE is motivating for all students, including lower ability students typically uninterested in classroom activities. Six out of seven experimental students scoring less than 35% on the content pre-test improved their content knowledge above that level, while only two of five control students did so. Some of this motivation undoubtedly stems from the novelty of using a game-like computer environment during class time and from the adoption of an avatar character. However, strong engagement persisted through detailed work in the Lab Notebook after this novelty effect had worn off.
Learning Outcomes
We found that students discovered multiple intriguing situations in the MUVE to investigate. In our seventh grade classroom, five different hypotheses about these situations emerged, with posited causes ranging from population density to immigration to water pollution. Another finding is that the MUVE seemed to have the most positive effects for students with high perceptions of their own thoughtfulness of inquiry. These students, on average, scored higher on the post content test, controlling for SES, science GPA, ethnicity, and content pre-test score.
Another outcome involved students' perceptions of their teacher's role in the classroom. By the end of the study, experimental group students perceived their teachers as pressing them less for understanding than at the beginning. When coupled with their increased academic efficacy and their teacher's facilitative role, this shift may indicate students assumed more responsibility for learning, a desirable result.
One concern regarding MUVE-based curricula is the reliance in this medium on reading and writing, which is problematic for some students. Our design was directed to minimize problems of this type, and our data show that there were no differences between ESL and non-ESL students in performance, indicating that this issue is moot when the medium is constructed appropriately. Our outcomes also suggest types of teacher professional development to aid in making MUVEs more effective.
Conclusion
To date, no one has studied whether MUVEs are a type of educational application suited to use in schools, linked to core curriculum. Our research to date demonstrates that MUVEs seem quite feasible as an addition to more conventional kinds of computer-based instruction or integrated into the curriculum. High levels of student motivation, improvements in academic efficacy, coupled with good usability and a lack of major implementation problems are a promising beginning for this emerging interactive medium.
Overall, these findings encourage further refinement and experimentation with curricular MUVEs as a learning modality that can help teachers reach students struggling with motivation, self-worth, and lack of content knowledge. These data are promising, but not conclusive about this curriculum's educational value or the effectiveness of MUVEs for learning. By examining student interactions with the pilot curriculum, we saw ways to strengthen our content and pedagogy. We also saw teachers struggle with facilitating the whole-class interpretive sessions that alternated with MUVE experiences, an indication that we need to extend our professional development experiences.