Technology Integration Research Review: Annotated Bibliography

Adams, W.K. (2010). Student Engagement and Learning with PhET Interactive Simulations (PDF). Presented at the International Conference on Multimedia in Physics Teaching and Learning. This study focuses on understanding how students use Physics Education Technology (PhET) simulations to construct a mental framework about concepts and the effect levels of guidance have on students' use of simulations. Hundreds of individual student interviews were conducted during which the students described what they were thinking as they interacted with simulations. Careful analysis reveals that showing the invisible and use of analogy both facilitate students' construction of their understanding, while the nature of guidance influences the amount of student engagement.

Arena, D.A. (2012). Commercial Video Games as Preparation for Future Learning [Abstract]. Stanford University, Stanford, CA. Doctoral dissertation, presented at the Games for Learning Conference, in the session on "Games in Schools," June 2012. In a randomized-control experiment, community-college students were assigned to one of three conditions: to play the video game Civilization for as much as 15 hours over five weeks, to play Call of Duty for as much as 15 hours over five weeks, or to play no game. All participants took a pretest which indicated no significant differences in their history knowledge. The participants then attended a World War II history slide show/lecture, followed by a posttest with content about World War II history. Students who had played computer games with thematic connections to World War II (e.g., thinking as a nation or being exposed to the historical context of World War II) significantly outscored those who hadn't played a game, indicating that the game-playing students learned and retained more information from the lecture. The research highlights the role that computer games play in creating informal knowledge, which creates better traction for school-based knowledge to take hold in memory. See a slide-show summary of this presentation.

Arici, A. (2008). Meeting Kids at Their Own Game: A Comparison of Learning and Engagement in Traditional and 3D MUVE Educational-Gaming Contexts [Abstract]. (Doctoral dissertation) Indiana University Bloomington, Bloomington, IN. Sixth-grade science classes taught by the same teacher were randomly assigned to either the video game Quest Atlantis (QA) or a traditional version of a two-week water-quality unit. Pretests showed no significant differences by condition. Posttests showed significant learning gains for both conditions, with the students in the QA condition scoring significantly higher than the students in the traditional condition, and retaining significantly more information at the time of a delayed posttest. Students in the QA condition were also more engaged, as demonstrated by qualitative analysis as well as the fact that approximately 75 percent of the students in the QA condition chose to complete optional activities in the game for no credit, whereas only four percent of the students in the traditional condition completed a similar optional assignment for extra credit.

Bai, H., Pan, W., Hirumi, A., and Kebritchi, M. (2012).> Assessing the Effectiveness of a 3D Instructional Game on Improving Mathematics Achievement and Motivation of Middle School Students [Abstract]. British Journal of Educational Technology 43(6), 993-1003. A total of 437 eighth graders were randomly assigned by classroom to the treatment group, which utilized the computer game DimensionM as a supplement to regular classroom instruction, or to the control group, which received regular class instruction without any computer activities. Results of the analysis on the pretest-posttest data revealed that the DimensionM game increased mathematical knowledge acquisition in algebra and maintained student motivation to learn, and suggest that the implementation of DimensionM can greatly benefit middle school students learning algebra.

Bakia, M., Shear, L., Toyama, Y., and Lassseter, A. (2012). Understanding the Implications of Online Learning for Educational Productivity (PDF). Washington, DC: U.S. Department of Education, Office of Educational Technology. Evidence from a 2009 meta-analysis by the Department of Education shows that hybrid models, which combine online curriculum with face-to-face teacher time, produce better outcomes than either face-to-face time alone or online learning alone. The authors conclude that the meta-analysis supports "redesigning instruction to incorporate additional learning opportunities online," but is based on a dearth of empirical research on K-12 learning environments. The hybrid model receives the strongest evidence, but much research remains to be done to define the conditions in which digital tools enhance learning -- for whom, in what contexts, and to what ends.

Barab, S.A., Scott, B., Siyahhan, S., Goldstone, R., Ingram-Goble, A., Zuiker, S., and Warren, S. (2009). Transformational Play as a Curricular Scaffold: Using Videogames to Support Science Education (PDF). Journal of Science Education and Technology, 18, 305-320. Students in the game-based version of the course performed significantly better on related standardized tests than their peers in textbook and descriptive-framing versions of the class.

Bebell, D., and O'Dwyer, L.M. (2010). Educational Outcomes and Research from 1:1 Computing Settings [Abstract]. The Journal of Technology, Learning, and Assessment, 9(1). This review synthesizes results from four multischool empirical studies of K-12 1:1 computer programs, which varied in their settings and unique attributes. Teachers played a critical role in the effective implementation of 1:1 programs. Schools with higher 1:1 implementation tended to have more committed leaders, greater teacher buy-in, preliminary professional development, and a commitment to the transformation of the student. Participation in the 1:1 programs was associated with increased student and teacher technology use, increased student engagement and interest level, and modest increases in student achievement.

Black, P., and Wiliam, D. (1998). Assessment and Classroom Learning [Abstract]. Assessment in Education: Principles, Policy & Practice, 5(1). A review of the literature on classroom formative assessment indicates that frequent feedback yields substantial gains in student learning.

Bransford, J.D., Brown, A.L., and Cocking, R.R. (2000). How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academy Press. Technology can be used to advance learning by bringing exciting curricula based on real-world problems into the classroom, providing scaffolds and tools to enhance learning, such as modeling programs and visualization tools, giving students and teachers more opportunities for feedback, reflection, and revision, and building local and global communities that include teachers, administrators, students, parents, practicing scientists, and other interested people expanding opportunities for teacher learning.

Caldwell, J.E. (2007). Clickers in the Large Classroom: Current Research and Best-Practice Tips (PDF). CBE -- Life Sciences Education, 6(1), 9-20. Audience response systems (ARS), or clickers as they are commonly called, have been used in a variety of fields and at all levels of education, offering a tool for engaging students in a large classroom. When used in classes, clickers typically have either a benign or positive effect on student performance on exams, depending on the method and extent of their use, and they create a more positive and active atmosphere in a large classroom. These systems are especially valuable as a means of introducing and monitoring peer learning methods in a large lecture classroom. So that the reader may use clickers effectively in his or her own classroom, the report provides a set of guidelines for writing good questions and a list of best-practice tips culled from the literature.

Center for Children and Technology (2004). Television Goes to School: The Impact of Video on Student Learning in Formal Education (PDF). New York, NY: Education Department, Corporation for Public Broadcasting. A large body of research indicates that the type of television content children watch is a truer determinant of students' future academic success than the amount of time they spend watching. The report argues, "Children's viewing of educational television has been shown to support significant and lasting learning gains, while too much viewing of other types of programming may be associated with a lack of academic achievement." The report describes examples drawn from the research literature of educational video use across academic disciplines, including: science shows (e.g., Mr. Wizard, Bill Nye the Science Guy); news and drama shows with scientific themes; fictional and factual historical representations (e.g., BBC-produced Teacher' Notes, PBS-produced Renaissance); documentaries for geography; films, news stories, soap operas for writing and foreign language learning activities; and math learning shows (e.g., Sesame Street, Square One TV, Cyberchase). To enhance the educational effectiveness of video in the schools, educators are urged to promote active viewing and critical analysis of media texts, assign video as homework, use segments of no more than ten to 15 minutes, and use television programming as a bridge to public television, museums, community groups, cultural organizations, and professional and industry associations. Broadcasters are urged to use accompanying websites for teacher materials and to consider aligning with core curricula. The report concludes that a long history of research clearly shows that "video is now and will continue to be an effective, engaging, and essential tool in our nation's classrooms."

Cheung, A., and Slavin, R.E. (2011). The Effectiveness of Educational Technology Applications for Enhancing Mathematics Achievement in K-12 Classrooms: A Meta-Analysis. Baltimore, MD: Johns Hopkins University, Center for Research and Reform in Education. A meta-analysis (including 74 rigorous studies and over 56,000 K-12 students) found that mathematics computer applications produce a small but positive effect on mathematics achievement, and, specifically, programs that supplement traditional math instruction with additional instruction at students' individualized assessed levels of need showed greater effects on math achievement. These supplemental math programs (e.g., Jostens, PLATO, Larson Pre-Algebra, and SRA Drill and Practice) were equally effective across elementary and secondary levels, and using them for more than 30 minutes per week increased the beneficial impacts on math achievement.

Cheung, A., and Slavin, R.E. (2012). The Effectiveness of Educational Technology Applications for Enhancing Reading Achievement in K-12 Classrooms: A Meta-Analysis. Baltimore, MD: Johns Hopkins University, Center for Research and Reform in Education. This meta-analysis includes 84 rigorous studies (with over 60,000 K-12 participants) that examine the impacts of technologies designed to improve K-12 reading achievement. The largest effects on reading achievement scores were found for comprehensive models, including READ 180, Voyager Passport, and Writing to Read. Such models integrate computer and noncomputer instruction in the classroom with extensive professional-development support. For example, in READ 180 classrooms, "Each period begins with a 20-minute shared reading and skills lesson, and then students in groups of five rotate among three activities: computer-assisted instruction in reading, modeled or independent reading, and small-group instruction with the teacher" (p. 19). Accelerated Reader is a unique reading achievement program which students do not use but which teachers use to assess student progress and assign curriculum that is appropriate to the students' level. Accelerated Reader showed moderate effects on reading achievement. Supplemental computer-assisted instruction programs augment regular classroom instruction with additional instruction at students' individualized assessed levels of need. Consistent with previous research, supplemental programs (e.g., Destination Reading, Plato Focus, Waterford Early Reading Program, Headsprout, Academy of Reading, and LeapTrack) did not show meaningful effects on reading achievement for K-12 students. .

Curley, J., and Taylor, A., (2010). Exploring Government Through Interactive Games: Evaluation of iCivics Games: Executive Summary. Presentation at the We the People Coordinator Conference, June 2010.

Darling-Hammond, L., Zielezinski, M. B., & Goldman, S. (2014). Using Technology to Support At-Risk Students’ Learning. Stanford Center for Opportunity Policy in Education. A review of more than 70 studies that examined approaches to using technology for high school students that are at risk for failing courses or dropping out. The report finds that a successful educational technology program typically supports active learning, provides opportunities to explore and create content, and blends technology with face-to-face instruction.

Ertmer, P.A., and Ottenbreit-Leftwich, A.T. (2010). Teacher technology change: How Knowledge, Confidence, Beliefs, and Culture Intersect (PDF). Journal of Research on Technology in Education, 42(3), 255-284. The authors assert that teachers cannot be effective without integrating technology into their practice. To integrate technology effectively, teachers must apply knowledge of both their learners and the subject to select appropriate information and communication technology resources that enable their students to meet the required learning goals. Knowledge is not enough to change normal teaching routines; teachers must also have confidence in their own ability to integrate technology successfully and confidence that doing so will benefit student learning. Culture is the proposed vehicle for developing teachers' confidence and behavior change. The authors provide a list of research-based recommendations for facilitating teacher change through preservice and professional-development contexts (p. 266). These recommendations resonate around participating in professional learning communities and job-embedded training, which starts with small-scale forms of technology integration that address immediate needs and yield positive experiences. The authors define "good teaching" as "teaching that facilitates student learning by leveraging relevant ICT resources," and they argue that this new definition should be nurtured and embraced in cultures where teachers learn and work.

Finkelstein, N.D., Adams, W.K., Keller, C.J., Kohl, P.B., Perkins, K.K., Podolefsky, N.S., and Reid, S. (2005). When Learning About the Real World is Better Done Virtually: A Study of Substituting Computer Simulations for Laboratory Equipment. Physical Review Special Topics -- Physics Education Research 1(1), 1-8. In this study, the direct-current-circuit laboratory was modified to compare the effects of using computer simulations with those of using real lightbulbs, meters, and wires. Two groups of students, those who used real equipment and those who used a computer simulation that explicitly modeled electron flow, were compared in terms of their mastery of physics concepts and skills with real equipment. Students who used the simulated equipment outperformed their counterparts both on a conceptual survey of the domain and in the coordinated tasks of assembling a real circuit and describing how it worked.

Fisch, S.M., Lesh, R. Motoki, E., Crespo, S., and Melfi, V. (2010). Children's Learning from Multiple Media in Informal Mathematics Education (PDF). Educational Broadcasting Corporation. This research uses Cyberchase (a multiple-media, informal mathematics project for eight- to 11-year-olds) to investigate synergy among multiple media components and how they interact to yield cumulative educational outcomes. The study incorporated both naturalistic and experimental methods. In the naturalistic setting, children who chose to use one form of Cyberchase media (e.g., television) were significantly more likely to use another (e.g., the Cyberchase website). During the experimental phase, children exposed to Cyberchase media improved significantly more in mathematical problem solving than children who were not. Finally, more consistent gains were observed among children who used multiple media. Children showed evidence of transfer of learning, not only from the treatment to the posttest, but also from one medium to another.

Fleischer, H. (2012). What Is Our Current Understanding of One-to-One Computer Projects: A Systematic Narrative Research Review [Abstract]. Educational Research Review, 7, 107-122. The article reviews cross-disciplinary empirical studies on 1:1 computer projects in school settings published in peer-reviewed journals between 2005 and 2010 and is the most recent review of 1:1 laptop research from 2006 to 2010. Overall, the prominent uses of laptops were for searching and exploration (i.e., using physics simulations), student expression (i.e., student presentations and written reports using PowerPoint and Word), communication (i.e., via email, discussion boards, and IM, between students and between students and teachers), and organization (e.g., OneNote helped students organize thoughts, plan, and complete homework). In general, studies found that laptops tended to improve student motivation and engagement in learning and that this was due to the students' greater sense of control over their learning experience. Students also showed use of laptops for learning outside of school. Studies suggest that 1:1 programs slightly improved students' writing and seemed to improve students' skills in using digital tools; there was some evidence that 1:1 programs very slightly improved students' scores on high-stakes tests. The introduction of 1:1 laptops changed the classrooms from traditional and lecture-oriented to more student-centered and constructivist learning environments. Teachers had mixed reactions to the change; some welcomed it, and others felt concerned about how it would affect their practice. Schools that provided teachers with the most support in integrating technology were the most successful. The author points out that as schools award contracts to computer vendors such as Apple, Dell, and HP, market interests influence the body of research on 1:1 laptop programs, and despite a great deal of research over the five years studied, there remains a lack of understanding of the value added by the 1:1 programs for students' knowledge formation and the teaching practices that support such knowledge formation.

Foster, A. (2011). The Process of Learning in a Simulation Strategy Game: Disciplinary Knowledge Construction [Abstract]. Journal of Educational Computing Research, 45(1) 1-27. This research provided one view of learning through the situative perceptive. The learning was conceptualized as construction of disciplinary knowledge while valuing the content and game and showing the process of learning through navigational strategies and player types from a representative simulation strategy game. Finally, this research showed that despite doubts regarding learning "school" content through games, participants developed their knowledge in core economics concepts as well as skills with technology. The combination of quantitative and qualitative analysis of data allowed researchers to develop a richer conception of how this learning occurs and the kinds of player characteristics that support the process of learning despite a small sample size.

Gerard, L.F., Varma, K., Corliss, S.B., and Linn, M.C. (2011). Professional Development for Technology-Enhanced Inquiry Science [Abstract]. Review of Educational Research, 81(3), 408-448. A literature search using the keywords technology, professional development, and science identified 360 studies from the past 25 years, 43 of which included multiple data sources and reported results for teachers and/or students. Findings suggest that professional-development programs that engaged teachers in a comprehensive, constructivist-oriented learning process and were sustained beyond one year significantly improved students' inquiry-learning experiences in K-12 science classrooms. In professional-development programs of one year or less, researchers encountered common technical and instructional obstacles related to classroom implementation that hindered success. Programs varied most considerably in terms of their support for teachers' use of evidence to distinguish effective technology-enhanced practices.

Hargittati, E. (2010). Digital Na(t)ives? Variation in Internet Skills and Uses Among Members of the "Net Generation" (PDF). Sociological Inquiry, 80(1), 92-113. This study finds that even when controlling for Internet access and experiences, people differ in their online abilities and activities. Higher levels of parental education, being a male, and being white or Asian American are associated with higher levels of Web-use skill. This study surveyed a first-year class at a public research university. A bivariate analysis to illustrate the relationship between variables was conducted, along with an ordinary least squares regression to look for predictors of skill level and diversity of Web usage while controlling for various social and use context factors.

Helsper, E., and Enyon, R. (2010). Digital Natives: Where is the Evidence? [Abstract]. British Educational Research Journal, 36(3), 503-520. This paper shows that breadth of use, experience, gender, and education are just as, if not more, important than generation in explaining how people become digital natives. Data used in the study was from a 2007 Oxford Internet Survey (2,350 respondents and a response rate of 77 percent). The multistage probability sample surveys were carried out face to face.

Innovative Teaching and Learning Research (2011). Findings and Implications (PDF). A survey of innovative teaching practices across seven countries found that innovative teaching varies more within schools than it does between schools. Teachers used Information Communication Technology (ICT) most commonly to present information, while students most commonly used ICT to find information, practice routine skills, and take tests. Such practices do not necessarily support higher-order thinking skills, such as critical thinking, complex problem solving, collaboration, creativity, or technological fluency. Examples of activities that integrate ICT to advance 21st-century skills include: analyzing data or information, writing and editing stories or reports, creating multimedia presentations, using and creating simulations or animations, collaborating with peers on learning, and working with others outside of class. Teachers who use innovative teaching methods are becoming more common but were few and far between, and opportunities for students to develop problem-solving and collaboration skills remain quite low across the seven countries studied. In order to increase innovative teaching practices, the report recommends increased collaboration among teachers, a school culture that offers a common vision of innovation and support for new types of teaching, and professional development that provides teachers opportunities to experiment and apply innovative teaching methods.

Kay, R.H., and LeSage, A. (2009). Examining the Benefits and Challenges of Using Audience Response Systems: A Review of the Literature (Abstract). Computers & Education 53, 819-827. The authors conducted a review of research on audience response systems (ARS) and conclude that the evidence supports benefits of ARS, including improvements to the classroom environment (increases in attendance, attention levels, participation, and engagement), learning (interaction, discussion, contingent teaching, quality of learning, and learning performance), and assessment (feedback, formative, and normative). Challenges for teachers using ARS include time needed to learn and set up the ARS technology, creating effective ARS questions, adequate coverage of course material, and ability to respond to instantaneous student feedback. Student challenges include adjusting to a new method of learning, increased confusion when multiple perspectives are discussed, and negative reactions to being monitored. However, more research is needed to develop effective practices.

Kebritchi, M., Hirumi, A., and Bai, H. (2010). The Effects of Modern Mathematics Computer Games on Mathematics Achievement and Class Motivation [Abstract]. Computers & Education, 55(2), 427-443. This study examined the effects of a computer game, DimensionM, on public high school students' mathematics achievement and motivation. A total of 193 students and 10 teachers participated in this study. The teachers were randomly assigned to experimental and control groups. A mixed method of quantitative analysis and interviews were used with multivariate analysis of co-variance to analyze the data. The results indicated significant improvement in the achievement of the experimental group versus the control group. No significant improvement was found in the motivation of the groups. Students who played the games in their classrooms and school labs reported greater motivation compared to the ones who played the games only in the school labs. Prior knowledge, computer skill, and English-language skill did not play significant roles in achievement and motivation of the experimental group.

Klisch, Y., Miller, L.M., Wang, S., and Epstein, J. (2012). The Impact of a Science Education Game on Students' Learning and Perception of Inhalants as Body Pollutants (Abstract). Journal of Science Education and Technology, 21(2), 295-303. The online science education game Uncommon Scents was developed to teach middle school students about the biological consequences of exposure to toxic chemicals in an environmental science context, as well as the risks associated with abusing these chemicals as inhalants. Middle school students (n = 444) in grades six through eight participated in the study consisting of a pretest, three game-play sessions, and a delayed posttest. After playing the game, students demonstrated significant gains in science content knowledge, with game usability ratings emerging as the strongest predictor of posttest content knowledge scores. The intervention also resulted in a shift to more negative attitudes toward inhalants, with the most negative shift occurring among eighth graders and posttest knowledge gains as the strongest predictor of attitude change across all grade levels.

Klisch, Y., Miller, L.M., Beier, M.E., and Wang, S. (2012). Teaching the biological consequences of alcohol abuse through an online game: Impacts among secondary students. Life Sciences Education, 11(1), 94-102. A multimedia game was designed to serve as an intervention that aligned with National Science Content Standards, attempting to convey knowledge about alcohol consumption and positively impact adolescents' attitudes toward science. In a pretest/delayed posttest design, middle school and high school students, both male and female, demonstrated significant gains on measures of content knowledge and attitudes toward science. The best predictors of these outcomes were the players' ratings of the game's usability and satisfaction with the game. The outcomes suggest that game interventions can successfully teach standards-based science content, target age-appropriate health messages, and impact students' attitudes toward science.

Law, N., Pelgrum, W.J., Plomp, T. Eds., (2006). Pedagogy and ICT Use in Schools Around the World: Findings from the IEA SITES 2006 Study Series. University of Hong Kong. Results of a cross-national study found that teachers' self-perceived competence in using technology and the amount of their training in uses of technology for instruction were associated with greater use of technology for instruction.

Lemke, C., Coughlin, E., and Reifsneider, D. (2009). Technology in Schools: What the Research Says (PDF). Culver City, CA: Commissioned by Cisco. This report reviews evidence on K-12 classroom technology use and is organized according to media platforms: interactive whiteboards, classroom response systems (clickers), video games, simulations, modeling, augmented reality, virtual worlds, mobile devices, data analysis tools, calculators, 1:1 ratio of computers to students, computer-assisted instruction (where a computer presents instruction or remediation), virtual learning, and educational television. Educational television and video games (e.g., DimensionM, MathBlaster, Quest Atlantis) had consistent and rigorous evidence supporting small, significant increases in learning (when implemented with fidelity and appropriate shifts in teaching). However, the authors acknowledge that much remains to be understood in terms of identifying the gaming attributes that contribute to lasting gains in student learning. The authors found incomplete but promising evidence in support of calculators, virtual learning (i.e., geographically separated teacher and learner), modeling (e.g., STELLA, Model-IT, My World, StarLogo, Worldmaker), simulations (e.g., Civilization, Supercharged), and data analysis/visualization tools (i.e., tools for conducting original research). Mobile devices (to facilitate collaboration) and augmented reality (to improve understanding and thinking skills, e.g., Mad City Mystery) showed promising results. Calculators promote mathematics learning when they are used during instruction and assessment and integrated for over nine weeks; however, calculators can negatively impact learning if students do not know many multiplication facts. Clickers were found to promote student engagement; however, the authors caution that the teaching and learning strategies used with clickers are more effective than the clickers themselves in promoting student achievement. These strategies include: checking for real-time student understanding of content being taught, diagnosing student misconceptions and misunderstandings, displaying responses of the group to trigger discussion and reflection, gathering formative data to guide instruction, and saving time in administering and scoring quizzes.

Levin, T., and Wadmany, R. K. (2008). Teachers' Views on Factors Affecting Effective Integration of Information Technology in the Classroom: Developmental Scenery. Journal of Technology and Teacher Education, 16(2), 233-263. This exploratory, longitudinal study examined six teachers' views on the factors that affect technology use in classrooms, studying teachers of grades 4-6 for three years, as a group and as individual case studies. Three case studies were selected for analysis to explore how changes in the teachers' educational views and practices resulted from their exposure to teaching and learning with technology. The case studies reveal that teachers must be involved in at least two radical changes: They must learn to use technology, and they must fundamentally change how they teach. The study encourages educators and teacher educators to be sensitive to the influences that teachers' educational views (on learning, teaching, and technology) have on their practices and on their capabilities and need to interact with others (authority figures and colleagues). The authors recommend that professional development for technology integration should consider providing both individual and group learning opportunities, which show respect and appreciation for teachers' unique interpretations of using technology, and that teachers also reflect on their own beliefs, knowledge, and experiences.

Li, Q., and Ma, X. (2010). A Meta-Analysis of the Effects of Computer Technology on School Students' Mathematics Learning (PDF). Educational Psychology Review, 22(3), 215-243. This meta-analysis examines the impacts of technology on K-12 math learning and finds it can have generally positive effects for technology integration. In particular, use with special needs students as opposed to general education students had a stronger effect, as did use with elementary students compared to secondary students. The greatest effects occurred when teachers used constructivist as opposed to traditional teaching methods. The paper also describes several studies that examined virtual manipulative tools in mathematics classrooms. These studies generally found positive impacts of such tools on student achievement in and attitude toward mathematics. When virtual manipulatives were used in combination with physical manipulatives, researchers also found positive results. The paper also reviews research on The Adventures of Jasper Woodbury, reporting that the program had a positive impact on students' attitudes toward math, problem-solving skills, and math learning.

Light, D., and Polin, D.K., Center for Children and Technology. (2010). Integrating Web 2.0 Tools into the Classroom: Changing the Culture of Learning (PDF). New York, NY: Education Department, Corporation for Public Broadcasting. Twenty-two schools were observed and over 30 educators were interviewed and observed, to document Web 2.0 and social networking technology use in classrooms across the United States. As the paper is descriptive, only hypotheses are offered. The paper provides a trove of Web resources, while describing how teachers use them in K-12 classrooms. In general, interactive and asynchronous features of Web 2.0 tools seem to extend and deepen the educational environment when they facilitate meaningful communication among teachers, students, parents, and larger communities toward authentic goals.

Linebarger, D. L. (2009). Evaluation of the Between the Lions Mississippi Literacy Initiative 2007-2008 (PDF). Children's Media Lab, Annenberg School for Communication, University of Pennsylvania. A treatment, maintenance, and control group existed. The treatment group was teachers beginning use of Between the Lions material, while the maintenance group was teachers who had previously used the material. Both the treatment and maintenance groups received mentoring, with the treatment group receiving 96 hours and the maintenance receiving 24 hours. No child had been introduced to the material in the classroom prior to the experiment. Teachers using the material and receiving mentoring support positively changed the literacy environment, the general classroom environment, the literacy activities, and the language, literacy, and curriculum. Children between 46 and 59 months in the treatment group scored higher on most indicators than children between 46 and 59 months in the maintenance and control groups. Older children in the maintenance group scored higher than older children in the treatment and control group. The study also concluded that the favorable changes in daily environment and teacher behavior were linked to positive changes and accelerated growth of at-risk preschoolers' early literacy skills.

Livingstone, S. (2008). Internet Literacy: Young People's Negotiation of New Online Opportunities (PDF). In T. McPherson (Ed.), Digital Youth, Innovation, and the Unexpected (pp. 101-122). Cambridge, MA: The MIT Press. This chapter is about a case study of three children and their Internet usage over time, along with a literature review on both the digital divide and literacy. Internet skills are conceptualized as a form of literacy. The article challenges myths about the "cyberkid" or the "digital generation" in order to point out that Internet usage and literacy vary among children. Society must first recognize this variance in order to support children's Internet literacy through design, education, and regulation.

Marsh, J.A., Pane, J.F., and Hamilton, L.S. (2006). Making Sense of Data-Driven Decision Making in Education Evidence from Recent RAND Research (PDF). Four studies provide evidence that illuminates data-driven decision making (DDDM) practices in a variety of contexts across the country. They included three statewide samples in one case, large districts in a second, small districts in a third, and a large educational management organization in the fourth. Finally, like most of the literature to date on DDDM, these studies are primarily descriptive and do not address the effects of DDDM on student outcomes. Together they create a foundation for ongoing and future research on the topic by helping to understand how data are being used, the conditions affecting use, and issues that arise in the process -- information that will be crucial for effective implementation of DDDM and evaluations of its effects.

Martin, T., and Schwartz, D.L. (2005). Physically Distributed Learning: Adapting and Re-interpreting Physical Environments in the Development of Fraction Concepts (PDF). Cognitive Science, K. 587-625. Five studies examine how interacting with the physical environment can support the development of fraction concepts.

McKagan, S.B., Perkins, K., Dubson, M., Malley, C., Reid, S., LeMaster, R., and Weiman, C.E. (2008). Developing and Researching PhET Simulations for Teaching Quantum Mechanics (PDF). American Journal of Physics. 76(1) 406-417. The research looked at 18 simulations on quantum mechanics designed to improve learning from the Physics Education Technology (PhET) Project. Several key features help students build mental models: visual representations that cannot be directly observed, interactive environments, connections to everyday life, and efficient calculations so students focus on concepts. This paper provides an overview of the PhET simulations, research on their effectiveness, and insights about student thinking.

McLeod, S. (2005). Data-Driven Teachers (PDF). School Technology Leadership Initiative, University of Minnesota.

Means, B., Padilla, C., and Gallagher, L. (2009). Use of Education Data at the Local Level: From Accountability to Instructional Improvement (PDF). Washington, DC: U.S. Department of Education, Office of Planning, Evaluation, and Policy Development. A total of 36 schools provided the case-study data for this report from 2005-2008. The report describes features of data systems and data usage that are essential to yielding value for students and educators. These features are described as follows: data, regular meetings, training, support, and leadership and alignment. Data: Timely, credible, interim assessments that generate actionable data were among the most powerful strategies districts used and key to engaging teachers in using data. State test results are often not useful to teachers because they arrive too late to inform instructional decisions, and in many cases, teachers cannot access student performance by content standard. States and test vendors should improve their efforts to ensure that schools have access to actionable student assessment results in time to inform their planning activities. District data systems should also support routine evaluation of instructional programs, practices, and decisions by linking student participation in specific endeavors with standards-based assessments. "Just 42 percent of districts have systems that can generate reports showing student performance linked to participation in specific instructional programs." Regular meetings: Schools and districts that are leaders in data-driven decision making set time aside within the regular work week for the analysis of student data and use student data to develop instructional decisions. Training: The report emphasizes "the use of data in decision making cannot have a positive impact on instruction without a linkage to effective instructional practices." Yet, teachers often lack ideas about how to teach differently based on student assessment results. Many teachers in the case studies responded to student data by grouping students according to their performance level and readdressing content that a majority of students understood poorly. Case studies also showed that data discussions have become an accepted method in teacher professional development, planning, and collaboration time. Support: School-based "data coaches" help teachers to interpret data and link data to instructional decisions. In a number of the case study schools, data coaches brought valuable insight to teachers, especially in the area of early literacy. Fifty percent of districts say they have provided data coaches for at least some of their schools, and 32 percent say that they have done so for all of their schools. Teachers also benefit from opportunities to examine student data with their colleagues, but they "only want to do so if they feel confident that they will not be opening themselves up to harsh judgments." Small groups of teachers who typically work together as part of a grade-level, department, or project team appeared to work best for data-analysis meetings. "It is also important to separate data reflection activities from performance management activities (which could affect salary or job status)." Leadership and alignment: According to the report, a criterion for hiring school leaders should be their ability to lead schools in continuous improvement processes informed by data. School leaders play an important role in modeling the use of data and in developing school practices where teachers are expected to use data to guide their instruction. School leaders should participate in ongoing training in using data to inform school improvement and instructional decision making and in motivating their staff to engage in these practices. District leadership is also critical to ensuring that districts have interim assessment tools and practices that foster data use. District policies that require interim assessments should not have contradictory pacing requirements that prohibit teachers from going back to reteach content that the students have not yet mastered. District policies should reflect an achievable plan for teachers to cover district curriculum. Surveys from district staff indicate a need for exemplary models of how to analyze student data to determine which practices work best for which students, adapt instructional practices to meet students' individual needs, and develop curriculum-embedded formative assessments. Finally, "many states have assembled collections of digitized resources for planning and implementing instruction around their state standards." Ideally, local interim assessment data would be linked with instructional resources geared to state standards and state data systems so that "teachers using a district data system to examine students' performance on a specific standard would be just a click or two away from instructional resources for that standard."

Means, B., Toyama, Y., Murphy, R., & Baki, M. (2013). The effectiveness of online and blended learning: A meta-analysis of the empirical literature. Teachers College Record, 115(3), 1-47. This 2013 meta-analysis examines 45 studies of online, blended learning, and face-to-face programs and found that purely online learning is equivalent to face-to-face instruction in effectiveness, while blended approaches are more effective than both online and face-to-face instruction.

Means, B., Toyama, Y., Murphy, R., Bakia, M., and Jones, K. (2009). Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies (PDF). Washington, DC: U.S. Department of Education, Office of Planning, Evaluation and Policy Development. A systematic search of the research literature from 1996 through July 2008 identified more than a thousand empirical studies of online learning. Analysts screened these studies to find those that contrasted an online to a face-to-face condition, measured student learning outcomes, used a rigorous research design, and provided adequate information to calculate an effect size. As a result of this screening, 51 independent effects were identified that could be subjected to meta-analysis. The meta-analysis found that, on average, students in online learning conditions performed better than those receiving face-to-face instruction. The findings suggest that the positive effects associated with blended learning should not be attributed to the media, per se. An unexpected finding was the small number of rigorous published studies contrasting online and face-to-face learning conditions for K-12 students.

Miller, L. M., Chang, C. , Wang, S., Beier, M.E., and Klisch, Yvonne. (2011). Learning and Motivational Impacts of a Multimedia Science Game [Abstract]. Computers & Education. 57(1) 1425-1433. The power of a Web-based forensic-science game to teach content and motivate careers in science, technology, engineering, and math (STEM) was tested among secondary students. More than 700 secondary school students were exposed to one of the three Web-based forensic cases for approximately 60 minutes. Gain scores from pretest to a delayed posttest indicated significant gains in content knowledge. In addition, the game's usability ratings were a strong predictor of learning. A positive relationship between role-play experience and science career motivation was observed, which suggests a role for authentic virtual experiences in inspiring students to consider STEM careers.

Miller, L.M., Moreno, J., Estrera, V., Lane, D. (2004). Efficacy of MedMyst: An Internet Teaching Tool for Middle School Microbiology (PDF). Microbiology Education, 5, 13-20. The central hypothesis examines whether brief exposure to a Web adventure format containing virtual lab experiments and computer games within an engaging story line can impact student learning. An episodic adventure series, MedMyst focuses on infectious diseases and the microbes that cause them. In the online adventure, the player (student) enters a futuristic world in which he or she becomes a "reconstructor," a member of an elite team charged with preventing the spread of infectious disease. The series consists of three "missions," each lasting approximately 30 to 40 minutes and designed to address a limited set of learning objectives. Middle school students, classroom teachers, scientists, and clinicians assisted the game development process. A field test involving over 700 students from nine schools assessed the knowledge gains attributable to playing MedMyst. Gain scores from pretest to posttest indicated that middle school students retained important information by interacting with the online material for as little as 30 minutes per adventure; however, gains for high school students were less persuasive, perhaps indicating a different learning tool or content is required for this age audience.

Miller, L., Moreno, J., Willcockson, I., Smith, D., Mayes, J. (2006). An Online, Interactive Approach to Teaching Neuroscience to Adolescents [Abstract]. Life Sciences Education, 5(2), 137-143. At the Rice University Center for Technology in Teaching and Learning, a project was undertaken through a Science Education Drug Abuse Partnership Award (R25 DA15063) from the National Institute on Drug Abuse to inform adolescents about the neurobiology of substance abuse and the current research dealing with a class of drugs known as club drugs. Problem-based learning, multimedia pedagogy, and the National Science Content Standards were integrated to produce The Reconstructors, an episodic series. A field test of students from five schools assessed the retention of content after playing The Reconstructors series titled Nothing to Rave About. Gain scores indicated that middle school students' knowledge about club drugs and the basic neuroscience concepts that explain their effects improved significantly.

Miller, L., Schweingruber, H., Oliver, R., Mayes, J., Smith, D. (2002). Teaching Neuroscience Through Web Adventures: Adolescents Reconstruct the History and Science of Opioids [Abstract]. The Neuroscientist 8(1):16-21. This is an experimental project to transmit aspects of problem-based learning and the National Science Content Standards through an episodic Web-based adventure series titled Medicinal Mysteries from History. Through the use of multimedia technologies, middle school students enter a futuristic world in which they become "reconstructors," members of an elite scientific unit charged with recovering lost medical knowledge about analgesic drugs. Two of the four episodes have been evaluated through a comprehensive review process involving middle school students, teachers, neuroscience researchers, and clinicians. Analysis of the pretest and posttest scores demonstrated significant knowledge gain that validly can be attributed to use of the game. These data provide evidence that science content can be transmitted through innovative online techniques without sacrificing compelling content or effective pedagogical strategies.

Moeller, B., and Reitzes, T. (2011). Integrating Technology with Student-Centered Learning. Education Development Center, Inc. (EDC). Quincy, MA: Nelle Mae Education Foundation. The report identifies effective ways that technology may be used to personalize a student's learning experience and examines the integration of computer- and Web-based tools, applications, and games.

Paulsen, C.A., and Bransfield, C.P. (2010). Evaluation of Design Squad, Season 3: Final Report (PDF). Concord, MA: Concord Evaluation Group, LLC. Concord Evaluation Group conducted an experiment (randomized block design) to evaluate the impacts of the Design Squad materials on learning and attitudes of teachers and students. A total of 559 students at eight middle schools participated. The specific materials tested included a teacher's guide, video animations, live-action videos and video profiles, and an interactive online game. Teachers were encouraged, but not required, to use the website for any other resources on an as-needed basis. Teachers reported that they used Design Squad to supplement their units on buoyancy, difference in gases, electricity and circuits, force, graphing, gravity, lift, mass motion, surface area, and volume. Teachers reported that Design Squad enabled them to encourage problem solving and teamwork among students, as well as their desire use inquiry-based learning in the classroom. Students exposed to Design Squad demonstrated greater gains than students who were not exposed to Design Squad and learned more about key science constructs (i.e., electrical circuits, sound, Newton's laws, force, and air pressure). Teachers in the treatment group reported that they observed positive changes in their students' behaviors after using Design Squad.

Perkins, K.K., Adams, W.K., Wieman, C.E., and PhET Team. (2006). PhET's Research-Based Guidelines for Design and Use of Interactive Simulations [Abstract]. Bulletin of the American Astronomical Society, 38, 977. The article presents existing literature and interview results from over 165 students who used PhET simulations. The article presents guidelines for designing effective simulations and activities to engage students in effective learning.

Prince, D. L., Grace, C., Linebarger, D.L., Atkinson, R., and Huffman, J.D. (2002). Between the Lions: Mississippi Literacy Initiative - A Final Report to Mississippi Educational Television (PDF). This study examined whether integrating the viewing of Between the Lions television episodes into classroom curriculum had an effect on early literacy skills. The treatment group watched two episodes weekly, read a book related to the episode, and then participated in a hands-on activity to reinforce the theme or skills in the episode. The results indicate the Between the Lions television series could be a meaningful part of reading interventions and is potentially most beneficial in kindergarten.

Rockman et al. (2010). PBS Kids iPod App Study: Executive Summary (PDF). iPod touch devices with the applications Super Why and Martha Speaks: Dog Party were given to 90 children ages 3-7. Parents completed daily observations as well as prestudy and poststudy surveys, while the children were given prestudy and poststudy tests. The study found average gains of 20 percent on the short and comprehensive vocabulary assessments for Martha Speaks.

Smith, M.K., Wood, W.B., Adams, W.K., Wieman, C., Knight, J.K., Guild, N., and Su, T.T. (2009). Why Peer Discussion Improves Student Performance on In-Class Concept Questions (PDF). Science, 323(5910), 122-124. This study finds that using audience response system (ARS) with peer discussion improves the understanding of science concepts among students in an undergraduate genetics course. Students answered in-class conceptual questions individually using clickers, then discussed with their neighbors, and then revoted on the same question or a conceptually similar question. Whether the question was the same or conceptually similar, correct answers increased following peer discussion, indicating that it was an effective method for engaging and addressing students' underlying misconceptions.

Squire, K., Giovanetto, L., Devane, B., and Durga, S. (2005). From Users to Designers: Building a Self-Organizing Game-Based Learning Environment (PDF). TechTrends, 49(5), 34-42. Using design-based methodology, the researchers studied cognitive consequences of playing the historical simulation game Civilization III. Eleven students, 80 percent of whom were African-American and most of whom were socioeconomically disadvantaged, attended two-hour sessions, twice per week, and were observed for five weeks. The students began playing the historical simulation game Civilization III in pairs. Students talked as they played, asking questions, quizzing each other, offering suggestions, and generally helping one another play. Working in pairs also led to better reflections and less confusion in figuring out the game. The authors argue that the students showed increased knowledge of maps, timelines, and historical terms, as well as flexible, systemic knowledge of a game system.

Tamim, R.M., Bernard, R.M., Borokhovski, E., Abrami, P.C., Schmid, R. F. (2011). What Forty Years of Research Says About the Impact of Technology on Learning: A Second-Order Meta-Analysis and Validation Study [Abstract]. Review of Educational Research, 81(1), 4-28. . A second-order meta-analysis of 25 meta-analyses encompassing over 1,000 studies and 40 years of research on technology and classroom learning found that the use of technology in the classrooms shows a small to moderate positive effect on student learning, as compared to technology-free traditional instruction. Using technology to support knowledge formation had a greater effect than using technology for the presentation of content. Teacher effectiveness and fidelity of technology implementation may have a greater effect on student learning than the nature of the technology intervention.

Thissen-Roe, A., Hunt, E., and Minstrell, J. (2004). The DIAGNOSER Project: Combining Assessment and Learning [Abstract]. Behavioral Research Methods, Instruments, and Computers 36(2), 234-240. The article explains the use of the DIAGNOSER and physics learning. DIAGNOSER is an Internet-based tool for classroom instruction in science and math. The tool allows for individually tailored feedback through computer-administered quizzes and a database of student responses.

U.S. Department of Education, Office of Educational Technology. (2010). Transforming American Education: Learning Powered by Technology: National Education Technology Plan. The plan calls for using technologies already prevalent in our personal and professional lives to transform public education by improving student learning, scaling best practices, and using data for continuous improvement. The plan outlines a vision "to leverage the learning sciences and modern technology to create engaging, relevant, and personalized learning experiences for all learners that mirror students' daily lives and the reality of their futures. In contrast to traditional classroom instruction, this requires that we put students at the center and empower them to take control of their own learning by providing flexibility on several dimensions." The plan also calls for "connected teaching," in which educators connect to "resources and expertise that improve their own instructional practices and guide them in becoming facilitators and collaborators in their students' increasingly self-directed learning."

Warschauer, M., and Matuchniak, T. (2010). New Technology and Digital Worlds: Analyzing Evidence of Equity in Access, Use, and Outcomes (PDF). Review of Research in Education, 34(1), 179-225. This review takes a broad perspective on issues of technology and equity for youth in the United States, illustrating how issues of access, use, and outcomes are intertwined. The authors start with access, considering whether diverse groups of youth have digital media available to them and how technological and social factors support or constrain their access. They then analyze the ways in which diverse youth use new media for education, social interaction, and entertainment. Finally, the authors consider the gains achieved by diverse groups through the usage of new media as measured by academic achievement, acquisition of 21st-century learning skills, and participation in technology-related careers.

Wieman, C., Adams, W., Loeblein, P., Perkins, K.K. (2010). Teaching Physics Using PhET Simulations (PDF). Physics Teacher, 48(4), 225-227. This article highlights ways to use PhET simulations in teaching based on research and experiences in high schools and colleges. PhET simulations are tools that enhance a well-designed curriculum and teacher efforts but cannot replace them. The simulations need to be integrated with the curriculum with appropriate activities.

Wieman, C., Adams, W.K., Perkins, K.K. (2008). PhET: Simulations That Enhance Learning (PDF). Science, 322, 682-683. This article highlights ways to use PhET simulations in teaching based on research and experiences in high schools and colleges. PhET simulations are tools that enhance a well-designed curriculum and teacher efforts but cannot replace them. The simulations need to be integrated with the curriculum with appropriate activities.

Young, M.F., Slota, S., Cutter, A.B., Jalette, G., Mullin, G., Lai, B., Yukhymenko, M. (2012). Our Princess Is in Another Castle: A Review of Trends in Serious Gaming for Education [Abstract]. Review of Educational Research, 82(1), 61-89. The authors define digital learning games as those that "target the acquisition of knowledge as its own end, and foster habits of mind and understanding that are generally useful within an academic context." After an exhaustive search, they find promising but inconclusive evidence that games promote learning in some K-12 education contexts. The strongest evidence for promoting learning was found in language learning in video games, physical education and tactile video games (e.g., Wii), and history (e.g., role-playing video games). The review found little support for their value in science and mathematics. This is partially attributed to the lack of empirical research and also to the ways that the formal school environment differs from that of naturalistic game playing.

Zucker, A.A., and Light, D. (2009). Laptop Programs for Students (PDF). Science, 323, 82-85. This article reviews the research on 1:1 computing programs in schools, where 1:1 is defined as students having ownership of the computer. Providing computers to schools increases the technology skills of teachers and students in both the developed and the developing world. Laptop programs increase students' engagement with academic work and school, improve technology skills, and have positive effects on students' writing. Research in many nations suggests that laptop programs will be most successful as part of comprehensive initiatives that also address changes in education goals, curricula, teacher training, and assessment.

Go to the first section of the Tech Integration Research Review, Introduction and Learning Outcomes.