The ICT Impact Report

EUN has published a major report on ICT impact on schools in Europe. The report, written in the framework of the ICT Cluster of the European Commission, draws on the evidence from 17 recent impact studies across Europe. It identifies 12 major findings concerning the impact of ICT on teaching and learning, identifies three types of barriers and makes 13 recommendations to policy-makers, educators and researchers.


The report is written by Anja Balanskat, Roger Blamire, Stella Kefalla

Full Report

The “impact studies’ in focus include a variety of studies, large and small scale, national and European, and cover evaluations of ICT programmes, inspection reports, specific technology interventions, research reviews, European comparisons and case studies. The major focus of the report was to look at impact of ICT in schools on two main areas learning and learners and teachers and teaching. Further to this, it shows the barriers for effective ICT use as identified in the studies. The report likewise tackles the research methods used to reveal impact in the two areas of learning and teaching.

Based on the body of evidence the report then raises a couple of key questions to be addressed in the near future, such as: Is it sound policy to concentrate resources on ICT for those subjects and sectors where results are proven? or How can we tackle the growing divide between e-mature and e-immature schools?
In the final section recommendations are given to policy makers, schools and research and development. These recommendations emerged from extensive discussions with the ICT cluster members and other ICT in education experts and point to possible solutions and actions to be taken on Member State and European level. In an overview table, the report also lists the core review studies according to their scope, research methods and the specific areas of impact in the two areas of learning and teaching, such as students’ motivation and engagement, the personalization of learning or the emergence of new pedagogical practices. This can help policy makers to identify studies most relevant to their interest.

Impact of ICT on learning and learners

ICT and pupil attainment

Six studies show statistical evidence that ICT can enhance attainment in subjects. UK’s largest impact study shows a raise in subject performance through ICT use in English, science and design, and technology. Also specific ICT uses, such as interactive whiteboards in the UK, had a positive effect on pupil’s performance in literacy, mathematics and science tests compared to students in other schools. They especially improved the performance of low achieving pupils in English and impact was greatest on writing. Another large impact study in the UK, which looked at ICT impact from an economic angle, confirms ICT investment impacts positively on educational performance in primary schools, particularly in English and less so on science but not in mathematics. On an international level, the analysis of the OECD PISA results indicates that longer use of computers by students is related to better results in mathematics in PISA results. As regards better results in national test, two other UK studies show that ICT can make a difference. Broadband access in classrooms is one necessary condition to benefit from new technologies for learning. It results in significant improvements in pupil’s performance in national tests taken at age 16. Overall, evidence from the studies reviewed shows that attainment improves as a result of embedding ICT into teaching and learning. Schools with higher levels of ematurity demonstrate a more rapid increase in performance scores than those with lower levels.

Most opinion based studies investigating ICT impact on student performance, such as the elearning Nordic study, published in 2006, give a positive picture with teachers being convinced that pupil’s subject related performance and basic skills (calculation, reading and writing) as well as educational achievements improve.

Wider benefits for learners

An overwhelming majority of studies reviewed in this report confirm wider positive benefits of ICT on learning and learners, such as motivation and skills, concentration, cognitive processing, independent learning, critical thinking and teamwork. Increased motivation goes together with a positive learning attitude and leads for example to more attention during lessons with students being more involved in the learning activities. The fact that ICT enhances a more student-centred learning approach is often cited as among its most important benefits. ICT can benefit likewise academically strong and weak students as well as students with special needs. Studies also reveal that the benefits can not only remain technology driven but should be more intentionally exploited following a pedagogical approach. Case studies show, for example, that teamwork does not automatically means increased collaboration. Many tasks which teachers called collaborative merely involved pupils working alongside one another rather than jointly addressing a problem.

Impact of ICT on teaching and teachers

Despite the growing body of evidence on the impact of ICT use on learners, whether it will deliver its potential depends to a large extent on how teachers use ICT within the teaching and learning process. As the evidence shows impacting on teachers’ practice have been proven to be a difficult endeavour. Whereas teachers estimate a high impact of ICT on learning and learning outcomes, the perceived impact on teaching methodologies is seen much more moderate.

Most progress has been made in recent years in raising teachers’ positive attitude towards ICT by realising its value for learning through more experience and embedded use. Teachers increasingly use ICT to prepare their work more efficiently and achieve time gains. As the latest Eurobarometer benchmarking survey (published in September 2006) 90% of teachers in Europe already use ICT to prepare their lesson.

There is also evidence of changes in roles of teachers either forced by the technology itself or more actively steered by teachers. In changing the teacher–student relationship, as part of the new educational paradigm, the most difficult process for teachers is to give up control and have more trust in students planning their work independently.

Literature stresses the importance that each use of ICT needs a pedagogical approach to improve learning. On the other hand the overwhelming body of evidence shows that the majority of teachers have not yet embraced new pedagogical practices. Teachers do not feel confident yet in exploiting ICT to support new approaches in teaching. Most of the teachers are still in stage of using ICT to enhance existing pedagogical practice. Current pedagogy is subject centred, and uses ICT for differentiation and project based teaching in more advanced cases. Collaboration between students is not yet sufficiently exploited.
An important research finding is that ICT impacts most in e-mature schools and with e-confident teachers, suggesting that once the foundations are laid the benefits will be considerable. The challenge is therefore to enable all teachers and schools to reach e-maturity.

Research methods to reveal impact

The Impact report shows that the evidence available from impact studies or the evaluation of government initiatives is unevenly spread across Europe. There is a predominance of UK research in the field, studies from the Nordic countries, The Netherlands and Estonia, whereas little is known from some other European countries such as France, Spain, Germany, Greece and most Eastern European Countries.

Currently, there are two main approaches to identify impact on a larger scale. In the UK approach focus is on proving the causal relationship between ICT and better learning outcomes in national tests (measurable systemic indicators), whereas the Nordic impact approach is on the perception of teachers and learners.
Both approaches have their pros and cons, the first tries to establish a statistically relevant association between two variables ICT and higher achievements to show the actual impact. On the other hand, inferring a causal relationship between ICT and pupil achievement from simple correlations can be misleading. Many unobserved factors may influence better learning results in national tests. Isolating the variable that caused the impact from other factors is problematic in education.

Studies that base their findings on perceptions of teachers’ students and parents have its limitations with regards to the actual impact on teachers which can be in sharp contrast to what is perceived. However, they are an important part of the evidence base in the field as they document the views teachers and students hold and express about the consequences using ICT in schools.

With results gained from specific technical interventions limitations remain in the generalisation and transferability of these findings in a wider context where the project specific conditions can not be met. Larger European comparative studies such as Eurydice or the Eurobarometer Benchmarking survey provide important baseline data, but lack to give more qualitative insights into how ICT is exploited for learning in the different European education systems.
The review also shows that current education systems hinder ICT impact. Correspondingly impact studies and evaluations often measure against traditional systems, where the potential of ICT can not be fully exploited. This poses a challenging question, Are researchers looking at the wrong outcomes? Are policy makers clear or realistic about what they expect the results of ICT investment to be?

The impact report therefore argues that a comprehensive body of evidence coming from various sources is needed to inform policy makers as well as finding ways to research “unexpected” outcomes. A combination of different methods e.g. enhance quantitative data with qualitative data increases the validity and reliability of the body of evidence.

Emerging questions based on the evidence

Based on the findings the impact report raises a number of key questions to be addressed. As regards to the positive findings in certain subjects (English as a home language, science) or levels of education (in primary schools); would it be therefore justified to concentrate resources for ICT on those subjects? Or should secondary schools be more remodeled like primary schools to take account of the greater impact in primary schools?
On the other hand to what extent are results transferable or are they contextually dependent? Can we deduce for example that investment in ICT in French schools will yield similar gains in test scores in French primary schools?
Moreover, the picture of evidence is only representative for the countries in focus. These are quite e-mature countries on a wider European scale; there are still large differences between countries. UK and Denmark are for example countries where almost all teachers use ICT as a teaching aid as opposed to countries such as Greece or Latvia where only 36% and 35% use it. What about evidence in those countries? And how can we gather it?

Recommendations

Finally, the report outlines specific recommendations for actions at national and European level in the areas of policy, schools and research and development. The recommendations emerged from the main results of the review and follow up discussion with ICT cluster members and the European Commission to have a real impact on the education systems in Europe.

Policy

Future key aspects in the field of policy making are to support the transformation process and management of change, of which ICT is and enabler and amplifier. New competencies, such as team work, independent learning or higher order thinking skills are part of a new education context with ICT and should be formally taken up in national curricula and assessment schemes. Concerning the continuous professional development of teachers, it will be important in the future to implement new forms of training; training that place in workplace environment, is part of peer learning, is much more personalized and adapted to the individual needs and wishes of teachers and in the long term helps to support lifelong learning of teachers. Initial teacher training, not tackled in this review, is also seen as an important area for improvement in the future. The evidence shows that ICT impacts most with e-mature schools. Policy makers should therefore work towards ensuring that the majority of schools (80 per cent by 2010 for example) reach this point. The value of high quality digital content, support services and highly motivated teachers in schools are essential in this respect.

Schools

As the latest evidence confirms teachers that assess to experience a more positive impact of ICT are most likely to be found in schools where headmasters have used ICT to support the development of the schools values and goals. If the ICT strategy is integrated into the overal’s strategy ICT has the greatest potential to act as a catalyst for change. Furthermore schools should capitalize on positive attitudes and support teachers to underpin ICT use with a pedagogical approach. This can be achieved by hand on practical training, providing easy to use ICT based materials, peer learning, securing a reliable infrastructure and easy access to research findings.

Research and Development

Quite a number of research related recommendations were drawn with the goal to support policy making more effectively. In the future, there is a need to evaluate more concretely school contexts, learning processes and teaching practices to show under which circumstances ICT based activities can enhance learning. To create closer links between research and practice it is recommended to combine more fundamental, small scale research or research focused on specific ICT tools with research that is much more linked to practice. One important finding is the limited access to national research. National research into ICT impact needs to be commissioned in some countries first and where already available made accessible, not only English speaking. Therefore it is recommended to set up mechanisms at national and European level that allow us to know better the results of such research and overcome language barriers in the field. This should include various types of research small and large scale, qualitative and quantitative research. Policy makers also need to broaden the current approach to identify impact, from measuring against well established policy goals towards more independent research to reveal the unexpected outcomes of ICT use in schools.

Full report (pdf - 1Mb)
Executive summary (pdf - 308Kb)

6 March 2007

idium webpublisering