Log in

"Measuring" the impact of BYOD at your school?

  • Public
Started by Steve Trotter 20 Nov 2014 10:44am () Replies (7)

Should we (school's) measure the impact devices and the learning that they allow have?

What should we measure?

How should we measure it?



  • Nathaniel Louwrens (View all users posts) 20 Nov 2014 11:50am ()

    Hi Steve

    Interesting questions.

    My first thought is, What is the purpose of measuring? What do we hope to gain from the measuring?

    I'm not against measuring and actually think it's useful to gather information on the impact of our classroom practice and this may include using various devices etc.

    However... I guess it depends on how we look at the devices.

    If we look at the devices to determine whether we continue with them or go back to older, more traditional ways of education, then I guess we would need to measure the impact.

    Has anyone measured the impact of using pens or pencils?

    What about the old overhead projectors?

    Is it the device that is or isn't have an impact on learning or is it the teacher, the practice, the student or another factor (since we know that there are a lot of different factors - internal and external to school - that can affect learning?


    I think it's important to gather data around aspects of education in order to reflect on, adjust, improve our practice and ultimately the outcomes for students. But we can measure and collect data for everything. We need to know why we're collecting it and what we might do with the data once we've collected it.

  • Keryn Davis (View all users posts) 20 Nov 2014 2:00pm ()

    Great discussion here.

    I'd like to add another question - can the impact be 'measured'?

    The outcomes, particularly if these are dispositional in nature (e.g. curiosity, thinking, creativity etc), aren't necessarily 'measurable' in the traditional sense, and will require different ways of approaching the task of considering the impact on learning.

    Perhaps what we need to do is make meaning of the impact, or explore questions such as 'what do we know about impact the use of devices is having on learning?'. If we do this we might need to 'measure' effectiveness in terms of meaningfulness... authenticity... trustworthiness... motivation... ?

    Just some thoughts.

  • Steve Trotter (View all users posts) 20 Nov 2014 2:46pm ()

    Yes Keryn - the impact is much wieder than the core subject areas! 

    It somewhat reminds of the staffroom debates around assessing Key Competencies when they were first introduced.

    Many schools created ways to measure the progress of KCs and some schools set 'tagets' around them (rightly or wrongly). 

    If we worked out ways to measure KCs we could equally work out how to measure curiosity, thinking, creativity etc?

  • Steve Trotter (View all users posts) 20 Nov 2014 2:12pm ()

    Thanks for you ideas Nathaniel,

    Yes, we can not measure the impacts of a 'device' - only of the learning that happens as a result of having the device available.

    My current thinking is schools need to determine their goals for how devices will improve student outcomes - such as an increase of;

    • effectiveness of teacher feedback,
    • student ownership of learning,
    • digital citizenship awareness 
    • writing achievement. 

    Nothing stands in isolation. Obviously other factors also influence the outcomes of above. But, this is no different to other initiatives where schools have to 'justify' the spending of money - e.g. when I was a ALIM (Accelerated Learning in Maths) teacher I had to 'measure' the success of a group of targeted students for the Ministry. 

    Maybe if as schools we took the initiative and demonstrated the success that BYO programmes have to the government, they would be forced to supply devices for all students and greater PD for teachers, rather than the current 'user pays' model!

  • Rosie Powell (View all users posts) 20 Nov 2014 3:10pm ()

    We have been BYOD for a year now, our main focus was with writing this year, and we can definitely see improvements.

    One of the big things for us was that we want to be using the best tool for the job and if pen and paper proved to be better for writing then we could go back to that.

    I can also see that there are many ways where the potential that BYOD offers is not measurable, but still adds another dimension to the students learning.

  • Tessa Gray (View all users posts) 27 Nov 2014 2:00pm ()

    Hi Steve, great post…the age ole argument about measuring the impact of e-learning on learning has arisen again. It’s a very complex construct and you’re not alone in your thinking. Thanks Rosie, Keryn and Nathaniel for your thinking too. smiley

    The following thread might be of interest to you. How do you know if ICT is making a difference - and does it matter?

    It’s not written in the context of BYOD, but it does debate similar arguments around proving the investment that e-learning devices can/do have a positive influence and impact on learning.

    Similar debate around modern learning environments touches on the perceived value and impact of physical spaces as well as the use of resources, including technologies. MLE- Do these actually raise student achievement?

    As you’ve already noted, it’s much bigger than assessing tools themselves.

    Check list There are many variables to teaching and learning, which makes us wonder what exactly we’re assessing and how can we demonstrate the tools have made a difference. The Alternatives to ICT skills checklists post touches on the e-Learning Planning Framework, rubrics and e-competencies which might be a useful reference too.

    Sorry if I’ve muddied the waters Steve!

Join this group to contribute to discussions.