Slide 1: Why is evaluation so important? Grace with magnifying glass by Thinkcage Bronwyn Hegarty 2007 - some rights reserved
Slide 2: Overview In this session the importance of evaluation and the what, why and how of evaluation will be covered. The relevance of evaluation for education and educational design will also be mentioned (ADDIE model). Bronwyn Hegarty 2007 - some rights reserved
Slide 3: Evaluation - is it really necessary? Bronwyn Hegarty 2007 - some rights reserved
Slide 4: What is evaluation? • Jot down some answers to this question. • Think about some situations where you might need to do an evaluation. Bronwyn Hegarty 2007 - some rights reserved
Slide 5: ADDIE model of instructional design Bronwyn Hegarty 2007 - some rights reserved
Slide 6: What are you evaluating? Chicken hat by KB35 Bronwyn Hegarty 2007 - some rights reserved
Slide 7: Check this out • Please take time to view this video clip YouTube - Flight Of The Conchords - Bowie Song at: http://tinyurl.com/2oc9zb • Is this good quality? • Why is it good? • Why is it not so good/ Bronwyn Hegarty 2007 - some rights reserved
Slide 8: Context of the evaluation • • • • • • • A course Resource Teaching strategies Learning activities Group interactions The teacher Technologies Slayers of second life by cogdogblog Bronwyn Hegarty 2007 - some rights reserved
Slide 9: feedback characteristics Assessment Learning outcomes Teacher constructivist Scenario-based New course Interactions Group work Teaching/learning strategies Reflective writing discussions What am I evaluating? email resource Learning activity DVD- skills Creating wiki page quiz Digital images Internet tools Technologies e-portfolio The purpose of the evaluation influences the methods to be used Bronwyn Hegarty 2007 - some rights reserved
Slide 10: Why evaluate? Evaluation checks quality and success. • • • • Cost-effectiveness - ROI Satisfaction of the learners Decision-making Teaching effectiveness The key to your success by Clearly Ambiguous Bronwyn Hegarty 2007 - some rights reserved
Slide 11: Why are you evaluating? promotion Professional development enjoyment achievement $$$ Performance review low Drop out Student satisfaction organisational Decisions teaching Completion rates Why are you evaluating? departmental success Industry needs demand Measure of effectiveness Impact Review teaching Learning New skills Transfer of knowledge experts To check success Bronwyn Hegarty 2007 - some rights reserved
Slide 12: How do you evaluate? "When the cook tastes the soup, that’s formative; when the guests taste the soup, that’s summative." (Robert Stakes) Poor evaluation can lead to disappointment and bad quality Bronwyn Hegarty 2007 - some rights reservedby Jon McLean Toy pileup
Slide 13: Formative Types of evaluation Usability - navigation - relevance - access Review - expert opinion How are you evaluating? Don’t forget about needs analysis Types of evaluation Summative Maintenance - viability Effectiveness - meets learning outcomes Impact - users transfer learning Bronwyn Hegarty 2007 - some rights reserved
Slide 14: Evaluation methods There are many ways to measure something Measuring time by Aussiegall Multiple methods will allow you to examine the situation from several angles (triangulation) Hegarty 2007 - some rights reserved Bronwyn
Slide 15: Data collection sampling methods and instruments • • • • • • Observation Questionnaires Focus groups Expert review Checklists Feedback in a discussion forum Bronwyn Hegarty 2007 - some rights reserved
Slide 16: Evaluation plan It is a very good idea to design and plan your evaluation carefully - rationale and purpose - decisions and context - type of evaluation - big picture questions - data collection methods and analysis - timeline & milestones Bronwyn Hegarty 2007 - some rights reserved
Slide 17: Evaluation report • • • • • • • • • • • Includes the following: Executive summary Introduction - background and rationale Purpose (aim) Decisions – target audience Big picture questions Methods – data collection instruments Results Discussion and recommendations References Appendices – samples of instruments, data Bronwyn Hegarty 2007 - some rights reserved
Slide 18: Summary • The type of evaluation depends on the context and purpose • Important part of educational design, development and implementation • Plan the evaluation • A multiple methods approach provides more perspectives • Choose data collection methods & instruments carefully – analysis of results • Prepare an evaluation report Bronwyn Hegarty 2007 - some rights reserved
Slide 19: Resources • Evaluating the impacts of eLearning (WikiEducator, 2007) (http://tinyurl.com/25r4lm) • Evaluation cookbook (Learning technology dissemination initiative) (http://tinyurl.com/n5efa) • Evaluation Tools (Center for Education Integrating Science, Mathematics and Computing (CEISMC), Georgia Tech College of Sciences) ( http://tinyurl.com/27693m) • Evaluation in action – online information literacy modules at: http://oil.otago.ac.nz • Polybase – survey section – sample questions Bronwyn Hegarty 2007 - some rights reserved
Slide 20: Go forth and evaluate Big smile by Kansir Bronwyn Hegarty 2007 - some rights reserved