Artifact 8 - Formative Evaluation Report
AECT Standard 4: Professional Knowledge and Skills
Context & Conditions
This artifact is a formative evaluation report that was done in EDET 793 on an earlier graduate student in the Learning Design and Technologies program at the University of South Carolina eLearning module. We were asked to write an evaluation report on the strengths and weaknesses of the initial design for the ASSURE eLearning module. Additionally, we were asked to do a small field study and use an SME to help gather quantitative and qualitative data on the initial eLearning module. By collecting the data from the small group, the SME, and identifying the strengths and weaknesses that my group noticed, we were able to evaluate the effectiveness of the original eLearning module and add suggestions on how to fix it. When going through this process, I had a good understanding of the ADDIE model, the MRK model, and The Kirkpatricks Model that can be used to help improve the learner's experience, but I noticed that I was a little bit intimidated by the evaluation process. The reason is that before this class, many of my other courses never did the evaluation process. So going into this project, I had a big learning curve that I had to overcome with my group, to instead of designing and developing our own eLearning module, to instead test and evaluate what the eLearning module could do by itself.
Scope
The purpose of this artifact was to evaluate an earlier graduate student eLearning module on the ADDIE Model of Instructional Design. We were asked to identify strengths and weaknesses that we noticed (as a group) in the module. Afterwards, we were tasked to collect our own quantitative and qualitative data by creating pre and post assessments that our SME and our participants were asked to take. By gathering the data from the small group, the SMEs, and our own analysis of the module, we’re able to decide what was effective and what needed work based on quantitative and qualitative data, and solutions on how to fix it. This project was not part of any curriculum or unit; however, if we were teaching a class on evaluations in instructional design, this would be a great example of how to correctly perform an evaluation on a previous eLearning module. The skills that I was able to use include having an in-depth understanding of what the ADDIE model was, understanding the MRK, Mayer’s Multimedia of Learning, universal design for learning (UDL), and the Kirkpatricks' model. By having a grasp on these different models and theories, we were able to identify the strengths and weaknesses of the eLearning module. Furthermore, when recommending solutions to improve the effectiveness of the module, leave suggestions that would improve the learner’s experience.
Role
This was a group project performed in EDET 793 on a previous graduate student’s eLearning module. We, as a group, equally distributed our work when working together, and we were all instructional designers. We did have a subject matter expert (SME), Dr. Tang, a professor for the Learning Design and Technologies program at the University of South Carolina. By creating and Dr. Tang completing an SME observation form on the eLearning project, we were able to gather data on what worked well on the module and suggestions to fix. Since this was a group project, we were able to come together and work together to develop our formative evaluation on the project. I worked on taking on the data from the pre- and post- assessments to create conclusions from the data to improve the experience, suggestions, and any minor errors that needed to be fixed.
Instructional Design
This artifact aligns with the ADDIE model, with the focus on the evaluation process. During this artifact, we were asked to develop an evaluation plan based on a previous graduate student’s eLearning module. The evaluation process focuses on the effectiveness of the instructional design and notes on how to enhance the instructional design. By doing this, my group and I were able to evaluate small groups and a subject matter expert to identify the weaknesses and strengths of the design and how to improve it for future usage.
This artifact aligns with Morrison, Ross, and Kemp’s (MRK) instructional design model because it focuses on the formative evaluation process in the instructional design phase. According to Morrison, they state, “Formative evaluations are used to provide feedback to designers for making course improvements (Sterbinsky & Ross, 2005). These evaluations take place as instruction is “forming,” and thus they precede the development of the final version of the instructional unit or course.” (Morrison et al., 2013) By performing a formative evaluation, my group was focused on improving the eLearning module based on the strengths and weaknesses that were consistent in the creation. Furthermore, instructions are always changing, and what worked with one group of students may not be beneficial for the next group, which means evaluating and improving our strategies to support a diverse group of learners.
Related Performance Indicator
Collaborative Practice – Candidates collaborate with their peers and subject matter experts to analyze learners, develop and design instructions, and evaluate their impact on learners.
Reflection
I really enjoyed creating this artifact and the process of creating this artifact. While there was a lot of typing involved, being able to use the skills that we have learned in previous courses to research and analyze data to improve someone else’s creation was fun. I was able to learn and experience the E on the ADDIE module, which is the evaluation section. Unlike every other course, this is the first time that I have experienced taking a module and evaluating how it performs, and suggestions to recommend improving. There are things that I would like to have changed, such as the number of participants who were asked to participate and the different educational backgrounds. While the eLearning module is targeted towards graduate students in instructional design programs, if an eLearning module is effective, any person would technically be able to learn from the module if done properly.
This artifact was created in my current capstone course, which means I have an advanced understanding of the ADDIE model, the MRK, the foundations of instructional design, UDL, and research skills. One thing that was new to me and my group members was performing the evaluation process. My group members and I both ran into the problem of confusing the terms evaluation and assessment because of our background in education, and in education, we use those terms interchangeably. In instructional design, you cannot because it means and the processes are different. It was a learning curve for me to take on the lens of how to take the data and create solutions that would improve the learning experience for the learners.
This aligns with AECT 2012 Standard 4 on collaboration practice because I worked with a wonderful group of classmates to create this extensive evaluation format. By working together, we were able to get our SME to experiment on participants, gather the data from the participants and our SME, and infer what the data means. We were also able to bring different skills, such as being able to analyze, graphic design skills, an extensive in-depth understanding of the ADDIE model, and we all have a background in education, so an interest in how to improve our learners’ comprehension and retention rates. By having peers who were just as motivated as me to create a good formative evaluation on a previous student’s eLearning module, we were able to design effective and useful solutions that would improve the overall experience.