|Go Back| Summative Evaluation

Formative Evaluation

After creating a rough draft of the Edmodo training, a formative evaluation was conducted in order to improve the instructional unit. The decision was based upon the data gathered from the formative evaluation. Three steps were applied to evaluate this training. First, a Subject Matter Expert (SME) revised the draft materials and provided feedback. After making suggested changes to the instructional unit, another evaluation was taken place with a member of the intended audience. A teacher from Myra Green Middle School completed the training. Instructional feedback was obtained from this evaluation and immediate changes were made. Finally, the revised training was available for a scheduled small group evaluation. Five teachers completed the training in a face to face environment. After receiving feedback and collecting data from observation made, complications were presented. These complications changed my plans and timelines. The observations and reflections of the formative evaluation from these three stages are indicated below.

Subject Matter Expert Review (SME)

After creating a work breakdown structure (WBS) of the instructional unit, the SME studied it to make his observations and provide me with feedback. I created a very detailed WBS of the instructional unit which helped me to visualize all the steps, check the instructional material, and catch the gaps missing in the plan. I created this by hand so it was possible to make immediate changes by erasing, and adding new information. I scanned my WBS and emailed to Mr. Perales, a middle school History teacher who has successfully integrated Edmodo in his class since two years ago. After reviewing the WBS, we had a conversation over the phone about the gaps and missing material. The following are observations made by Mr. Perales

First of all to concentrate only on the most important tools used in Edmodo. It will be impossible to cover all the Edmodo tools in a 50 minute time period. Initially I had seven steps to cover
Add an introduction to the training; the audience needs to understand the reason, objectives and goals of the training.
Make sure teachers are advised to own and have access to an email account before taking the training.
Once teachers have created Edmodo account, ask them to connect with you through Edmodo to provide guidance on any given day, or to collaborate with this tool.

Immediately I took note on the feedback provided and edited my WBS to carry on a plan with better fundamentals. After this review, he asked to show him the final product to continue giving me feedback. He expressed enthusiasm and was happy I considered Edmodo; since his knows this Web 2.0 tool has been of great help to communicate with his students. He said students love this tool, especially because it looks like a very popular networking social site; Facebook.

One to One Evaluation

The one to one evaluation was conducted with an English teacher from San Juan, TX. After building a basic website for the Edmodo training in Weebly, I emailed the link to Mrs. Salinas. The training was still incomplete; since it still needed the quizzes and rubrics, but she was aware of that and decided she was going to concentrate on the design functionality. At this point I had finished the introduction and steps 1-3. This evaluation was completed at the teacher’s house, and emailed me back with her observations.

She really liked the Weebly as a site builder; she said it makes it look professional
Suggested to remove unnecessary clipart and images; she said it was too crowded, that too many images could distract the learner and lose concentration on the topic.
She was glad to read the introduction with the goals and objectives of the lesson; but she recommended embedding a short entertaining video as a grabber attention.
Even though the steps links are located at the top of all the pages; she advised me to add more buttons at the bottom of the training, because pages were too long to scroll all the way up in order to continue to the next step.
Add at least a short video at the end of the page incase teachers still don’t understand the instructions given with the screenshots.

After carefully reading Mrs. Salinas suggestions, I accessed the training online and observed her feedback. I agreed on the excessive clip arts and pictures. I guess I got excited with clip arts and started uploading them to the training. I erased several and only left the ones that are used as indicators for information or questions. I started searching for a good short YouTube video so I could embed it to the training, and found more than one, but of course only kept one. I also created links at the bottom of every page to make sure learner is comfortable in navigating the online training. To be honest I didn’t like the idea of recording tutorial videos for every step since I was using screenshots for that same purpose. Later I was thinking that providing videos could be convenient to many teachers who decide to watch videos instead of reading. I started recording videos immediately after these observations.

Small Group Evaluation

The last evaluation was scheduled for the online training in a face to face environment. This evaluation was taken place at my house with five female teachers from the intended audience. A day before evaluation, they received an email with the Entry Skills: have basic computer skills, be able to manipulate mouse functions such as copy and paste, be able to navigate the internet and own an email account.

All five teachers are computer literate so the Entry Skills were no problem. None of these teachers have used Edmodo; some of them have heard of it but never tried it before. Every teacher brought their own laptop, earphones, and notebooks to take personal notes. We put together two big tables and set up the living room for a learning oriented setting. I have fiber optic wireless Internet, so speed was excellent. I have a personal projector which I used to project the YouTube video as an attention grabber. Edmodo was introduced in the video and right after the video; I explained in brief, the training objectives and gave a tour of the training. After these I asked if they needed any clarifications but they all were already with their earphones following the online training. I started making my observations by walking around and taking notes.

All five teacher were engaged in the online training
They answered anticipation guide fast with no questions or problems
I had provided a hard copy of the flow chart, but none of them were using it
The training took 55 minutes long
Training went very smooth, although sometimes they were chatting about nonrelated topics that I had to re-direct them. That situation makes me think I will have that problem on the in-service day at the library. All the teachers from Myra Green have really good relationships and are often joking and playing around. I will need to think on how to address a situation like this so they can all successfully complete the training on time.
They finished training and were very excited to receive a Certificate of Completion.
After training, they shared their views on this new learned tool. They expressed they are anxious to implement it this coming year, since they know students will love it. They liked the idea of communicating with parents as well.
Three of the teachers downloaded the Edmodo app to their smartphone to get more familiar with it.
I’m glad these teachers made it to the evaluation so they can be of help at the actual training, since they now have prior knowledge of it.

Results, Reaction and Action

After the SME evaluation and the one-to-one evaluation helped to see gaps that I couldn’t identify before. I made immediate changes after I their constructive feedback, which it helped to improve the instructional unit. After completely finishing the Edmodo online training, I conducted the small group evaluation, which was a success. Overall, the online training went smooth with the instructional guidance.

However, there are some complications presented after successfully finishing the group evaluation. Edmodo website decided to change their look! This change was on July 15, 2013. They changed their look and some features. When I found out about this change I felt frustrated and a bit disappointed because I had just finished my last evaluation. I didn’t know what to do with the online training, since it was done with screenshots and videos from the “Old” Edmodo. I know I could just leave the online training without any changes to be submitted as the Capstone Project. However, I know this online training will be utilized in an actual teacher in-service in August 19-23, 2013. If I leave everything the way it is, it would cause confusion with the learners and definitely would not accomplish my instructional goals and objectives.

I decided to re-take screenshots and re-record videos with the new Edmodo template. I know that as Instructional Designers we have to be ready for any unplanned changes and have always a plan B available. I’m working hard every day in finishing the last steps of the actual training. It’ll be finished before the deadline.

In conclusion, the three conducted evaluations were clarity, impact and feasibility. The evaluations helped me to improve the online training. The results motivated me to create good learning material.

I have assumed the responsibility of staying current with trends in educational technologies, therefore modifications for this project will be ongoing. As of now, modifications made were based on the feedback and recommendations received from both the formative and summative evaluations. Revisions include:

Removing unnecessary clip art and illustrations
Changing font letter for a more readable one
added more buttons for easier navigation
added a video at the end of every lesson
removing unnecessary notes
created the quiz online
embedded quiz to the same page at the end of the lesson instead of adding a link
added shortcut links at bottom of each lesson

Go Back| Summative Evaluation