I have evaluated the community’s reaction to my activity by collecting feedback through survey monkey on what worked and what didn’t work (or could have been better) for participants. This is level one of evaluation in the Kirkpatrick model. I also completed my own notes, self reflections and observations at this reaction level.
Although I have not rigorously evaluated level two “to what extent did participants improve knowledge and skills and change attitudes as a result of the training (activity)”, (unknown author, Business Performance Ltd). There was some anecdotal evidence of learning through both comments made on the forum and comments made in the level one survey monkey evaluation feedback. For example in the discussion thread one participant said to another “Thanks Pam, Glad to find out about the howstuffworks site and to be reminded of merlot.com – which I had entirely forgotten about”. In the survey for the question, what worked well for you – one participant answered “learning of new sites that community member’s use”.
I have not evaluated the activity to level 3 (change of behaviour) or level 4 (results/return on investment) as either more time or a larger number of participants would be required to fully develop a community for sharing websites.
Effectiveness of Community Activity in terms of purpose
The purpose for my activity was for participants to share useful links that relate to online learning (that all audiences can use e.g. teachers, tertiary educators and vocational educators) and how they can be used to help with online learning.
The purpose was achieved as over 17 favourite elearning websites were shared by participants of the activity. One participant gave this feedback – “A simple activity but useful – maybe include more about how the sites are used in practice”. This feedback shows the activity instructions could have been refined to make the activity even more useful to participants. It could have been improved if there was a requirement to include a brief description of the website shared and how it could be used by others for elearning.
The effectiveness of communication and collaboration within the activity
Some aspects of communication and collaboration were effective in the activity as the participants shared sites that they used and what they liked about their site. There was also some evidence of the beginning of participant-to-participant collaboration as seen by the comments “Hi Wendy I have not seen this site before, it looks really good. I will use this next semester when we are linking our science to practice. Pam” and “Thanks Pam
Glad to find out about the howstuffworks site, and to be reminded of merlot.com – which I had entirely forgotten about. Cheers Fred”.
Unfortunately about half of the people had posted once, did not return to post again. One participant suggested that I could have improved the facilitation by sending email prompts and reminders. Sherry et al (2001) state that a mixture of probes and supportive comments help to extend conversation, and I based my responses to posts on this. I tried to prompt within the forum this however was only effective in extending conversation when participants returned to the forum to read my response to posts.
Issues which arose for learners and the facilitator during the activity are identified (if any)
A participant emailed me being unable to join the Wiki site where the links were collated. I emailed back instructions for how to access the Wiki, and if unable to access it to let me know and I would send an invite through the Wiki. In hindsight it would have been easier for participants if I sent out the invitation to join through the Wiki or provided some clear instructions on the Wiki of how to join. Sending out invitations through the Wiki would have encouraged more membership as under half of participants from the blackboard forum became members of the Wiki.
Strategies used by the facilitator to anticipate and/or deal with issues are identified
One issue that I identified before planning and submitting instructions for my activity was to make sure the location of the activity was accessible and easy to find to maximise participation. My strategy for this was to take the discussion and sharing of sites activity to the participants’ location i.e. blackboard. Rather than make the participants come to my activity elsewhere on a blog or wiki. This made the activity easy for the participants to locate and can be seen by the fact that eight people participated in the blackboard discussion forum whereas only three people joined my Wiki that collated all of the links (although it’s also possible that participants may have viewed the Wiki but chosen not to join as members).
Areas for further development or application of activities are identified
I will continue to add links for elearning and invite others to do the same on the Wiki. This is a resource that can grow and benefit all participants interested in elearning as we learn more and share resources with each other.
To develop the library of links it would be more useful to have a brief description of what each site is useful for, so participants can explore the sites that will be of most benefit for them.
Areas for further development of own facilitation skills are identified
I received positive feedback from participants for my facilitation such as “you were consistent in replying and commenting”. I was conscious of setting a positive tone to value everyone’s contribution to encourage more interaction.
The discussion and posts did reduce towards the end of activity. Part of the facilitator’s role is to motivate and “engage the learner”, (Australian Flexible Learning Framework – Effective Online Facilitation). Perhaps I could have motivated and engaged participants more if I had sent follow up emails asking them to check out the new links posted and encouraged replies and feedback to other people’s posts.
Australian Flexible Learning Framework. Effective Online Facilitation. Downloaded 25 June 2010 from http://pre2005.flexiblelearning.net.au/guides/facilitation.html
Left, P (2010). Evaluating online community activities. Downloaded 25 June 2011 from http://www.verso.co.nz/learning-communities/780/evaluating-online-community-activities/
Sherry, L, Tavalin, F & Billig, SH 2001, Good Online Conversation: Building on Research to Inform Practice, Journal of Interactive Learning Research. Vol 11 (1).
Unknown author, Business Performance Ltd (nd). Why Measure Training Effectiveness? Downloaded 36 May 2011 from http://www.businessperform.com/workplace-training/evaluating_training_effectiven.html