Page links
Site links
Remote links
RSS feed
This site now has a valid RSS v2.0 feed:
download
This site now has a valid RSS v2.0 feed:
download
I asked faculty in the following classes if I could use the FAST online assessment tool to survey the students in their classes for their opinions about Moodle:
| Division: Natural Sciences | |||
|---|---|---|---|
| Faculty | Course Id | Course Name | number students |
| Brent Smith | BIO 455 | Population & community Ecology | 16 |
| Brent / Jon Branstrator | ENPR 111 | Environmental Science | 54 |
| Jennifer Ziebarth | MATH 420 | Abstract Algebra | 11 |
| Jennifer Ziebarth | MATH 180 | Calculus A | 37 |
| Division: Social Sciences | |||
| Kathy Milar | PSYCH 342 | Experimental Psychology | 47 |
| Michael Jackson | PSYCH 351 | Qualitative research Methods | 21 |
| Mark Pearson | MGMT 110 | Info Tech & Soc | 22 |
| Division: Fine Arts | |||
| Julia May | ART 382 | Art of the Americas | 39 |
| Julia May | ART 282 | Survey of Western Art | 40 |
| Division: Humanities | |||
| Micelle Patterson | HIST 121 | US to 1865 | 41 |
| Sandrine Sanos | ES | History modern anti-semetism | 18 |
| total | 346 | ||
A total of 346 students were canvassed by email from 8th to 10th December after I had obtained permission from the class teachers. 93 students responded, a 27% success rate, and there was at least one respondent in each of the courses canvassed [comments:Q15].
Questions one and two seeked to discover whether the student was new to Moodle, and if not, how he/she compared it to what was used beforehand. Questions three and four followed this further asking whether students found Moodle easy to use ('intuitive') and whether they considered it an asset to the course (which also depends upon how the teacher used Moodle). 'Ease of use' is not neccessarily equivalent to 'intuitive' — a system may be easy to use but not accomplish what the teacher desired — but I wanted the students to consider internal consistency in how they did things with the Moodle system. There was no formal instruction offered to students and question five asked whether students would have liked such a session. While question four looked at how the course was presented with Moodle, question six seeked to discover whether students considered Moodle to be a help or hindrance to their learning efforts. Questions 7 - 10 looked at how students made use of the Moodle system. Question seven looked at frequency of use outside class time, question eight at whether students found the 'home page' arrangement useful (course contents presented in a Weekly or Topic outline), and question nine asked how students used the uploaded files in Resources. Finally, question ten sought to discover whether student would (hypothetically) like to post assignments online. Questions eleven and twelve asked for written feedback about Moodle likes and dislikes (Q11) and whether students felt that Moodle could have been used in other ways (Q12). Given some previous comments about the performance of the Moodle system questions 13 and 14 concerned the responsiveness of the system in general (Q13) and from the student dorm rooms (Q14). Finally, students were asked to give their course code so that we could check that all the courses surveyed had respondents.
| Questions | Strongly Disagree | Somewhat Disagree | Not Applicable | Somewhat Agree | Strongly Agree |
|---|---|---|---|---|---|
| Q1. Which Course Management system did you use before using Moodle in this class ? (If more than one pick the one you're most familiar with) | 0 | 20 | 14 | 12 | 47 |
| Course Compass | CHEF | Moodle | Other (such as Blackboard) | None | |
| Q2. If you used a different CMS beforehand how would you compare it to Moodle. | 0 | 2 | 65 | 17 | 9 |
| Strongly prefer the other system | Mildy prefer the other system | I am using Moodle for the first time / no opinion | Mildly prefer Moodle | Strongly prefer Moodle | |
| Q3. Overall, would you agree that you found Moodle intuitive to use? | 1 | 5 | 1 | 72 | 14 |
| Q4. Overall, would you agree that using Moodle was an asset to your course? | 1 | 6 | 1 | 55 | 30 |
| Q5. What sort of formal instruction on using Moodle would you have liked? | 3 | 1 | 8 | 21 | 60 |
| I needed an in-depth session | I needed a solid overview | N/A (I did have a session to learn Moodle) | Some introduction would have helped | I did not need any since Moodle was so easy to use | |
| Q6. How did Moodle help or hinder your learning progress with the class? | 1 | 4 | 19 | 47 | 22 |
| Having to use Moodle made the class much more difficult for me | I made progress in spite of having to use Moodle | Neutral. I could have done the same without it | Moodle was useful but not essential | Using Moodle greatly enhanced my learning | |
| Q7. How frequently did you use Moodle outside of class time? | 7 | 6 | 46 | 29 | 5 |
| 1 Only when I was sent an announcement via email | 2 Only when told during class to check it | 3 1 & 2 & when I needed to do an assignment | 4 1 & 2 & 3 & occasionally to see latest activity | 5 I checked it every day regardless | |
| Q8. Do you agree that it was helpful to have the course contents viewable in the weekly or topic outline? | 2 | 6 | 13 | 48 | 24 |
| Q9. How much use did you make of the files posted into Resources ? | 9 | 24 | 18 | 39 | 3 |
| 1. No use | 2 Only when I was explicitly told to | 3 2 & to do assignments | 4 2 & 3 & for revision for Quiz | 5 2 & 3 & 4 & after each class session | |
| Q10. If you did not submit your assignments online would you like to do this? | Yes | 48 | No | 45 | |
| Q11. Which aspect / feature of Moodle did you like the best? The least? | Written response | ||||
| Q12. Are there other ways you can think of that Moodle could have been used in the course? | Written response | ||||
| Q13. Generally when you used the Moodle system how did you rate its responsiveness? | 3 | 15 | 30 | 37 | 8 |
| Far too slow | Slowish | Good enough | Reasonably responsive | Very responsive | |
| Q14. Do you agree that Moodle was just as responsive from your dorm room as in the Lab? | 8 | 15 | 32 | 31 | 7 |
Prior to the start of the fall semester 2004, 50% of the students surveyed were completely new to online Course Management Systems [Q1:chart], 22% had used Chef, 15% had used Moodle and 13% other systems such as Course Compass based on Blackboard, in other courses before the Fall 04 semester. Thus there was a preponderance of novice students.
Question 2 wished to elicit from students who had used other systems beforehand how they rated Moodle in comparison. If we leave aside those who had not used a CMS before (50% of the total) we find that a mere 4% mildly preferred the other system they used (2 students), 39% didn't care either way (18), but 37% mildly preferred Moodle (17) and 20% strongly preferred Moodle (9). Thus Moodle was preferred by 57% of students who had used other course management systems as well. This is consonant with the survey of students using Moodle and Chef at the same time in different courses [link]. There, a total of 63% of students thought that Moodle was easier to use than Chef. What we can conclude from this is that on the whole students who have used course management systems prefer Moodle to others. We will explore some of the nuances of student's response to the Moodle system in questions 11 & 12.
The vast majority of students found Moodle intuitive to
use [Q3:chart] but much depended upon how the teacher
organized the course within Moodle. Thus, tellingly, one student wrote
that it was natural and simple to use
As long as the teacher keeps it updated and site listings organized.
and
another student complained that it was not clear where to put
what. so unclear what was due, when and where to put it.
[Q3:Comments].This
latter is a fault with the instructions the teacher gave (or didn't give)
for how to respond to the given assignment. [I did address this issue
in the winter Moodle workshop where we practised writing instructions
which were directive and not vague]. However, most students agreed with
the sentiment expressed thus, I found it very easy to use without
a lot of instructor guidance, and I'm not very good with computers.
[Q3:Comments].
Even though Moodle itself was deemed to be user friendly, some students
expressed the need for some sort of formal instruction, so, I
wish that I was told how to use moodle more clearly at the outset of
the course.
[Q3:Comments]; maybe
this would be another topic to cover in the Technology Literacy sessions
for freshmen. Moreover, the scope of a Course Management System (or,
better, Online learning Environment) is by no means obvious and one student
commented that It did not seem
intuitive what its powers were, and how much the professor was actually
using.
. Here again, this could be addressed within the context
of Technology Literacy. Finally, some students commented on the Moodle
interface itself pointing out some interface design flaws based
on Fitt's Law
and requesting that the resources button be more
prominent (no), and one student suggested borrowing a feature of Blackboard when
I used blackboard I knew that there was a way to post notes to the prof.
and other students and a way for them to respond.
[Q3:Comments].
This may be a feature which we could work on with Earlham programmers.
A massive 92% of students agreed that using Moodle was an asset to their
course [Q4:chart] with 60% in somewhat agreement and
32% in strong agreement. Thus, most students had positive comments
such as it kept things organized and sorted
, It
was SUCH an asset to have access to all of her lectures, images, and
additional resources.
, and it certainly made handing in
assignments easier.
[Q4:Comments].
The pedagogic value of online resources was apparent from comments such
as these, If it weren't for powerpoints on
Moodle, I would have failed numerous tests!
, We would have
had to take at least twice as much time in class without the powerpoint
presentations on moodle.
,
and When it came time to study for a test all of the lecture
notes that were on powerpoint were available and it was easy to recall
class discussion based on the slides
but nevertheless one or two
disliked the extra work which was entailed in reading websites and online
articles [Q4:Comments]. Once again, the
way in which Moodle was used conditioned the response from students;
thus The more the teacher is comfortable with it and uses it,
the more helpful it is to the students.
, and I wish the
professor had continued to put our grades online
and there was
what one student termed the one page problem
, where (ie.
the professor put everything on one page)
[Q4:Comments].
Students also alluded to problems inherent in posting large Acrobat pdf
files where paper was wasted when students printed everything out; however,
PDF files are awkward to handle, and a lot of paper and ink is wasted
printing off images. ...
When there were 50 images for a test, and i did not watn to just study
off the screen, then i would have to print all 50 images.
. There is an
inherent tendency to print pdf rather than view or read on screen but it's easy
to select individual pages to print and if the Labs had duplex printing enabled
paper would be saved. Thus we have a training as well as an equipment
issue.
When asked what sort of formal instruction (if any) students would have
liked (question 5) approximately two thirds said that they didn't need
any instruction [Q5:chart] but 23% said that some (formal)
instruction would have helped their usage of Moodle. One student (in
the Q4
section) pointed out that he had figured things out through
trial and error
from a previous course that used Moodle and that
his rating of 5 on this question ("not needing any instruction on using
Moodle") should not be taken as a complete endorsement of
the ease of use
and that There is still plenty of streamlining
that it needs
. Thus, there is a fairly strong case for covering
the basics of Moodle (enrolling in a course, accessing resources, uploading
an assignment, contributing to a discussion forum) in the Technology
Literacy sessions for incoming freshmen.
Student responses to question six (does Moodle help or hinder your learning) were split into roughly three groups [Q6:chart]. Half the respondents (51%) considered Moodle to be useful but not essential, but a quarter (24%) thought that it greatly enhanced their learning (an encouraging statistic), while 20% were neutral and reckoned they could have done the same without it. Thus even in this early trial with most faculty using Moodle for the first time the student's perceptions of the pedagogical value of Moodle were on the whole very positive. This suggests that with perhaps more guidance for 'best practices' with using Moodle there is potential for a consistent positive influence on course pedagogy.
For a faculty using Moodle for the first time there is not just the question of how to use the system, but also of what to use and when to employ it. In the Information Technology and Society course I discovered that students need to be directed explicitly to access their Moodle course; they would not check Moodle regularly as they would their email. This was borne out by responses to question 7. 8% of respondents only accessed the Moodle course when they received an announcement in their email (postings made by the teacher to the 'News Forum' get emailed to all students enrolled in the course) and 6% used it only when explicitly directed so to do during class time [Q7:chart]. However, 50% accessed their Moodle course in order to do an assignment and 30% would even check occasionally to see whether there was anything new. A mere 5% checked it every day as they would their email. This pattern of usage was confirmed by the answers to question nine. Fully a quarter of students used Moodle only when their teacher explicitly told them so to do [Q9:chart]; this implies that if these students were not told to use Moodle to revise for a quiz or to research for an assignment they would not have done so! However, an extra 19% used their initiative to look on Moodle for resources relating to an assignment, and 42% would do this together with using Moodle resources to revise for a quiz or exam [Q9:chart]. Here again, a mere 3% checked for new resources independently of proximate need.
Unlike some other systems (eg Chef) Moodle allows the teacher to input and display the contents of the course and to customise the course layout. 50% of students agreed that having the course work scaffolded in this way was helpful [Q8:chart] and another quarter of the students strongly agreed that this was helpful. This confirms the pedagogic value of the Weekly or Topic outline and also suggests that it's important to make this outline clear and comprehensible.
The response to question 10 was interesting
[Q10:chart]; students were equally split in wanting
to submit assignments online and not wanting to do this. Written
responses [Q10:comments] mirrored this
result. About a third of the comments pointed out that they already submitted
assignments online and that it was fun
, it worked
great
, and I really liked it
. Saving paper was given
as a reson for submitting assignments online, It
is absolutely imperative that we reduce the paper usage on campuses across
the world and Moodle is an integral asset in accomplishing these goals
.
Some students expressed a strong preference for turning in hard copy
assignments for similar reasons to this: I
feel like turning in a hard copy is a way to make sure that there is
no fault except the student's for not getting in an assignment on time.
One
student commented that i want a hardcopy back, with comments
for all my work
and this is certainly one area where an improvement
is needed in the Moodle system (the ability of a teacher to return a
graded assignment). Technical
issues were also given for why online submissions were not a good idea;
for example, my
math homework is usually hand-written, which wouldn't work well online.
(but
Moodle can accomodate Tex math scripting as Jennifer Ziebarth is discovering
in her Calculus B class), similarly worksheet assignments were not suitable
for online submission, and formatting and display issues were cited as
concerns as was the use of Microsoft software, I do not use Micro$oft
word
.
{As an aside, there are a lot of advantages to not using
Microsoft software such as Internet Explorer, Word, Powerpoint, Excel,
with Moodle or any other online system [link]}.
Earlham students are nothing if not opinionated and so it was no surprise
that 70% of the total number of respondents gave their likes and dislikes
of the Moodle system (question 11, Q11:comments).
Taking the positive comments first we can discern some themes. Students
really liked having resources available online so that they could cover
things missed in class or study the material for exams (this was especially
the case with Powerpoint presentations - many students commented on the
value of having them available outside of class time). However, one
student did whine about the extra information in Powerpoint notes being
included in the exam (evidently he did not review the presentations online
before the exam). The availablity, ease of use, immediacy and accessibility
of the course on Moodle were positive factors that were mentioned; students
appreciated having all the course material (especially homework assignments)
in one place (centrally
located
, localized
) together with the straightforward
user interface. In particular, the Course Outline display was liked, The
approachable quality of class content online was what i liked.
.
But the user interface did come in for some criticism; design flaws
were mentioned and one student commented about the colours used; why
that heinous pink. it makes me anxious. it looks like errant pepto-bismaly
epidermis
. {Yes, we need to pay some attention to the styling
of our installation - colour scheme and text fonts & sizes}. Ability
to easily upload assignments was mentioned several times as a plus and
availability of assignment grades online was a common theme, eg assignments
- submitting with this system was easy and useful, grades - nice to
be able to check and keep up with status
. Students appreciated
the savings that accrued from being able to access resources online — good
way to cut overhead on course costs
— and they also appreciated
the fast response to questions, the fact that I could ask questions
and ppl would send me answers back faster than if I had tried to call
.
Another pedagogically powerful feature which was liked was the online feedback
to assignments. Finally, for one student the best part of using Moodle
had little directly to do with class; My
favorite part of moodle was definitely the profile option. I found it
so liberating to post a photograph to represent my online persona! I
also enjoyed viewing the "avatars" of
others, not to mention being able to see when the last time everyone
logged on was.
— {I think that this 'social' use of Moodle
is the way to get students 'hooked' into the system. The question would
then become - what features should we look for to encourage the formation
of a dynamic online community?}.
Some negative aspects ascribed to Moodle [Q11:comments]
were a consequence of how teachers used it; thus the complaints
that grades were not kept up or posted at all. One student appreciated
this; would have liked all class notes to be there before during
and after the pertinent class but that was up to the teacher's disgression
(sic)
,
but another complained about The fact that my late assingments
wouldn't be graded.
!! Other students expressed a willingness to
read resources online to conserve paper but were concerned about the
wastage of paper when they had to print out because of eye strain. {Printing
is really an issue that needs to be addressed}.
technical difficulties
proved to be bothersome; some students
did not have the Microsoft Office suite on their machines and so could
not access Word documents, others were annoyed by the
security check my computer has to do through before loading each page
— Internet
Explorer doesn't handle the secure site properly — and the fact that word,
excel & powerpoint files were displayed within Internet Explorer was also
aggravating to some— Word documents should not display when I click
them. They should download, and not be displayed in-line
. {Both of these
issues are a 'feature' of Internet Explorer and the solution is to use Mozilla
Firefox}. Students pointed out some design flaws; the poor design of the gradebook
in terms of display,
there needs to be like an
excel-looking matrix with just the dates
, and the way it works,
inability to know whether or not an assignment has been graded, or the teacher
simply hasnt gotten to it
{a 'hold/release' mechanism would be useful where
faculty could 'hold' the graded assignments and then 'release' them when they
have all been completed. Also, a feedback note on the assignment page with assignment
seen, currently grading, etc}. A major flaw in Moodle is the absence of group
working features, thus one student hated not being able to post things or
communicate with group members easily
and another felt
it would be nice if moodle had drop boxes like chef
. A
frequent theme was a perceived slowness in loading pages or general sluggishness
of the system some of which was a consequence of the teacher uploading large
Acrobat or Powerpoint files which then took a long time to display,
but the Moodle system also did not scale well with displaying large amounts of
information. For example on the Weekly outline for Information Technology and
Society course all users had to switch to 'show individual week' view because
with each week taking between 1 & 2 printed pages the total size was too
large to display on a single page — 'show all weeks' option displayed only
up to half way through week 7. Similarly students were unable to see all their
journal entries because the display showed the whole of the instructor's prompt
as well as their response for each Journal entry— when we view the
journal page it shows the entire journal and feedback instead of just a
link to the journal. It takes a long time to load. If it was a link it
would be better
. Finally, students wanted better integration with Earlham's
systems— no link to moodle from earlham website
— and one
student expressed a desire for a universal login feature which would be supplied
by a 'portal' system My least favorite part of moodle was that I had to log
in every single time. After the first few times, typing in my password got so
tiring that if it weren't for the awesome services moodle provided, I wouldn't
have had the heart to do it.
Question 12 asked students to think of other
ways that Moodle could have been used for their course. A persistent
theme was posting grades and keeping them updated (5 responses) — Maybe
posting our test grades, both letter and percent and having an available
reference to be able to refer back to of what the range of points was
for say an A or B or C etc
— and students were also keen to have more
ways to communicate with each other;
to discuss with other students out of your view. like a student only message
board. but really, viewable to nonstudents.
interactive discussions with T.As!!!!
and
I think it would
have been helpful to have postings from those in the class (questions
or concerns) and then responses from other classmates and/or TAs and
the prof when they had time.
Discussion groups, open chat forums and
'drop boxes' for group work were among the ideas that students came up with.
Despite prior comments about problems with sluggish performance, responses
to question 13 were upbeat [Q13:chart].
A third of students considered Moodle to have 'good enough' response
time, while 40% thought it 'reasonably responsive' and 9% very responsive.
Finally, only 19% of students rated overall response as 'slow'. This
pattern was mirrored in responses to question 14 41%
agreed that Moodle was as responsive from dorm as lab and 25% disagreed
[Q14:chart]. However, the written comments showed
widespread dissatisfaction with internet response from the dorms and
college houses; wireless connections were perceived as being inferior
(i have wireless in a campus house, which never is as responsive as
the landlines the computer labs have.
The campus houses need more reliable internet. I
want to use ethernet again.
) and one student commented on the chronic problem
of students' machines infected with viruses, worms, backdoor apps etc — Traffic
can be seen from infected Windows machines all the time,
and NetSquid seems to not catch them
... Lines are permanently
saturated with activity on all student lines ...
Methinks it is time for ECS to drastically reduce the number of
infected machines.
Clearly if we expect Moodle to be more widely used outside
of class some improvement needs to be made in bandwidth available to students
accessing the Moodle server.
Clearly, the use of Moodle in this pilot period has been a success from
the student's point of view. Despite some negative vibes only one student
expressed opposition to any form of online course system; i did not
like using a system like moodle. not necessaryily moodle spacificaly
but more as a general dislike for such a system.
Moodle has the potential
to become as useful, universal and indispensible as email. Moodle as
an online learning environment has the potential radically to change
teaching at Earlham. However, to make it a success of this venture we
need to pay attention to the context of which the system is a part.
