CHEF Pilot : Faculty Evaluation

All five faculty who participated in the trial (Wes Miller, Jennifer Jayne, Mark Pearson, Amy Mulnix, Monteze Snyder) completed a survey form.

Crude results are presented in the spreadsheet.

• Did you ask students to use this tool?

  

  Schedule, Assignments and Resources were the tools that most faculty asked students to use. Nobody asked them to use News or Help and Web Content and Drop Box were not popular either. Discussion and Chat tools were requested by 3 of 5 faculty.

• Did you employ this tool in managing the course?

  

  The answers to this question took the same form as before with Schedule, Resources and Assignments being the most popular and News and Help not used at all. However, fewer faculty used Discussion and Chat tools in managing their course than had asked students to use them inside the course.

• Did you find this tool pedagogically helpful?

  

  All faculty found the Resources tool pedagogically helpful, but one faculty who had asked students to use the Assignments tool nevertheless did not find that it was helpful. Morever, two out three faculty who asked students to use Discussion and Chat tools also did not find them helpful for teaching their courses. However, the faculty who used web content and drop box did find it helpful to teaching.

6. What was the coolest thing CHEF did for your course?
   “decreased paperwork / let organize resources”, “allowed instant access to extra material” , “made collection of assignments easy”.
   This question was hoping to probe what got the faculty excited about CHEF and it seemed that the course management aspect rather than the teaching possibilites was what faculty thought was “cool”.

7. CHEF as an empowering tool.
   “detailed feedback in the text and their papers; this is a primary reason I wanted CHEF. Papers then returned on line” — this was the sole detailed response.
   The faculty person giving this response discovered that CHEF offered this capability and found it pedagogically empowering. This is a good example of how faculty can start off with simple uses of a CMS and then discover teaching opportunities made possible by the capabilities of the software.

8. & 9 Problems with the CHEF system.
   “schedule didn't work; can't copy assignments from one course to another easily. Having grade sheet in new version does help”.
   “figuring out how to post web resources - this was not obvious”, “not being able to use the 'back' button”
   “easy to 'freeze' files if not careful in storing”. “The lack of interconnectedness between the modules — clicking on an assignment in the schedule tool did not take you to the actual assignment but just to its description. Sometimes there was slow upload of pdf files and erratic response time due to network sluggishness”.
   The problems were of two classes:

   1. Usability.
      Faculty who weren't used to using web based applications (such as web mail) were perplexed by the number of steps neccessary to upload files and were also confused by the non-functioning 'back' button. Clearly, some training in the use of Chef would be needed before roll out. Experienced faculty were exasperated by the lack of basic functionality such as copying courses and not being able to move smoothly from one tool to another.
   2. Reliability.
      There were a number of surprising reliability issues with the system - at one point duplicating more than one assignment at a time was guaranteed to generate a 'horrible exception error' and need cleaning up. These were resolved with bug fix updates to the CHEF system.

10. In terms of managing a course, did CHEF help materially?
    “Somewhat, in terms of helping to encourage students to submit their work in a timely manner”. “Yes. Few handouts, easy access to references.” “Cut down on [student] copying”. “Did help get out handouts.” “Swings and roundabouts. It was good to be able to put up assignments and resources, but the schedule and announcements were a frustrating waste of time [see above].”

11. Can you envision how a CMS (eg CHEF) could be of great use in managing a course?
    “Quizzing and grading would be what I'm looking for.” “Excellent way to get info to students.” “Automatic grade sheet. Ease of submitting and returning assignments.” “Out of classroom discussion holds some possibilities.”
    Currently, a quizzing / assessment tool is a major omission from the CHEF system and the grading tool does not accumulate grades between assignments.

12. Pedagogically, did the use of CHEF have an influence on how you taught the course?
    “No, but had I required participation in the discussion section, it might have.” “Yes.” “I used websites in class more than I might have normally.” “No.” “It allowed students a single point of access to resources and their blogs.”
    Faculty seemed to find CHEF worth the effort and that it improved class pedagogy.

13. Can you envision areas where a CMS (CHEF or other) might change your teaching and allow you to do things differently?
    “Prepare for class based on work they submit before class.” “Potentially all quizzes and exams on a CMS save class time.” “With online autograded quizzing and comprehensive gradebook a CMS would be worth my time.”
    Here we can see faculty getting glimmers of the potential for a CMS to change the way they teach their classes.