Students’ Course Evaluations (Particularly Regarding #EDCMOOC)

On the last face-to-face day of my onground class last week, I asked students to complete the official university course evaluations (which I will receive sometime in the spring), but also my own unofficial evaluation, in the form a Survey Monkey survey with discursive (rather than quantitative) responses.

I asked them some general questions about the class, including:

  • What aspects of the class contributed most to your learning?
  • What would you change about the course and why?
  • Please assess yourself as a student. What could you have done better/more/less?

These three questions were the ones that had responses related to the EDC MOOC. The course lasted 15 weeks, and the first 10 weeks involved non-MOOC work. The last five were raken up by the EDC MOOC.

A total of 33 students completed the survey.

I found the responses to the second question above (what would change about the course and why?) to be most interesting. 22 of the 33 students mentioned the EDC MOOC in their response, though one of them spoke of it postively, and several others spoke in mixed terms (found some positive things about our using EDC MOOC, but would not have included it in the course).

Of the negative comments, I found a few trends:

  • Several students found the EDC MOOC “confusing.” I get the sense that this was because of its open structure, with students having choices about what to focus on, and having few formal requirements from the MOOC instructors, making EDC MOOC more like a cMOOC than the xMOOCs that are more common, and making EDC MOOC unlike their typically college classes. It would be interesting to see how they react to a more xMOOC-like course, with more imposed structure.
  • Several also used the word “confusing,” but in a different way, highlighting the confusion between using the EDC MOOC materials, and my own Course Management System. This seems less an issue with the EDC MOOC itself, and more an issue about how I integrated the two. Integration is, of course, a frequent problem with blended/hybrid MOOCs/distributed flips.
  • A few thought the MOOC was “too large,” which I found interesting, given that they could (and did) personalize it any way they liked. I think this goes back to the “intimidation” I discussed earlier, and some students’ uneasy feeling that their words would be seen by 10,000 people from all over the world.
  • A few thought the MOOC was “extra work.” This was another aspect of the integration problem by me (though one student seemed to equate the MOOC with “extra” work on top of our regular class work, which is strange, because we didn’t really have any additional work once the MOOC started).

Several were also very pleased with the MOOC content, and with the large size.

And interestingly, with the last question I asked (their self-assessment) a number of them admitted that they kind of blew off the MOOC and didn’t take it as seriously as they could/should have. I think this is partly the nature of the MOOC, which stressed independence (and which I also emphasized, stepping back more than I might have in a different course), though this could also be a matter of timing (the last few weeks of the semester put other demands on them, and the MOOC was a convenient place to put in less work). Still, this is ultimately another integration problem, if they were able to put in less work but still manage to meet the requirements.

I have not done much analysis of these students responses, other than the above. I should also note that they were purposely open-ended, and I do not have any quantitative evaluations of the MOOC aspect of the course. But I think the responses are valuable. I have included them below:

What aspects of the class contributed most to your learning?

[5 MOOC-related responses]

  •  The weekly detailed instruction helped. Blackboard was exquisitely organized and the MOOC was “the cherry on top” of this learning to bring it all together and use the information I learned.
  •  in a way, the mooc videos contributed the most helpful and fun part of the class, even though i would not reccomend MOOC to anyone due to the confusion.
  • I really liked the MOOC and that the class was mainly conducted online it allowed me to be responsible for my own learning with the guidance of the teacher
  • I think that the group discussions contributed to a lot of learning but being able to socialize and communicate with a vast amount of people on the MOOC was a really big contribution to learning.
  • The online mooc assignments with the youtube videos.

 What would you change about the course and why?

[22 MOOC-related responses. 1 was positive, several were mixed positive and negative]

  •  I would change the EDCMOOC learning, because I found it to be boring and confusing
  • I would take out the Mooc, as cool as some of the videos were it was very difficult to operate around the site and was very annoying using both.
  • If I have to change one thing about the course, I would not do the ED-MOOC. I thought some of the material in that portion was a bit repetitive, and I thought it was a little confusing having to use website in addition to using blackboard to post assignments as well. I liked the information that the MOOC provided, but I think that it would be just as effective to learn it and not have to use the MOOC system.
  • I did not like participating in the MOOC. I did not find it enjoyable and thought I would have gotten more out of doing other activities and discussing more in class. While I enjoyed the videos in the MOOC, I think it was simply too large.
  • I would not change anything about the course except for the inclusion of the EDC MOOC into the curriculum. The MOOC did not really help in my learning of any of the course material, and although the videos and content we explored online was beneficial and interesting, I think we could have learned more without the inclusion of the MOOC.
  • It was a well orchestrated course that I would not change. Balanced well and the MOOC was perfectly set.
  • I would take MOOC away, I did not really take anything away from doing this and it did not help me as much as our normal class lectures did because it was confusing.
  • I would change the amount of time for the MOOC and/or take the MOOC out. I was not a fan.
  • EMOOC wasnt my forte. The videos had a very interesting concept but everything seemed so difficult to comment.
  • the MOOC i think it was i bit confusing and felt a bit weird taking a class with so many people
  • NO MOOC.
  • I wouldnt use Mooc, i thought how you had run the course through blackboard was more than enough, and you can take the videos from mooc, or similar and make your own type of course, i feel that MOOC is extra, and to be honest a hassle.
  • Honestly, I was no a fan of the MOOC.
  • I would not do the EDCMOOC again. It was too confusing, but I did learn a lot about technology.
  • Perhaps not doing the MOOC as it is very confusing to navagate.
  • I would probably change the MOOC because it was really confusing.
  • As interesting as it was, I would eliminate the MOOC from the course. I understand that it had to do with the values part of the course, but it kind of took away from what we did prior to it.
  • Next time, I would not use MOOC in the course because it took away from how businesses use social media. It focused more on how we as humans view social media, but I feel like we see these changes in our everyday lives.
  • The only thing that I might have changed in the course if having to do EDCMOOC. Although, I did learn new things and it was interesting, I did not think it was necesssary to do.
  • Final project, have edcmooc be the final project only, no additional.
  • I would change the lectures and probably scrap the online Mooc class. I did [not?] find the Mooc class interesting or worth the time.
  • the MOOC was unnecessary, too much extra work and really did not contribute to learning. Was kind of a waste of time.

 Please assess yourself as a student. What could you have done better/more/less?

[9 MOOC-related responses]

  • I think that I could have put a little more thought and effort into some of my discussion posts. All together I think I did a pretty good job, but some of them I have to admit I slacked off on a little and did not put in my full effort. In the MOOC, i should have strayed away from commenting on only the YouTube videos (in addition to the discussion posts on the forum), but it was the only thing I was comfortable with. Other than that, I would like to think that I did a good job in this class; I always had my posts done on time with over the minimum word count and I tried to be the best student that I could possibly be in all of my work.
  • I think I did a fair amount of work for the class. I could have focused more time on the MOOC to maybe get more out of the experience.
  • Overall I think I did okay in this class. I worked really hard on the blog posts and group projects, but I could have participated more in the MOOC and the MOOC Final Project. With online classes I don’t seem to really take them seriously and with 10,000 students in the MOOC who’s really look at my stuff.
  • I could have done more with the MOOC and got really involved but it really just didnt catch my attention. Other than that i believe i did well in class.
  • I could have spent more time with the MOOC instead of dismissing it.
  • I could have been a little better with watching every video on MOOC to its extent I somewhat skimmed through videos to get the important pieces out and may have missed information and for the classes discussion boards I could have contributed to conversation or started conversations by commenting on other students posts.
  • As a student I could have read more comments and discussion forums on the MOOC website to get a better understanding of what each video was trying to say. Sometimes I would watch a video two or three times and still be unsure of the message.
  • could have read and participated more in Edc mooc.
  • I think I am a very good student I am just spread thin as far as my workload with all my classes. Adding on a MOOC was unnecessary

The Role of the Teacher in a Blended MOOC #EDCMOOC

As I am wrapping up my experience with the EDC MOOC, and considering how it will affect the way I approach blended MOOCs in the future (and I do intend to try this again), I have been thinking about the role that the teacher plays in such a course. It’s been a question that I have been considering since before the EDC MOOC even began.

Now, it’s important to point out that the EDC MOOC is unlike any other MOOC that I have participated in. I’ve completed several, and started far more without completing them, taking courses from Coursera, Canvas Network, Open2Study, and Alison. Each MOOC provider has its own approach and mission, and each individual course/instructor has its own objectives and methods, but pretty much all of them have involved some degree of the “sage on the stage” approach. That is, there is some kind of direct instruction from the professor(s) of the course, typically in the form of video lecture (though sometimes through text, or some combination of the two).

EDC MOOC is definitely not like that. There is no real lecturing, no easily-followable structure, no easy guide to doing what needs to be done in order to learn what you are supposed to learn. I, personally, find all of that a strength. My students, however, were less enamored, and probably would have preferred a more traditionally-structured course. My point is, EDC MOOC is not necessarily a model MOOC, at least for the kind of blended MOOCs I will probably construct from here on. And so the role I play as a teacher in future blended MOOCs will not be the role I played here.

That said, the more open structure of the EDC MOOC did offer me a good sense of some of the tasks I need to focus on when I do try this again, precisely because it was so different.

So here are my thoughts on the role I have to play in a blended MOOC:

1. I have to be a Curator

As I discussed in my last post, others who have attempted Blended MOOCs have had problems integrating the MOOC’s content with their own course objectives. I had a slightly easier time that most, because of the more open nature of the EDC MOOC and the fairly open nature of the unit that I was attempting to integrate the MOOC into.

But it showed me that as a teacher of a blended MOOC, I have to make my own objectives the priority. I have to be able to distinguish what fits and what doesn’t, and guide my students (require them, really) to read, watch, write, and discuss certain things but not others. They will likely miss out on some things by my doing this. So I can encourage them to read, watch, write, and discuss those things that look interesting beyond what is required. I am realistic enough to know that most are unlikely to do this, but I can encourage it anyway.

My own objectives must rule, however. The MOOC is one of several tools that I will use; it is unlikely to be the only tool. And just like assigning certain textbook chapters, but not others, I can ask students to focus on certain aspects of the MOOC, but not all.

2. I have to be a Team Teacher

Being a Curator will likely be easier in a course with the EDC MOOC’s open structure. But not all MOOCs (in my experience) have such a structure, and some build on its content week by week. In other words, I can’t necessarily have students do weeks 1 and 2 of a MOOC, but skip week 3, because if we did, they’d be lost in week 4. If that turns out to be the case, then my job is to be a Team Teacher.

Let me explain: I have had the opportunity to team-teach several courses in my career, where I have worked with one or more instructors in teaching the course. I was warned early on that, if I am team-teaching with someone else, my work is not cut in half. On the contrary, if I’m doing it right, my work will be doubled. This is because I will need to know my own contribution to the course, but also know my partner’s contribution well enough to challenge it, or at least offer a different perspective on it. The strength of team-teaching comes from its dialogic nature, and having students see the push-and-pull of ideas from two or more people.

And so it would be in a MOOC. If I found that I was unable to avoid it, I would try to take extra care to engage with it. I would do this anyway, no matter what part of the MOOC we were dealing with. But if it was a part that I would have avoided anyway, I think I would highlight the avoidance, and let students know why I would have not included it.

This is something I didn’t do in the EDC MOOC. Each week, I required students to watch and read certain things, but not others. I think i should have talked more about why. The reasons were typically practical — just not enough time to read and watch and discuss everything — but especially for things I avoided because of the content, I think I should have addressed them more directly. That would have been instructive for them to see some dialogue between my ideas and the ones presented.

(I’m still working through this idea.)

3. I have to be a Participant-Observer

One thing that has frustrated many people who have attempted blended MOOCs is the lack of access to material, assuming they are teaching in real-time. Like everyone else who signs up for the course, the instructor of the Blended MOOC gets access to the material on Day 1, and often does not get access to subsequent weeks’ material until the first day of that week. This makes it very difficult to plan ahead. Again, I was lucky with the EDC MOOC, having signed up for (but not finishing) the first iteration of the course, and having access to the entire course from the first day.

But both of those situations are rare. More likely, I will start the MOOC on the same day as my students. This could be very frustrating for many teachers. It takes extra time to be even a half a step ahead of students, which will be necessary if I want to be a Curator and a Team Teacher, deciding which parts are important and which are not.

Still, I think this weakness could be a major strength in the right hands. Seeing me struggle with the subject matter in real time could be a great model for students, and they see my own though process.

I think of this as similar to the kind of participant-observation that I did years ago in qualitative research, becoming part of the group I was studying, but also being just a little apart from it. In a blended MOOC classroom, I would be a student, signed up for the MOOC, seeing things from the student’s perspective — a participant. But at the same time, I am still a teacher, removed just a little from the situation — an observer, with enough knowledge and skill to be able to see what is going on with  my students and help out.

A participant-observer is never fully part of the group she studies, always just a little bit apart, and a teacher in a blended MOOC will never be fully a student, always in a position of power, always able to decide what to curate (in fact, always the one who decided to use a MOOC in the first place), but also always able to see things, at least a little bit, from a students’ perspective.

4. I have to be a Learning Specialist

As I mentioned in an earlier post, I think it’s important for me to teach my students how to learn. MOOCs in general, and especially the EDC MOOC, are very different from the kind of classes they typically take, even those that have taken hybrid/blended or fully-online courses. They require different skills, and I don’t know if students always have them. As great as many of them are with technology, they are often First-Page-of-Google-Results people. They want their information quick, they want it easy, and they get frustrated if they don’t get it that way.

The EDC MOOC did not make for quick-and-easy learning. It required a lot of independence, a lot of sifting and searching, and it rewarded people who were willing to be that way. Not everyone was willing to be that way, including my students.

That’s where my participant-observation, my real-time teacher/student role, comes in — by modeling for them how to learn. As a teacher of my own course, I can explain how to learn. But I can’t really model. In my own course, I’m always the teacher. In a blended MOOC, at least part of the time, I am also a student. I can show them how to find things, where to look, how to read and analyze, how to be frustrated and work through it, where to go for help.

This is, I admit, not an easy thing to do, especially when taking so much time just to learn the content of the course. there’s an extra step to explaining it all, too. But I think, in the end, it’s probably worth it.

I don’t think I did this as well as I could have in the EDC MOOC. I think that was partly because of the nature of the course. I wanted my students to experience that openness on their own, and to do the searching and sifting. They did, but only as much as they were required to. Which is understandable. In the future, I’ll be more of a presence, especially in the modelling of how to learn.

These are my initial thoughts on the role of the teacher in a blended MOOC. I don’t think I could do all four in a single MOOC experience, because some of those roles are going to contradict one another. But they’ll serve as a good guide for me the next time I do this. And there will be a next time.

I still have more thoughts on my EDC MOOC experience. They are on the way soon.

#EDCMOOC: Distributed Flips and Blended MOOCs

It’s been a couple of weeks since I posted (Thanksgiving holiday and end-of-semester craziness), but I’ve been thinking a lot about comments made during the Week 3 Google Hangout about Blended MOOCs and Distributed Flips. I am calling my experiment a Blended MOOC; Jen referred to work by Mike Caufield and Amy Collier with their term, “Distributed Flip.” They are similar terms — using all or part of a MOOC (or other online resources, since DF is a  broader practice involving resources in addition to MOOCs) in an onground or face-to-face class.

Using the MOOC in this way certainly has some challenges. Caufield outlines a lot of them in this post from his Hapgood blog; I’ve experienced some of them myself.

They include the difficulties of matching up schedules: a MOOC will not always run on the same schedule as an onground class. This is certainly the case, and I was very fortunate to have the EDC MOOC run when it did, coinciding almost exactly with the last five weeks of my own class. My class ends next Wednesday, the same day the Peer reviews are done. The final exam for my class is the following Monday, just in time for a final reflective essay on their MOOC experience. As I said, I am fortunate that it worked out so well. Looking at the lists of Coursera courses, there aren’t many that offered the same experience of timing, as well as relative brevity (five weeks allowed me to teach 10 weeks of my own material, building up to the MOOC experience).

Another issue brought up by Caufield: mismatch of course content. As higher education instructors, we have historically had much control over our own course’s content; bringing in someone else’s content can create challenges. Hamish pints out in the Hangout that textbooks get used often enough without complaint, though I would suggest that even that involves some choice, both in the textbook itself and in the parts of the book that get assigned. A MOOC is a much larger structure within the class, with video lectures (often), readings, discussions, and other materials. Assigning selected chapters of a textbook is easier than assigning selected units of a MOOC. There is too much material integrated into the MOOC units to separate them out. The instructor needs to decide to give up much more control over content when using a MOOC.

Again, I was fortunate with the EDC MOOC. The synching of content wasn’t perfect, but I am teaching a class that allows some flexibility — enough that I could use the MOOC to meet my course objectives. My course is a capstone for our general education program. The first tier of the program teaches skills: writing, quantitative reasoning, languages, technical skills, etc. The second tier involves application of those skills in exploring thematic areas. Think “applying” and “analyzing” in Bloom’s Taxonomy. My course, as a third tier capstone, involves taking all of the knowledge from those first two tiers and using it to explore a particular list of values. In a way, this is the kind of “Evaluating” that takes place in Bloom (along with some “Creating,” in other parts of the course). The EDC MOOC, with its discussion of the role of technology in our past, present, and future lives, fit well with the values discussions, as students were able to look critically at the way technological change affected both their own individual lives and the culture(s) they exist in.

A final challenge mentioned by Caufield: MOOCs rely on Discussions, and students don’t often engage in them when they are involved in a Blended MOOC/Distributed Flip. Caufield and a couple of colleagues asked some instructors who had tried the BM/DF, and found that the median number of Forum visits (not posts, but visits) was two per student. (That matches up with my own experience as a student in MOOCs, apart from this experience as a teacher: Discussion Forums tend to populated by a small but committed group.)

I dealt with this issue in my own class in a simple way: I required them to be part of discussions, and to cut and paste their posts in our class’s CMS so I could see that they had actually done it. They were required to post twice per week, and had the choice to post to an EDC MOOC Discussion Forum, the #EDCMOOC Twitter stream, the (unofficial)  EDCMOOC Facebook page, the Google Plus page attached to the Hangouts, in YouTube comments (tagged), or as part of a blog they wrote themselves. As Caufield notes, the “massive” aspect of a MOOC is a distinguishing feature that makes a MOOC different from other digital resources. I didn’t want the opportunity for my students to discuss issues with people from all over the world to go to waste.

To be honest, I had mixed success with this. Students did meet the requirement, and posted twice each week. But many were simply meeting the requirement, posting something along the lines of “Wow, this video was really strange, but it says a lot about technology and how we use it.”

But many went deeper, especially as the weeks went on. Their thoughts on the videos and readings were more complex; they tried different venues; they joined conversations, and started some of their own; they generally took more chances, acted less intimidated, and used the MOOC the way it was intended. Not all of them, to be sure, but I can say most of them did.

Again, I was lucky in this respect — the EDC MOOC lended itself to discussion. If I was teaching a different type of course, with different subject matter, they may very well have turned to me with questions, rather than asking their MOOCmates in India or Spain.

I have so much more to say about this experience: about the ultimate success of my Blended MOOC experiment; about my role as a teacher; about the kind of “repurposing” that Hamish, Jeremy, and Jen discussed in the Hangout; about my future plans for using Blended MOOCs (I’m staying with that term, with all due respect to Any Collier’s “Distributed Flip”). I plan to address them here soon.