Catholic Distance Learning Network - Register for a Course

Instructional Designers

As part of our leadership initiative, our instructional designers will post a new thought on assisting the process of instructional design each day throughout the course. For the sake of shortening this page, the instructional design page for June 2007 has been moved.

Thursday, July 26, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


		For my very last post of the online course on online teaching and learning, I'd like to simply thank the eleven participating faculty and seven instructional designers (not including myself) who involved themselves with this project over the summer of 2007. I'll be in a small cabin on a lake in Estes Park, Colorado, for the remainder of the summer. Alas, it will be the first time in almost a decade that I'll find myself without Internet access for so many days. It's been most edifying working with the course professor, Jim Rafferty, and with my colleagues at other seminaries and out of the various technology departments in the schools of education from which our instructional designers came. God's blessings to you all, and don't forget to check out the Catholic Distance Learning Network's August 2007 Workshop hosted by Sherry Kennedy Brownrigg, beginning on Monday, August 6.

Wednesday, July 25, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


This penultimate post will consider a final aspect of peer review, which, of course, like all these others we've considered these past eight weeks, is a topic not exhausted by the posts I've provided already. Peer review, when we speak of it in academic circles, is a process that involves one's work being scrutinized by equals within a discipline.		Courtesy of Nick D. Kim (reposted from Arizona State University's Haydel Lab)
When we submit our materials to vetted journals, for instance, we expect to receive our reviewers comments as a helpful means by which to strengthen our work for eventual, it is to be hoped, publication. The same holds true for when our colleagues review our course offerings and provide us with meaningful feedback, if not from a disciplinary vantage point as we'd find within the professional organizations to which we belong, then from an interdisciplinary one, such as we find within the Catholic Distance Learning Network. We are a community of scholars who are also teachers, and it is from our common pedagogical interests that we find our epistemological authority in the reviewing of one another's course syllabi. Our students, by contrast, don't have the same advantage in their reviewing of one another as we do, for the simple reason that they're not yet experts within their disciplines or within the meta-professional arenas in which they're being asked to evaluate one another. Student peer review, for that reason, needs continuous guidance and support, and it may seem, as a result, that it's more trouble than it's worth. Never let that discourage you, however, as you're ultimately developing within your students the skills necessary for them to move from knowledge to evaluation on Bloom's chart.

Tuesday, July 24, 2007 -- Dr. Sebastian Mahfood (share your thoughts)

Reviewing all the Criteria to make a Successful Launch. Courtesy of NASA.

As we engage in peer review of one another's projects, let's try to systematize the process a bit since the only thing left to us at this point in the semester is a summative evaluation. First thing we want to do is give our reviewers the criteria by which we would like to be reviewed. There are, naturally, a few objective criteria -
1) our course goals,
2) our activities (including lecture material, readings, and accountability exercises) in relation to our course goals, &
3) our assessment and evaluation standards for our activities.

There are also subjective criteria, which could include project clarity (what seems quite clear to one might not seem clear to another), project doability (which answers the question of which level of prerequisites is essential for a person to engage a given project as a starting point in your course - for instance, we often assign research papers and grade students on their composition rather than on their content, the natural prerequisite being a facility in writing, not just in writing within the discipline; likewise, a project in ethics would presuppose some understanding of moral philosophy or theology as would a project in fundamental theology presuppose some understanding of historical philosophy, etc. Project doability, then, asks whether the implicit prerequisites can actually be presupposed on the knowledge and skills level as well as on the practical applications level.), and the like. In short, we'd provide our reviewers with a checklist of things that are important to us and ask them to evaluate the list on a Likert agreement scale concerning whether we adequately meet our own goals (like the scale that Jim's been providing each week). If we can provide such a tool in each of our discussion fora, then all that's left for us to do is to consider the feedback we receive as potential formative materials that can assist in the design of our project prior to our students' enrollment in the course. There's also the feedback we can give to others who are ready to receive it. Comb through the projects that are posted and see what you might do to help their authors improve upon their design - even to the point of proposing additional criteria by which you're able to measure them.

Monday, July 23, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


If you scroll down to the post of July 9, 2007, you'll see Kenrick's four-fold assessment strategy developed through attention to formative, continuous, summative, and recursive processes. Peer review can benefit from an adaptation on this model. Left to our own devices, most peer review exercises we assign will be summative in nature.		Courtesy of Nick D. Kim (reposted from Arizona State University's Haydel Lab)
Students will complete their written assignments or activities and share them with their classmates for the sake of polishing. While there are quite a number of advantages to this method, not the least of which being that it ensures a student reads his own work through the eyes of another person before submitting it for our review, use of summative peer review alone is limited by its being product- rather than process-oriented. Insertion of the formative and continuous methods of peer review - which entail students being actively involved in reading one another's proposals and abstracts and engaging at significant points along the way in focused review of particular sections of a colleagues work make of peer-review a transactive rather than transmissive operation. There is little room in a given project or activity for recursive peer review - if only for the reason that a number of data points concerning a student's work need to be collected before original opinions can be developed - but it may be possible to meaningfully apply the method of revisiting comments made incipiently in light of later development; otherwise, I'd try to incorporate recursive review into the eportfolio evaluations where students can examine the comments made throughout the duration of the class in light of the progress a given student has made. This method is most soundly applied, of course, in peer review over time where, for instance, students move through a four-year program of study in stable cohorts, something that isn't found in discrete online courses like the ones the CDLN will be offering.

Sunday, July 22, 2007 -- Dr. Sebastian Mahfood (share your thoughts)

"The Death of Socrates" by Jacques-Louis David (1787)

Before moving further into the methods for graded peer review, I'll write a little on what I call 'natural review.' Within any given class, students are going to develop affinities for one another based on shared interests and capacities. Early in the semester, especially if they're uncertain about a professor's grading criteria, some or all of them might form study groups on their own in order to reinforce one another's learning.

I can give an example from a course of study in which I'm presently involved, the online Masters of Arts in Philosophy program out of Holy Apostles College & Seminary (Cynthia Toolin's institution) in Cromwell, CT. A group of us students in the fall of 2006 thought that a student support structure would be a meaningful addition to the program and so formed the Dead Philosophers Society as a way to share our work with one another and help each other prepare for the final exams each of the courses requires. This summer, I'm taking two courses, Logic and Natural Law, and recruited two of my fellow logic students into the Society. The small group of three tries to meet every Sunday night for an hour where we discuss the exercises we've posted throughout the week and work through any differences in opinion. If a teacher notices groups like this forming on their own, he or she should encourage (even promote) their development throughout the class, finding ways in which to incorporate their structures into the course (e.g., each study group leads a panel discussion on a given lesson). By plugging these natural groups into the life of the class, the teacher solves one of the group formation problems found in the science of grouping we discussed earlier.

Saturday, July 21, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


Because the peer review process is such a helpful tool in both assessment and evaluation, it is something that could be studied in context with every topic we've covered over the past seven weeks of this course. We have, in fact, encouraged you to review the work of your classmates in their project fora, but we've never articulated a systematic process by which you were expected to go about doing it. My thoughts over the coming week will concern themselves with what peer review actually entails and what it ultimately offers to the course or program that meaningfully engages it. In the previous post, I pointed to the problem of having only a single evaluator/ assessor in the person of the teacher responsible for having designed and implemented the course. The advantage to having multiple opinions is, relative to the problem of having only one, rather obvious, but only if those opinions are rightly formed based on the
criteria established and promulgated through a well-designed rubric that indicates to the reviewer what he or she should be looking for in the evaluation of a student's eportfolio or in the assessment of a course of study. Without a standard upon which to base their evaluations, the student peers are just as likely to score a project excellent as they are to score it below average based on personal caprice. The same is true, moreover, for your faculty peers who might look askance at what is coming out of your class as somewhere between excellence in teaching and learning and the pedagogical equivalent of herding cats. The short lesson for today, then, is that of peer review for purposes of student evaluation rather than for purposes of course assessment, and I'll state it briefly so as to prepare the way for the final handful of thoughts - assign no more than two accountability partners to improve the quality of each peer review and give the reviewers a grade for reviewing based on its own rubric of expectations.

Friday, July 20, 2007 -- Dr. Sebastian Mahfood (share your thoughts)

	Student accomplishment of the eportfolio is measured in terms of growth as evidenced by the reflections and syntheses integrated throughout the semester into the final product. A danger exists in student evaluation in that determinations of growth via eportfolios are often qualitative instead of quantitative, meaning there is a likelihood the teacher will exercise a bias towards his or her personal preferences or tastes in the evaluation of any given eportfolio. I, for instance, have an appreciation for aesthetics in eportfolio design and would be predisposed to see greater evidence of a student's growth through the packaging of the eportfolio though I know to be aware that what I'm really after is content and capacity.
A second danger exists in course assessment as teacher bias might cause an unintentional misinterpretation of the qualitative data in a way that demonstrates the success of the course to meet its own goals and the program goals. This usually happens when the teacher-qua-assessor skips the process of problematizing those areas that appear successful to determine ways in which to strengthen the course the next time it is taught and justifies those areas that appear less successful on the basis of student inexperience rather than as a result of flaws in course design. Ways to minimize the margin of error in the evaluation of student growth (and in the assessment of course effectiveness -- again, two different things) can be found in the development of objectively ordered rubrics (to which the students will have access at the beginning of the project) and in the formation of a peer-review accountability group (in one instance, on the part of the students who have taken the course who evaluate one another according to the rubric, and in another instance, on the part of disciplinary or interdisciplinary colleagues on the faculty level who can do likewise). The rubric should be applied at least three times by the responsible groups during the course of the semester in order to get enough data points to identify trends in student work. In developing this process-based measurement rubric, the teacher has to be clear about the kinds of areas he or she wants to measure to determine the capacity of the learners to adequately engage the teaching and learning environment, for instance knowledge of the curriculum (which entails a demonstration of skills in analysis and synthesis), the ability to work in groups (which entails social responsibility), meaningful engagement on a discussion board, and the like. Students will know what a teacher values by what a teacher endeavors to measure. As long as the measurement scales are consistently applied across the diversity of student work within any given class, there will be sufficient accuracy in the process to make it worth the investment in time.

Thursday, July 19, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


The elements of an eportfolio should be outlined on the assessment rubric the students receive as part of their instructions on eportfolio development. Since it is likely that the student's first attempt at eportfolio development will be in the course that you are teaching, it is necessary that you provide an easy checklist of what it is you expect the student to include (not necessarily accomplish, which we'll discuss in tomorrow's post) as part of the package he or she is designing. The essential elements of the eportfolio, because the eportfolio is direct assessment material and tied to the course and program goals, include but are not limited to the following:
Statement of Purpose - this is a formative reflection piece concerning how the student interprets his or her motivation in taking the course with the mission statement of the course itself as outlined on the syllabus. This is generally written the first week of class and will be part of the introductory webpage. Summary of activities - this is a list of activities in which the student was engaged during the life of the course, and it can include class work (e.g., the student's involvement with online lectures as it led to discussion and participation in assigned activities), group work on specific projects, engagement in online exams, development of research papers, and the like. This summary is like a table of contents and should include very little reflective description on the work done as that will be taken care of in the bulk of the eportfolio, which is the next item. Assignments - these are the actual assignments the student has completed during his or her time in the course, and they should have the instructor's comments attached along with a short response from the student to each of those comments. Each assignment section should also include a short reflection on the specific connection between the student's work on the assignment and the course goals with which the assignment was associated. Conclusion - this is a summative reflection piece concerning how and in what spirit the student has completed the course. It should be written with the formative reflection piece in mind and is the space in which the student can express what he or she feels were the strengths of the course and areas in which he or she feels the course could have better expressed the student's motivation in having taken it. Students should conclude this reflection with the impact the course has had on them and what they might do in the future with what the course has provided them.

Wednesday, July 18, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


Arabella - space spider Courtesy of NASA		The eportfolio for a given course of study, if it involves only that course of study, will necessarily exclude materials outside of that course of study. If we merely want our students to package their work for the course we're teaching, then there are a few meaningful ways to make that happen. The best, because it provides universal access, involves the student's developing his or her materials on the web and the teacher's collecting the links to each student's home page on a single 'splash' page accessible through the course template. We've seen lots of examples of that already - of splash pages that merely house a collection of links to student pages. If students are not familiar with web-building, of course, they can build their materials in online areas that make it easy for them to create a presence on the web -- places like MySpace, FaceBook, or MoTime.
A less useful way to develop the online portfolio is for the student to send all the materials to the teacher for the teacher to upload them (which puts the burden for updating on the teacher rather than the student). A third way, not as effective as the first but far better than the second, is to use the course template and provide each student with his or her own forum in the discussion area. That forum will become the house for the student's eportfolio and can be synched with the student pages area so that fellow students can visit each other's personal page and be directed from there into the student's discussion forum. Beyond these three methods, the eportfolio starts to become less dynamic - for instance, the student could just email a .zip file containing all his or her contents to the other students in the class, but that would necessitate the student's sending email updates of his or her progress to everyone (either in plenary or in small groups) throughout the semester. Finally, the idea of the eportfolio could be jettisoned altogether in a face-to-face classroom in favor of the physical version, which a student actually brings to class to share with others during groupwork and hands over to the teacher for periodic review and evaluation.

Tuesday, July 17, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


If we want our students to be able to create portfolios of their work, it is meaningful for us to model that activity in the publication of our own portfolios. Our portfolios, of course, are qualitatively different from the portfolios that students will produce for their courses or their programs because ours are comprehensive curriculum vitae inclusive of the professional work we feel defines our vocations as teachers. In a sense, then, our C.V.'s are extra-curricular and extra-programmatic, extending in scope beyond a single project goal or institutional vision.
My online portfolio, for instance, contains not only my summary of activities and publications, but also examples of written work during my doctoral program and links to the syllabi for the classes I taught as an adjunct at various community colleges during my early years in St. Louis. Were I to update this portfolio, I'd collect within it representative samples of my publications and materials from the current classes I teach. To make it an authentic portfolio, moreover, I'd include reflections on my work and develop an upgradeable vocational mission statement along with, perhaps, a regular blog or vlog journal (like the page I've been creating here for the past six weeks) that explains evolutions in my understanding of my continuing role as a teacher and a learner. In brief, then, if we're beginning work on the development of a website, we're already starting in on the development of our own eportfolio. If we remain conscious of our own goals of self-representation within and beyond the institutions to which we're attached, we'll be able to pass that consciousness along to our students who are trying to define themselves through their own eportfolios in relation to the institution (and later beyond) to which each is attached.

Monday, July 16, 2007 -- Dr. Sebastian Mahfood (share your thoughts)

Courtesy of Dr. Helen Barrett's site on developing e-portfolios.

As explained in the introduction to student portfolios, each portfolio constitutes a collection of student work over time. The value of portfolios is that they can be used to directly assess whether a course or program is doing what it claims to be doing in the real lives of its greatest constituency - the students. Aside from having the students collate their materials for you, you're asking them to reflect on the value of their work to their own learning goals and to recognize, as it were, the connections between their course or program activities and the course or program goals.

One way in which we have addressed this need at Kenrick has been in the development, first, of sacred scripture portfolios, and, second, of portfolios that engage the learning experiences in other discrete courses. In the fall of 2003, we incorporated into our strategic planning the idea to develop a comprehensive portfolio requirement in cyberspace. This is a feasible goal with 100% of the student body trained on web-development, but we are still in the process of systematizing the structure so that it runs programmatically rather than discretely. To accomplish this goal, we needed to develop a summative evaluation process and tie the portfolio structure into a special seminar alongside the structure such a seminar would have already developed for masters theses and comprehensive exams. Since developing the summative evaluation seminar with the third year theology group in the fall of 2003 (a seminar which produced its first fruits in the ordination class of 2005), we have achieved a 98% completion rate for the masters of arts degree, which for the past three years has been given concurrently with the masters of divinity. The challenge for us now will be to subsume this process within a comprehensive collection of student works. Educause, the same company that approves all .edu websites, has written more on this point in a summer 2004 article entitled "The 'Sticky e-Portfolio System: Tackling Challenges and Identifying Attributes."

Sunday, July 15, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


Our grading individually designed projects and journal postings is easier than our grading collaborative work for the reason that we have to express our evaluatory justice not only to the group but also to each of the individuals within the group. As Antonio Gramsci has taught us, in every group there are intellectuals, opinion leaders to whom others in the group look up for advice. The corollary to this principle is that in every
group, there are also dullards and slackers, opinion laggards to whom others in the group simply don't look for advice. The various reasons for this phenomenon are as diverse as the reasons for opinion leadership, but if any given small group might have at least one of these folks, then that person generally likes group work as a way to minimize his or her own efforts on course activity and still receive high marks for the slack his or her teammates are willing to pull in order to keep up their own marks. Sometimes, of course, the person on the team who dissociates him- or herself from the others is the brightest who simply doesn't want to work at the pace of the slower group members or finds in the direction they want to take an impractical route that he or she would not have taken. In short, even teams that demonstrate functionality by their ability to muscle their way through assignment dates might be comprised of members who deserve a grade less than that being given to the finished product. To be just, then, the teacher ought to rely on a multi-tiered evaluation system - with the whole grade given for the assignment being an average of a group grade and an individual grade. One way to do this is to assign process grades that culminate in a product grade (e.g., grading smaller parts of a larger project - part A is worth 10% of the total, part B 25%, etc.). For each part the students do in common, they have to demonstrate evidence of collaborative activity via a log on their activities (and a percentage of the final grade -- say 10%, just enough to knock a person down a letter grade -- can be based on peer-evaluations; that is, the aggregate grade each student gives to another within his or her group is factored into the total project grade), and they have to submit individual papers based on their group work. Dr. Anne Marie Kitz's prophetic literature students, for instance, work in groups to create a common project and presentation, but each member of the group is responsible for his or her own 10-page analysis, a method I used in my fall 2006 graduate writing seminar where the students in the class were divided into four groups for the purpose of developing their materials and then had to submit individual 3-page papers.

Saturday, July 14, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


		Aside from the two or three accountability exercises in which a student will be engaged over the duration of a given course, the unique feature of the online course is the discussion board and the way dialogue develops within a forum and across a field of fora. While what has concerned us previously has been the idea of assessing whether the forum itself is working to achieve the ends for which it was created, what concerns us now is the idea of assigning some kind of raw score to the quality of student postings. In addressing how we measure whether one student's contribution is an A or a B or worse, we have to go back to the idea of the rubric. We likely already have a standard that we use to determine a student's grade for written work, but we just as likely expect students to know the difference between an A and a B in the written work they submit (as evidenced by the lack of rubrics on course syllabi in schools of higher education).
Whatever standard we use, we have to be conscious of the fact that a discussion board is a collaborative event, not merely a place for students to turn in their assignments. For that reason, it's not just the content a student posts in response to a given prompt that has to be evaluated, but also the frequency and quality of that student's interaction with other students on the discussion board. In the cyberethics class I teach, my students receive up to 15 points for each weekly reflection they post, up to 10 points for each weekly activity they choose (they can choose for credit up to three of the nine or ten I post in a week), and up to 3 points for each response they make to another student (they can respond for credit to up to five different students). I provide them with a quick-glance rubric that encourages them to demonstrate their 'critical thinking' skills (thanks, Eileen, for having already shared with the group what this concept means) by emphasizing exploration, analysis, and application in both their original postings and in their responses to others.

Friday, July 13, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


After promulgating the bases of evaluation, best articulated through a rubric designed for that purpose, the teacher usually has no further input into the evaluation process until the final product is received. If we follow the four-fold assessment process for activity design (whatever the activity may be), we can find value in it for the way in which we evaluate students performing those activities. To do this, we need to shift from product-based activities to process-based activities, or activities that allow for multiple points of evaluation through the duration of their existence within the teaching and learning community.
An example of process-based evaluation can be found in the way in which we engage student writing assignments, from short reflection papers to masters theses. During the spring of 2000, I had the opportunity to teach a developmental writing class at St. Louis Community College and led my students through a graduated-writing project on the state of technology in the United States. As the students developed their paragraphs, they were given the opportunity to peer-review one another and receive immediate feedback from me. At the end of their first draft, I'd 'enter' their text and provide a comprehensive sentence-by-sentence evaluation on each of their paragraphs. They'd then receive a temporary grade with the option of revision for up to a letter grade higher. In Fr. John Paul Heil's and Dr. Anne Marie Kitz's sacred scripture classes, the idea was modified to provide a graduated research project with the opportunity for peer review and faculty collaboration throughout a process characterized by milestones of achievement. My adult learning & technology class turns in a portion of its work each week for peer review and teacher evaluation. Finally, but far from the last of my examples on things like this, my cyberethics class is required to write a three-page analysis of a particular cyberethical issue of their choosing, but they have to do so in five parts - each part of which is subject to review and revision by me and their classmates prior to its culminating in a finished product ready for a process-based presentation in which each student takes the role of discussion leader to walk his or her presentation group through a paper they've already read using a three-fold process of explaining why he or she pursued the topic, what he or she learned from the pursuit, and where he or she might take it in the future. Breaking activities into discrete units of evaluation and institutionalizing a peer review process into them strengthens the final product and more carefully engages the students in the important steps leading up to its accomplishment.

Thursday, July 12, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


Image courtesy of Ben's Guide		Student evaluation is different from course assessment in the sense that we're now switching from reviewing whether the structures we've put in place are adequate for the task we're
giving them to reviewing how well a student is able to perform within a given structure ideally suited for its task. In our face-to-face classes, we presuppose a lot about students entering graduate studies, some of which includes the ability a student ought to have to write a cohesive argument articulating some theological point. We often assign our students grades on two things, then, the first being the meta-professional skills involved in the students' writing ability, and the second being the professional skills involved in students' ability to adequately express course content. Because what bears out in practice is often not what is promised incipiently, we may find ourselves having to teach our students the meta-professional skills of how to write within our disciplines in addition to our teaching them our disciplines. By analogy, a similar process is at play in online teaching where we're evaluating a student's capacity to adequately engage the course in whatever assignment we've created for them to demonstrate their facility with the materials. When we evaluate an online group activity, we're still evaluating two things, but, in this case, the first is the meta-professional skill of collaborating with others (usually asynchronously) and the second is the professional skill of engaging course content. This dual-teaching will not always be the case, however, as students will develop their facility with the former over time. The questions in the meantime come down to which of the colorful letters above we assign to a student, and, bearing the aforesaid in mind, what it is we're actually grading when we make our decision to judge a piece of work as an A or a B or worse. It is for this reason that grading rubrics (a custom design for which can be found at Rubistar) ought to be promulgated and the standards for evaluation objectively defined on them. Students need to know on every given activity the criteria under which they're being measured to avoid equivocation in the evaluation process. Some specific methods on how to accomplish that will follow this week.

Wednesday, July 11, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


For the final thought on course assessment before I shift into student evaluation, I'd like to focus on the weekly lecture itself. The idea of the lecture is derived from a professor's need to provide an analysis and synthesis of a given set of materials for the purpose of fulfilling the goals of the course in accordance with the goals of the program under which he or she is teaching. This need in a face-to-face course bumps against a temporal reality - there is a finite period during which we can convey an interpretive context for the materials we're assigning the students. This is the root of the transmissive model, for we always have more material to convey than we have time in which to convey it.		Rev. Michael John Witt [lecturing on Islam] Courtesy of Kenrick-Glennon Seminary
Jim Rafferty has a favorite quote where the irrational fear of not covering all the material is called anapolsteraphobia, an attitude that is categorically opposed to the transactive model that engages students in a dialogue throughout the process of their encounters within the teaching and learning environment. One of Kenrick's examples of someone endeavoring to reconcile the transmissive need with the transactive model is found in the person of Rev. Michael John Witt, who has posted 160 hours of Church history audio on his website. Because Rev. Witt has a finite time in his face-to-face covers to cover his material, he assigns the lectures as part of his homework requirement. Through the process of dialogue during class time, he's able to assess their impact on the students who arrive prepared to actively discuss the content rather than merely to listen to it as it is conveyed. In the online courses that Rev. Witt teaches, he doesn't have the face-time to engage the students in synchronous dialogue, but he does have a whole week's worth of discussion board time to engage them. Whatever isn't working in the lecture as a means by which to transform students into co-producers of course content can be developed through his interaction with them on the discussion board. He began developing creative ways to use his materials online, such as breaking up his lectures with links pages so that students can pause the audios and quickly get background material that they might be lacking to help them better understand the syntheses they contain. I learned from this myself and moved from providing complete lectures to cutting up my lecture materials into smaller chunks - offering some by print, some by audio, some by video. I also formatted each lecture within various contexts - course overviews, announcements, discrete activities, course projects, thoughts pages (like this one), etc. This enabled me to provoke student feedback on an as-you-go sort of basis. In this way, I found that students were demonstrating a capacity to engage discrete points in ways they'd never demonstrated a capacity to engage complete lectures.

Tuesday, July 10, 2007 -- Dr. Sebastian Mahfood (share your thoughts)

Photo Courtesy of Citizens Health Care (The dots could be students!)

Depending upon the online class I'm teaching, I like to use some method (see the July 7 posting) to divide my students into working groups. Once grouped, my students are allowed to work on the principle of subsidiarity, meaning that I don't actively interfere in the management or structure of their group as far as its normal operations are concerned.

Each group has the capacity to take care of most of its own problems, and my entry into each of them only comes through invitation where the group has decided that it doesn't have the resources by which to handle a given difficulty in its self-administration. A concrete example of this would be when one student is doing nothing (in spite of repeated efforts of the group members) to support the other two or three students who are actively trying to bring closure to a project. My intervention, then, is to find out the intentions of the truant -- either to bring him or her back into the life of the group or pull him or her out altogether. Assessing group work, then, from the administrative angle, is different from evaluating a group's accomplishments. It involves ensuring that the group is functional (if not relational) and is doing the task for which it was formed. One method might include a routine spot-survey where you ask each of the members of a group to assess his or her own levels of commitment and the levels of commitment he or she perceives from others. One student might believe he or she is contributing a lot while the other students might perceive him or her as a slacker. Another method might include the implementation of a graduated project design where certain smaller elements are due at various times and their failure to materialize is indicative of problems within the group structure. Ultimately, in this kind of assessment, you're trying to determine whether the group structure is working or not for the purpose of accomplishing the activity that meets a particular course goal, so, in a sense, if you find that the group structure is not working, you end up with two questions instead of one -- the first concerns what prompted the breakdown in the group (e.g., are there personality or epistemological differences that prevent a group's self-actualizing around its activity, is the activity viable for group work, etc.), and the second concerns whether the activity is viable at all (even if done by individuals not assigned to groups) for accomplishing the course goals. It may be that you have to answer that second question prior to your answering the first, and one way to do it will be to see if there are similar problems in other groups or give people a choice to do the activity in a group or do it alone and take data on both kinds of engagement.

Monday, July 9, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


If we're engaging in transactive learning processes, we also have to engage in transactive assessment processes. To wit, you want to 1) know what it is you expect to receive as a tangible demonstration of a student's ability to adequately express his or her knowledge, and you want to 2) incorporate student feedback into the assessment grid you're structuring. In developing accountability exercises for your course, furthermore, you want to start with your end in mind based on the course goals, which are tied to the program goals, which are tied to the institutional mission.		Courtesy of Nasa
Once you have an idea of the first point, all you have to do is work backwards in establishing the structure of the activity that will provide you with that demonstration. The thing that you're looking for when creating an activity is knowledge of whether or not the activity does the task for which it is designed - that of demonstrating competency of one or more course goals. To do that, you need a rubric, some kind of essential elements list against which you can check student progress within an activity. To include the second point, you'll want to share your rubric with your students and let them know that the data you collect from their efforts via this rubric is not meant to evaluate their work as students but the adequacy of the activity itself as a means of expressing the course goals. It usually takes me two semesters of data collection before I can be certain an activity works in the way it is intended to, and that is because I run through a two-fold assessment structure designed as follows: 1) Formative assessment is what you do in the planning stages of a given learning activity. You decide upon the goals and purpose of the activity and test the methods to see if they fulfill those purposes. It continues throughout the learning activity -- when you interact with students on their developing projects, you're engaging in a kind of shaping behavior. You're also taking data on how they're responding to the kind of assignment that you're providing so that you can improve the way in which you articulate the needs of the project in the moment. 2) Summative assessment occurs once a project is finished -- how well did students respond to the environment you established and how well did your environment meet their needs. It continues at the end of the second semester working through this activity -- how well did the suppositions you made at the end of the first semester continue to be true for the group reaching the end of the second semester. Maybe your evaluatory remarks were not able to be developed sufficiently with only one semester's experience. If you were to change your evaluatory assumptions from a semester previous, how would that change the way you view the ability of your environment to respond to the needs of the students? At this point, you're back to formative assessment and the process starts over so that you can always maintain the integrity of your process through the evolving needs of your student population and institutional ethos.

Sunday, July 8, 2007 -- Dr. Sebastian Mahfood (share your thoughts)

The idea of assessment can be broken into two parts - course assessment and student evaluation. Whenever we engage in an assessment process, therefore, it is important for us to distinguish exactly what it is we're trying to measure - how well the course is able to accomplish its goals or how well the students are able to respond to the activities we've articulated as demonstrations of those goals. Often, we can see problems in the former through the latter (e.g., if most students fail a test or do poorly structuring a paper, then that's indicative of a problem in the way in which the course is structured, or the pre-requisites established, or something else external to a student's raw abilities). My thoughts the first part of this week will predominantly fall in the area of course assessment and in the second part of this week in the area of student evaluation. To begin, then, let's look at a basic component - which is a bridge from last week - the discussion board.

Since all of the activity in an online course occurs through mediated means (and mostly on the course template), the discussion board is the primary vehicle through which to assess student interaction and participation. In a structural assessment, then, all we want to do is determine whether the discussion board is working as a means of providing a dialogic forum. If it is not, then what we want to determine is a method by which we can either accomplish the goal we set for ourselves or reframe the goal to make it more realistic for the needs of the learning community and for the needs of the program within which that learning community is enrolled. An example of one kind of assessment is called a discursive assessment because it is an analysis of the discourse within a forum set up for a given purpose. What we discovered by doing one was that because the students had no experience with online forums, they were using the discussion board merely as a place to turn in assignments and not as a place to engage in meaningful dialogue with one another. If we wanted them to do that, we'd have to teach them how.

Saturday, July 7, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


Since we have invested our efforts this week discussing a teacher's interaction with individuals within an online class and with the class as a whole, I would like to engage this thought on a teacher's interaction with groups established within the class. Just like in our face-to-face classes, we sometimes find it useful to divide the students into small groups around a specific purpose and have those groups in some way report back in plenary session. This is the real value of having students meet in class face-to-face, for it gives us the opportunity to actively engage them as co-producers of the teaching and learning environment. It so happens, on occasion, that as lecturers we are slaves to our content transmission and have very little time left over after lecturing to involve the students in any meaningful group work.		A free small group (3-person) meeting area.
		Credit goes to Elizabeth Osika for finding this service.
This is where our posting our lectures as audio files in online courses really comes in handy, for we can assign students to listen to it outside of class in order to clear the time and space for in-class activities. This idea is adapted from what actually occurs in cyberspace where the teacher's lectures are already posted and reviewed by the students at the beginning of each week. The entirety of the course can be developed, then, on group-based activities, but the students need some vehicle through which to form themselves in and as a group. A few suggestions on how to group: (1) let the students group themselves by interest level (since they don't know each other well enough at the beginning to group themselves by personal affinities) - I do this by asking the students to pick one of three or four ideas to comment on. Once they all post a comment, they've self-selected, at the same time, their groups (e.g., everyone who selected idea # 1 upon which to comment will work together on the follow-up project). (2) Let the students group themselves by times of availability - I do this by asking the students to pick one of three times they're available for synchronous interaction. All the students who choose, say, Saturday at 9:00 am, are in a group. (3) Assign groups based on statements of interest submitted at the beginning of a given week's study or based on the level of ability you perceive across the spectrum of student knowledge and capacity predicated upon a diagnostic survey given at the beginning of the course - I do the former in my writing classes by making four or five classifications for student papers, dropping all of them into slots depending upon their quality, and choosing one from each slot to form a group of diverse talents and abilities.

Friday, July 6, 2007 -- Dr. Sebastian Mahfood (share your thoughts)

"You guys are my witnesses . . . He insinuated that ZFC set theory is superior to Type Theory!"
(Courtesy of NASA)

If our learning communities express their dynamism partly through opinion leadership, then they also have to form their own internal structures for conflict resolution. In face-to-face communities, we can read conflict resolution through the discipline of pragmatics, or discourse theory, which is "concerned with bridging the explanatory gap between sentence meaning and speaker's meaning" (Wikipedia). Pragmatic competence is demonstrated by our ability to interpret meaning even in moments of equivocation, by our ability to see the real reason behind a statement or outburst. Pragmatics contains within it a number of principles, one of which is the politeness principle - the idea that people (1) will not go out of their way to cause discord within a social body and (2) will endeavor to bring harmony to the group when it is within their power to do so.

In face-to-face encounters, applications of the politeness principle for the purpose of conflict resolution are easier in the sense that the immediacy of the moment provides a context for people to work through their difficulties with one another. In asynchronous encounters, on the other hand, people may not notice disharmony within the group for hours or days while misunderstandings fester. Students may not realize that they are oppressing each other or even how they are oppressing each other, at which point the teacher as facilitator of dialogue needs to become part of the historical reality of the discussion. As important as a teacher's intervention is the method by which the intervention occurs - not only for the immediate resolution of the problem at hand but also for the modeling that a teacher is able to provide. A couple of basic strategies include (1) email the disputants privately and ask them to seek clarity with one another outside the discussion board and post evidence of their having come to terms with one another on the discussion board afterwards, (2) if the teacher is the one who is misunderstood, the teacher should apologize for the misunderstanding and seek resolution and clarity outside the discussion board -- under no circumstances should the teacher involve him- or herself in a public brawl. The point is that the discussion board is a place for people to engage in meaningful discussion (even in moments of disagreement and debate as long as it contributes to the purpose for which the forum was created), but it is never a place for people to actively pursue a conflict that is damaging to the greater community of which they are a part. If you notice that happening (and sometimes it takes you by surprise), fall into conflict resolution mode.

Thursday, July 5, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


In Antonio Gramsci's Prison Notebooks, we learn that there are intellectuals within every social group -- in groups of beggars, janitors, trash collectors, construction workers, teachers, politicians, lawyers, doctors, and even online students -- and that these intellectuals are opinion leaders to whom others within the group look up for advice. Everett Rogers writes in his book Diffusion of Innovations, furthermore, that behind the implementation of every new idea is a cadre of opinion leaders that operate at different levels within the organization undergoing change. Gabriel Tarde, in his 1903 The Laws of Imitation, is more succinct in saying that "[e]very herd of wild cattle has its leaders, its influential heads."		Some free multi-user audio chat tools: http://www.there.com/ (this one has cartoon avatars) http://www.paltalk.com/ (this one has video) http://www.skype.com/ (this one is straight voice)
To effect development, then, within an online classroom community, it is essential that we first take the responsibility for nurturing the new community into being and, second, allow that community to develop its own character so that we may then engage in the process of inculturation within it. That means understanding opinion leadership. Opinion leaders, of course, will not always be those who are on the vanguard of any new movement. They, in fact, have their position as opinion leaders because they can be counted on to have their pulse on the needs of the community -- needs that might not call for changes. In some cases, then, the opinion leaders might be laggards on some issues, and it will take an approach of critical mass within the group to so upset the equilibrium that opinion leaders begin to explore the merits of the prevailing view in order to protect their positions as opinion leaders (cf, also, Thomas Kuhn's Structure of Scientific Revolutions). As concerns the vanguard and the laggards on any new idea, furthermore, the online teacher ought to invest him- or herself in the study of the extremes in order to best understand the impact on the middle. The reason for this can be found in Marshall McLuhan's work where he writes that our technologies are extensions of ourselves in the world, and that for every extension or amplification of the human person, there is some kind of amputation that occurs. A car, he explains, is an extension of our foot -- we became a nation of drivers, but we lost the ability to walk long distances. Text is an extension of the eye, but we traded our acoustic consciousness for a visual consciousness. The thing to understand is what we amputate by accepting the amplification, and we have to minister to the perceptions of not only all stakeholding groups but also of all constituencies within any given stakeholding group whether what we stand to gain outweighs what it is we lose. This is something that opinion leaders do unconsciously, and we have to emulate that unconscious habit in order to emerge as opinion leaders in communities that we have formed but that are not really our own.

Wednesday, July 4, 2007 -- Dr. Sebastian Mahfood (share your thoughts)

	Get your Own Meebo Account		Happy Birthday, U.S.! Today, 'we' are 231 years old, and by 'we,' I mean those of us who became citizens sometime between 1776 and now. We imagine ourselves to be in solidarity with one another on a day like this, and we embrace the sense of community that our common affinity provides. This idea of affinity as a binding element in community is something that translates well into our work online as we come together in communion with others in diaspora around the country for a common and specific purpose. The catch is, of course, getting people to buy into the idea of their entering into a relationship when they sign up for an online course.
Flags are symbols of the communities of which we imagine ourselves to be a part. For more on this, see Benedict Anderson's work on imagined communities, in which he argues that a nation "is imagined because the members of even the smallest nation will never know most of their fellow-members, meet them, or even hear of them, yet in the minds of each lives the image of their communion."

Tuesday, July 3, 2007 -- Dr. Sebastian Mahfood (share your thoughts)

Courtesy of NASA		Both active and passive engagement on a discussion board or in a chat room are responsible for developing a group's ethos. Active engagement is demonstrated by the quantity of postings one makes whereas passive engagement, otherwise known as lurking, is invisible to the community but can manifest itself in the quality of the participant's posts. There is room in every discourse for both active and passive engagement on the part of its participants, for it is through active engagement that things actually get done within a forum and through passive engagement that those things are demonstrative of reflection and informed judgment. To get the most out of a discussion forum, then, participants should be encouraged to post early in order to help structure the format of the discussion and to read the posts of others without responding to each one immediately to better enable them to shape the developing content. This kind of dialogic shaping is how a community forms online, and it builds upon its formation from one forum to another, from one thread to another.
An academic community is different from what I call a feral community in the sense that academic communities are self-contained, enabling all of their members to participate from the very beginning of a conversation in developing their structure and content. Feral communities, or communities in the wild, are those found online in medias res, where the newcomer was not present at the beginning of a given topic (or even at the beginning of the formation of the community into which he or she is entering) and is, therefore, required to lurk for a while in order to understand the nature of the community in which he or she is about to participate. A student learning how to interact as a formator within an academic community, then, develops the skills he or she will need in the inculturation of the Christian message within cyberspace. (For more on this, see Sr. Angela Ann Zukowski's chapter "Inculturation in Cyberspace" in The Gospel in Cyberspace: Nurturing Faith in the Internet Age co-authored with Fr. Pierre Babin.)

Monday, July 2, 2007 -- Dr. Sebastian Mahfood (share your thoughts)


A distinction can be drawn between immediate authenticity and mediate authenticity in the respective functions that synchronous and asynchronous forms of communication have. Immediate authenticity is that which demonstrates no mediation between dialogic partners. It used to be referred to in the language of face-to-face interaction. With the rise of mediated communication technologies, however, this term has been progressively broadened, and mediating tools that bridge space but not time (like the telegraph, the telephone, or even the audio/visual multimedia chat room) have come under its evolving definition of the immediate being 'that which is synchronous.' To say something is 'immediate,' otherwise, is really to say that nothing stands in a mediating position between point A and point B.		Courtesy of NASA "Authentic, genuine, real, veritable share the sense of actuality and lack of falsehood or misrepresentation. Authentic carries a connotation of authoritative certification that an object is what it is claimed to be" (Dictionary.com).
Mediate authenticity is that which demonstrates mediation between dialogic partners. This has always been referred to in the language of some medium serving as the device through which exchange is made. With the rise of mediated communication technologies, however, this term has been narrowed, and mediating tools that bridge time but not necessarily space (like letters, books, journal articles, or even emails and discussion board postings) have remained under its evolving definition of the mediate being 'that which is asynchronous.' To say something is 'mediate,' otherwise, is really to say that something stands in a mediating position between point A and point B. This distinction is meaningful for us who work in cyberspace in the sense that it helps us understand that the difference between chat rooms (synchronous) and discussion boards (asynchronous) is that of time. The synchronous encounter is that which calls upon the immediacy of the moment without much pause available for reflection on or composition of an idea before it is disseminated into the community. The asynchronous encounter is that which relies upon a mediate intervention to enable pause for reflection on or composition of an idea before its dissemination into the community. Each carries within itself its own authenticity.

Sunday, July 1, 2007 -- Dr. Sebastian Mahfood (share your thoughts)

Augusto Boal and Sebastian Mahfood (2003) Click here for a FireFox friendly audio I give this audio to my cyberethics students week 2.		After having met Augusto Boal, the founder of Theatre of the Oppressed, at a Pedagogy and Theatre of the Oppressed conference in Omaha, Nebraska, in the spring of 2003, I developed the idea that one of his methods would be ideally suited for my cyberethics students. Invisible theatre is a way of teaching others about social justice without putting oneself in danger for doing so. Boal, who was regularly beaten by the Brazilian authorities whenever he would try to raise the consciousness of factory workers about their plight, discovered a method to make social justice theatre invisible in public places.
I explored the idea of invisible theatre in cyberspace through interactive role playing in an online community called ActiveWorlds, which is a place that my students and I were used to visiting during online office hours. In each session, we engaged in role play for the benefit not only of our class but also for others in the chatroom. The object was that those participating would pretend not to know each other and form a community around the topic of discussion. My avatar tossed out an idea for a story -- a girl decides not to mow her grass because she hears that behind every blade of grass is an angel coaxing it to grow. Another person added on to the story. Finally, someone in the group announced that the story would be useful for an English paper due on Monday. The person who was planning on stealing it then said that the teacher puts the stories on the Internet. When my avatar protested that it's his (my!) story and that he (I) wanted to sell it, we actually began having an interesting discussion. The role play ended when the person agreed to tell the story of what happened in the chatroom instead. Our hope in performing this skit was that others would realize the rights of the individual in a social setting -- maybe they wouldn't realize it right then, but the next time they had an opportunity to take an idea someone else formed, they'd think twice. For more on Augusto Boal's methods of interactive theatre, see the Wikipedia article that discusses his work or google him. To go to Dr. Laura McCammon's paper cited in the audio, click her name.

Instructional Designers

Last Updated July 9, 2007. © Catholic Distance Learning Network.