Module 2 of 4
Module 4 of 4
MODULE 4 OF 4
MODULE OVERVIEW
In this module we will examine transactive assessment and evaluation and theory, explore surveys and questionnaires, discuss goals and objectives, examine Brookfield’s Critical Inquiry Questionnaire, discuss eportfolios, and using a template to create a syllabus.
ASSIGNMENTS/ACTIVITIES OVERVIEW (details are listed below)
- Create a form to assess
- Create a syllabus
- Participation posts
ANNOUNCEMENT
This document contains all the information needed to successfully complete the
fourth of four modules for the online certification.
TERMINOLOGY
- Assessment: examining data against expected outcomes (below, we apply these terms to student assessment, but some may also be useful in programmatic evaluation)
- Direct assessment – measuring the quality of student learning based on their performance
- Formative Assessment – "assessment that is specifically intended to provide feedback on performance to improve and accelerate learning" (Sadler, 1998, qtd. in Nicol and Milligan, 2006, p. 65)
- Indirect assessment – measuring the quality of student learning based on how they opine on survey questions
- Self-Assessment - a formative-assessment process by which students assess their own work against general guidelines or a specific rubric provided by their instructor prior to any summative assessment on the instructor's part; this can be done prior to or following a peer review process. Self-assessment involves students "both in identifying the standards/criteria that apply to their work and in making judgements [sic] about how their work relates to these standards" (Boud, 1986 and 2000, qtd. in Nicol and Milligan, 2006, p. 66).
- Summative assessment – measurement of student performance at the conclusion of a learning experience
- Transactive assessment – a formative process of assessment that measures the quality of student learning based on their interactions through the course materials with the course professor and one another, often done in conjunction with self-assessment [William Myers' use of the term "constructive assessment" adequately conveys this concept.]
- Transmissive assessment – often a summative process of assessment that measures the quality of student learning based on their ability to respond to information received from the instructor and the course texts
- Capstone - a summative assessment and/or evaluation method usually completed at the end of a student's program of study but can be deployed as early as the beginning or middle (e.g., an ePortfolio, comprehensive exam preparation, or thesis development)
- Criteria – pre-determined performance indicators
- ePortfolios - collections of student work in an online environment manipulable by both the student and his or her advisors for the purpose of student assessment and programmatic evaluation
- Evaluation – an equivocal term even when used separately from assessment
- Definition 1: determining the value of a given program for achieving its stated goals
- Definition 2: assigning a grade to student work
- Goals/Objectives– main points or intentions of a course
- Keystone – any of a series of assessments assigned at various points within a program of study meant to collectively determine programmatic effectiveness and/or a given student's capacity within a program of study
- Rubric – measurement guide containing a set of performance indicators
- Self-Study – the process by which institutions engage in a comprehensive review of their teaching and learning environment for the dual purpose of satisfying their accreditation agency(ies) during a comprehensive visit and strengthening programmatic effectiveness
- Syllabus– a general table of contents for a course
COMMENTARY/LECTURE:
A. ASSESSING AND EVALUATING
As late as the 1980s, assessment was focused on the presence of institutional resources – whether enough books of the right kind were available in the library, whether enough professors of sufficient diversity to make a viable faculty were available for teaching and research, and whether a sufficient budget was available to keep the school running – but by 1996, according to Bill Miller, Director of Accreditation and Institutional Evaluation at the Association of Theological Schools, "a greater emphasis [was placed] on analysis of the data and decision-making within the institution" (italics his). In any reaccreditation recommendation, Miller adds, the Commissioners "want to know whether the school understands what is happening. What is quality in a school? What [does a school] need to sustain and nurture it? What will [a school] do and how do[es it] know when [it's] successful in doing what [it] do[es]?" (See the Online Self-Study Workshop hosted by ATS.)
An answer lies in outcomes-based assessment, for we measure what we value. If our primary product is an educated student, then his or her performance outcomes become the "product" upon which we can ground an understanding of our program's quality. Programmatic evaluation, therefore, is substantively tied to student assessment.
B.
DIFFERENCES & SIMILARITIES AMONG THE TERMS "ASSESSING" AND "EVALUATING":
Assessment is the "systematic, on-going, iterative process of monitoring learning in order to determine what we are doing well and what we must improve." Assessment involves "observing, describing, collecting, recording, scoring, and interpreting information." Assessment is effective when it "is student centered …congruent with instructional objectives …relevant …comprehensive …clear (in purpose, directions, expectations) …objective and fair, …simulates "end" behavior/ product/ performance, …incites active responses, …shows progress/development over time." http://www.mcli.dist.maricopa.edu
What's the difference between assessment and evaluation? "The term assessment refers to the systematic gathering of information about component parts of the thing to be evaluated. The evaluation process is broader than assessment and involves examining information about many components of the thing being evaluated and making judgments about its worth and effectiveness." From http://www.mcli.dist.maricopa.edu
"What's the difference between assessment and grading? When instructors assess student performance, they're not placing value or judgment on it — that's evaluating or grading. They're simply reporting a student's profile of achievement." From:
http://teacher.scholastic.com
"42.7% of all statistics are made up on the spot." From the comedy of Steven Wright
Why assess?
Educators know how to assess and know that assessing is important – but educators may not realize that if they were to improve their assessing skills, their teaching potential would also improve, which will help their students become better learners and thinkers. According to Frederikson and Collins, "Assessment should not simply monitor achievement or report scores. Whether we are assessing to report to others or for ourselves, …assessment should lead to instructional action."
http://www.eduplace.com
One method to improve teacher assessing skills is to use technology to assess (i.e. use Excel to create a grade book, Word to create a quiz, a website to create a rubric). The technology is readily available and can help predict results and increase both student interest in learning and teacher interest in teaching. Technology can lessen the burden of test giving, test taking, and grading, and it may offer new and imaginative ways to measure student performance. (See the 2010 Revised ATS General Accreditation Standards regarding the integration of appropriate technologies in theological teaching and learning.)
Assessment is changing!
As Bill Miller noted above, assessment changed from resource-based assessment to outcomes-based assessment by the mid-90s of the 20th century. Now that we're in the second decade of the 21st century, we can look back over the past decade and a half and see that outcomes-based assessment itself has undergone change – we are no longer as concerned with what a student knows about a given course of study as we are with how that student is able to practically apply it.
A call for this shift in concern was sounded as early as 1998 by, among others, Norton and Wiburg when they wrote in Teaching with Technology that "[a]ssessment strategies need to shift from the assessment of a student's knowledge about a subject to an assessment of a student's ability to reason, think critically, and solve problems." What they were really calling for was an applications-based assessment process derived not from gauging the student's ability to process data transmitted from the instructor but from the capacity of the student to apply that data in a meaningful way. The pursuit of demonstrations of such applications evolved in online teaching and learning into transaction-based activities that engaged students and instructors through the materials made available to them. The new teaching was, therefore, transactive rather than transmissive in nature, so the new assessment model had to also become transactive.
In transmissive teaching and learning environments, assessment was considered to transpire after instruction, usually in the form of tests over lecture notes, course readings, and class discussions. In transactive teaching and learning environments, assessment is formative and an important part of lesson planning and implementation.
Transactive assessment cannot be delayed; instead, activities and projects that demonstrate desired outcomes and measure skills and knowledge, should be included from the start.
Under a transactive assessment model, students are not only co-producers of their own teaching and learning environments, but they also become co-assessors of their own outcomes. When students have a say in their own assessment they take ownership, perform better, develop pride in what they do, and become better communicators who will get in the habit of reflecting upon their own work. Students are more likely to meet the conditions for favorable completion of an assignment if they have a hand in developing the criteria for that assignment. Self-assessment helps students become focused learners who are able to think about what they have learned, question what is and isn't clear, and think critically to evaluate their work, which leads to students who become self-directed and active learners instead of passive listeners.
C.
ONLINE SURVEYS AND QUESTIONNAIRES
Survey Monkey and Formsite are two web sites that allow for creating forms, evaluations, surveys directly online. Both are free.
- SurveyMonkey
- Formsite
- CoffeeCup - For a small fee you can purchase a Flash based form builder that is easy yet powerful to use. The form program gathers the data in a database while keeping you informed by email when anyone submits the form.
You can generate tests, evaluations, and surveys using Blackboard test and survey managers.
D.
GRADING AS AN EVALUATORY METHOD
A transactive assessment model does not mean that students grade themselves. Grading is a form of evaluation meted out by the instructor based on a student's ability to respond against a given set of criteria provided in advance in the form of a grading rubric. Grades, furthermore, may reveal things that assist in transactive assessment, but they do not substitute for the process of transactive assessment. Among the things that grades may reveal are whether the content is presented properly, whether the lectures are relevant, whether the test material is applicable, and/or whether students are doing more than memorizing. In online teaching and learning environments, grading ought to be considered during all aspects of a lesson – such as its goals, objectives, planning stages, and method(s) of delivery – and in all aspects of course planning – such as textbook selection, lecture material, activities, projects, assignments, planning stages, and method(s) of delivery.
The grading process should, therefore, be ongoing as a factor of lesson and course evaluation, not just as a final product. (An example would be in the development of a research project – instead of simply assigning a grade to the final product on its due date, teachers can create incremental due dates factoring grades for various peer-reviewable components of the project. John Paul Heil and Anne Marie Kitz of Kenrick-Glennon Seminary in St. Louis accomplished something like this in their joint web-based exegetical research projects from fall 2002 to spring 2006. See Catholic Bible Association. Educators will be assured that what they are doing is working and know when and what adjustments need to be made for future classes if grading is included in all aspects of a lesson.
- For fun, take this survey (helpful to determine student tech experience)
E.
TECHNOLOGY ASSESSMENT TOOLS AND STRATEGIES
(Educator created / Student created)
Assessment situations might include the following:
- Instructors or students are asked to evaluate another instructor or student (peer review)
- Students are asked to complete an assessment project as an assignment, i.e. create a rubric Self-assessment or self-reflection by instructor for the instructor, or by students for themselves, or by students for their instructor, i.e. use word processing to journal or log
- Students might be asked to assess peers and create a method to do so
- Students might be asked by their instructor to create a form to assess their own work
- Students or teacher might want to assess a technology tool or software using technology
- Instructors might even want to create a method to assess a method of assessment
F. THE SEVEN PRINCIPLES OF GOOD FEEDBACK PRACTICE (Nicol and Milligan, 2006)
Good feedback practice…
- 1. 'helps clarify what good performance is (goals, criteria, expected standards)'
- 2. 'facilitates the development of reflection and self-assessment in learning'
- 3. 'delivers high-quality information to students about their learning'
- 4. 'encourages teacher and peer dialogue around learning'
- 5. 'encourages positive motivational beliefs and self-esteem'
- 6. 'provides opportunities to close the gap between current and desired performance'
- 7. 'provides information to teachers that can be used to help shape the teaching'
G. SELF-ASSESSING
Self-assessment is meaningful for students endeavoring to develop competencies in the production and dissemination of their own work, yet the skill has to be cultivated through encouragement and instruction. An assumption that precedes any effort of an instructor to assist students in the development of self-assessment strategies is that students are capable of doing so competently.
We have to make this assumption in their favor if we expect them to one day be able to operate their own ministries without teacher supervision, and it is better they learn such strategies under our supervision than otherwise.
The entire weight of Malcolm Knowles's writings on andragogy and adult learning also fall into play here. Knowles makes the following assumptions in The Adult Learner (6th edition):
- The learner's need to know -- the learner has a practical necessity
- The learner's self-concept -- the learner has a sense of his or her own presence in the world and is very self-aware
- The learner's prior experiences -- the learner brings to the learning environment a wealth of life experiences that may facilitate his or her learning
- The learner's readiness to learn -- the learner that approaches a subject on his or her own has a certain readiness to learn
- The learner's orientation to learning -- the learner has developed a particular proclivity toward learning
- The learner's motivation -- the learner has an intrinsic motivation
If we accept these assumptions, then we have to also give our students an opportunity to build on them. --The Four Steps of the Student Learning Assessment Loop by William R. Myers (p. 20)
H. GRADING RUBRICS
Grading rubrics are quick ways to show students what they are doing or not doing within a given assignment. Each rubric is tailored to the specific activity for which it is used. Below are two grading rubrics – the first was created for a discussion board prompt asking faculty to identify one activity in which they engage in transmissive evaluation and explain how they might change their evaluation method into one that is transactive while the second is specific to peer responses on a discussion board.
A rubric is helpful to both instructor and student: it is a simple way to set up grading criteria for assignments or can be used as a self-assessing tool or check list. “A rubric defines in writing” what is expected to get a “specific grade” from uen.org/rubric/html/know
Short Activity Assessment Rubric:
In this module we will examine transactive assessment and evaluation and theory, explore surveys and questionnaires, discuss goals and objectives, examine Brookfield’s Critical Inquiry Questionnaire, discuss eportfolios, and using a template to create a syllabus.
This document contains all the information needed to successfully complete the fourth of four modules for the online certification.
- Direct assessment – measuring the quality of student learning based on their performance
- Formative Assessment – "assessment that is specifically intended to provide feedback on performance to improve and accelerate learning" (Sadler, 1998, qtd. in Nicol and Milligan, 2006, p. 65)
- Indirect assessment – measuring the quality of student learning based on how they opine on survey questions
- Self-Assessment - a formative-assessment process by which students assess their own work against general guidelines or a specific rubric provided by their instructor prior to any summative assessment on the instructor's part; this can be done prior to or following a peer review process. Self-assessment involves students "both in identifying the standards/criteria that apply to their work and in making judgements [sic] about how their work relates to these standards" (Boud, 1986 and 2000, qtd. in Nicol and Milligan, 2006, p. 66).
- Summative assessment – measurement of student performance at the conclusion of a learning experience
- Transactive assessment – a formative process of assessment that measures the quality of student learning based on their interactions through the course materials with the course professor and one another, often done in conjunction with self-assessment [William Myers' use of the term "constructive assessment" adequately conveys this concept.]
As late as the 1980s, assessment was focused on the presence of institutional resources – whether enough books of the right kind were available in the library, whether enough professors of sufficient diversity to make a viable faculty were available for teaching and research, and whether a sufficient budget was available to keep the school running – but by 1996, according to Bill Miller, Director of Accreditation and Institutional Evaluation at the Association of Theological Schools, "a greater emphasis [was placed] on analysis of the data and decision-making within the institution" (italics his). In any reaccreditation recommendation, Miller adds, the Commissioners "want to know whether the school understands what is happening. What is quality in a school? What [does a school] need to sustain and nurture it? What will [a school] do and how do[es it] know when [it's] successful in doing what [it] do[es]?" (See the Online Self-Study Workshop hosted by ATS.)
An answer lies in outcomes-based assessment, for we measure what we value. If our primary product is an educated student, then his or her performance outcomes become the "product" upon which we can ground an understanding of our program's quality. Programmatic evaluation, therefore, is substantively tied to student assessment.
Assessment is the "systematic, on-going, iterative process of monitoring learning in order to determine what we are doing well and what we must improve." Assessment involves "observing, describing, collecting, recording, scoring, and interpreting information." Assessment is effective when it "is student centered …congruent with instructional objectives …relevant …comprehensive …clear (in purpose, directions, expectations) …objective and fair, …simulates "end" behavior/ product/ performance, …incites active responses, …shows progress/development over time." http://www.mcli.dist.maricopa.edu
What's the difference between assessment and evaluation? "The term assessment refers to the systematic gathering of information about component parts of the thing to be evaluated. The evaluation process is broader than assessment and involves examining information about many components of the thing being evaluated and making judgments about its worth and effectiveness." From http://www.mcli.dist.maricopa.edu
"What's the difference between assessment and grading? When instructors assess student performance, they're not placing value or judgment on it — that's evaluating or grading. They're simply reporting a student's profile of achievement." From:
http://teacher.scholastic.com
"42.7% of all statistics are made up on the spot." From the comedy of Steven Wright
Why assess?
Educators know how to assess and know that assessing is important – but educators may not realize that if they were to improve their assessing skills, their teaching potential would also improve, which will help their students become better learners and thinkers. According to Frederikson and Collins, "Assessment should not simply monitor achievement or report scores. Whether we are assessing to report to others or for ourselves, …assessment should lead to instructional action."
http://www.eduplace.com
One method to improve teacher assessing skills is to use technology to assess (i.e. use Excel to create a grade book, Word to create a quiz, a website to create a rubric). The technology is readily available and can help predict results and increase both student interest in learning and teacher interest in teaching. Technology can lessen the burden of test giving, test taking, and grading, and it may offer new and imaginative ways to measure student performance. (See the 2010 Revised ATS General Accreditation Standards regarding the integration of appropriate technologies in theological teaching and learning.)
Assessment is changing!
As Bill Miller noted above, assessment changed from resource-based assessment to outcomes-based assessment by the mid-90s of the 20th century. Now that we're in the second decade of the 21st century, we can look back over the past decade and a half and see that outcomes-based assessment itself has undergone change – we are no longer as concerned with what a student knows about a given course of study as we are with how that student is able to practically apply it.
A call for this shift in concern was sounded as early as 1998 by, among others, Norton and Wiburg when they wrote in Teaching with Technology that "[a]ssessment strategies need to shift from the assessment of a student's knowledge about a subject to an assessment of a student's ability to reason, think critically, and solve problems." What they were really calling for was an applications-based assessment process derived not from gauging the student's ability to process data transmitted from the instructor but from the capacity of the student to apply that data in a meaningful way. The pursuit of demonstrations of such applications evolved in online teaching and learning into transaction-based activities that engaged students and instructors through the materials made available to them. The new teaching was, therefore, transactive rather than transmissive in nature, so the new assessment model had to also become transactive.
In transmissive teaching and learning environments, assessment was considered to transpire after instruction, usually in the form of tests over lecture notes, course readings, and class discussions. In transactive teaching and learning environments, assessment is formative and an important part of lesson planning and implementation.
Transactive assessment cannot be delayed; instead, activities and projects that demonstrate desired outcomes and measure skills and knowledge, should be included from the start.
Under a transactive assessment model, students are not only co-producers of their own teaching and learning environments, but they also become co-assessors of their own outcomes. When students have a say in their own assessment they take ownership, perform better, develop pride in what they do, and become better communicators who will get in the habit of reflecting upon their own work. Students are more likely to meet the conditions for favorable completion of an assignment if they have a hand in developing the criteria for that assignment. Self-assessment helps students become focused learners who are able to think about what they have learned, question what is and isn't clear, and think critically to evaluate their work, which leads to students who become self-directed and active learners instead of passive listeners.
Survey Monkey and Formsite are two web sites that allow for creating forms, evaluations, surveys directly online. Both are free.
- SurveyMonkey
- Formsite
- CoffeeCup - For a small fee you can purchase a Flash based form builder that is easy yet powerful to use. The form program gathers the data in a database while keeping you informed by email when anyone submits the form.
You can generate tests, evaluations, and surveys using Blackboard test and survey managers.
A transactive assessment model does not mean that students grade themselves. Grading is a form of evaluation meted out by the instructor based on a student's ability to respond against a given set of criteria provided in advance in the form of a grading rubric. Grades, furthermore, may reveal things that assist in transactive assessment, but they do not substitute for the process of transactive assessment. Among the things that grades may reveal are whether the content is presented properly, whether the lectures are relevant, whether the test material is applicable, and/or whether students are doing more than memorizing. In online teaching and learning environments, grading ought to be considered during all aspects of a lesson – such as its goals, objectives, planning stages, and method(s) of delivery – and in all aspects of course planning – such as textbook selection, lecture material, activities, projects, assignments, planning stages, and method(s) of delivery.
The grading process should, therefore, be ongoing as a factor of lesson and course evaluation, not just as a final product. (An example would be in the development of a research project – instead of simply assigning a grade to the final product on its due date, teachers can create incremental due dates factoring grades for various peer-reviewable components of the project. John Paul Heil and Anne Marie Kitz of Kenrick-Glennon Seminary in St. Louis accomplished something like this in their joint web-based exegetical research projects from fall 2002 to spring 2006. See Catholic Bible Association. Educators will be assured that what they are doing is working and know when and what adjustments need to be made for future classes if grading is included in all aspects of a lesson.
(Educator created / Student created)
Assessment situations might include the following:
- Instructors or students are asked to evaluate another instructor or student (peer review)
- Students are asked to complete an assessment project as an assignment, i.e. create a rubric Self-assessment or self-reflection by instructor for the instructor, or by students for themselves, or by students for their instructor, i.e. use word processing to journal or log
- Students might be asked to assess peers and create a method to do so
- Students might be asked by their instructor to create a form to assess their own work
- Students or teacher might want to assess a technology tool or software using technology
- Instructors might even want to create a method to assess a method of assessment
Good feedback practice…
- 1. 'helps clarify what good performance is (goals, criteria, expected standards)'
- 2. 'facilitates the development of reflection and self-assessment in learning'
- 3. 'delivers high-quality information to students about their learning'
- 4. 'encourages teacher and peer dialogue around learning'
- 5. 'encourages positive motivational beliefs and self-esteem'
- 6. 'provides opportunities to close the gap between current and desired performance'
- 7. 'provides information to teachers that can be used to help shape the teaching'
Self-assessment is meaningful for students endeavoring to develop competencies in the production and dissemination of their own work, yet the skill has to be cultivated through encouragement and instruction. An assumption that precedes any effort of an instructor to assist students in the development of self-assessment strategies is that students are capable of doing so competently.
We have to make this assumption in their favor if we expect them to one day be able to operate their own ministries without teacher supervision, and it is better they learn such strategies under our supervision than otherwise.
The entire weight of Malcolm Knowles's writings on andragogy and adult learning also fall into play here. Knowles makes the following assumptions in The Adult Learner (6th edition):
- The learner's need to know -- the learner has a practical necessity
- The learner's self-concept -- the learner has a sense of his or her own presence in the world and is very self-aware
- The learner's prior experiences -- the learner brings to the learning environment a wealth of life experiences that may facilitate his or her learning
- The learner's readiness to learn -- the learner that approaches a subject on his or her own has a certain readiness to learn
- The learner's orientation to learning -- the learner has developed a particular proclivity toward learning
- The learner's motivation -- the learner has an intrinsic motivation
If we accept these assumptions, then we have to also give our students an opportunity to build on them. --The Four Steps of the Student Learning Assessment Loop by William R. Myers (p. 20)
Grading rubrics are quick ways to show students what they are doing or not doing within a given assignment. Each rubric is tailored to the specific activity for which it is used. Below are two grading rubrics – the first was created for a discussion board prompt asking faculty to identify one activity in which they engage in transmissive evaluation and explain how they might change their evaluation method into one that is transactive while the second is specific to peer responses on a discussion board.
A rubric is helpful to both instructor and student: it is a simple way to set up grading criteria for assignments or can be used as a self-assessing tool or check list. “A rubric defines in writing” what is expected to get a “specific grade” from uen.org/rubric/html/know
Short Activity Assessment Rubric:
CATEGORY |
1 |
2 |
3 |
Matching activity to transmissive assessment method |
neither the activity nor the method to assess is clear from the description provided |
either the activity or the method to assess is clearly stated, but not both |
both the activity and the method to assess are clearly stated |
Matching activity to transactive assessment method contrast |
neither the activity nor the method to assess is clear from the description provided no real explanation of the difference between the transmissive and transactive methods is provided |
either the activity or the method to assess is clearly stated, but not both the explanation of the difference between the transmissive and transactive methods is unclear |
both the activity and the method to assess are clearly stated the explanation of the difference between the transmissive and transactive methods is clear |
Total Points |
|||
Student Response Assessment Rubric:
| CATEGORY | 1 |
2 |
3 |
Student responses |
Response added nothing new, such as simply agreeing with the original posting |
Response added something new by way of parallel information or personal experience, but it did not indicate a clear understanding of the expressed viewpoint/summary |
Response advanced the conversation, added parallel information or personal experience, and indicated a clear understanding of the expressed viewpoint/summary |
Total points |
|||
Visit RubriStar's web site. While it is primarily designed for K-12 teachers, the demonstrations it provides might be helpful when creating a rubric for higher education. Once on the site, scroll down and click on one of the blue buttons for a rubric template, or on create at the top. (You may have to register, but it is free.)
I. EPORTFOLIOS
Portfolios are collections of student work over time (within a given course or across a number of courses within a given program of study). Each portfolio is a demonstration of student accomplishment in responding (more narrowly) to the course goals and (more broadly) to the program goals. In essence, the portfolio is hard evidence of the student's contribution to and experience of a given course or program and is considered a direct means of programmatic assessment.
An ePortfolio might be created using a PowerPoint template, might be created using an online application for a fee, or not if not electronically created, a portfolio might be hard copies in a binder.
For a detailed description of the values and norms inherent in the use of portfolios, see Truman State's Portfolio Assessment program. While a portfolio provides meaningful data concerning a student's growth within a program, then, it is really geared to assess the teaching and learning environment within which that student has been formed.
The ePortfolio differs from the text-based portfolio in a couple of significant ways - one, in the way it is generated, and, two, in the way it is distributed. In the first case, the ePortfoliois developed on a web page, which enables it to take advantage of appropriate use of multimedia in the packaging of course content. Students can provide audio or video reflections on those areas of their work that they selected as representative of their experience within a course or program in light of the course or program goals they have to prove they have addressed. In addition to providing links to the work they upload to their sites, students may also provide interpretive contexts for external links to various entities online that supported their work. In the second case, the ePortfolio is universally accessible to those responsible for its review, and this includes a student's peers and a program's review board. The semi-public nature of the ePortfolio increases student attentiveness to its development as a self-evaluation designed to be used as an institutional assessment tool.
When we ask our students to develop ePortfolios, we are actually asking them to do two things - the first is to demonstrate a facility with the various technologies necessary to post materials online, and the second is to collate examples of their learning that evidence their responsiveness to the course (and, ultimately, program) goals.
At the beginning of a given course (or program), then, students have to be told that this portfolio will be a portion of their course (or program) requirement and that it will merely entail the gathering and packaging of evidence specific to the course goals. Students will pay much more attention to their course goals (and their relation to the program goals) if they have this in mind, and they'll actually be the ones collecting much of the assessment data for the instructor (or dean) to use within the context of course or institutional assessment. Students will also develop a stronger understanding of the relevance of any given course to the overall program in which they're involved and of the overall program to the vocation for which they're studying.
J. SYLLABUS
(A completed syllabus will be the final project for the course)
A syllabus is a formal table of contents for a course and will most likely be students’ first contact with you as an instructor (the welcome document will be the next).
K. MORE ON GOALS AND OBJECTIVES:
Sometimes goals and objectives are used interchangeably – this depends on syllabus criteria of the teaching establishment. When writing goals and objectives in the syllabus, it is important to note that goals are where we want to be, and objectives are methods or steps we should take to get there. Goals and objectives tell students what you want from them and what the course will offer. Use active measurable verbs such as ‘explain’ and ‘demonstrate’ – not words like ‘learn.’
Visit the WriteExpress site for a list of active verbs.
New terminology has evolved for writing objectives relative to technology:
‘Navigating’, ‘wrap text’, ‘use the ... menu bar Atool palette,’ ‘import’ or ‘export’ Agraphics Apictures), ‘attach documents’, ‘cut, copy, paste’, are some new additions to the action verb list for writing realistic objectives pertaining to technology. The general rule, of course, is to choose words when writing objectives that involve technology and words that are appropriate to the action.Think of what it is you want your students to do during the course. A general rule is to examine the activity or software that online students are using, decide on objectives and learning outcomes, choose a tech tool and method to assess, then choose wording.
L. STANDARDS – THE MATCH BETWEEN GOALS AND OBJECTIVES
The standards used when teaching is dependent upon the teaching establishment.
- In teaching about the use of the Internet online, we use as our standards those available from the International Society for Technology in Education (ISTE), which are called the National Educational Technology Standards (NETS) for Teachers. When you teach about theology, your institution likely has its own standards – perhaps the Program for Priestly Formation or Pastores dabo vobis or some other like documents.
Macros standards like those provided by ISTE or by the Association of Theological Schools help orientate us in our decision-making of what goes into a given syllabus because we, ultimately, want to be able to demonstrate through the courses that we teach that we are meeting some of the standards of the agencies that accredit us.
Given what you now know about how these standards can affect your course design, you will want that understanding reflected in the kind of syllabus you put together, derived not in superfluous technological integration but in a very real and positive response to the nature of the course you teach.
M. FORMATTING AND WRITING COURSE CONTENT IDEAS, COMMENTS, SUGGESTIONS, FYI
- Make print size large enough for all ages
- Create weekly course documents as short as possible
- Use bullet points for lists
- Highlight or bold important information
- Use no more than two colors - web addresses show two colors once visited so course content immediately starts off with at least two colors
- Use no more than two different font types, styles, and formatting or the content will look too busy
- Use white space when possible to avoid the sense that words are crammed together
- Don’t make students search for information in a different place every week (assignments, deadline dates, etc) – create consistent course documents– use a template (more on templates later)
- Weekly documents should follow the course objectives and provide information sequentially
- Provide deadlines
- Create headers and footers so students will know where they are when they open course documents - students often take more than one online class at a time, sometimes from different universities
- Do not provide multiple link options for students to reach the same area - create one link to assignments and one to lecture material, i.e. if two links are needed, make sure that the links have the same name (this will eliminate emails from students asking if they have completed all necessary work for the week)
- All course documents should be accessible from the home page and labeled properly - and do not label discussion links with ‘cute’ names such as tyme-out or coffee-klatch – if two discussion links are necessary - one for informal discussion and one for assignment discussions, i.e., name them appropriately
Personal experience: One fall, a single eight-week course of 16 students created nearly 350 personal questioning emails to the instructor during the first week of class - many could’ve been avoided if the instructor had posted clear instructions from the start (remember that each email should be answered personally)
ASSIGNMENTS
Submit each of the following using the module 4 assignment post
- Create a form/questionnaire/survey for any type of assessment and post a link to the form/questionnaire/survey
- create and attach a rubric (five or six points only) – use Rubistar as shown above or Excel to create a rubric
- Open the Syllabus Template, modify it according to the needs of your class, and submit the completed syllabus as an attachment to the module 4 assignment post.
PARTICIPATION
Suggestions – use the module 4 comment post for the following:
- Share what you learned in this module – something you didn’t know, something you forgot until you read this module, something surprising.
- Provide details of when (or if) you might want to use e-portfolios in an online class.
- Examine the changes in the General Institutional Accreditation Standards and the revised ISTE NETS Standards and reflect on whether these standards cause you to rethink your course syllabus.
SUGGESTED READINGS, RESOURCES, PODCASTS:
- An interesting formula for weighting grades
- Assessing Effectiveness of Student Participation in Online Discussions
- Association of Theological Schools technology standards revisions from the 2010 Biennial meeting in Montreal, Canada.
- Assoc. of Theological Schools (ATS) - Participation Rubric
- ATS Online Assessment Resources – Assessment Basics
- Blackboard Learn instructor guide
- Brookfield Classroom Critical Incident Questionnaire. This is a quick assessment tool asking students to identify their impressions of a given course module or activity. Review Stephen Brookfield's Critical Incident Questionnaire, which, while designed for classroom use, could be used in an online course. Look at the questionnaire and notice what Brookfield is trying to do with this instrument. Post a short response on the Module 7 discussion forum concerning how you might use this same idea in an online environment where there is no face-to-face contact.
- "Building an Online Course for the Catholic Distance Learning Network: Teaching Theology and Science in Cyberspace" by Sebastian Mahfood and Michael Hoonhout
- Closing the Assessment Loop: Nurturing Healthy, On-going Self-evaluation in Theological Schools by William R. Myers - book
- ePortfolio has been used to describe collections of student work at a Web site (about ePortfolios)
- ePortfolios samples
- e-portfolio site - LiveText
- Essential Elements, 87-94
- Evaluating grading and scoring by Houghton Mifflin
- Graduate Research & Writing Seminar Portfolio at Kenrick-Glennon Seminary
- International Society for Technology in Education National Educational Technology Standards - ISTE NET
- Module 7 Podcast (4:47)* by Mr. David Harrison, CDLN course designer - text version
- Online Self-Study Workshop
- PDF primer on all things group assessment
- Prezi presentation that shows the concept of whole-part and manipulation of content “Does perspective matter?” Directions: 1) Once the site opens, enlarge the presentation to full-screen by hovering over More (lower right-hand corner) and clicking on Fullscreen. 2) You can either step through the presentation manually by clicking on the right-facing arrow in the middle of the bottom portion of your screen, or going back to More and selecting Autoplay. 3) Watch the presentation as it dramatically illustrates the differences between Prezi and PowerPoint. 4) At any point you can click on an area to zoom in or out, and resume the pre-made sequence from where you left off.
- Rubric for Course Postings and Writing Projects - Kenrick-Glennon Seminary
- Rubric for Masters Theses and Comprehensive Exams - Kenrick-Glennon Seminary
- Scroll to locate the section titled: Design an effective syllabus.
- Some examples of syllabi created for an online course:
- Book of Revelation - Rev. Tommy Lane, Mt. St. Mary's, Emmitsburg, MD
- Islam through Catholic Eyes - Rev. Michael John Witt and Sebastian Mahfood, OP, Kenrick-Glennon Seminary, St. Louis, MO
- Technology, Ethics, and Society - David Harrison, Webster University, St. Louis, MO
- Dante's Divine Comedy - Sebastian Mahfood, OP, Holy Apostles College & Seminary, Cromwell, CT
- Theory and Learning Resources, Pedagogy, etc.
- "Theory and Practice of Online Learning" Ed. by Terry Anderson and Fathi Elloumi. (esp. Chapters 1, 5, 11)
- USAbility - Site Design - Writing for the Web
- Writing a syllabus – course goals, etc.