As noted in Chapter V.4, reviews of all academic programs are required every ten years. To help chairs and administrators who face the daunting task of preparing a self-study, department chairs and administrators familiar with the process offer the suggestions found below. Departments may consult with their Associate Dean for guidance or assistance throughout this process.
Preparing for the Self Study:
|
"Start early. We had one faculty member, with the help of one staff member in odd hours, collating all the material for the appendices. He estimated he spent dozens of hours pulling it together." |
Reviews are expected to take place every ten years, so departments should have a general idea of when they can expect to fall due for review in the College's review cycle. If a department doesn't have a record of the last review, the chair should contact the Dean's office to obtain a copy of the previous self study and review committee report, if desired. (Both of these documents, if available, are supplied to departments when the Dean asks the department to undertake a self-study.)
|
"It would have helped if we collected information on a more regular basis." |
Good record-keeping is an essential component of the program review process, since a department's self-study is usually founded upon the data located in the department's files. Many of the materials listed in the "Materials for Review Committees" are records departments should keep as part of their normal order of business. Gathering the data for the self-study and the review committee's visit is easier if records are complete and well-organized. (The Office of Quality Improvement has been a useful resource for departments that need to get a grip on basic records management.)
Chairs should consider establishing a procedure for noting information regarding departmental initiatives that may make interesting or necessary additions to a future self-study. Consider this an investment in the future, since the faculty members who perform the self-study will appreciate having a "For Future Academic Program Review" file. Such a file may be particularly useful for drafting the portion of the self-study that describes what the department has done since the last review. Contents of such a file might include lists of special achievements or awards earned by students, faculty and staff; informal histories provided by retiring faculty or staff members; notes about factors affecting enrollment (ISIS, touchtone crises, new FIGs initiatives, etc).
The Self-Study Process:
|
"Delegate and share tasks. Consider dividing the appendix-information-gathering jobs among several faculty/staff." |
As many members of the department as possible should be involved in this process; this involvement should, however, suit the temperament and needs of the department. Some departments find it worthwhile to establish an ad hoc self-study committee. Others choose not to create a new committee, preferring instead to delegate responsibility for various portions of the report to existing committees.
Consideration should be given to involving staff members and students in the process. In many reviews, the department administrator plays a key role in obtaining and organizing documents, and helping the chair move the process along.
Responsibility for the self-study should not be delegated to non-faculty members of the department. Nor should the self-study be delegated to an outside consultant, although some departments have called upon consultants for assistance in the process.
Sources of Data for the Self-Study:
Student Data
 |
The L&S Profile is available through the UW-Madison Data Warehouse. The Profile includes ten-year trend data on major enrollments, degrees granted, student credit hours taught, etc, as well as information for the same period on budget funds, number of faculty and staff, etc. Access to the Profile is limited to authorized users. Contact Anne Gunther (gunther@ls.admin.wisc.edu) for more information. | ||||||||
|
Some
student data reports need to be obtained from the UW Data Warehouse on a
regular basis, since some of the information tables contain only the current
semester's data. Departments may want to pull reports each semester
on the following topics:
|
||||||||
|
The Graduate School posts information for students on various aspects of academic programs. This information can also be filed for future use in an Academic Program Review. See http://www.wisc.edu/grad/education/acadprogs.html for more information. | ||||||||
|
Degree trends are monitored by the Office of the Provost. They are posted online at http://apa.wisc.edu/degrees_DegreesTrends.html | ||||||||
|
The Office of the Registrar Issues several annual enrollment reports that include departmental breakdowns of student characteristics. See the Student Tabulations website, at http://jumpgate.acadsvcs.wisc.edu/registrar/faculty/enrollment_reports/erindex.html, for more information. |
The sources of information cited above, of course, only offer a glimpse into a department's student profile; however, this information can be used to consider whether there are enduring or emerging trends of interest to the department, or whether curricular revisions are warranted or, if already made, are having the desired effect.
|
Assessment of Student Learning Outcomes: |
Another important source of information about your department can come from your efforts to assess student learning outcomes. Simply put, "assessment" in this area is merely an explicit effort to define goals, measure progress toward the program's level of achievement of those goals, consider the results, and finally, make any changes that seem to be warranted. For assessment of the undergraduate major, this might mean defining competence in key areas within a field; locating the courses in which students are expected to gain that knowledge; evaluating students' acquisition of that information; and, based on what one learns from the students as a group, taking appropriate action. It is important to remember that most assessment strategies look at what student performance reveals about the program, curriculum or course as a whole; unless measures are taken to ensure consistency in evaluation across the program, individual student performance indicators (grades) do not usually provide general information about the program.
Student Outcomes Assessment is a very useful tool for program review. Several departments actively engaged in assessment programs have incorporated information about their assessment activities into their self-studies, particularly in cases where information obtained has guided their decisions about curricular goals, student interests or alumni contacts. In addition, this data provides good evidence that departments are taking a thoughtful look at their educational processes.
It is important to note that assessment in L&S is based on the presumption that every department will approach assessment in the way that best suits its own mission, culture, disciplinary expectations and learning goals established for the students served. To that end, every department has an assessment plan that has been devised to suit its own needs and resources. (Every department has an assessment plan on file with the dean's office.) A wide variety of direct and indirect measures can (and should) be used to assess student learning; those measures should be employed on a schedule that best fits the department's cycle of activities - it's usually easier on departments to implement a small piece of the overall plan each year rather than try to do every piece, every year. Our philosophy is that, while we expect every degree-offering department and program in the College to engage in student outcomes assessment, there are no "one-size-fits-all" assessment plans, tools or schedules in L&S.
If you anticipate that your department will be reviewed in coming years, you have time to plan ahead. Confer with other departments whose strategies for assessing student learning suggest models (departmental assessment reports, including a grid showing which assessment tools are being used in L&S, are posted online at http://www.wisc.edu/provost/assess.html). Consider applying for a University Assessment Council grant to redirect your assessment efforts or revitalize your current assessment program. Consult the LEAD center, the Office of Quality Improvement, or the UW-Survey Center to see what professional resources are available. (It's important to remember, however, that assessment practices that are developed and maintained within the department generally have a longer lifespan and greater impact--and are more cost-effective--than those programs undertaken episodically or that are farmed out to professionals.) Last but not least, feel free to contact Elaine Klein (klein@ls.admin.wisc.edu), who serves as the College's "point person" on assessment.
Departmental Data
|
For a university and college-wide perspective, you may want to look at the annual Data Digest produced by the Office of Budget, Planning and Analysis. |
|
Most academic areas have some means of gauging the standing of their department relative to departments at other institutions. These range from NRC rankings or other formal, national surveys of programs conducted by professional organizations, to informal solicitations of opinion conducted by individual department chairs. Review committees assume that all programs at UW-Madison are usually "among the best" -- but it offers committees more assurance to hear that from independent sources in addition to the department's faculty. |
The Self-Study Document:
|
"Be realistic about the weaknesses as well as the strengths of the dept. A total whitewash will be seen through by the review committee (who are, after all, other faculty who know how this can be done) and will make them skeptical about the validity of all the other material presented." |
The value of the Self-Study is that the department can take an honest look at itself, consider its future plans, and seek advice from knowledgeable peers. The Self-Study document is not intended to serve as a public relations or marketing piece: the goal is to candidly assess the department's past and current strengths and weaknesses with an eye toward the future. In this respect, the L&S program review process goes far beyond the mandated review of degree programs, looking at those programs as an element within the larger whole of the department or program.
The department chair should involve as many members of the department as possible in creating the self-study. Both the academic program (teaching, research, and outreach) and the administrative and collegial functioning of the department should be considered. A good self-study is a road map to guide the department in its development over the next ten years (although it is still possible to get lost!)
|
"Some information about the history of your department or program is useful to the committee, whose members are unlikely to know details or even major events that have shaped your department and so may not understand why its formal and informal structures work the way they do." |
Self-Study Appendices:
The Appendices for the Self-Study contain the materials upon which assertions made in the self study are based. Broadly defined, this might include tables, charts, diagrams, or lists too unwieldy to include in the body of the document; committee reports; policy documents; catalog entries; timetable information; web pages, etc. (These items can be referenced in the self study by appendix number.) For example, if the department claims a particular national ranking, the source should be cited and included in the appendix.
The "Site Visit":
As noted above, a list of "Materials for Review Committees" is sent with the Dean's letter asking a department to begin its self study. Prior to the visit, departments should consult this list to ensure that the review committee has access to all the information--faculty vitas, in particular--it will need during its deliberations.
|
"The program chair should be encouraged to check with the External Review chair right from the start--and throughout the visit--to make sure that he/she is providing what is wanted. Our review committee wanted information beyond what was on the standard list." |
Departments should remember that review committees consist of colleagues of good will, who hope that their contributions will be of help to the department. Their role is not to serve as auditors looking for mistakes, but rather, to be advisors who may offer assistance. (And in many cases, members of review committees are ready to learn from the good examples found in the departments they visit.)
|
"We got some really good advice from our review committee, even before they wrote up their report. If there are things you would like to fix but don't know how to, don't be scared of representing that to the committee and seeking their advice. The great thing about review committees is that they are made up of folks from different departmental cultures; maybe one does something a different way that would work for your department, too." |
|
"The Chair should speak candidly with the committee. Some information can be conveyed orally, esp. if it is sensitive." |
Logistics and Schedule:
|
"Provide a convenient location for the search committee and, if necessary, its subgroups, to meet and to interview. (It saves time to bring interviewees to the committee, rather than move the committee to the interviewees)." |
|
"Work with the chair of the review committee to schedule interviews, and remember to allow the committee time between interviews with different constituencies to summarize and reflect on what members have learned." |
|
"Consider
scheduling classroom visits for the committee. Invite them to
a seminar or talk. Think of ways to help them observe the ordinary
life of the department, |
|
"Schedule time with the chair at the beginning and end of the site visit to answer any specific questions the review committee may have. This can clear up confusion before they write their report." |
|
"Since everyone wants to get things done, and it's grueling for everyone, stick to the schedule, make sure you have break times, and don't make your committee stay too late in the evening." |
Never underestimate the power of food, particularly when trying to persuade students to participate:
|
"Treat your review committee well. Seriously. They liked it that we fed them (coffee and kringle for an early morning meeting; pizza with undergraduate majors), and it was not just bribery--it was part of the larger tone of caring for the folks we are in contact with that we successfully conveyed to the committee. " |
|
"Encourage graduate students and undergraduates to meet with members of the committee --providing refreshments may help encourage them." |
Response to the Review Committee Report:
Every department reviewed is provided with a copy of its review committee's report. Departments are asked to correct any errors in the factual information found in the report; they are not required to correct errors of impression or interpretation. This response is appended to the review committee report when it is reviewed by the Academic Planning Council, so they may be in full possession of the correct information. No action on the self-study or review committee report is required of the department until after all documents have been reviewed and discussed by the Academic Planning Council, which advises the Dean about how best to respond.
Final Thoughts:
Every self-study will, of necessity, be performed differently by each department that finds itself under review. While all self-studies are guided by a consistent set of questions, departments submit their answers in concert with their own departmental "personalities", which cannot (and probably should not) be contained by a single program review formula.
Regardless of mandated requirements for review, periodic and careful self-scrutiny is a healthy practice, and, ideally, the process of seriously engaging a department in a careful survey of its mission, goals, achievements and future plans will have benefits that transcend the remaining stages of the review process. The most useful self-studies are those that extend beyond the boundaries of the requirement for review, in whatever direction may suit the department.
|
"I learned a lot out of the process in the end, even given all the work, and have no regrets." |
Return to V.4 "Academic Program Review Guidelines"
First published September 2001; revised by Elaine Klein.
Suggestions for additions to this page may be sent to Elaine at kleine@ls.admin.wisc.edu.
Last updated: 02/24/03 .
It is the nature of this document to be a work in progress.
The L&S Handbook is produced and hosted by the UW-Madison College of Letters and Science.
South Hall, 1055 Bascom Mall, Madison WI 53706
This document is updated annually each summer; however, significant additions may be made at any time. Departments
will be alerted to these when they are made. Most recent annual update: August 2006.
Contact: Elaine M. Klein, Webmaster/Editor, L&S Online Handbook
klein@ls.admin.wisc.edu or (608) 265-8484
©2000-2006 UW System Board of Regents