Data-Based Decisions 2024

After many years of assessing our graduate program via indirect means (quality and quantity of applicants, completion percentage, course reviews, employment of former students), in 2021 the History Department began a direct assessment program of its History MA/MS degrees and its Ancient Languages and Cultures MA degree. We assess our primary learning objectives using an interrater reliability method in which two evaluators (not on the student’s committee) evaluate the program’s learning objectives for a substantial sample of students’ final thesis (Plan A) or principal Plan B project or paper (we sample over 50 percent of completions). Each major learning objective is assessed for each paper or project, to track if students are exceeding expectations, meeting expectations, their project needs work to meet expectations, and did not complete. These papers/projects are the culmination of each student’s MA/MS program, and thus provide the best overview if the program as a whole is accomplishing its learning objectives.

The full data set is available on the department’s webpage (and individual evaluators’ rating are available upon request). In summary, since 2021 we have evaluated 24 theses or Plan B projects. For Historical Knowledge, there were 17 reviews of exceeds expectations, 28 meets expectations, and 3 needs works (93.8 percent success rate of meeting expectations or higher). For Historical Thinking (Historiography) there were 16 reviews of exceeds expectations, 27 meets expectations, and 5 needs works (89.6 percent success rate of meeting expectations or higher). For Historical Thinking (sources and evidence) there were 18 reviews of exceeds expectations, 20 meets expectations, and 8 needs works (82.7 percent success rate of meeting expectations or higher). For Historical Skills there were 22 reviews of exceeds expectations, 25 meets expectations, and 1 needs works (97.9 percent success rate of meeting expectations or higher). In total, for all of our learning objectives, there were 173 reviews of meets expectations or higher out of 190 total reviews, for a 91.1 percent success rate.

The Graduate Director, James Sanders, met with the History Department Chair, Ravi Gupta, to discuss these results in September of 2024. There was agreement that the program as a whole was succeeding in ensuring students met our learning objectives, but that some improvements in promoting timely completion of project approvals should be considered. The History Department convened on October 16, 2024 to review this data. There was agreement that given the high success rate, we are confident our program is working to enable our students to meet and even exceed the learning objectives established for our MA/MS programs.

In the meeting, we discussed how we have used past assessment data (if indirect measures) to improve our program. The department’s most important decision was to add a new required class, HIST 6030 (Research Seminar) to our core sequence (HIST 6000 – Historical Methods and Research and HIST 6010 - History and Theory).

In the meeting, the department also discussed how to ensure Thesis or Project Proposals are defended during the second semester of the first year. A suggestion was made to pull funding from students who do not defend their thesis proposal, but the counterargument was made that this step seems too drastic and may actually further delay the student from making progress. The proposal is now under consideration by the graduate committee. The department did make a data-based decision to coordinate and streamline the thesis/project proposal writing between 6000, 6010, and 6030.