SciTrain University: A Promising Practice in Project Evaluation

SciTrain University (SciTrain U) provides both in-person and web-based training for science, technology, engineering, and mathematics (STEM) faculty. Funded as an Office of Postsecondary Education demonstration project SciTrain U is hosted by the Georgia Institute of Technology and University of Georgia. The project is designed to provide educators with the tools necessary to implement Universal Design for Learning (UDL) principles in college classrooms and other learning environments.

SciTrain U has two major components: in-person workshops and a website. The in-person workshops educate faculty about disability awareness, working with students with disabilities, and accessible pedagogy. The workshops, developed in conjunction with STEM faculty, are led by an experienced instructional technologist. The website hosts online courses to disseminate SciTrain U resources at other institutions, covering topics including transitioning to college, universal design of learning, specific disabilities, and disability laws. Faculty and instructors can utilize the materials for their own benefit as well as teach on the topics at their own institutions.

SciTrain U evaluation efforts documented the effectiveness of the project while allowing timely and responsive changes to be made to their trainings. To document the effectiveness of SciTrain U, considerable project resources were used for evaluation and assessment using a mixed-methods approach that utilized surveys with faculty, students, and website users as well as classroom observations, online journals, and focus groups. At the outset, a group of meaningful indicators were developed. Data collected related to three tracks: referenced program performance indicators, knowledge synthesis, and program implementation measures.

Longitudinal data collected about a group of faculty participants included: performance and demographic data for students with disabilities enrolled in their classes, classroom observations, and online journals about their experiences with the project. Due to low numbers, the performance and demographic data was not found to be statistically significant, but did suggest that the project made a positive difference in earning passing grades and completing courses for students with disabilities. Classroom observations suggested that over time participants' courses became more accessible. In addition, increased accessibility of class notetaking and electronic learning support corresponded to when workshops were held on those topics. In online journals, instructors shared methods they were using and student receptivity to those methods. An analysis of the journal entries placed faculty into three categories: enthusiasts, skeptics, and incremental adopters.

As evaluation data emerged, project personnel worked to use it to strengthen their project by making modifications to curriculum, instruction, technical assistance, and other areas. Both a relatively low enrollment of students with documented disabilities and difficulty securing longitudinal participants were limitations of the evaluation of SciTrain U. As a result, methods were developed to recruit students to SciTrain U affiliated courses and to improve outreach to potential faculty participants via faculty champions.

SciTrain U is a promising practice because of the use of a mixed-methods and multi-faceted evaluation approach that led to timely and effective changes in trainings.

This article was based on information reported in Moon, N.W., Utschig, T.T., Todd, R.L., and Bozzorg, A. (2011). Evaluation of Programmatic Interventions to Improve Postsecondary STEM Education for Students with Disabilities: Findings from SciTrain University. Journal of Postsecondary Education and Disability, 24(4), 331-348 and developed through the RDE Collaborative Dissemination Project (National Science Foundation Research in Disabilities Education Award #HRD-0929006) and Demonstration Projects to Ensure Students with Disabilities Receive a Higher Education (Office of Postsecondary Education, U.S. Department of Education, Grant P333A080022) .