IOTTA

 DESCRIPTION 

The Impact of Training & Technical Assistance (IOTTA) is a self-report survey instrument designed to assess participant perceptions of the quality and impact of training and/or technical assistance (TA) efforts. The measure includes two forms: a post-event form that is completed by participants immediately following a training or upon completion of TA, and a follow-up form that is typically completed 6 to 8 weeks later to assess sustainment of skills, mastery, and behavior change. The IOTTA can be administered as a web-based or paper and pencil survey. Both the post-event and follow-up versions of the form take approximately 5 minutes to complete.  

USES 

The post-event survey is designed to assess participant satisfaction with various aspects of a training or TA event, the importance of the training or TA goals, the anticipated level and type of impact that the training or TA will have, their current mastery of the training concepts, and their anticipated mastery 6-8 weeks in the future. 

The follow-up survey asks participants to rate their current mastery of the training/TA content, the amount and type of practice-level change that resulted from the training/TA, drivers and barriers to integrating the information and skills into their practice, and the overall "worthwhile-ness" of the training. 

LEARN MORE

PSYCHOMETRICS, RELIABILITY, and VALIDITY 

A recent study focused on the reliability and validity of the IOTTA. The study examined IOTTA data that came from two separate sources: A sample of 9330 behavioral health providers who participated in a Wraparound training hosted by the National Wraparound Implementation Center (NWIC), and a sample of 900 professionals who received training or TA by the Northwest Mental Health Technology Transfer Center (MHTTC). Exploratory factor analyses (EFAs) of both the post-event and follow-up versions of the IOTTA suggested that items grouped together as expected. For the post-event survey, three factors emerged: 1) Anticipated impact, 2) Perceived quality, and 3) Content mastery/competence. In addition, single items assessed participant confidence in integrating new information into their work and the degree to which training/TA material was different from current workplace practices. For the follow-up survey, five factors emerged: 1) Type of impact, 2) Change in practice, 3) Implementation support, 4) Content mastery/competence, and 5) Implementation barriers. Follow-up analyses suggested strong internal consistency of each of the above-mentioned factors, meaning that the items comprising each factor were related to one another.  

Evidence for concurrent validity was found in results showing that IOTTA scores from two months post-training correlated with similar items from another scale. For construct validity, post-event IOTTA ratings predicted two-month follow-up outcomes as expected. Furthermore, IOTTA ratings of two-month impact were higher for trainees who received coaching and consultation after the initial training. Such results indicate that the IOTTA is measuring the constructs it was designed to assess, including likelihood of impact of training or TA immediately after training and actual impact of training or TA two months after receipt of training or TA. 

For a more detailed summary of methods and results from this study of the IOTTA, please click here.