When my family moved from Italy to Brooklin, NY, around 1900, they brought their pasta sauce recipe. As a child, my favorite memory was the smell of this sauce in the house. As an adult, I make the same sauce for my two boys. Over 100 years later, this recipe still brings joy to our family members. There are ways to tailor it to your specific taste. However, there are vital ingredients (e.g., garlic, capers) that you must use and in a specific order (simmer the garlic first until you can smell it, then add the capers). You must remember the key ingredients and their sequence to have a sauce my family would recognize. Fidelity to this process is essential for cooking these meals and other significant tasks.
Atule Gawaunde, a surgeon and author of The Checklist Manifesto, says that checking process fidelity is important in many settings, including treatings hospital patients. For example, one study showed that a patient in an intensive care unit (ICU) requires 178 treatment steps (e.g., administering medication) daily. He went on to say that ICU’s have about an 86% survival rate. Many of these deaths are based on errors that could have been avoided if care teams had used simple checklists to guide their care. Having a checklist of the steps you need complete can help you in situations where you want to prevent errors.
While there are several types of fidelity, I focus on the notion of adherence for this blog: how well the team sticks with the plan, assessment, or curriculum. These types of fidelity tools typically involve the systems you need to support the intervention (e.g., leadership teams, access to resources), accessing data to guide implementation (e.g., identifying the problem, progress monitoring), and the selection of evidence-based practices. These checklists help teams develop action plans for their work and determine the degree to which they adhere to a model. Some tools may focus on one type of intervention (e.g., behavior, academics) and/or a specific level of intervention (e.g., schoolwide, groups of students). The tool you use depends on the type of intervention you are implementing. In many cases, these fidelity tools are self-assessments completed by leadership teams, sometimes with the guidance of external support personnel (e.g., external coaches).
Examples of MTSS Fidelity Measures
If you are focusing on implementing multi-tiered systems of support, a great place to start would be the Center on Multi-Tiered Systems of Support MTSS Fidelity of Implementation Rubric. This tool consists of steps to support the implementation of MTSS. The tool is general enough that it could be used to support a variety of MTSS approaches (e.g., academics, behavior). In particular, I appreciate the section, “MTSS School Infrastructure and Support Mechanisms.” Teams can use this portion of the tool to build the capacity of their system (e.g., prevention focus, leadership team) to implement MTSS effectively.
Check With Your District or State
Sometimes states and districts have specific tools they want teams to use to check fidelity and action plan. For example, Michigan’s MTSS Technical Assistance Center uses the Reading Tiered Fidelity Inventory (R-TFI) (link) to check the fidelity of staff’s implementation of academic support for reading. The State of Montana has a MTSS tool (link) that is also general enough to support most school and districts for planning. When possible, check with your state, region, or district to see if there is a tool you might use to guide your MTSS work.
Content and Tier Specific Tools
As with the Reading Tiered Fidelity Inventory (R-TFI) (link) mentioned above, some tools are specific to MTSS approaches (e.g., academics, behavior), or specific tiers of support (e.g., tier-one schoolwide). For example, if your school implements social and emotional learning, your team might consider using the CASEL Implementation Rubric (link).
Schools interested in supporting schoolwide approaches to trauma-informed care might consider the Trauma Sensitive Schools Fidelity Tool from the State of Wisconsin (link).
There are also various fidelity tools that teams can use to support schoolwide positive behavior support. The National Center on PBIS has numerous research-validated tools to track fidelity for all tiers of support (link). PBISApps (link) is an excellent site with digital versions of these tools you can use with your teams. Further, you can use their reports to track your progress and make decisions over time.
Additionally, some tools focus on specific tiers of intervention. For example, Measuring Fidelity of Core Features of Tier 2 Systems and Practices in Schools (link), developed by National Center on PBIS, is an excellent resource on implementing tier two supports with fidelity. You can also check out the Team Initiated Problem Solving Fidelity Checklist Tier 2 (TIPS-FC -T2) that teams can use to guide their processes (link). The TFI mentioned above also includes sections for all three tiers of MTSS for behavior.
Example of Planning with Fidelity Data
Here is an example of practicing action planning based on a fidelity tool for PBIS.
This graph comes from a high school team who used the Schoolwide Evaluation Tool (link). The teams used the data from this tool to measure key features of PBIS. From research, we know that having a score of 80% on the teaching section and an overall score of 80% meant the school was implementing schoolwide PBIS with fidelity. Based on these data, here are a few questions to ask yourself as a team member.
- What are the strengths related to implementation?
- What are the areas for improvement?
- Is this school implementing schoolwide PBIS at full strength?
Now, you can check your work. Based on the data, here is what the team determined related to each question above.
Areas of strength for the school included:
- Defining expectations (e.g., having 3-5 positively stated expectations, posted expectations in key locations)
- Acknowledging students’ appropriate behavior
- Having clear systems for responding to student behavior (e.g., determining classroom vs office-managed behaviors)
- Using data for decision making
Area for improvement included:
- Explicitly teaching expectations
- Access to district level and/or external coaching
The next question to ask is, so what? What should be the response of the leadership team to these data? In the case of this school, the team provided specific professional development on teaching expectations to ninth-grade level teachers. They focused on this group because the fidelity data indicated a need for improved teaching of expectations. A review of the office discipline referral data (ODR) indicated that most of the referrals came from freshmen students related to respect and responsibility. Using both data in combination helped the team to efficiently identify areas of support for their staff and students (see this link for more information on this specific example).
Using fidelity data to plan interventions is a bit like the board game of Clue – it was the butler, with the nightstick in the pantry. In this case of schoolwide efforts, the next steps involved preparing the instructors to teach expectations to first-year students related to respect and responsibility. The fidelity data help the staff see what to do (e.g., support the teaching of expectations). The office discipline referral data helped the team see who to support. Without the fidelity data, the staff might have generated solutions that would not address the underlying problem. Without the ODR data, the team could not have been as efficient in knowing where to spend their limited time and energy.
Additional Resources
Following the COVID-19 pandemic, I find many schools are interested in teaching their students expectations related to appropriate behavior. My favorite tool for teams to review as they start this work is the Team Implementation Checklist (v 3.1) (link). This very efficient tool can help teams guide and monitor their implementation of schoolwide PBIS, particularly at tier one.
There are additional resources that can support your team’s understanding of the fidelity of implementation. First, the IRIS Center at Vanderbilt University has an online module on implementing evidence-based practices with fidelity (link). The center offers certificates of completion you can use towards your professional development (link). Another great resource is the National Center on PBIS’s Collecting Fidelity Data to Support and Sustain PBIS/MTSS in Schools (link). The guide helps districts and state-level teams use fidelity data to guide their interventions.
While I cannot share my family’s pasta sauce recipe (I have been sworn to secrecy), I can tell you that using fidelity tools can help teams stay on track with their schoolwide interventions. I would love to hear about your journey using fidelity instruments to guide your interventions. Please leave a comment below with your thoughts!