According to AOTA, “Evidence-based practice (EBP) is based on the integration of critically appraised research results with the clinical expertise, and the client’s preferences, beliefs and values” (American Occupational Therapy Association, 2017). Essentially, it is a complex balancing act between the clinician’s knowledge and experience, high-quality research, and client factors. Maintaining equilibrium between these three factors is crucial to ensuring high-quality, evidence-based occupational therapy practice.
- By focusing too closely on research results, a clinician risks ignoring client preferences and discounting the value of their own clinical expertise.
- By relying too heavily on personal and professional experience, a clinician may fail to remain abreast of relevant research and acknowledge client opinions and preferences.
- By concentrating too much on client factors, a clinician may reject or ignore clinical experiences or evidence that doesn’t precisely match the client’s background or health condition.
Although the definition of evidence-based practice is clear, the implementation of evidence-based practice is much more difficult.
Barriers to Evidence-based OT Practice
In the book Evidence-based Practice for Occupational Therapists (2007), author M. Clare Taylor perfectly summarizes the challenge that evidence-based practice presents. She wrote:
“There is a growing body of evidence (e.g. Wiles & Barnard 1998; Upton 1999b; Welch & Dawson 2006) that therapists are less willing to act on research evidence than they are to act on other forms of evidence to review or change their practice. Upton (1999b) found, in descending order of willingness to act, that therapists would act on information from the follow sources:
- own practice and experience
- colleagues from the same profession
- line manager
- journal articles
- clinical effectiveness facilitator
- colleagues from different professions
- the internet
when reviewing or changing their practice. From this evidence there would appear to be a mismatch between the accepted hierarchy of evidence for EBP and the value therapists place on various sources of evidence.
Perhaps one of the biggest challenges to the development of [evidenced-based OT] is this reliance upon experiential knowledge. However, the definition of [evidence-based OT] cited earlier emphasizes the use of three strands of evidence when making intervention decisions. The evidence-based occupational therapist should draw on her or his own experiential evidence, on evidence and information from the client and on a critical review of the research evidence…the skill is to synthesize all of this evidence and to make sure that all three strands are utilized rather than [relying] on experiential evidence alone.”
Taken together, the data from professional literature, published studies, and my own professional and clinical experience suggest that while many therapists understand the value of evidence-based practice, they do not consistently use an evidence-based approach to practice. It is also apparent that there are many therapists who do not appear to understand the meaning or importance of EBP, and these individuals may be damaging our professional reputation – or worse, clients – by using unproven interventions with unpredictable outcomes.
This struggle between overreliance on experience and the skilled integration of multiple types of evidence in OT practice that Taylor described is exemplified in the following social media “case study” that will be the focus of this post.
Evidence Based OT Practice: A Social Media Study
The screenshot below was originally shared in the Pediatric Occupational Therapy Facebook group, and it was the spark that ignited a heated discussion about evidence-based practice – specifically, what constitutes “evidence,” the use of evidence in OT practice, and the importance of fact-checking and clinical reasoning when making treatment decisions. The resulting dialogue demonstrates just how charged and complex the conversation about EBP continues to be in everyday practice and professional circles.
Although it is only a single discussion thread, this online debate was a largely accurate representation of the current state of evidence-based practice in occupational therapy and many other health professions. Additionally, it provides eye-opening, concrete examples of the aforementioned barriers to evidence-based practice from the perspectives of everyday practitioners.
Note: I have preserved the sequence and overall flow of the discussion while removing some unrelated comments for the ease of reading.
After the link was posted, the first commenter stated:
“I would be cautious in recommending this intervention. It is not considered evidence-based. If a family is considering doing it, it is our duty as the professional to let them know it is considered experimental. Maybe this has to do with the lack of training courses as well.”
In response, another individual referenced several other controversial, newer, or minimally-researched treatments that are considered by many OT professionals to be “experimental” and cited their widespread use as a reason to continue using them:
“Same with [Masgutova], the Astronaut Program, [The Alert Program], [Interactive Metronome], [Integrated Listening Systems], I could go on and on. I am always on this thread saying evidence based! But, Wilbarger does have a long history of practice and I have met so many therapists, teachers, and families swearing by it and I have seen it work myself. Unfortunately in OT, there is very little of what we do, especially in [pediatrics] with research to back it. If we used that rationale all the time, there would be very little treatment modalities.”
Another practitioner appeared to agree with her, saying “I find the research comment that comes up almost comical. When I was in school 20 years ago there wasn’t enough research to even support the effectiveness of OT interventions in many ways…I find minimizing new therapy techniques due to lack of specific research techniques to be limiting to our field.”
Unfortunately, these commenters and several others clearly demonstrated the way in which many OT practitioners fail to use a well-balanced, evidence-based approach to OT intervention and choose instead to place more confidence in personal experiences or anecdotes than high-quality research or other reputable sources.
Advocating for Evidence-Based Practice: Advancing Patient and Professional Outcomes
At this point, another group member acknowledged the limitations of research in OT but argued that it is still crucial to use the best evidence available to make treatment decisions:
“…A lack of evidence is not evidence the intervention is lacking, only that its efficacy has not yet been tested or established by research methods. You are right in that if we only ever used [clinical research] evidence based interventions we would have a very narrow practice. But to suggest weak evidence is strong is very wrong. And to suggest peer review gives credit is also dangerous. We need to combine evidence (at all levels) with our clinical wisdom.”
As the conversation continued, more therapists chimed in with reasons why the treatment in the original post was not evidence based and why the original poster and others should be wary:
- “If you look deeper [into a research article linked on the Masgutova website], this is not strong evidence in a respected journal. Many are published in ‘pay to publish’ journals. Just because something is published or peer reviewed doesn’t mean the conclusions it comes to are sound.”
- “I’m EXTREMELY skeptical about a link that takes me right to a sales page. Reminds me of a day-long Brain Gym training that was nothing but a thinly disguised sales pitch for much more costly trainer training.”
- “We need to be critical consumers of research to continue to move our profession forward. Many of the articles presented in that link are theoretical, not actually something that contributes to establishing an evidence base. And there appears to be minimal research into efficacy that (1) addresses functional performance changes related to occupational performance and (2) have been done by independent researchers who aren’t affiliated with Masgutova. Those two things should be red flags for OTs…in the meantime, we can say we’ve seen results anecdotally, but we can’t say that the intervention is evidence based.”
- “Please be aware that what is presented [on the Masgutova website] is heavily self-promotional and a marketing tool (while looking scientific). Very few [studies] appear peer reviewed – the [study] on EEG findings makes sweeping statements based on a single case, [and] the rest are postings of non-published cases, anecdotes, and heavily biased blog material.”
- “Most sites touting reflex integration have few or no primary research references and mainly blog articles, magazine references, infomercials, or just anecdotes. We need to support our role [in] OT treatment with critical analysis and current science.”
Soon the discussion turned toward the complexity of evidence-based practice and individual clinicians’ understanding of what that meant for everyday practitioners. It was encouraging to see the number of important points commenters made about how complex evidence-based practice can be:
- “[The original post and ensuing discussion are] a real case in point for how establishing evidence is a complex and skilled process, we need to critically evaluate the methods, the review process, [and] how it fits with other relevant research.”
- “Evidence-based practice has to include our clinical experience as well. All levels of evidence have a place. Research is important and should influence practice in a thoughtful way. I would not exclude trying a new approach that could help a client for lack of research nor would I persist with a new method that doesn’t work for my client, no matter how much research indicates efficacy.”
- “I’m far more cautious about a new approach that touts an evidence base which is weak, than a clinically well tried approach which is poorly researched and that fact is acknowledged.”
As the discussion wound down, it became clear that there was a divide between clinicians who continued to believe in the efficacy of the treatment based on the website’s provided “literature” and personal anecdotes and those who remained highly skeptical of it. However, the well-reasoned and intelligent statements from multiple clinicians advocating for a multi-pronged approach to evidence-based practice demonstrate that it is crucial to understand the complexity and value of multiple types of evidence for providing high-quality, patient-centered, and effective OT services.
Ignoring Evidence-based Practice: The Potential to Cause Patient and Professional Harm
Disappointingly, despite the many comments advocating for a more evidence-based approach to practice, other participants in the conversation continued to cite the “lack of evidence and research in OT” to support their continued use of treatments that have limited evidence for their effectiveness.
It is not the use of “untested” interventions that is the most alarming in these scenarios – every treatment has been “untested” at some point, and clinicians can collect valuable data on outcomes for clients who receive new or emerging interventions that can help inform practice. Rather, it is these clinician’s seeming inability to critically analyze evidence, overreliance on personal experience and outdated information, and outright dismissal of available evidence or cautionary advice to justify their use of fad “interventions” with little or no proven impact on functional performance that is most distressing and potentially dangerous.
In addition to relying too heavily on anecdotal experience, individuals who appear to be unfamiliar with the multifaceted nature of evidence-based practice frequently used a bandwagon approach (“Everyone is using it, so it must be OK”) and the “lack of evidence for OT” excuse (which I disprove here) as the basis for clinical decision-making regarding the use of reflex integration, Astronaut Training, the Masgutova method, and other controversial interventions.
It’s true that dismissing new approaches or interventions simply because they are new and as-of-yet untested can limit treatment options that may be beneficial for clients. However, it is important to recognize that many healthcare interventions are not “minimized” because they are new, but because they are untested or unproven. Just as physicians and public health officials are cautious when recommending newly-developed drugs or treatments, occupational therapy practitioners should take a measured, evidence-based approach to using any interventions without sufficient evidence to determine clear risks and benefits to clients.
New treatment approaches should never be outright dismissed, but it is the practitioner’s responsibility to educate patients about the emerging nature of the treatment, carefully review all available evidence, and monitor closely for negative outcomes and functional outcomes linked to changes in occupational performance. By ignoring this responsibility, OT practitioners are taking a dangerous approach to professional practice with potential ramifications at the individual, population, and even policy levels as our professional credibility is damaged or diminished by the use of ineffective or sham interventions.
Conclusion
In the Facebook discussion above, multiple clinicians relied too heavily on personal experience and weak evidence to support the use of unproven occupational therapy interventions. However, the conversation also highlighted other professionals’ use of a more evidence-based approach to intervention that incorporated all three elements of the evidence-based practice triad – high-quality research or clinical evidence, solid clinical reasoning, and knowledge of client factors:
Again, this isn’t to say that clinicians should never try new treatments with clients just because they don’t have multiple research studies backing them up. However, it is important to implement the principles of evidence-based practice when trying new interventions – by conducting a thorough review of available information and literature, thinking critically about the intervention’s mechanisms and goals, and considering how it will impact a client’s occupational performance before and after implementing it.
This discussion highlights major hurdles to evidence-based practice that exist among OT professionals with a wide variety of experience, practice settings, and education backgrounds. The following are several recommendations for improving your and others’ understanding of and implementation of evidence in practice:
- Have ongoing, open dialogue with colleagues and others about their practice decisions
- Increase awareness about what constitutes high-quality evidence versus marketing materials or propaganda
- Improve education about implementing evidence-based practice effectively (i.e. blending clinical reasoning, patient values, and quality research evidence)
- Reduce or eliminate the use and promotion of interventions that deviate greatly from the foundational values of OT and do not significantly impact patient participation in or return to meaningful roles, routines, and occupations
In the future, I hope to see more occupational therapy practitioners taking the time and effort to identify, analyze, and apply high-quality evidence across practice settings and patient populations. Ultimately, dialogues like the one above indicate that we are making progress towards this goal, but that we still have a ways to go.
Gotta Be OT 