Tag Archives: evidence based practice

Evidence Based OT Practice, Part IV: Fact or Fad? A Case Study with Bal-A-Vis-X

4

This post is part of a series on evidence-based practice in OT. Read the previous posts in the series here, here, and here.


Many rehab therapists have fallen prey to using fad treatments that have little or no evidence to support them. There continue to be many lively Facebook debates and polarized opinions about various occupational therapy interventions. Those who believe the intervention is effective make a point of highlighting the many clients they’ve treated who have experienced significant gains, and non-believers turn their noses up and ignore on any evidence presented by those on the other side. As I’ve stated in previous posts about evidence-based practice, the issue in these cases is not whether to use new interventions without a ton of research evidence behind them, but the lack of clinical judgment and application of evidence-based practice principles when choosing and using OT interventions.

 

For example, in one setting where I worked I witnessed a department of over 50 experienced and intelligent clinicians buy into the promises of an intervention with only weak evidence to support it due to its extreme popularity in the pediatric professional sphere. No matter that this intervention required costly trainings and materials, was unlikely to be generalizable outside of a highly specific context, and was barely occupation-based – therapists wanted to learn it anyway! I was shocked that the department was spending limited continuing education funds on a popular pseudoscience program rather than a well-researched, evidence-based training that could help clinicians obtain better functional outcomes for young clients. Unfortunately, this is the case in many settings and with many practitioners.

 

The goal of this post is to closely examine one such program and break down how principles of evidence-based practice can be used to help make a determination about the validity, clinical applications, and utility of this program for occupational therapy practitioners. Read on to learn more about whether the widely-used Bal-A-Vis-X program is an evidence-based intervention or a highly-flawed fad!

Continue reading

Advertisements

Evidence Based OT Practice, Part III: How to Become an Evidence-Based OT Practitioner

Evidence Based Practice Series (1)

Current literature indicates that many healthcare practitioners – including OT practitioners – are failing to identify or utilize high-quality, up-to-date sources of evidence and are instead relying on personal experience, popular media, and outdated treatment protocols to provide intervention.

 

I have observed firsthand the difficulty of integrating evidence into practice during my OT education and employment experiences. During one of my fieldwork placements, a veteran fieldwork educator with 15+ years of experience gave the same home exercise program (HEP) to every client she saw, despite their diverse diagnoses and abilities. Although one could argue that upper body strengthening may be generally helpful for any person who is recovering from illness or injury, I began to wonder whether or not giving the same handout and Theraband to every client was actually benefiting them. In short, what was the evidence supporting this practice?

 

The fieldwork educator certainly wasn’t harming any of the clients by giving them a generic home exercise program. But not causing harm isn’t a particularly good reason to use a treatment or intervention. By that logic, you could provide any treatment whatsoever as long as you aren’t hurting your patients! It seemed to me like this supervisor had simply included “home exercise program” as part of her routine, rather than carefully considering each person’s condition, learning style, or level of health literacy and incorporating this information into her plans of care.

 

I don’t fault my fieldwork educator for her actions; since joining the workforce I’ve experienced firsthand how challenging it can be to ensure that the interventions I provide are evidence-based. It’s exhausting to spend all day treating patients, answering emails, talking with interdisciplinary team members, documenting, and going to meetings – some days it feels like Googling an unfamiliar diagnosis is the most than I have the time or energy to do. Despite this, I push myself every week to read an article, pick a coworker’s brain, talk with a mentor, or find another way to continue incorporating evidence into my everyday practice. And I’ve found that while it’s not always easy, it is always worth it to find evidence to support the interventions I provide and grow my knowledge base and clinical skills.

 

Why Is It So Hard to be an Evidence-based OT?

There are a plethora of reasons why many occupational therapists struggle to consistently utilize evidence within their daily practice, such as:

Continue reading

Barriers to EBP: A Social Media Study

Evidence Based Practice Series.png

According to AOTA, “Evidence-based practice (EBP) is based on the integration of critically appraised research results with the clinical expertise, and the client’s preferences, beliefs and values” (American Occupational Therapy Association, 2017). Essentially, it is a complex balancing act between the clinician’s knowledge and experience, high-quality research, and client factors. Maintaining equilibrium between these three factors is crucial to ensuring high-quality, evidence-based occupational therapy practice.

EBP Pyramid Diagram

 

  • By focusing too closely on research results, a clinician risks ignoring client preferences and discounting the value of their own clinical expertise.
  • By relying too heavily on personal and professional experience, a clinician may fail to remain abreast of relevant research and acknowledge client opinions and preferences.
  • By concentrating too much on client factors, a clinician may reject or ignore clinical experiences or evidence that doesn’t precisely match the client’s background or health condition.

 

Although the definition of evidence-based practice is clear, the implementation of evidence-based practice is much more difficult.

 

Barriers to Evidence-based OT Practice

In the book Evidence-based Practice for Occupational Therapists (2007), author M. Clare Taylor perfectly summarizes the challenge that evidence-based practice presents. She wrote:

“There is a growing body of evidence (e.g. Wiles & Barnard 1998; Upton 1999b; Welch & Dawson 2006) that therapists are less willing to act on research evidence than they are to act on other forms of evidence to review or change their practice. Upton (1999b) found, in descending order of willingness to act, that therapists would act on information from the follow sources:

  • own practice and experience
  • colleagues from the same profession
  • line manager
  • journal articles
  • clinical effectiveness facilitator
  • colleagues from different professions
  • the internet

when reviewing or changing their practice. From this evidence there would appear to be a mismatch between the accepted hierarchy of evidence for EBP and the value therapists place on various sources of evidence.

Perhaps one of the biggest challenges to the development of [evidenced-based OT] is this reliance upon experiential knowledge. However, the definition of [evidence-based OT] cited earlier emphasizes the use of three strands of evidence when making intervention decisions. The evidence-based occupational therapist should draw on her or his own experiential evidence, on evidence and information from the client and on a critical review of the research evidence…the skill is to synthesize all of this evidence and to make sure that all three strands are utilized rather than [relying] on experiential evidence alone.”

Taken together, the data from professional literature, published studies, and my own professional and clinical experience suggest that while many therapists understand the value of evidence-based practice, they do not consistently use an evidence-based approach to practice. It is also apparent that there are many therapists who do not appear to understand the meaning or importance of EBP, and these individuals may be damaging our professional reputation – or worse, clients – by using unproven interventions with unpredictable outcomes.

 

This struggle between overreliance on experience and the skilled integration of multiple types of evidence in OT practice that Taylor described is exemplified in the following social media “case study” that will be the focus of this post.

 

Evidence Based OT Practice: A Social Media Study

The screenshot below was originally shared in the Pediatric Occupational Therapy Facebook group, and it was the spark that ignited a heated discussion about evidence-based practice – specifically, what constitutes “evidence,” the use of evidence in OT practice, and the importance of fact-checking and clinical reasoning when making treatment decisions. The resulting dialogue demonstrates just how charged and complex the conversation about EBP continues to be in everyday practice and professional circles.

 

Although it is only a single discussion thread, this online debate was a largely accurate representation of the current state of evidence-based practice in occupational therapy and many other health professions. Additionally, it provides eye-opening, concrete examples of the aforementioned barriers to evidence-based practice from the perspectives of everyday practitioners.

 

Note: I have preserved the sequence and overall flow of the discussion while removing some unrelated comments for the ease of reading.

EBP Facebook Original Post

After the link was posted, the first commenter stated:

“I would be cautious in recommending this intervention. It is not considered evidence-based. If a family is considering doing it, it is our duty as the professional to let them know it is considered experimental. Maybe this has to do with the lack of training courses as well.”

 

In response, another individual referenced several other controversial, newer, or minimally-researched treatments that are considered by many OT professionals to be “experimental” and cited their widespread use as a reason to continue using them:

“Same with [Masgutova], the Astronaut Program, [The Alert Program], [Interactive Metronome], [Integrated Listening Systems], I could go on and on. I am always on this thread saying evidence based! But, Wilbarger does have a long history of practice and I have met so many therapists, teachers, and families swearing by it and I have seen it work myself. Unfortunately in OT, there is very little of what we do, especially in [pediatrics] with research to back it. If we used that rationale all the time, there would be very little treatment modalities.

 

Another practitioner appeared to agree with her, saying “I find the research comment that comes up almost comical. When I was in school 20 years ago there wasn’t enough research to even support the effectiveness of OT interventions in many ways…I find minimizing new therapy techniques due to lack of specific research techniques to be limiting to our field.

 

Unfortunately, these commenters and several others clearly demonstrated the way in which many OT practitioners fail to use a well-balanced, evidence-based approach to OT intervention and choose instead to place more confidence in personal experiences or anecdotes than high-quality research or other reputable sources.

 

Advocating for Evidence-Based Practice: Advancing Patient and Professional Outcomes

At this point, another group member acknowledged the limitations of research in OT but argued that it is still crucial to use the best evidence available to make treatment decisions:

“…A lack of evidence is not evidence the intervention is lacking, only that its efficacy has not yet been tested or established by research methods. You are right in that if we only ever used [clinical research] evidence based interventions we would have a very narrow practice. But to suggest weak evidence is strong is very wrong. And to suggest peer review gives credit is also dangerous. We need to combine evidence (at all levels) with our clinical wisdom.

 

As the conversation continued, more therapists chimed in with reasons why the treatment in the original post was not evidence based and why the original poster and others should be wary:

  • “If you look deeper [into a research article linked on the Masgutova website], this is not strong evidence in a respected journal. Many are published in ‘pay to publish’ journals. Just because something is published or peer reviewed doesn’t mean the conclusions it comes to are sound.”
  • I’m EXTREMELY skeptical about a link that takes me right to a sales page. Reminds me of a day-long Brain Gym training that was nothing but a thinly disguised sales pitch for much more costly trainer training.”
  • We need to be critical consumers of research to continue to move our profession forward. Many of the articles presented in that link are theoretical, not actually something that contributes to establishing an evidence base. And there appears to be minimal research into efficacy that (1) addresses functional performance changes related to occupational performance and (2) have been done by independent researchers who aren’t affiliated with Masgutova. Those two things should be red flags for OTs…in the meantime, we can say we’ve seen results anecdotally, but we can’t say that the intervention is evidence based.
  • “Please be aware that what is presented [on the Masgutova website] is heavily self-promotional and a marketing tool (while looking scientific). Very few [studies] appear peer reviewed – the [study] on EEG findings makes sweeping statements based on a single case, [and] the rest are postings of non-published cases, anecdotes, and heavily biased blog material.”
  • “Most sites touting reflex integration have few or no primary research references and mainly blog articles, magazine references, infomercials, or just anecdotes. We need to support our role [in] OT treatment with critical analysis and current science.”

 

Soon the discussion turned toward the complexity of evidence-based practice and individual clinicians’ understanding of what that meant for everyday practitioners. It was encouraging to see the number of important points commenters made about how complex evidence-based practice can be:

  • “[The original post and ensuing discussion are] a real case in point for how establishing evidence is a complex and skilled process, we need to critically evaluate the methods, the review process, [and] how it fits with other relevant research.”
  • Evidence-based practice has to include our clinical experience as well. All levels of evidence have a place. Research is important and should influence practice in a thoughtful way. I would not exclude trying a new approach that could help a client for lack of research nor would I persist with a new method that doesn’t work for my client, no matter how much research indicates efficacy.”
  • “I’m far more cautious about a new approach that touts an evidence base which is weak, than a clinically well tried approach which is poorly researched and that fact is acknowledged.”

 

As the discussion wound down, it became clear that there was a divide between clinicians who continued to believe in the efficacy of the treatment based on the website’s provided “literature” and personal anecdotes and those who remained highly skeptical of it. However, the well-reasoned and intelligent statements from multiple clinicians advocating for a multi-pronged approach to evidence-based practice demonstrate that it is crucial to understand the complexity and value of multiple types of evidence for providing high-quality, patient-centered, and effective OT services.

Ignoring Evidence-based Practice: The Potential to Cause Patient and Professional Harm

Disappointingly, despite the many comments advocating for a more evidence-based approach to practice, other participants in the conversation continued to cite the “lack of evidence and research in OT” to support their continued use of treatments that have limited evidence for their effectiveness.

 

It is not the use of “untested” interventions that is the most alarming in these scenarios – every treatment has been “untested” at some point, and clinicians can collect valuable data on outcomes for clients who receive new or emerging interventions that can help inform practice. Rather, it is these clinician’s seeming inability to critically analyze evidence, overreliance on personal experience and outdated information, and outright dismissal of available evidence or cautionary advice to justify their use of fad “interventions” with little or no proven impact on functional performance that is most distressing and potentially dangerous.

In addition to relying too heavily on anecdotal experience, individuals who appear to be unfamiliar with the multifaceted nature of evidence-based practice frequently used a bandwagon approach (“Everyone is using it, so it must be OK”) and the “lack of evidence for OT” excuse (which I disprove here) as the basis for clinical decision-making regarding the use of reflex integration, Astronaut Training, the Masgutova method, and other controversial interventions.

It’s true that dismissing new approaches or interventions simply because they are new and as-of-yet untested can limit treatment options that may be beneficial for clients. However, it is important to recognize that many healthcare interventions are not “minimized” because they are new, but because they are untested or unproven. Just as physicians and public health officials are cautious when recommending newly-developed drugs or treatments, occupational therapy practitioners should take a measured, evidence-based approach to using any interventions without sufficient evidence to determine clear risks and benefits to clients.

New treatment approaches should never be outright dismissed, but it is the practitioner’s responsibility to educate patients about the emerging nature of the treatment, carefully review all available evidence, and monitor closely for negative outcomes and functional outcomes linked to changes in occupational performance. By ignoring this responsibility, OT practitioners are taking a dangerous approach to professional practice with potential ramifications at the individual, population, and even policy levels as our professional credibility is damaged or diminished by the use of ineffective or sham interventions.

 

Conclusion

In the Facebook discussion above, multiple clinicians relied too heavily on personal experience and weak evidence to support the use of unproven occupational therapy interventions. However, the conversation also highlighted other professionals’ use of a more evidence-based approach to intervention that incorporated all three elements of the evidence-based practice triad – high-quality research or clinical evidence, solid clinical reasoning, and knowledge of client factors:

 

Again, this isn’t to say that clinicians should never try new treatments with clients just because they don’t have multiple research studies backing them up. However, it is important to implement the principles of evidence-based practice when trying new interventions – by conducting a thorough review of available information and literature, thinking critically about the intervention’s mechanisms and goals, and considering how it will impact a client’s occupational performance before and after implementing it.

 

This discussion highlights major hurdles to evidence-based practice that exist among OT professionals with a wide variety of experience, practice settings, and education backgrounds. The following are several recommendations for improving your and others’ understanding of and implementation of evidence in practice:

  • Have ongoing, open dialogue with colleagues and others about their practice decisions
  • Increase awareness about what constitutes high-quality evidence versus marketing materials or propaganda
  • Improve education about implementing evidence-based practice effectively (i.e. blending clinical reasoning, patient values, and quality research evidence)
  • Reduce or eliminate the use and promotion of interventions that deviate greatly from the foundational values of OT and do not significantly impact patient participation in or return to meaningful roles, routines, and occupations

 

In the future, I hope to see more occupational therapy practitioners taking the time and effort to identify, analyze, and apply high-quality evidence across practice settings and patient populations. Ultimately, dialogues like the one above indicate that we are making progress towards this goal, but that we still have a ways to go.

Evidence Based OT Practice, Part I: Why OTs Should Stop Saying “There’s No Evidence for What We Do”

1

Like most OT practitioners, I’ve had wonderful and terrible experiences with evidence-based practice. I’ve worked with clinicians across the spectrum – from those who are up-to-date experts in their practice area to therapists whose only sources of evidence were Pinterest and occasional chats with coworkers.

 

As I’ve started working and participating in more professional dialogues about occupational therapy practice, I’ve noticed a disturbing trend. In every setting where I’ve worked or done fieldwork, in online forums (including this blog), and at every conference I’ve heard people saying, “There’s no evidence for anything we do in OT!”

 

Just as an example, consider the two images below, both from online discussions I’ve seen or participated in this past year:

EBP Blog Comment Screenshot

Source: WTF Blog Post Comment Section

 

EBP Facebook Comment Screenshot

Source: Pediatric Occupational Therapists Facebook Group

 

At first I was quite confused by these statements. After all, I spent two years in a master’s program learning about different types of evidence that validate and legitimize the practice of occupational therapy – so how could these people say there wasn’t any evidence for OT intervention? It didn’t make any sense.

 

But as I joined the workforce and began spending more time at professional conferences, team meetings, and other events I noticed that few people cited anything other than their own experience, outdated trainings, or anecdotes from friends to back up what they were doing with their clients. Despite the professional push for evidence-based practice, countless news stories about the value and impact of OT, and my professors’ unending lists of references and clinical experience with the methods and information I learned, it seemed that many of the clinicians I interacted with did not use or even believe in a multifaceted, evidence-based approach to practice.

 

Continue reading