Decoding Research Claims: How Do We Know What Works?
Whether we’re talking about a new medication or a different psychotherapy technique, at some point you’ll hear the phrase “studies show” that it works. Or you may have read that “research indicates” that a particular intervention (for example, practicing yoga to help improve mood) is effective. Perhaps you’ve wondered about this: what kind of studies? How can researchers determine whether someone’s mood has improved? We live in a fascinating and miraculous moment in history where Google puts an amazing amount of information quite literally at our fingertips. But when it comes to making mental health care decisions, how do you comb through it all?
Mental Health Research: Effectiveness
When we talk about treatment options, the two most important concerns we all share are 1) that it is effective, and 2) that it is safe. How is effectiveness measured in studies that try to evaluate different treatment approaches? When you are exploring the effectiveness of a blood pressure medication, it seems pretty simple: you measure blood pressure before and after using the medication. Assuming nothing else changed, you can say that the medication caused whatever changes are measured. It isn’t quite so easy with mental health—measuring someone’s anxiety level or depression can’t be done by “objective” means. In other words, there is no blood test or simple measurement that can be taken that will tell you about your mood or emotions.
Questionnaires or surveys, sometimes called inventories or clinical scales, are used to measure the effectiveness of an intervention. Commonly, a questionnaire such as the Beck Depression Inventory is given to the study participants before the intervention and again after the intervention. If the score on the questionnaire changes enough, for enough different study participants, the intervention would be considered effective.
While a great deal of research goes into creating accurate and reliable surveys, they rely on the person filling them out to do so accurately. And the measurement of emotions is tricky at best—“how sad do I feel today” may be a difficult question to answer. Sometimes the surveys are “self-administered” meaning you do it yourself. For some studies, the surveys are administered by the researchers. There are scales that measure anxiety, depression, psychosis and many other symptoms or conditions.
But Couldn’t You Just Feel Better For Other Reasons?
This leads us to a complex and often frustrating situation—for patients, therapists and doctors. How do you know whether it was that specific intervention that made you feel better? What if during the study time period, you got a new job or fell in love or won the lottery? Ok, not many people win the lottery, but you can see the problem—studies occur during the course of people’s lives and study participants are affected by many life events in addition to and often at exactly the same time as the specific intervention the researchers are trying to measure. This is where things get really tangled up and tough to tease apart.
There’s another confounding factor in mental health research, which involves the study participants’ attitude. Hope and faith are strong human emotions that are believed to actually change brain chemistry. This means that when you are depressed, your depression has a negative impact on your brain chemistry, but if you are given a cup of tea and told “this tea will improve your mood” your hope and faith that it will work may improve your mood even if the tea is just hot water with a drop of food coloring in it! Of course what I’m talking about here is the placebo effect—the idea that your hope, faith, or belief is the actual active ingredient and that the medicine is not what’s helping.
A particular type of study is designed to take the placebo effect into account. You may have heard of a “double-blind, placebo-controlled” study. This is when researchers try a medication on a large group of people split into two groups: a “control group” (they receive a medication or technique that doesn’t have any active ingredient in it, a sugar pill, for example) and a treatment group that receives the medication or treatment. These types of studies, when well designed and carried out, help to remove the placebo effect. Researchers design these types of studies to test the effectiveness of all sorts of therapies, medications, and alternative techniques. These studies are considered the gold standard in medical research—they do a great job of helping clarify just what is an effective intervention, and what you don’t need to waste money trying.
Some clinics or rehab facilities advertise that they use “evidence-based” interventions. This means that they look to recent, well-designed research studies for the best ideas about what works, and then use those methods. Evidence-based interventions increase your chances of getting better; they stack the deck in your favor because there is research to indicate that what you and your therapist are doing actually works. And feeling happier, functioning better, and living well are always the ultimate goals.