Categorizing Scientific Studies: From Practice-Changing to Meaningless MOOPs

The skeptical cardiologist typically spends many hours a week scanning scientific publications, looking at past research, and analyzing relevant cardiovascular information.

Within a few minutes of looking at a study, I have a pretty good idea of which of three categories to put them in.

Practice-Changing Studies

The most exciting category is “Studies Which Will Change My Clinical Practice and Impact My Patients.” Unfortunately, these are few and far between.

When these have come along in the last 10 years in the cardiovascular arena I have generally written about them in detail on this site. 

Examples of this include the SPRINT study in hypertension, the emergence of TAVR for the treatment of aortic stenosis, the PREDIMED study in nutrition, and the ISCHEMIA trial in coronary artery disease.

Of note, all of the above-mentioned studies were high-quality randomized controlled trials (RCTs) and therefore represent science’s best method of knowing the truth.

Confirmation of My Current Practice

Many studies fall into the category of “Studies which confirm my current clinical practice”, meaning they confirm what I had concluded based on the previous scientific information on a clinically relevant topic.

If such a study is of note because it differs from guidelines or mainstream thinking I typically have written an update of my prior articles. This happens quite frequently in the nutrition field where the scientific literature has evolved and now is in conflict with seemingly ossified mainstream dogma and guidelines.

An example of this in the field of preventive cardiology was published this week in one of the JACC journals and is entitled “When Does a Calcium Score Equate to Secondary Prevention?: Insights From the Multinational CONFIRM Registry.”

My liberal clinical use of and strident advocacy for utilizing scans coronary artery calcium scans to better characterize individual risk for atherosclerotic cardiovascular disease has been consistently ahead of guidelines and this study supports my clinical approach. However, because this study adds little or nothing to the field or changes my clinical practice I wouldn’t write about it.

Of course, being a good scientist means looking out for studies that don’t confirm my current perspective on cardiovascular disease and avoiding “confirmation bias.”

Meaningless and/or Overblown Observational Studies

The vast majority of scientific publications these days fall into the category I will term “Meaningless and/or Overblown Observational Publications” (MOOPs) and these, as many have noted, tend to create media headlines and social media buzz.

These studies are easy to do and a profusion of low-quality scientific journals, eager for content snap, them up. Likewise, online and print journalists, eager for content, clicks, and readers, create click-bait headlines and articles that frequently overstate the significance of the MOOP.

Academic scientists are frequently complicit in promoting the significance of these very insignificant papers because their careers and advancement are dependent on publication and recognition.

This process in the last 10 years has substantially contributed to a public that has lost confidence in science as a guide to their health.

The field of nutritional epidemiology, the area that transformed me into a skeptical cardiologist, is particularly susceptible to the MOOP but we’ve also seen it most recently and prominently in COVID-19-related publications on long COVID, Paxlovid therapy, masking and vaccine effectiveness.

It is very tempting (and disappointingly common in those scientists/doctors who are biased) to endorse a MOOP if it supports your viewpoint and savagely critique MOOPs that don’t.

I try hard to approach all MOOPs with the same “unbiased” critical approach.

For example, although I advocate that my sedentary patients increase their weekly minutes of moderate to vigorous exercise (and I have written tons on the importance of maintaining cardiofitness as we age), I have to admit the data supporting this (with rare exceptions) comes almost exclusively from observational studies.

A recent study (also published in JACC) on this topic was summarized by a Dutch media outlet as follows:

A new study by researchers at Radboud University Medical Center has found you don’t need to take 10,000 steps a day to reduce your chances of premature death. The magic number is 7,126.

The paper was not a “new” study it was a meta-analysis, a lumping together, of 12 previously performed MOOPs involving over 110,000 people. 

I’m happy to see the 10000-step myth debunked. I’m good with the idea supported by this paper that those who are really sedentary should start moving around more but to be consistent I have to point out these data are observational and therefore, fairly meaningless.

Here is the “Central Illustration*” from the paper. Look at the nice graphs which show a dramatic reduction in mortality as one proceeds from 2,000 steps to around 6000 steps.

Those graphs and these analyses, however, are only showing associations, the cause of which is unclear. Proponents of more steps would like you to believe that increasing your step count from 2000 to 6000 will result in a 50% reduction in your chances of dying over a 5-year period. However, if we are honest and consistent we have to state clearly that they don’t prove that.

Those individuals who choose to or can only walk the remarkably low amount of 2000 steps per day are radically different from those who choose to or are capable of walking 7126 steps per day. While some of these differences can be accounted for (age, sex, hypertension, etc.), many can’t. These are unmeasured and, therefore confounding variables.

Thus, we cannot state that this paper proves that increasing your step count reduces your chances of premature death. To prove this statement we would have to take two groups of very similar individuals and randomly assign one group to increase their step count and then assess death rates after a period of time. This experiment, a randomized controlled trial (RCT), would be a very strong indicator of causality.

It turns out an RCT on this topic was published in 2020 but no longevity experts mention it because it was negative. The authors of “The effect of exercise training for five years on all-cause mortality in older adults—the Generation 100 study: randomised controlled trial” found no difference in its primary end-point of overall mortality over 5 years in 1600 patients receiving active exercise training versus those told to follow the national guidelines of 150 minutes of moderate exercise weekly.

The Other Categories

There are a few other categories that medical publications fall into.

One that I find particularly irritating is “The Worthless Cardiac Imaging Study.”

Having spent a decade in academia doing research and writing papers primarily in the area of cardiovascular imaging I am very aware of the limitations and the limited value of single-center studies on new imaging techniques/processes/measurements. 

These studies are very easy to do, can easily find a low-rent medical journal that will publish them and add little to nothing to our clinical knowledge and practice. I won’t dwell on this category because the vast majority are not publicized widely and are of little interest to doctors and patients.

Another category is “Studies That Only I am Likely to be Interested in.” “Longitudinal Changes in Left Ventricular Diastolic Function in Late Life: The ARIC Study” is unlikely to be heavily tweeted (or Xed?) and an overblown headline related to it won’t be noted in the NY Times.

However, I am fascinated by age-related changes in diastolic function because a lot of the research I performed in my early career focused on Doppler-echocardiographic assessment of how the left ventricle relaxes and fills with blood.

I have previously written on the vagaries of the clinical diagnosis of diastolic function. Part of the difficulty cardiologists have in making sense of diastolic function is due to the longitudinal changes we see late in life. The speed at which the left ventricle relaxes declines progressively with aging and I theorize that this is the major cause of the age-related decline in cardio-fitness with aging.

Moopingly Yours,

ACP